System

Abstract

A system is provided.SOLUTION: A system comprising: means for an employee with a disability to input and register his / her own skill information; means for using a generative artificial intelligence model that generates an individualized support tool based on the skill information; means for providing the generated support tool to the employee; means for generating and providing a guideline for appropriate communication between the employee and employees around the employee; and means for proposing an appropriate guidance method as support for on-the-job training and tracking a training process.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] As the legal employment rate for employees with disabilities increases, companies must provide an environment in which people with disabilities can work comfortably. However, because the type and severity of disabilities vary from person to person, support tailored to each individual is required. Despite this situation, uniform support tools and methods are often insufficient, so there is a need for systems that are effective for both companies and employees. Furthermore, there is a need for specific support to ensure appropriate communication between employees and their colleagues and to provide effective on-the-job training. [Means for solving the problem]

[0005] The present invention is a system that includes a means for inputting and registering skill information for employees with disabilities, a means for using a generative artificial intelligence model to generate personalized assistance tools based on the skill information, a means for providing the generated assistance tools, a means for generating and providing guidelines for appropriate communication between the employee and other employees, and a means for proposing appropriate training methods to support on-the-job training and tracking the training process. The generative artificial intelligence model takes into account the employee's skill information and the type and degree of disability to generate personalized assistance tools and step-by-step training procedures. This provides a system that can provide an environment in which employees with disabilities can immediately contribute to work and improve productivity across the entire company.

[0006] "Employee" refers to anyone who works for a company or organization, including employees with disabilities.

[0007] "Skill information" refers to the specific abilities, knowledge, experience, and related information possessed by an employee.

[0008] "Assistive devices" are tools and applications used by employees to perform their work, and are provided to meet the individual needs of the employee.

[0009] A "generative artificial intelligence model" refers to AI technology that automatically generates assistive devices based on input data and provides services tailored to individual needs.

[0010] "Communication guidelines" are guidelines and rules that allow employees and those around them to communicate smoothly with each other.

[0011] "On-the-job training (OJT)" refers to a training method in which employees acquire the skills necessary for their jobs through actual work.

[0012] "Instructional procedures" refer to step-by-step instructions or methods provided as part of on-the-job training that show employees how to perform specific tasks.

[0013] A "training process" refers to a series of training activities that an employee undertakes to improve their job skills, including tracking their progress. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective. The system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between employees and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process.

[0036] System configuration

[0037] 1. Input and registration of employee skill information

[0038] User: Enter their skill information (e.g., type and degree of disability, programming experience, etc.) through a dedicated application or website.

[0039] Device: Validate input and check skill information format.

[0040] Terminal: Sends skill information to the server.

[0041] 2. Activation of the generative AI model and generation of the assistive device

[0042] Server: Receives skill information sent from the device.

[0043] Server: Registers skill information in the database and launches the generative AI model.

[0044] Server: Generates personalized assistive devices based on the user's skill information and the type and degree of disability.

[0045] Server: Sends the generated assistive devices to the user's device.

[0046] 3. Provision and use of assistive devices

[0047] Terminal: Receives and stores the generated assistive devices.

[0048] Device: Provide the assistive device to the user in an appropriate format (e.g., audio description, text, visual aid).

[0049] User: Performs daily tasks using the assistive devices provided.

[0050] 4. Creation and provision of communication guidelines

[0051] Server: Generates communication guidelines appropriate for the user and their surrounding employees.

[0052] Server: Sends guidelines to the user's and nearby employees' devices.

[0053] Device: View and apply received guidelines.

[0054] 5. Supporting OJT and tracking progress

[0055] Server: Generates OJT training procedures based on the user's skill information and job requirements.

[0056] Server: Sends step-by-step instruction instructions to the user's device.

[0057] Terminal: Displays the received instruction procedure and provides it to the user.

[0058] User: Follow the provided instructional procedures and conduct OJT.

[0059] Terminal: Records the user's training progress in real time and sends it to the server.

[0060] Server: Records and analyzes progress in a database.

[0061] Server: Creates training progress reports and provides them to managers and companies.

[0062] Specific examples

[0063] New employee A with a visual impairment

[0064] 1. Enter and submit skill information

[0065] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[0066] Device: Checks the data and sends it to the server.

[0067] 2. Receiving and registering skill information

[0068] Server: Receives information sent from the device and registers it in a database.

[0069] 3. Generation of assistive devices

[0070] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[0071] Server: Sends tools to the device.

[0072] 4. Use of assistive devices

[0073] Terminal: Receive and save tools.

[0074] User: Launches the tool and performs programming tasks.

[0075] 5. Communication support

[0076] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[0077] Terminal: Receives the guidelines and displays them to employees.

[0078] Users: Communicate according to guidelines.

[0079] 6. Support for OJT

[0080] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[0081] Device: Save the guide and provide it to Person A.

[0082] User: Use the guide to progress through the task.

[0083] In this way, this system provides effective support for employees with disabilities and companies, creating an environment in which employees can function immediately as contributions to the workplace.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, a screen for entering your skill information will appear.

[0087] Step 2:

[0088] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[0089] Step 3:

[0090] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[0091] Step 4:

[0092] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[0093] Step 5:

[0094] Server: Activates a generative AI model based on the registered skill information. The AI ​​model then begins the process of generating personalized assistive devices tailored to the user's needs.

[0095] Step 6:

[0096] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[0097] Step 7:

[0098] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[0099] Step 8:

[0100] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[0101] Step 9:

[0102] User: Activates the received assistive device and uses it for their own work. Any problems or feedback that arise while using the assistive device can be reported to the server via the device.

[0103] Step 10:

[0104] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[0105] Step 11:

[0106] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[0107] Step 12:

[0108] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[0109] Step 13:

[0110] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[0111] Step 14:

[0112] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[0113] Step 15:

[0114] Server: Sends the generated OJT instruction procedures to the user's terminal.

[0115] Step 16:

[0116] Terminal: Receives the instructional steps and displays them to the user.

[0117] Step 17:

[0118] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[0119] Step 18:

[0120] Terminal: Records the progress of work in real time and sends it to the server.

[0121] Step 19:

[0122] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[0123] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company.

[0124] Example 1

[0125] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] In conventional employment environments for employees with disabilities, there is often a lack of support to fully utilize the skills of the employees themselves, and appropriate communication methods and effective training methods are often lacking. As a result, it takes time for employees to become immediately effective, which can lead to a decrease in productivity throughout the organization. The purpose of this invention is to solve these problems and provide an environment in which employees with disabilities can perform their duties quickly and effectively.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0128] In this invention, the server includes a means for recording the progress of the employee's training and transmitting it to the server in real time, a means for generating a training progress report and providing it to a manager or company, and a means for generating and providing guidelines for appropriate communication between the employee and other employees around him / her. This makes it possible to grasp the training progress in real time and provide appropriate communication and instruction methods so that the employee can quickly become a work-ready employee.

[0129] "Employee" refers to a working individual employed by a company or organization.

[0130] "Skill information" refers to detailed data such as an employee's abilities, experience, and knowledge.

[0131] A "generative artificial intelligence model" refers to a machine learning model that automatically generates specific outputs based on input data.

[0132] "Assistive devices" refers to tools, software, equipment, etc. that are useful to employees in carrying out their work.

[0133] "Communication guidelines" refer to guidelines and rules for employees and those around them to communicate appropriately.

[0134] "On-the-job training (OJT)" refers to practical training that takes place at the workplace.

[0135] "Training progress report" refers to a report that records and analyzes the progress of an employee's training.

[0136] "Real-time" refers to data and information being processed and provided almost immediately.

[0137] This invention is a support system for improving the employment environment for employees with disabilities, providing maximum support so that employees can work immediately. This system is composed of the following elements:

[0138] Input and registration of employee skill information

[0139] Users use a dedicated application or website to input their own skill information (e.g., type and degree of disability, programming experience, etc.) This data is received by the terminal, and after format confirmation and matching, it is sent to the server.

[0140] Activation of generative artificial intelligence models and generation of support tools

[0141] The server stores the received skill information in a database and activates a generative AI model. The AI ​​model generates personalized assistive devices (e.g., programming tools with voice feedback) based on the received skill information and the type and degree of disability. The generated assistive devices are sent from the server to the device.

[0142] Provision and use of assistive devices

[0143] The device receives the generated assistive devices, stores them, and provides them to the user in an appropriate format (e.g., audio guide, text, visual aids). The user uses the provided assistive devices to perform daily tasks.

[0144] Creation and provision of communication guidelines

[0145] The server generates communication guidelines suitable for the user and the surrounding employees, and transmits the guidelines to the terminals of the user and the surrounding employees. The terminals display and apply the received guidelines.

[0146] Supporting on-the-job training and tracking progress

[0147] The server generates OJT training procedures based on the user's skill information and job requirements, and sends the step-by-step training procedures to the user's device. The device displays the received training procedures and provides them to the user. The user follows the provided training procedures to carry out the OJT. The device records the training progress in real time and sends it to the server. The server records the progress in a database, creates a progress report, and provides it to managers and companies.

[0148] As a concrete example, consider the case of a new employee, Mr. A, who is visually impaired.

[0149] The user uses Mr. A's dedicated app to enter information about his visual impairment and programming experience.

[0150] The terminal checks the entered data and sends it to the server.

[0151] The server uses a generative AI model to generate programming tools with voice feedback for visually impaired people and sends them to the device.

[0152] The terminal receives the tool and provides it to the user. Person A uses the tool to perform his / her work.

[0153] The server generates appropriate communication methods for employees with visually impaired people and sends them to their terminals.

[0154] The terminal receives the guidelines and displays them to the employee. Mr. A and the employee communicate according to the guidelines.

[0155] The server generates a step-by-step audio guide for the programming task and sends it to the device. The device saves the guide and provides it to Person A. Person A uses the guide to progress through the task.

[0156] Example prompt sentence:

[0157] "Please create assistive tools to help new employees with visual impairments to smoothly carry out programming tasks."

[0158] "Please suggest effective ways for deaf people to communicate with their superiors."

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] Input and registration of employee skill information

[0162] The user opens a dedicated application or website.

[0163] The user fills out an input form with the type and degree of disability and skill information (e.g., programming experience, languages ​​fluent, etc.).

[0164] (Input) Type and degree of disability, skill information

[0165] (Processing) The terminal receives the input, performs format validation and matching, and checks that required fields are filled in.

[0166] (Output) Confirmed skill information

[0167] (Operation) The terminal sends the confirmed skill information to the server.

[0168] Step 2:

[0169] Activation of generative artificial intelligence models and generation of support tools

[0170] The server receives the skill information sent from the terminal.

[0171] (Input) Confirmed skill information

[0172] (Processing) The server stores the received information in a database and activates a generative AI model. Information about the user's skills and the type and degree of disability is obtained from the database, and a prompt is generated based on this information. The prompt is input into the generative AI model, which then generates an individualized assistive device.

[0173] (Output) Generated support equipment

[0174] (Operation) The server sends the generated support tool to the user's terminal.

[0175] Step 3:

[0176] Provision and use of assistive devices

[0177] The terminal receives the assistive tool transmitted from the server.

[0178] (Input) Generated support equipment

[0179] (Processing) The device stores the received assistive devices and provides them to the user in an appropriate format (e.g., audio guide, text, visual aid).

[0180] (Output) Assistive devices provided in a format that can be used by the user

[0181] (Action) The user uses the provided assistive device to perform daily tasks.

[0182] Step 4:

[0183] Creation and provision of communication guidelines

[0184] The server generates communication guidelines suitable for the user and the employees around him.

[0185] (Input) Information about the user and surrounding employees

[0186] (Processing) The server generates guidelines and transmits them to the terminals of the user and nearby employees.

[0187] (Output) Generated communication guidelines

[0188] (Operation) The terminal displays the received guidelines and applies them to the user and the employees around him / her.

[0189] Step 5:

[0190] Supporting on-the-job training and tracking progress

[0191] The server generates OJT training procedures based on the user's skill information and job requirements.

[0192] (Input) User skill information, job requirements

[0193] (Processing) The server generates step-by-step instruction procedures and sends them to the user's terminal.

[0194] (Output) Generated teaching steps

[0195] (Operation) The terminal displays the received training procedure and provides it to the user. The user follows the provided training procedure and performs OJT. The terminal records the user's training progress in real time and sends it to the server. The server records the progress in a database, creates a training progress report, and provides it to the manager or company.

[0196] (Application example 1)

[0197] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0198] In order for employees with disabilities to be immediately productive in the workplace, it is essential to provide them with individual assistive devices and appropriate communication support. However, conventional systems were unable to provide detailed support tailored to the type and degree of disability, and it was difficult to provide sufficient support in special situations such as driving assistance. This resulted in employees with disabilities being unable to fully utilize their abilities, and companies were ultimately unable to maximize their productivity. The purpose of this invention is to solve these problems.

[0199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0200] In this invention, the server includes: means for a disabled employee to input and register his / her skill information; means for using a generative artificial intelligence model to generate personalized assistive devices and driving assistive devices based on the skill information; means for providing the generated assistive devices and driving assistive devices to the disabled employee; means for generating and providing guidelines for appropriate communication between the disabled employee and other employees around him / her; means for suggesting appropriate teaching methods to support on-the-job training and tracking the training progress; means for providing real-time assistance during driving using audio feedback and visual aids; and means for tracking the progress of driving tasks in real time and providing improvement suggestions and further assistance as needed. This enables disabled employees to work efficiently while receiving assistance tailored to their individual needs, and makes it possible to provide sufficient support, especially in special situations such as driving assistance.

[0201] "Employees with disabilities" refers to workers who have physical or sensory disabilities and require special assistance tailored to their individual needs.

[0202] "Skill information" refers to information about an employee's skills and experience, as well as the type and degree of disability.

[0203] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data and generates appropriate output for a specific purpose.

[0204] "Assistive devices" refers to assistive devices and software that employees with disabilities use to perform their daily work or specific tasks.

[0205] "Driver assistance devices" refers to devices and systems that are specifically designed to assist employees with disabilities while driving a motor vehicle.

[0206] The "Communication Guidelines" provide guidelines for smooth communication between employees with disabilities and those around them.

[0207] "On-the-job training (OJT)" refers to a training method in which employees acquire skills through actual work.

[0208] "Voice feedback" refers to a function that uses voice to convey information and instructions while working, etc.

[0209] "Visual aids" refers to devices or software that provide visual support.

[0210] "Real-time support" refers to support provided in response to the current situation.

[0211] "Driving task progress tracking" refers to monitoring and recording the progress of driving tasks in real time.

[0212] "Improvement suggestions and further support" refers to suggestions for improvements or problems found during the driving task or work, and providing additional support as needed.

[0213] The present invention is a system for supporting disabled employees in working efficiently and safely in a work environment, and particularly focuses on driving assistance for autonomous vehicles. Specific embodiments for implementing the present invention will be described below.

[0214] Hardware and software used

[0215] Hardware

[0216] Smartphone: Used by the employee to input skill information and receive feedback while driving.

[0217] Smart glasses: As a visual aid, they display driver assistance information while driving.

[0218] Head-mounted display: Displays assistance information and provides audio guidance while driving.

[0219] In-car robots: Providing real-time assistance while driving.

[0220] software

[0221] Generative AI model: Generates personalized assistive devices and driving assistance tools based on employee skill information.

[0222] System configuration

[0223] 1. Input and registration of employee skill information

[0224] The user uses a smartphone to input information about their skills and the type and degree of their disability.

[0225] The device checks the input, formats the skill information, and sends it to the server.

[0226] 2. Generation of assistive devices and driving assistive devices

[0227] The server receives the skill information sent from the terminal and registers it in a database.

[0228] A generative artificial intelligence model is activated to generate personalized assistive devices and driving assistive devices based on the employee's skill information and the type and degree of disability.

[0229] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot.

[0230] 3. Provision and use of assistive devices and driving aids

[0231] The terminal receives and stores the generated equipment.

[0232] The device provides the user with assistive devices in the form of audio guidance, visual aids, haptic feedback, etc.

[0233] The user uses the assistive device to perform work or drive.

[0234] 4. Creation and provision of communication guidelines

[0235] The server generates and transmits communication guidelines suitable for the user and the employees around him / her.

[0236] The terminal receives and displays the guidelines.

[0237] 5. Supporting OJT and tracking progress

[0238] The server generates OJT instruction procedures based on the skill information and job requirements and transmits them to the terminal.

[0239] The terminal displays the instruction procedure and provides it to the user.

[0240] The user follows the instruction procedure and performs OJT.

[0241] The device records the training progress in real time and transmits it to the server.

[0242] 6. Tracking driving task progress and suggesting improvements

[0243] The server tracks the progress of the driving task in real time and generates improvement suggestions or further assistance as needed.

[0244] The terminal provides the generated suggestions and assistance methods to the user.

[0245] Specific examples

[0246] In the case of visually impaired driver B

[0247] 1. Enter and submit skill information

[0248] User B uses a smartphone to enter information about his visual impairment and his level of vision.

[0249] Terminal: Checks the data and sends it to the server.

[0250] 2. Receiving and registering skill information

[0251] Server: Receives information sent from the terminal and registers it in a database.

[0252] 3. Generation of assistive devices and driving assistive devices

[0253] Server: Uses a generative AI model to generate a voice feedback system for the visually impaired.

[0254] Server: Sends the system to the smartphone.

[0255] 4. Use of assistive devices

[0256] Smartphone: Receives and stores voice feedback system.

[0257] User: Activates smartphone and uses voice feedback while driving.

[0258] 5. Communication support

[0259] Server: Generates and transmits appropriate communication methods for visually impaired people.

[0260] Device: Receives and applies guidelines, for example by alerting the driver to the sound of approaching vehicles.

[0261] 6. Support for OJT

[0262] Server: Generates step-by-step audio guidance for the driving task and sends it to the device.

[0263] Device: Save the guide and provide it to Person B.

[0264] User: Use the guide to safely complete the driving task.

[0265] Example prompts to input to the generative AI model

[0266] To the generative AI model:

[0267] Please create a driving assistance tool for a visually impaired user who needs to drive a car. The user has almost lost all of his / her sight. The tool needs to provide a proper audio feedback system and assist with obstacle detection and communication with other vehicles. Thank you in advance.

[0268] This system will help employees with disabilities to fully utilize their abilities, and will also enable companies to maximize their productivity.

[0269] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0270] Step 1:

[0271] Entering and submitting skill information

[0272] The user uses a smartphone to input their own skill information and the type and degree of disability. The device checks the input, formats the skill information, and sends it to the server. In this step, the input data includes the user's level of vision, programming experience, and required assistive functions. The device verifies the input data, formats it into the appropriate format, and transfers it to the server.

[0273] Step 2:

[0274] Receiving and registering skill information

[0275] The server receives the skill information sent from the terminal and registers it in the database. Specifically, it saves the received information in an SQL database and confirms that the skill information has been properly registered.

[0276] Step 3:

[0277] Generation of assistive devices and driving aids

[0278] The server launches a generative AI model to generate personalized assistive devices and driving aids based on the employee's skill information and the type and severity of their disability. At this stage, the AI ​​model inputs prompts and runs a process to generate assistive devices optimized for the employee's needs. Skill information and disability details are used as input data, and personalized feedback systems and visual aids are provided as output.

[0279] Step 4:

[0280] Providing the generated assistive devices

[0281] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot. Specifically, the generated digital feedback system, audio guide, etc. are transmitted to the terminal in file format and stored on the terminal.

[0282] Step 5:

[0283] Use of assistive devices

[0284] The device stores the generated assistive devices and converts them into a format to provide to the user. The user activates a smartphone or head-mounted display and uses audio guidance and visual aids to perform tasks or drive. For example, the user activates a voice feedback system stored on the smartphone and receives audio guidance while driving.

[0285] Step 6:

[0286] Creation and provision of communication guidelines

[0287] The server generates and transmits communication guidelines appropriate for the user and the employees around them. The data includes guidelines and examples of appropriate communication methods between employees. This is compiled into a document and sent to the terminal so that the employees around them can apply it.

[0288] Step 7:

[0289] Supporting on-the-job training and tracking progress

[0290] The server generates OJT training procedures based on skill information and job requirements and sends them to the terminal. It lists the step-by-step training procedures and provides them to the employee in the appropriate order. The terminal also records the user's progress in real time and sends it to the server. The server stores this progress in a database and generates progress reports as needed.

[0291] Step 8:

[0292] Tracking driving task progress and suggesting improvements

[0293] The server tracks the progress of the driving task in real time and generates suggestions for improvement or further assistance as needed. For example, feedback and suggestions for improvement from the obstacle detection system while driving are generated in real time and provided to the user's device, allowing the user to take immediate action.

[0294] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0295] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. This system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system provides more appropriate support tailored to the user's condition.

[0296] System configuration

[0297] 1. Input and registration of employee skill information

[0298] User: Enters their own skill information (e.g., type and degree of disability, past work experience, skills, etc.) through a dedicated application or website.

[0299] Device: Validate input and check skill information format.

[0300] Device: Converts skill information into JSON format and sends it to the server.

[0301] 2. Activation of the generative AI model and generation of the assistive device

[0302] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[0303] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[0304] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[0305] 3. Collecting emotional information using an emotion engine

[0306] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's tone of voice and facial expressions to determine their emotional state.

[0307] Terminal: Collects emotional information recognized by the emotion engine as data and sends it to the server.

[0308] 4. Provision and use of assistive devices

[0309] Server: Receives the emotion information sent from the emotion engine and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[0310] Terminal: Receives and stores the generated assistive devices. The assistive devices are displayed on the interface for easy access by the user.

[0311] User: Performs daily tasks using the assistive devices provided.

[0312] 5. Creation and provision of communication guidelines

[0313] Server: Generates communication guidelines appropriate for the user and their surrounding workers, including communication methods and considerations for visually impaired people.

[0314] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[0315] Device: View and apply received guidelines.

[0316] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[0317] 6. Supporting OJT and tracking progress

[0318] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[0319] Server: Sends the generated OJT instruction procedures to the user's terminal.

[0320] Terminal: Receives the instructional steps and displays them to the user.

[0321] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[0322] Terminal: Records the progress of work in real time and sends it to the server.

[0323] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[0324] Specific examples

[0325] New employee A, who has visual impairment and high stress

[0326] 1. Enter and submit skill information

[0327] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[0328] Device: Checks the data and sends it to the server.

[0329] 2. Receiving and registering skill information

[0330] Server: Receives information sent from the device and registers it in a database.

[0331] 3. Generation of assistive devices

[0332] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[0333] Server: Sends tools to the device.

[0334] 4. Stress assessment using an emotion engine

[0335] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[0336] Device: Sends data on high stress conditions to the server.

[0337] 5. Adjustment of assistive devices

[0338] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[0339] Server: Sends appropriately adjusted assistive devices to the device.

[0340] 6. Use of assistive devices

[0341] Terminal: Receives the adjusted assistive device and provides it to Person A.

[0342] User: Launches the tool and performs tasks.

[0343] 7. Communication Support

[0344] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[0345] Terminal: Receives the guidelines and displays them to employees.

[0346] Users: Communicate according to guidelines.

[0347] 8. Support for OJT

[0348] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[0349] Device: Save the guide and provide it to Person A.

[0350] User: Use the guide to progress through the task.

[0351] In this way, this system provides effective support for employees with disabilities and companies, creating an environment where employees can immediately function as a valuable asset. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[0352] The processing flow will be explained below.

[0353] Step 1:

[0354] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, the skill information input screen will be displayed.

[0355] Step 2:

[0356] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[0357] Step 3:

[0358] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[0359] Step 4:

[0360] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[0361] Step 5:

[0362] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[0363] Step 6:

[0364] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[0365] Step 7:

[0366] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[0367] Step 8:

[0368] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[0369] Step 9:

[0370] On the device: The emotion engine is activated to recognize the user's emotions in real time. The emotion engine analyzes voice tone and facial expressions to determine the user's emotional state (e.g., stress, excitement, concentration, etc.).

[0371] Step 10:

[0372] Terminal: Recognized emotion information is sent as data to the server.

[0373] Step 11:

[0374] Server: Receives emotional information and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[0375] Step 12:

[0376] Server: The adjusted assistive device is sent back to the user's device.

[0377] Step 13:

[0378] Device: Receives and saves the adjusted assistive devices. Notifies the user of the adjustments and provides feedback using icons and messages.

[0379] Step 14:

[0380] User: Uses the provided assistive devices to perform daily tasks. Any problems or feedback that arise during use are reported to the server via the device.

[0381] Step 15:

[0382] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[0383] Step 16:

[0384] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[0385] Step 17:

[0386] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[0387] Step 18:

[0388] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[0389] Step 19:

[0390] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[0391] Step 20:

[0392] Server: Sends the generated OJT instruction procedures to the user's terminal.

[0393] Step 21:

[0394] Terminal: Receives the instructional steps and displays them to the user.

[0395] Step 22:

[0396] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[0397] Step 23:

[0398] Terminal: Records the progress of work in real time and sends it to the server.

[0399] Step 24:

[0400] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[0401] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company. Furthermore, by combining it with an emotion engine, it is possible to dynamically adjust the support content according to the employee's emotional state, achieving more effective support.

[0402] Example 2

[0403] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0404] For many companies, providing appropriate assistive devices and a comfortable working environment for employees with disabilities is a difficult task. It is necessary to accurately grasp the employee's skill information, generate and provide customized assistive devices based on that information, and provide support that responds to fluctuations in their emotional state. However, current systems cannot effectively solve these issues.

[0405] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for an employee to input and register his / her own skill information, a means for generating an individualized assistive tool using a generative artificial intelligence model, and a means for providing the generated assistive tool to the employee. This makes it possible to generate and provide assistive tools tailored to the individual needs of employees with disabilities. In addition, by including a means for using a terminal equipped with an emotion engine that recognizes and collects the user's emotional information in real time and a means for adjusting the assistive tool based on the collected emotional information, it becomes possible to provide support that responds to changes in the employee's emotional state. This makes it possible to provide an environment in which employees can work comfortably and efficiently.

[0406] "Employee" refers to a person who is employed by a company or organization, and specifically includes people with disabilities.

[0407] "Skills information" refers to information including an employee's specific skills, past work experience, educational background, and the type and degree of disability.

[0408] A "dedicated application" is software designed to provide specific operations or functions, and provides an interface for users to input skill information.

[0409] "Generative AI model" refers to an AI technology for automatically generating assistive devices and other suggestions based on a worker's skill information and needs.

[0410] "Assistive devices" refers to the assistive devices and software tools that employees with disabilities need to perform their work efficiently and comfortably.

[0411] "Terminal" refers to an electronic device such as a computer, smartphone, or tablet, which is a device that allows a user to use this system.

[0412] "Server" refers to a central computer system that stores, processes data, and responds to various requests.

[0413] "Emotion engine" refers to artificial intelligence technology that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[0414] "Communication guidelines" refers to documents and tools that explain the guidelines and methods for employees and those around them to communicate appropriately.

[0415] "On-the-job training (OJT)" refers to a training method in which employees learn and improve their skills through actual work.

[0416] "Progress" refers to information that indicates the progress of an employee in completing a task or training.

[0417] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. The system uses a generative artificial intelligence model that generates personalized assistive devices based on the employee's skill information, which is input through a dedicated application or website. It also includes an emotion engine that collects the user's emotional information and adjusts the assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, supporting on-the-job training (OJT).

[0418] Hardware and software used

[0419] 1. Input and registration of employee skill information

[0420] The user enters skill information using a dedicated application (e.g., a smartphone app) or website.

[0421] The terminal verifies the entered skill information, converts the skill information into JSON format, and sends it to the server.

[0422] 2. Activation of the generative AI model and generation of the assistive device

[0423] The server receives the skill information in JSON format and registers it in a database. The technology used is a database management system (DBMS).

[0424] Based on the registered skill information, a generative artificial intelligence model (e.g., TensorFlow or PyTorch) is launched to generate personalized assistive devices that meet the user's needs.

[0425] The server transmits the generated assistive tool to the user's terminal.

[0426] 3. Collecting emotional information using an emotion engine

[0427] The device launches an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's voice tone and facial expressions.

[0428] The device transmits the collected emotion information to the server.

[0429] 4. Provision and use of assistive devices

[0430] The server adjusts the assistive device based on the emotion information and sends the adjusted assistive device to the user's terminal.

[0431] The terminal provides the user with the adjusted assistive device, which the user uses to perform the task.

[0432] 5. Creation and provision of communication guidelines

[0433] The server generates communication guidelines suitable for the user and surrounding employees and transmits them to the terminal.

[0434] The terminal displays guidelines, which employees follow when communicating.

[0435] 6. Supporting OJT and tracking progress

[0436] The server generates OJT training procedures based on the employee's skill information and job requirements.

[0437] The server sends the generated OJT procedure to the user, and the terminal displays it.

[0438] The user follows the instructional procedures to carry out OJT, records their progress through the terminal, and the server receives the data and tracks their progress.

[0439] Specific examples

[0440] New employee A, who has visual impairment and high stress

[0441] 1. Enter and submit your skill information:

[0442] User: Person A uses a dedicated app to enter that he or she is visually impaired and has programming experience.

[0443] Device: Checks this data and sends it to the server.

[0444] 2. Receiving and registering skill information:

[0445] Server: Receives information sent from the device and registers it in a database.

[0446] 3. Generating assistive devices:

[0447] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[0448] Server: Sends tools to the device.

[0449] 4. Stress assessment using emotion engine:

[0450] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[0451] Device: Sends data on high stress conditions to the server.

[0452] 5. Adjustment of assistive devices:

[0453] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[0454] Server: Sends appropriately adjusted assistive devices to the device.

[0455] 6. Use of assistive devices:

[0456] Terminal: Receives the adjusted assistive device and provides it to Person A.

[0457] User: Launches the tool and performs tasks.

[0458] Prompt Sentence Examples

[0459] "I am visually impaired and have three years of programming experience. Please suggest the best assistive device for me."

[0460] "Currently in a state of high stress. Please adjust your support equipment accordingly."

[0461] In this way, this system provides effective support for employees with disabilities and companies, and plays an important role in creating an environment where employees can function immediately as a workforce. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[0462] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0463] Step 1:

[0464] The user enters skill information. Using a dedicated application or website, employees enter their skill information (type of disability, skills, work experience, etc.). The entered information is saved in fields on the form. Specific rules apply to the format of the entered data.

[0465] Input: User's skills (e.g., "I am visually impaired and have 3 years of programming experience").

[0466] Output: Skill information saved in the input form.

[0467] Step 2:

[0468] The terminal validates the input data. The terminal validates the input data to ensure it conforms to the required format. Once validation is complete, it converts the data into JSON format.

[0469] Input: Skill information saved in the input form.

[0470] Data processing: Format check and validation, conversion to JSON format (e.g., {"Disability type": "Visual impairment", "Skills": "Programming", "Years of experience": 3}).

[0471] Output: Validated data in JSON format.

[0472] Step 3:

[0473] The device sends JSON format data to the server. The validated JSON data is sent to the server using an HTTP request.

[0474] Input: Validated data in JSON format.

[0475] Data operations: creating and sending HTTP requests.

[0476] Output: Skill information data sent to the server.

[0477] Step 4:

[0478] The server receives the skill information in JSON format and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[0479] Input: Submitted skill information in JSON format.

[0480] Data processing: INSERT operations into the database (e.g., registering data into the skill information table).

[0481] Output: Skill information registered in the database.

[0482] Step 5:

[0483] The server launches the generative AI model, which starts the process of generating appropriate assistive devices based on the skill information registered in the database.

[0484] Input: Skill information registered in the database.

[0485] Data computation: Initialization and execution of generative AI models.

[0486] Output: The process of creating a personalized assistive device begins.

[0487] Step 6:

[0488] The server generates assistive devices. Using generative AI models, it generates personalized assistive devices tailored to the user's needs, such as programming tools with voice feedback for the visually impaired.

[0489] Input: The parameters required for the generation process.

[0490] Data computation: Generation of assistive devices based on AI models.

[0491] Output: Individualized assistive devices.

[0492] Step 7:

[0493] The server sends the generated assistive device to the user's device, and returns the link or file of the generated assistive device to the device as an HTTP response.

[0494] Input: The generated assistive device.

[0495] Data operations: Creating and sending HTTP responses.

[0496] Output: Assistive devices sent to the user's device.

[0497] Step 8:

[0498] The device provides the assistive tools to the user and displays them on the interface for easy access by the user.

[0499] Input: Assistive device sent from the server.

[0500] Data calculation: Display settings for assistive devices.

[0501] Output: The assistive device provided to the user.

[0502] Step 9:

[0503] The user uses the assistive device. The user performs daily tasks using the provided assistive device.

[0504] Input: Assistive devices provided to the device.

[0505] Data calculation: Use of assistive devices.

[0506] Output: A deliverable that is useful for business.

[0507] Step 10:

[0508] The device activates an emotion engine to recognize the user's emotions, analyzing voice tone and facial expressions to determine the user's emotional state in real time.

[0509] Input: User audio and video data.

[0510] Data Computing: Sentiment Analysis.

[0511] Output: Determined emotion information.

[0512] Step 11:

[0513] The device sends emotional information to the server, which converts the recognized emotional information into JSON format and sends it to the server as an HTTP request.

[0514] Input: Determined emotion information.

[0515] Data operation: Creating and sending JSON format.

[0516] Output: Emotion information sent to the server.

[0517] Step 12:

[0518] The server receives the emotional information and adjusts the assistive device. For example, if the user is feeling stressed, the server generates an assistive device with relaxing music or additional guidance.

[0519] Input: The submitted emotion information.

[0520] Data calculation: Adjustment of assistive devices.

[0521] Output: Coordinated assistive devices.

[0522] Step 13:

[0523] The server sends the adjusted assistive device to the user's device, and then sends the appropriately adjusted assistive device again as an HTTP response.

[0524] Input: Adjusted assistive device.

[0525] Data operations: Creating and sending HTTP responses.

[0526] Output: The adjusted assistive devices are sent to the device.

[0527] Step 14:

[0528] Providing users with assistive devices that are adjusted to their devices, providing a relaxing environment for users to work comfortably.

[0529] Input: Calibration aid sent from server.

[0530] Data calculation: Display settings for assistive devices.

[0531] Output: The adjustment aid provided to the user.

[0532] Step 15:

[0533] The server generates and provides communication guidelines for the user and their surrounding employees, and sends them to the terminal.

[0534] Input: Information about the user and surrounding workers.

[0535] Data arithmetic: generating guidelines.

[0536] Output: The generated guidelines.

[0537] Step 16:

[0538] The terminal displays the communication guidelines. The guidelines are displayed on the terminal so that the user and nearby employees can easily refer to them.

[0539] Input: Communication guidelines sent by the server.

[0540] Data calculation: Guideline display settings.

[0541] Output: The displayed guidelines.

[0542] Step 17:

[0543] The server generates and transmits OJT training procedures based on the employee's skill information and job requirements.

[0544] Input: User skill information and job requirements.

[0545] Data calculation: Generation of OJT instruction procedures.

[0546] Output: Generated OJT instruction procedures.

[0547] Step 18:

[0548] The terminal displays the OJT instruction procedure. The procedure for the user to carry out the OJT is displayed on the interface.

[0549] Input: OJT training procedure sent from the server.

[0550] Data calculation: Display settings for instructional procedures.

[0551] Output: Displayed OJT instruction procedures.

[0552] Step 19:

[0553] The user performs OJT by following the provided instructional procedures to carry out the tasks and recording the progress in real time via the terminal.

[0554] Input: OJT training procedures.

[0555] Data calculation: Performing tasks and recording progress.

[0556] Output: Real-time recorded progress.

[0557] Step 20:

[0558] The device sends the progress to the server. The device sends the progress data recorded in real time to the server.

[0559] Input: Progress data recorded in real time.

[0560] Data calculation: Sending progress data.

[0561] Output: Progress data sent to the server.

[0562] Step 21:

[0563] The server records the training progress in a database and generates a report. Based on the progress data, the server stores the training progress in a database and generates a progress report.

[0564] Input: The progress data submitted.

[0565] Data calculation: recording to database and generating reports.

[0566] Output: The generated progress report.

[0567] (Application example 2)

[0568] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0569] It is important to create an environment in which employees with disabilities can immediately contribute to the workplace, but it is difficult to properly manage employees' skill and emotional information and provide them with personalized assistive devices. There is also a need for systems that improve communication between employees and those around them and enable employees to perform their work efficiently in a mentally stable state. Furthermore, there is a need to effectively support on-the-job training (OJT) and track the training process. The purpose of this invention is to solve these problems.

[0570] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0571] In this invention, the server includes: a means for an employee to input and register his / her own skill information; a means for using a generative artificial intelligence model to generate personalized assistive devices based on the skill information; a means for providing the generated assistive devices to the employee; a means for generating and providing guidelines for appropriate communication between the employee and other employees around him / her; a means for suggesting appropriate teaching methods to support on-the-job training and tracking the training progress; and a means for collecting employee emotion information using an emotion engine and adjusting the assistive devices. This makes it possible to create an environment in which employees can work immediately.

[0572] An "employee with a disability" is an employee who has a physical or mental disability and requires special assistance or consideration to accommodate that disability.

[0573] "Skill information" refers to information such as the skills, experience, and qualifications that employees possess.

[0574] "Individualized assistive devices" are devices created based on the skill information of a specific employee and the type and degree of disability, with the aim of assisting the employee in their work.

[0575] A "generative artificial intelligence model" is an artificial intelligence model that generates an output (here, an assistive device) according to a specific purpose based on input data.

[0576] The "Emotion Engine" is a technology that analyzes and recognizes emotional states in real time from voice tone and facial expressions.

[0577] "On-the-job training (OJT)" is workplace training conducted through actual work, and is an educational method in which knowledge and skills required for work are acquired through practice.

[0578] "Communication guidelines" are guidelines that compile specific instructions and advice for achieving appropriate communication.

[0579] This invention is a support system for improving the work efficiency of disabled employees in the workplace. The system uses a generative AI model that generates personalized assistive devices based on the employee's skill information. It also combines an emotion engine to provide support tailored to the employee's emotional state.

[0580] System configuration

[0581] 1. Input and registration of employee skill information

[0582] Users use a dedicated application or website to enter their skills information (e.g., type and degree of disability, past work experience, skills, etc.).

[0583] The terminal checks the entered skill information, converts it into JSON format, and sends it to the server.

[0584] 2. Activation of the generative AI model and generation of the assistive device

[0585] The server receives the JSON-formatted skill information sent from the device, analyzes it, and registers it in a database that includes information such as the user ID, type of disability, and skill content.

[0586] The server activates a generative AI model based on the registered skill information and generates personalized assistive tools that meet the user's needs.

[0587] The server generates and sends personalized assistive devices to the device, for example, a visually impaired user is provided with an assistive device with audio feedback.

[0588] 3. Collecting emotional information using an emotion engine

[0589] The device is equipped with an emotion engine that recognizes the user's emotions in real time by analyzing the user's tone of voice and facial expressions to determine their emotional state.

[0590] The terminal transmits the emotion information recognized by the emotion engine to the server.

[0591] 4. Provision and use of assistive devices

[0592] The server receives the emotional information and adjusts the assistive devices according to the user's current emotional state, for example, providing relaxation tools and guidance if the user is in a high-stress state.

[0593] The terminal displays the adjusted assistive tool to the user, allowing the user to easily access it.

[0594] The user performs daily tasks using the provided assistive devices.

[0595] 5. Creation and provision of communication guidelines

[0596] The server generates communication guidelines suitable for the user and the surrounding employees, and transmits them to the terminals of the user and the surrounding employees.

[0597] The terminal receives the guidelines and displays them to the user and nearby workers.

[0598] 6. Supporting OJT and tracking progress

[0599] The server generates OJT training procedures based on the employee's skill information and job requirements. These training procedures are provided in a step-by-step format.

[0600] The server transmits the generated OJT instruction procedure to the user's terminal.

[0601] The terminal receives the instruction procedure and displays it to the user.

[0602] The user follows the provided instructional procedures to carry out OJT, recording the progress of the work and any questions they may have in real time via the terminal.

[0603] The terminal records the progress of the work in real time and transmits it to the server.

[0604] The server records the progress in a database and generates reports to provide to administrators.

[0605] Specific examples

[0606] For example, if new employee A has a visual impairment but has programming experience, he or she can use a dedicated app to input his or her visual impairment and skills. The device then sends the data to a server, and an AI model generates a programming support tool with voice feedback. If the emotion engine determines that A is under high stress, relaxing music is added. The user and other employees perform their work according to appropriate communication guidelines, and on-the-job training is also supported step-by-step.

[0607] Example prompt sentence:

[0608] Mr. A is visually impaired and has programming experience. Please create an effective programming assistant tool with audio feedback that does not rely on vision for him. Also, please provide soothing music when he feels high stress.

[0609] This system allows employees with a variety of disabilities to function immediately and provides an environment in which they can work more effectively and safely in the workplace.

[0610] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0611] Step 1:

[0612] Users use a dedicated application or website to enter their own skill information (e.g., type and degree of disability, past work experience, skills, etc.) The entered data is converted into JSON format and sent to the server.

[0613] Input: User skill information

[0614] Output: JSON formatted data

[0615] Step 2:

[0616] The server receives the skill information in JSON format sent from the device, analyzes the received data, and registers the employee's ID, type of disability, skill information, etc. in a database.

[0617] Input: Skill information in JSON format

[0618] Output: Employee information stored in the database

[0619] Step 3:

[0620] The server activates a generative AI model based on the registered skill information, which generates personalized assistive devices tailored to the user's needs.

[0621] Input: Employee information stored in the database

[0622] Output: personalized assistive devices

[0623] Step 4:

[0624] The server transmits the generated assist tool to the terminal, which displays the received assist tool data to the user, making it available for use.

[0625] Input: personalized assistive devices

[0626] Output: Assistive device data displayed on the user's device

[0627] Step 5:

[0628] The device activates an emotion engine that recognizes the user's emotions in real time by analyzing voice tone and facial expressions to determine the user's emotional state.

[0629] Input: User's voice tone and facial expression data

[0630] Output: Perceived emotional state

[0631] Step 6:

[0632] The device sends the emotional information recognized by the emotion engine to the server, which then adjusts the assistive devices based on this emotional information. For example, if the user is in a high-stress state, it will add tools or guidance to help them relax.

[0633] Input: Perceived emotional state

[0634] Output: Adjusted assistive devices

[0635] Step 7:

[0636] The server generates communication guidelines suitable for the user and the employees around him / her and transmits them to the terminal, which then displays the received guidelines to the user and the employees around him / her.

[0637] Input: Database and emotion information

[0638] Output: Communication Guidelines

[0639] Step 8:

[0640] The server generates on-the-job training (OJT) instructions based on the employee's skill information and job requirements. The instructions are generated in a step-by-step format and sent to the terminal. The terminal displays the received instructions to the user and makes them available.

[0641] Input: Employee information and job requirements stored in the database

[0642] Output: OJT training procedure

[0643] Step 9:

[0644] Users follow the provided instructional procedures to carry out OJT. Any progress or questions they may have are recorded in real time on their devices and sent to the server. The server then records the progress in a database, generates a report, and provides it to the administrator.

[0645] Input: OJT progress and questions

[0646] Output: Progress report

[0647] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0648] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0649] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0650] [Second embodiment]

[0651] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0652] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0653] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0654] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0655] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0656] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0657] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0658] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0659] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0660] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0661] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0662] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0663] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective. The system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between employees and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process.

[0664] System configuration

[0665] 1. Input and registration of employee skill information

[0666] User: Enter their skill information (e.g., type and degree of disability, programming experience, etc.) through a dedicated application or website.

[0667] Device: Validate input and check skill information format.

[0668] Terminal: Sends skill information to the server.

[0669] 2. Activation of the generative AI model and generation of the assistive device

[0670] Server: Receives skill information sent from the device.

[0671] Server: Registers skill information in the database and launches the generative AI model.

[0672] Server: Generates personalized assistive devices based on the user's skill information and the type and degree of disability.

[0673] Server: Sends the generated assistive devices to the user's device.

[0674] 3. Provision and use of assistive devices

[0675] Terminal: Receives and stores the generated assistive devices.

[0676] Device: Provide the assistive device to the user in an appropriate format (e.g., audio description, text, visual aid).

[0677] User: Performs daily tasks using the assistive devices provided.

[0678] 4. Creation and provision of communication guidelines

[0679] Server: Generates communication guidelines appropriate for the user and their surrounding employees.

[0680] Server: Sends guidelines to the user's and nearby employees' devices.

[0681] Device: View and apply received guidelines.

[0682] 5. Supporting OJT and tracking progress

[0683] Server: Generates OJT training procedures based on the user's skill information and job requirements.

[0684] Server: Sends step-by-step instruction instructions to the user's device.

[0685] Terminal: Displays the received instruction procedure and provides it to the user.

[0686] User: Follow the provided instructional procedures and conduct OJT.

[0687] Terminal: Records the user's training progress in real time and sends it to the server.

[0688] Server: Records and analyzes progress in a database.

[0689] Server: Creates training progress reports and provides them to managers and companies.

[0690] Specific examples

[0691] New employee A with a visual impairment

[0692] 1. Enter and submit skill information

[0693] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[0694] Device: Checks the data and sends it to the server.

[0695] 2. Receiving and registering skill information

[0696] Server: Receives information sent from the device and registers it in a database.

[0697] 3. Generation of assistive devices

[0698] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[0699] Server: Sends tools to the device.

[0700] 4. Use of assistive devices

[0701] Terminal: Receive and save tools.

[0702] User: Launches the tool and performs programming tasks.

[0703] 5. Communication support

[0704] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[0705] Terminal: Receives the guidelines and displays them to employees.

[0706] Users: Communicate according to guidelines.

[0707] 6. Support for OJT

[0708] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[0709] Device: Save the guide and provide it to Person A.

[0710] User: Use the guide to progress through the task.

[0711] In this way, this system provides effective support for employees with disabilities and companies, creating an environment in which employees can function immediately as contributions to the workplace.

[0712] The processing flow will be explained below.

[0713] Step 1:

[0714] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, a screen for entering your skill information will appear.

[0715] Step 2:

[0716] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[0717] Step 3:

[0718] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[0719] Step 4:

[0720] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[0721] Step 5:

[0722] Server: Activates a generative AI model based on the registered skill information. The AI ​​model then begins the process of generating personalized assistive devices tailored to the user's needs.

[0723] Step 6:

[0724] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[0725] Step 7:

[0726] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[0727] Step 8:

[0728] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[0729] Step 9:

[0730] User: Activates the received assistive device and uses it for their own work. Any problems or feedback that arise while using the assistive device can be reported to the server via the device.

[0731] Step 10:

[0732] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[0733] Step 11:

[0734] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[0735] Step 12:

[0736] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[0737] Step 13:

[0738] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[0739] Step 14:

[0740] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[0741] Step 15:

[0742] Server: Sends the generated OJT instruction procedures to the user's terminal.

[0743] Step 16:

[0744] Terminal: Receives the instructional steps and displays them to the user.

[0745] Step 17:

[0746] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[0747] Step 18:

[0748] Terminal: Records the progress of work in real time and sends it to the server.

[0749] Step 19:

[0750] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[0751] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company.

[0752] Example 1

[0753] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0754] In conventional employment environments for employees with disabilities, there is often a lack of support to fully utilize the skills of the employees themselves, and appropriate communication methods and effective training methods are often lacking. As a result, it takes time for employees to become immediately effective, which can lead to a decrease in productivity throughout the organization. The purpose of this invention is to solve these problems and provide an environment in which employees with disabilities can perform their duties quickly and effectively.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0756] In this invention, the server includes a means for recording the progress of the employee's training and transmitting it to the server in real time, a means for generating a training progress report and providing it to a manager or company, and a means for generating and providing guidelines for appropriate communication between the employee and other employees around him / her. This makes it possible to grasp the training progress in real time and provide appropriate communication and instruction methods so that the employee can quickly become a work-ready employee.

[0757] "Employee" refers to a working individual employed by a company or organization.

[0758] "Skill information" refers to detailed data such as an employee's abilities, experience, and knowledge.

[0759] A "generative artificial intelligence model" refers to a machine learning model that automatically generates specific outputs based on input data.

[0760] "Assistive devices" refers to tools, software, equipment, etc. that are useful to employees in carrying out their work.

[0761] "Communication guidelines" refer to guidelines and rules for employees and those around them to communicate appropriately.

[0762] "On-the-job training (OJT)" refers to practical training that takes place at the workplace.

[0763] "Training progress report" refers to a report that records and analyzes the progress of an employee's training.

[0764] "Real-time" refers to data and information being processed and provided almost immediately.

[0765] This invention is a support system for improving the employment environment for employees with disabilities, providing maximum support so that employees can work immediately. This system is composed of the following elements:

[0766] Input and registration of employee skill information

[0767] Users use a dedicated application or website to input their own skill information (e.g., type and degree of disability, programming experience, etc.) This data is received by the terminal, and after format confirmation and matching, it is sent to the server.

[0768] Activation of generative artificial intelligence models and generation of support tools

[0769] The server stores the received skill information in a database and activates a generative AI model. The AI ​​model generates personalized assistive devices (e.g., programming tools with voice feedback) based on the received skill information and the type and degree of disability. The generated assistive devices are sent from the server to the device.

[0770] Provision and use of assistive devices

[0771] The device receives the generated assistive devices, stores them, and provides them to the user in an appropriate format (e.g., audio guide, text, visual aids). The user uses the provided assistive devices to perform daily tasks.

[0772] Creation and provision of communication guidelines

[0773] The server generates communication guidelines suitable for the user and the surrounding employees, and transmits the guidelines to the terminals of the user and the surrounding employees. The terminals display and apply the received guidelines.

[0774] Supporting on-the-job training and tracking progress

[0775] The server generates OJT training procedures based on the user's skill information and job requirements, and sends the step-by-step training procedures to the user's device. The device displays the received training procedures and provides them to the user. The user follows the provided training procedures to carry out the OJT. The device records the training progress in real time and sends it to the server. The server records the progress in a database, creates a progress report, and provides it to managers and companies.

[0776] As a concrete example, consider the case of a new employee, Mr. A, who is visually impaired.

[0777] The user uses Mr. A's dedicated app to enter information about his visual impairment and programming experience.

[0778] The terminal checks the entered data and sends it to the server.

[0779] The server uses a generative AI model to generate programming tools with voice feedback for visually impaired people and sends them to the device.

[0780] The terminal receives the tool and provides it to the user. Person A uses the tool to perform his / her work.

[0781] The server generates appropriate communication methods for employees with visually impaired people and sends them to their terminals.

[0782] The terminal receives the guidelines and displays them to the employee. Mr. A and the employee communicate according to the guidelines.

[0783] The server generates a step-by-step audio guide for the programming task and sends it to the device. The device saves the guide and provides it to Person A. Person A uses the guide to progress through the task.

[0784] Example prompt sentence:

[0785] "Please create assistive tools to help new employees with visual impairments to smoothly carry out programming tasks."

[0786] "Please suggest effective ways for deaf people to communicate with their superiors."

[0787] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0788] Step 1:

[0789] Input and registration of employee skill information

[0790] The user opens a dedicated application or website.

[0791] The user fills out an input form with the type and degree of disability and skill information (e.g., programming experience, languages ​​fluent, etc.).

[0792] (Input) Type and degree of disability, skill information

[0793] (Processing) The terminal receives the input, performs format validation and matching, and checks that required fields are filled in.

[0794] (Output) Confirmed skill information

[0795] (Operation) The terminal sends the confirmed skill information to the server.

[0796] Step 2:

[0797] Activation of generative artificial intelligence models and generation of support tools

[0798] The server receives the skill information sent from the terminal.

[0799] (Input) Confirmed skill information

[0800] (Processing) The server stores the received information in a database and activates a generative AI model. Information about the user's skills and the type and degree of disability is obtained from the database, and a prompt is generated based on this information. The prompt is input into the generative AI model, which then generates an individualized assistive device.

[0801] (Output) Generated support equipment

[0802] (Operation) The server sends the generated support tool to the user's terminal.

[0803] Step 3:

[0804] Provision and use of assistive devices

[0805] The terminal receives the assistive tool transmitted from the server.

[0806] (Input) Generated support equipment

[0807] (Processing) The device stores the received assistive devices and provides them to the user in an appropriate format (e.g., audio guide, text, visual aid).

[0808] (Output) Assistive devices provided in a format that can be used by the user

[0809] (Action) The user uses the provided assistive device to perform daily tasks.

[0810] Step 4:

[0811] Creation and provision of communication guidelines

[0812] The server generates communication guidelines suitable for the user and the employees around him.

[0813] (Input) Information about the user and surrounding employees

[0814] (Processing) The server generates guidelines and transmits them to the terminals of the user and nearby employees.

[0815] (Output) Generated communication guidelines

[0816] (Operation) The terminal displays the received guidelines and applies them to the user and the employees around him / her.

[0817] Step 5:

[0818] Supporting on-the-job training and tracking progress

[0819] The server generates OJT training procedures based on the user's skill information and job requirements.

[0820] (Input) User skill information, job requirements

[0821] (Processing) The server generates step-by-step instruction procedures and sends them to the user's terminal.

[0822] (Output) Generated teaching steps

[0823] (Operation) The terminal displays the received training procedure and provides it to the user. The user follows the provided training procedure and performs OJT. The terminal records the user's training progress in real time and sends it to the server. The server records the progress in a database, creates a training progress report, and provides it to the manager or company.

[0824] (Application example 1)

[0825] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0826] In order for employees with disabilities to be immediately productive in the workplace, it is essential to provide them with individual assistive devices and appropriate communication support. However, conventional systems were unable to provide detailed support tailored to the type and degree of disability, and it was difficult to provide sufficient support in special situations such as driving assistance. This resulted in employees with disabilities being unable to fully utilize their abilities, and companies were ultimately unable to maximize their productivity. The purpose of this invention is to solve these problems.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0828] In this invention, the server includes: means for a disabled employee to input and register his / her skill information; means for using a generative artificial intelligence model to generate personalized assistive devices and driving assistive devices based on the skill information; means for providing the generated assistive devices and driving assistive devices to the disabled employee; means for generating and providing guidelines for appropriate communication between the disabled employee and other employees around him / her; means for suggesting appropriate teaching methods to support on-the-job training and tracking the training progress; means for providing real-time assistance during driving using audio feedback and visual aids; and means for tracking the progress of driving tasks in real time and providing improvement suggestions and further assistance as needed. This enables disabled employees to work efficiently while receiving assistance tailored to their individual needs, and makes it possible to provide sufficient support, especially in special situations such as driving assistance.

[0829] "Employees with disabilities" refers to workers who have physical or sensory disabilities and require special assistance tailored to their individual needs.

[0830] "Skill information" refers to information about an employee's skills and experience, as well as the type and degree of disability.

[0831] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data and generates appropriate output for a specific purpose.

[0832] "Assistive devices" refers to assistive devices and software that employees with disabilities use to perform their daily work or specific tasks.

[0833] "Driver assistance devices" refers to devices and systems that are specifically designed to assist employees with disabilities while driving a motor vehicle.

[0834] The "Communication Guidelines" provide guidelines for smooth communication between employees with disabilities and those around them.

[0835] "On-the-job training (OJT)" refers to a training method in which employees acquire skills through actual work.

[0836] "Voice feedback" refers to a function that uses voice to convey information and instructions while working, etc.

[0837] "Visual aids" refers to devices or software that provide visual support.

[0838] "Real-time support" refers to support provided in response to the current situation.

[0839] "Driving task progress tracking" refers to monitoring and recording the progress of driving tasks in real time.

[0840] "Improvement suggestions and further support" refers to suggestions for improvements or problems found during the driving task or work, and providing additional support as needed.

[0841] The present invention is a system for supporting disabled employees in working efficiently and safely in a work environment, and particularly focuses on driving assistance for autonomous vehicles. Specific embodiments for implementing the present invention will be described below.

[0842] Hardware and software used

[0843] Hardware

[0844] Smartphone: Used by the employee to input skill information and receive feedback while driving.

[0845] Smart glasses: As a visual aid, they display driver assistance information while driving.

[0846] Head-mounted display: Displays assistance information and provides audio guidance while driving.

[0847] In-car robots: Providing real-time assistance while driving.

[0848] software

[0849] Generative AI model: Generates personalized assistive devices and driving assistance tools based on employee skill information.

[0850] System configuration

[0851] 1. Input and registration of employee skill information

[0852] The user uses a smartphone to input information about their skills and the type and degree of their disability.

[0853] The device checks the input, formats the skill information, and sends it to the server.

[0854] 2. Generation of assistive devices and driving assistive devices

[0855] The server receives the skill information sent from the terminal and registers it in a database.

[0856] A generative artificial intelligence model is activated to generate personalized assistive devices and driving assistive devices based on the employee's skill information and the type and degree of disability.

[0857] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot.

[0858] 3. Provision and use of assistive devices and driving aids

[0859] The terminal receives and stores the generated equipment.

[0860] The device provides the user with assistive devices in the form of audio guidance, visual aids, haptic feedback, etc.

[0861] The user uses the assistive device to perform work or drive.

[0862] 4. Creation and provision of communication guidelines

[0863] The server generates and transmits communication guidelines suitable for the user and the employees around him / her.

[0864] The terminal receives and displays the guidelines.

[0865] 5. Supporting OJT and tracking progress

[0866] The server generates OJT instruction procedures based on the skill information and job requirements and transmits them to the terminal.

[0867] The terminal displays the instruction procedure and provides it to the user.

[0868] The user follows the instruction procedure and performs OJT.

[0869] The device records the training progress in real time and transmits it to the server.

[0870] 6. Tracking driving task progress and suggesting improvements

[0871] The server tracks the progress of the driving task in real time and generates improvement suggestions or further assistance as needed.

[0872] The terminal provides the generated suggestions and assistance methods to the user.

[0873] Specific examples

[0874] In the case of visually impaired driver B

[0875] 1. Enter and submit skill information

[0876] User B uses a smartphone to enter information about his visual impairment and his level of vision.

[0877] Terminal: Checks the data and sends it to the server.

[0878] 2. Receiving and registering skill information

[0879] Server: Receives information sent from the terminal and registers it in a database.

[0880] 3. Generation of assistive devices and driving assistive devices

[0881] Server: Uses a generative AI model to generate a voice feedback system for the visually impaired.

[0882] Server: Sends the system to the smartphone.

[0883] 4. Use of assistive devices

[0884] Smartphone: Receives and stores voice feedback system.

[0885] User: Activates smartphone and uses voice feedback while driving.

[0886] 5. Communication support

[0887] Server: Generates and transmits appropriate communication methods for visually impaired people.

[0888] Device: Receives and applies guidelines, for example by alerting the driver to the sound of approaching vehicles.

[0889] 6. Support for OJT

[0890] Server: Generates step-by-step audio guidance for the driving task and sends it to the device.

[0891] Device: Save the guide and provide it to Person B.

[0892] User: Use the guide to safely complete the driving task.

[0893] Example prompts to input to the generative AI model

[0894] To the generative AI model:

[0895] Please create a driving assistance tool for a visually impaired user who needs to drive a car. The user has almost lost all of his / her sight. The tool needs to provide a proper audio feedback system and assist with obstacle detection and communication with other vehicles. Thank you in advance.

[0896] This system will help employees with disabilities to fully utilize their abilities, and will also enable companies to maximize their productivity.

[0897] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0898] Step 1:

[0899] Entering and submitting skill information

[0900] The user uses a smartphone to input their own skill information and the type and degree of disability. The device checks the input, formats the skill information, and sends it to the server. In this step, the input data includes the user's level of vision, programming experience, and required assistive functions. The device verifies the input data, formats it into the appropriate format, and transfers it to the server.

[0901] Step 2:

[0902] Receiving and registering skill information

[0903] The server receives the skill information sent from the terminal and registers it in the database. Specifically, it saves the received information in an SQL database and confirms that the skill information has been properly registered.

[0904] Step 3:

[0905] Generation of assistive devices and driving aids

[0906] The server launches a generative AI model to generate personalized assistive devices and driving aids based on the employee's skill information and the type and severity of their disability. At this stage, the AI ​​model inputs prompts and runs a process to generate assistive devices optimized for the employee's needs. Skill information and disability details are used as input data, and personalized feedback systems and visual aids are provided as output.

[0907] Step 4:

[0908] Providing the generated assistive devices

[0909] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot. Specifically, the generated digital feedback system, audio guide, etc. are transmitted to the terminal in file format and stored on the terminal.

[0910] Step 5:

[0911] Use of assistive devices

[0912] The device stores the generated assistive devices and converts them into a format to provide to the user. The user activates a smartphone or head-mounted display and uses audio guidance and visual aids to perform tasks or drive. For example, the user activates a voice feedback system stored on the smartphone and receives audio guidance while driving.

[0913] Step 6:

[0914] Creation and provision of communication guidelines

[0915] The server generates and transmits communication guidelines appropriate for the user and the employees around them. The data includes guidelines and examples of appropriate communication methods between employees. This is compiled into a document and sent to the terminal so that the employees around them can apply it.

[0916] Step 7:

[0917] Supporting on-the-job training and tracking progress

[0918] The server generates OJT training procedures based on skill information and job requirements and sends them to the terminal. It lists the step-by-step training procedures and provides them to the employee in the appropriate order. The terminal also records the user's progress in real time and sends it to the server. The server stores this progress in a database and generates progress reports as needed.

[0919] Step 8:

[0920] Tracking driving task progress and suggesting improvements

[0921] The server tracks the progress of the driving task in real time and generates suggestions for improvement or further assistance as needed. For example, feedback and suggestions for improvement from the obstacle detection system while driving are generated in real time and provided to the user's device, allowing the user to take immediate action.

[0922] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0923] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. This system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system provides more appropriate support tailored to the user's condition.

[0924] System configuration

[0925] 1. Input and registration of employee skill information

[0926] User: Enters their own skill information (e.g., type and degree of disability, past work experience, skills, etc.) through a dedicated application or website.

[0927] Device: Validate input and check skill information format.

[0928] Device: Converts skill information into JSON format and sends it to the server.

[0929] 2. Activation of the generative AI model and generation of the assistive device

[0930] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[0931] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[0932] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[0933] 3. Collecting emotional information using an emotion engine

[0934] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's tone of voice and facial expressions to determine their emotional state.

[0935] Terminal: Collects emotional information recognized by the emotion engine as data and sends it to the server.

[0936] 4. Provision and use of assistive devices

[0937] Server: Receives the emotion information sent from the emotion engine and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[0938] Terminal: Receives and stores the generated assistive devices. The assistive devices are displayed on the interface for easy access by the user.

[0939] User: Performs daily tasks using the assistive devices provided.

[0940] 5. Creation and provision of communication guidelines

[0941] Server: Generates communication guidelines appropriate for the user and their surrounding workers, including communication methods and considerations for visually impaired people.

[0942] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[0943] Device: View and apply received guidelines.

[0944] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[0945] 6. Supporting OJT and tracking progress

[0946] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[0947] Server: Sends the generated OJT instruction procedures to the user's terminal.

[0948] Terminal: Receives the instructional steps and displays them to the user.

[0949] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[0950] Terminal: Records the progress of work in real time and sends it to the server.

[0951] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[0952] Specific examples

[0953] New employee A, who has visual impairment and high stress

[0954] 1. Enter and submit skill information

[0955] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[0956] Device: Checks the data and sends it to the server.

[0957] 2. Receiving and registering skill information

[0958] Server: Receives information sent from the device and registers it in a database.

[0959] 3. Generation of assistive devices

[0960] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[0961] Server: Sends tools to the device.

[0962] 4. Stress assessment using an emotion engine

[0963] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[0964] Device: Sends data on high stress conditions to the server.

[0965] 5. Adjustment of assistive devices

[0966] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[0967] Server: Sends appropriately adjusted assistive devices to the device.

[0968] 6. Use of assistive devices

[0969] Terminal: Receives the adjusted assistive device and provides it to Person A.

[0970] User: Launches the tool and performs tasks.

[0971] 7. Communication Support

[0972] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[0973] Terminal: Receives the guidelines and displays them to employees.

[0974] Users: Communicate according to guidelines.

[0975] 8. Support for OJT

[0976] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[0977] Device: Save the guide and provide it to Person A.

[0978] User: Use the guide to progress through the task.

[0979] In this way, this system provides effective support for employees with disabilities and companies, creating an environment where employees can immediately function as a valuable asset. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[0980] The processing flow will be explained below.

[0981] Step 1:

[0982] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, the skill information input screen will be displayed.

[0983] Step 2:

[0984] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[0985] Step 3:

[0986] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[0987] Step 4:

[0988] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[0989] Step 5:

[0990] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[0991] Step 6:

[0992] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[0993] Step 7:

[0994] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[0995] Step 8:

[0996] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[0997] Step 9:

[0998] On the device: The emotion engine is activated to recognize the user's emotions in real time. The emotion engine analyzes voice tone and facial expressions to determine the user's emotional state (e.g., stress, excitement, concentration, etc.).

[0999] Step 10:

[1000] Terminal: Recognized emotion information is sent as data to the server.

[1001] Step 11:

[1002] Server: Receives emotional information and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[1003] Step 12:

[1004] Server: The adjusted assistive device is sent back to the user's device.

[1005] Step 13:

[1006] Device: Receives and saves the adjusted assistive devices. Notifies the user of the adjustments and provides feedback using icons and messages.

[1007] Step 14:

[1008] User: Uses the provided assistive devices to perform daily tasks. Any problems or feedback that arise during use are reported to the server via the device.

[1009] Step 15:

[1010] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[1011] Step 16:

[1012] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[1013] Step 17:

[1014] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[1015] Step 18:

[1016] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[1017] Step 19:

[1018] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[1019] Step 20:

[1020] Server: Sends the generated OJT instruction procedures to the user's terminal.

[1021] Step 21:

[1022] Terminal: Receives the instructional steps and displays them to the user.

[1023] Step 22:

[1024] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[1025] Step 23:

[1026] Terminal: Records the progress of work in real time and sends it to the server.

[1027] Step 24:

[1028] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[1029] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company. Furthermore, by combining it with an emotion engine, it is possible to dynamically adjust the support content according to the employee's emotional state, achieving more effective support.

[1030] Example 2

[1031] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1032] For many companies, providing appropriate assistive devices and a comfortable working environment for employees with disabilities is a difficult task. It is necessary to accurately grasp the employee's skill information, generate and provide customized assistive devices based on that information, and provide support that responds to fluctuations in their emotional state. However, current systems cannot effectively solve these issues.

[1033] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for an employee to input and register his / her own skill information, a means for generating an individualized assistive tool using a generative artificial intelligence model, and a means for providing the generated assistive tool to the employee. This makes it possible to generate and provide assistive tools tailored to the individual needs of employees with disabilities. In addition, by including a means for using a terminal equipped with an emotion engine that recognizes and collects the user's emotional information in real time and a means for adjusting the assistive tool based on the collected emotional information, it becomes possible to provide support that responds to changes in the employee's emotional state. This makes it possible to provide an environment in which employees can work comfortably and efficiently.

[1034] "Employee" refers to a person who is employed by a company or organization, and specifically includes people with disabilities.

[1035] "Skills information" refers to information including an employee's specific skills, past work experience, educational background, and the type and degree of disability.

[1036] A "dedicated application" is software designed to provide specific operations or functions, and provides an interface for users to input skill information.

[1037] "Generative AI model" refers to an AI technology for automatically generating assistive devices and other suggestions based on a worker's skill information and needs.

[1038] "Assistive devices" refers to the assistive devices and software tools that employees with disabilities need to perform their work efficiently and comfortably.

[1039] "Terminal" refers to an electronic device such as a computer, smartphone, or tablet, which is a device that allows a user to use this system.

[1040] "Server" refers to a central computer system that stores, processes data, and responds to various requests.

[1041] "Emotion engine" refers to artificial intelligence technology that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[1042] "Communication guidelines" refers to documents and tools that explain the guidelines and methods for employees and those around them to communicate appropriately.

[1043] "On-the-job training (OJT)" refers to a training method in which employees learn and improve their skills through actual work.

[1044] "Progress" refers to information that indicates the progress of an employee in completing a task or training.

[1045] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. The system uses a generative artificial intelligence model that generates personalized assistive devices based on the employee's skill information, which is input through a dedicated application or website. It also includes an emotion engine that collects the user's emotional information and adjusts the assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, supporting on-the-job training (OJT).

[1046] Hardware and software used

[1047] 1. Input and registration of employee skill information

[1048] The user enters skill information using a dedicated application (e.g., a smartphone app) or website.

[1049] The terminal verifies the entered skill information, converts the skill information into JSON format, and sends it to the server.

[1050] 2. Activation of the generative AI model and generation of the assistive device

[1051] The server receives the skill information in JSON format and registers it in a database. The technology used is a database management system (DBMS).

[1052] Based on the registered skill information, a generative artificial intelligence model (e.g., TensorFlow or PyTorch) is launched to generate personalized assistive devices that meet the user's needs.

[1053] The server transmits the generated assistive tool to the user's terminal.

[1054] 3. Collecting emotional information using an emotion engine

[1055] The device launches an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's voice tone and facial expressions.

[1056] The device transmits the collected emotion information to the server.

[1057] 4. Provision and use of assistive devices

[1058] The server adjusts the assistive device based on the emotion information and sends the adjusted assistive device to the user's terminal.

[1059] The terminal provides the user with the adjusted assistive device, which the user uses to perform the task.

[1060] 5. Creation and provision of communication guidelines

[1061] The server generates communication guidelines suitable for the user and surrounding employees and transmits them to the terminal.

[1062] The terminal displays guidelines, which employees follow when communicating.

[1063] 6. Supporting OJT and tracking progress

[1064] The server generates OJT training procedures based on the employee's skill information and job requirements.

[1065] The server sends the generated OJT procedure to the user, and the terminal displays it.

[1066] The user follows the instructional procedures to carry out OJT, records their progress through the terminal, and the server receives the data and tracks their progress.

[1067] Specific examples

[1068] New employee A, who has visual impairment and high stress

[1069] 1. Enter and submit your skill information:

[1070] User: Person A uses a dedicated app to enter that he or she is visually impaired and has programming experience.

[1071] Device: Checks this data and sends it to the server.

[1072] 2. Receiving and registering skill information:

[1073] Server: Receives information sent from the device and registers it in a database.

[1074] 3. Generating assistive devices:

[1075] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[1076] Server: Sends tools to the device.

[1077] 4. Stress assessment using emotion engine:

[1078] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[1079] Device: Sends data on high stress conditions to the server.

[1080] 5. Adjustment of assistive devices:

[1081] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[1082] Server: Sends appropriately adjusted assistive devices to the device.

[1083] 6. Use of assistive devices:

[1084] Terminal: Receives the adjusted assistive device and provides it to Person A.

[1085] User: Launches the tool and performs tasks.

[1086] Prompt Sentence Examples

[1087] "I am visually impaired and have three years of programming experience. Please suggest the best assistive device for me."

[1088] "Currently in a state of high stress. Please adjust your support equipment accordingly."

[1089] In this way, this system provides effective support for employees with disabilities and companies, and plays an important role in creating an environment where employees can function immediately as a workforce. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[1090] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1091] Step 1:

[1092] The user enters skill information. Using a dedicated application or website, employees enter their skill information (type of disability, skills, work experience, etc.). The entered information is saved in fields on the form. Specific rules apply to the format of the entered data.

[1093] Input: User's skills (e.g., "I am visually impaired and have 3 years of programming experience").

[1094] Output: Skill information saved in the input form.

[1095] Step 2:

[1096] The terminal validates the input data. The terminal validates the input data to ensure it conforms to the required format. Once validation is complete, it converts the data into JSON format.

[1097] Input: Skill information saved in the input form.

[1098] Data processing: Format check and validation, conversion to JSON format (e.g., {"Disability type": "Visual impairment", "Skills": "Programming", "Years of experience": 3}).

[1099] Output: Validated data in JSON format.

[1100] Step 3:

[1101] The device sends JSON format data to the server. The validated JSON data is sent to the server using an HTTP request.

[1102] Input: Validated data in JSON format.

[1103] Data operations: creating and sending HTTP requests.

[1104] Output: Skill information data sent to the server.

[1105] Step 4:

[1106] The server receives the skill information in JSON format and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[1107] Input: Submitted skill information in JSON format.

[1108] Data processing: INSERT operations into the database (e.g., registering data into the skill information table).

[1109] Output: Skill information registered in the database.

[1110] Step 5:

[1111] The server launches the generative AI model, which starts the process of generating appropriate assistive devices based on the skill information registered in the database.

[1112] Input: Skill information registered in the database.

[1113] Data computation: Initialization and execution of generative AI models.

[1114] Output: The process of creating a personalized assistive device begins.

[1115] Step 6:

[1116] The server generates assistive devices. Using generative AI models, it generates personalized assistive devices tailored to the user's needs, such as programming tools with voice feedback for the visually impaired.

[1117] Input: The parameters required for the generation process.

[1118] Data computation: Generation of assistive devices based on AI models.

[1119] Output: Individualized assistive devices.

[1120] Step 7:

[1121] The server sends the generated assistive device to the user's device, and returns the link or file of the generated assistive device to the device as an HTTP response.

[1122] Input: The generated assistive device.

[1123] Data operations: Creating and sending HTTP responses.

[1124] Output: Assistive devices sent to the user's device.

[1125] Step 8:

[1126] The device provides the assistive tools to the user and displays them on the interface for easy access by the user.

[1127] Input: Assistive device sent from the server.

[1128] Data calculation: Display settings for assistive devices.

[1129] Output: The assistive device provided to the user.

[1130] Step 9:

[1131] The user uses the assistive device. The user performs daily tasks using the provided assistive device.

[1132] Input: Assistive devices provided to the device.

[1133] Data calculation: Use of assistive devices.

[1134] Output: A deliverable that is useful for business.

[1135] Step 10:

[1136] The device activates an emotion engine to recognize the user's emotions, analyzing voice tone and facial expressions to determine the user's emotional state in real time.

[1137] Input: User audio and video data.

[1138] Data Computing: Sentiment Analysis.

[1139] Output: Determined emotion information.

[1140] Step 11:

[1141] The device sends emotional information to the server, which converts the recognized emotional information into JSON format and sends it to the server as an HTTP request.

[1142] Input: Determined emotion information.

[1143] Data operation: Creating and sending JSON format.

[1144] Output: Emotion information sent to the server.

[1145] Step 12:

[1146] The server receives the emotional information and adjusts the assistive device. For example, if the user is feeling stressed, the server generates an assistive device with relaxing music or additional guidance.

[1147] Input: The submitted emotion information.

[1148] Data calculation: Adjustment of assistive devices.

[1149] Output: Coordinated assistive devices.

[1150] Step 13:

[1151] The server sends the adjusted assistive device to the user's device, and then sends the appropriately adjusted assistive device again as an HTTP response.

[1152] Input: Adjusted assistive device.

[1153] Data operations: Creating and sending HTTP responses.

[1154] Output: The adjusted assistive devices are sent to the device.

[1155] Step 14:

[1156] Providing users with assistive devices that are adjusted to their devices, providing a relaxing environment for users to work comfortably.

[1157] Input: Calibration aid sent from server.

[1158] Data calculation: Display settings for assistive devices.

[1159] Output: The adjustment aid provided to the user.

[1160] Step 15:

[1161] The server generates and provides communication guidelines for the user and their surrounding employees, and sends them to the terminal.

[1162] Input: Information about the user and surrounding workers.

[1163] Data arithmetic: generating guidelines.

[1164] Output: The generated guidelines.

[1165] Step 16:

[1166] The terminal displays the communication guidelines. The guidelines are displayed on the terminal so that the user and nearby employees can easily refer to them.

[1167] Input: Communication guidelines sent by the server.

[1168] Data calculation: Guideline display settings.

[1169] Output: The displayed guidelines.

[1170] Step 17:

[1171] The server generates and transmits OJT training procedures based on the employee's skill information and job requirements.

[1172] Input: User skill information and job requirements.

[1173] Data calculation: Generation of OJT instruction procedures.

[1174] Output: Generated OJT instruction procedures.

[1175] Step 18:

[1176] The terminal displays the OJT instruction procedure. The procedure for the user to carry out the OJT is displayed on the interface.

[1177] Input: OJT training procedure sent from the server.

[1178] Data calculation: Display settings for instructional procedures.

[1179] Output: Displayed OJT instruction procedures.

[1180] Step 19:

[1181] The user performs OJT by following the provided instructional procedures to carry out the tasks and recording the progress in real time via the terminal.

[1182] Input: OJT training procedures.

[1183] Data calculation: Performing tasks and recording progress.

[1184] Output: Real-time recorded progress.

[1185] Step 20:

[1186] The device sends the progress to the server. The device sends the progress data recorded in real time to the server.

[1187] Input: Progress data recorded in real time.

[1188] Data calculation: Sending progress data.

[1189] Output: Progress data sent to the server.

[1190] Step 21:

[1191] The server records the training progress in a database and generates a report. Based on the progress data, the server stores the training progress in a database and generates a progress report.

[1192] Input: The progress data submitted.

[1193] Data calculation: recording to database and generating reports.

[1194] Output: The generated progress report.

[1195] (Application example 2)

[1196] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1197] It is important to create an environment in which employees with disabilities can immediately contribute to the workplace, but it is difficult to properly manage employees' skill and emotional information and provide them with personalized assistive devices. There is also a need for systems that improve communication between employees and those around them and enable employees to perform their work efficiently in a mentally stable state. Furthermore, there is a need to effectively support on-the-job training (OJT) and track the training process. The purpose of this invention is to solve these problems.

[1198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1199] In this invention, the server includes: a means for an employee to input and register his / her own skill information; a means for using a generative artificial intelligence model to generate personalized assistive devices based on the skill information; a means for providing the generated assistive devices to the employee; a means for generating and providing guidelines for appropriate communication between the employee and other employees around him / her; a means for suggesting appropriate teaching methods to support on-the-job training and tracking the training progress; and a means for collecting employee emotion information using an emotion engine and adjusting the assistive devices. This makes it possible to create an environment in which employees can work immediately.

[1200] An "employee with a disability" is an employee who has a physical or mental disability and requires special assistance or consideration to accommodate that disability.

[1201] "Skill information" refers to information such as the skills, experience, and qualifications that employees possess.

[1202] "Individualized assistive devices" are devices created based on the skill information of a specific employee and the type and degree of disability, with the aim of assisting the employee in their work.

[1203] A "generative artificial intelligence model" is an artificial intelligence model that generates an output (here, an assistive device) according to a specific purpose based on input data.

[1204] The "Emotion Engine" is a technology that analyzes and recognizes emotional states in real time from voice tone and facial expressions.

[1205] "On-the-job training (OJT)" is workplace training conducted through actual work, and is an educational method in which knowledge and skills required for work are acquired through practice.

[1206] "Communication guidelines" are guidelines that compile specific instructions and advice for achieving appropriate communication.

[1207] This invention is a support system for improving the work efficiency of disabled employees in the workplace. The system uses a generative AI model that generates personalized assistive devices based on the employee's skill information. It also combines an emotion engine to provide support tailored to the employee's emotional state.

[1208] System configuration

[1209] 1. Input and registration of employee skill information

[1210] Users use a dedicated application or website to enter their skills information (e.g., type and degree of disability, past work experience, skills, etc.).

[1211] The terminal checks the entered skill information, converts it into JSON format, and sends it to the server.

[1212] 2. Activation of the generative AI model and generation of the assistive device

[1213] The server receives the JSON-formatted skill information sent from the device, analyzes it, and registers it in a database that includes information such as the user ID, type of disability, and skill content.

[1214] The server activates a generative AI model based on the registered skill information and generates personalized assistive tools that meet the user's needs.

[1215] The server generates and sends personalized assistive devices to the device, for example, a visually impaired user is provided with an assistive device with audio feedback.

[1216] 3. Collecting emotional information using an emotion engine

[1217] The device is equipped with an emotion engine that recognizes the user's emotions in real time by analyzing the user's tone of voice and facial expressions to determine their emotional state.

[1218] The terminal transmits the emotion information recognized by the emotion engine to the server.

[1219] 4. Provision and use of assistive devices

[1220] The server receives the emotional information and adjusts the assistive devices according to the user's current emotional state, for example, providing relaxation tools and guidance if the user is in a high-stress state.

[1221] The terminal displays the adjusted assistive tool to the user, allowing the user to easily access it.

[1222] The user performs daily tasks using the provided assistive devices.

[1223] 5. Creation and provision of communication guidelines

[1224] The server generates communication guidelines suitable for the user and the surrounding employees, and transmits them to the terminals of the user and the surrounding employees.

[1225] The terminal receives the guidelines and displays them to the user and nearby workers.

[1226] 6. Supporting OJT and tracking progress

[1227] The server generates OJT training procedures based on the employee's skill information and job requirements. These training procedures are provided in a step-by-step format.

[1228] The server transmits the generated OJT instruction procedure to the user's terminal.

[1229] The terminal receives the instruction procedure and displays it to the user.

[1230] The user follows the provided instructional procedures to carry out OJT, recording the progress of the work and any questions they may have in real time via the terminal.

[1231] The terminal records the progress of the work in real time and transmits it to the server.

[1232] The server records the progress in a database and generates reports to provide to administrators.

[1233] Specific examples

[1234] For example, if new employee A has a visual impairment but has programming experience, he or she can use a dedicated app to input his or her visual impairment and skills. The device then sends the data to a server, and an AI model generates a programming support tool with voice feedback. If the emotion engine determines that A is under high stress, relaxing music is added. The user and other employees perform their work according to appropriate communication guidelines, and on-the-job training is also supported step-by-step.

[1235] Example prompt sentence:

[1236] Mr. A is visually impaired and has programming experience. Please create an effective programming assistant tool with audio feedback that does not rely on vision for him. Also, please provide soothing music when he feels high stress.

[1237] This system allows employees with a variety of disabilities to function immediately and provides an environment in which they can work more effectively and safely in the workplace.

[1238] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1239] Step 1:

[1240] Users use a dedicated application or website to enter their own skill information (e.g., type and degree of disability, past work experience, skills, etc.) The entered data is converted into JSON format and sent to the server.

[1241] Input: User skill information

[1242] Output: JSON formatted data

[1243] Step 2:

[1244] The server receives the skill information in JSON format sent from the device, analyzes the received data, and registers the employee's ID, type of disability, skill information, etc. in a database.

[1245] Input: Skill information in JSON format

[1246] Output: Employee information stored in the database

[1247] Step 3:

[1248] The server activates a generative AI model based on the registered skill information, which generates personalized assistive devices tailored to the user's needs.

[1249] Input: Employee information stored in the database

[1250] Output: personalized assistive devices

[1251] Step 4:

[1252] The server transmits the generated assist tool to the terminal, which displays the received assist tool data to the user, making it available for use.

[1253] Input: personalized assistive devices

[1254] Output: Assistive device data displayed on the user's device

[1255] Step 5:

[1256] The device activates an emotion engine that recognizes the user's emotions in real time by analyzing voice tone and facial expressions to determine the user's emotional state.

[1257] Input: User's voice tone and facial expression data

[1258] Output: Perceived emotional state

[1259] Step 6:

[1260] The device sends the emotional information recognized by the emotion engine to the server, which then adjusts the assistive devices based on this emotional information. For example, if the user is in a high-stress state, it will add tools or guidance to help them relax.

[1261] Input: Perceived emotional state

[1262] Output: Adjusted assistive devices

[1263] Step 7:

[1264] The server generates communication guidelines suitable for the user and the employees around him / her and transmits them to the terminal, which then displays the received guidelines to the user and the employees around him / her.

[1265] Input: Database and emotion information

[1266] Output: Communication Guidelines

[1267] Step 8:

[1268] The server generates on-the-job training (OJT) instructions based on the employee's skill information and job requirements. The instructions are generated in a step-by-step format and sent to the terminal. The terminal displays the received instructions to the user and makes them available.

[1269] Input: Employee information and job requirements stored in the database

[1270] Output: OJT training procedure

[1271] Step 9:

[1272] Users follow the provided instructional procedures to carry out OJT. Any progress or questions they may have are recorded in real time on their devices and sent to the server. The server then records the progress in a database, generates a report, and provides it to the administrator.

[1273] Input: OJT progress and questions

[1274] Output: Progress report

[1275] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1276] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1277] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1278] [Third embodiment]

[1279] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1280] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1281] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1282] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1283] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1284] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1285] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1286] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1287] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1288] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1289] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1290] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1291] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective. The system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between employees and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process.

[1292] System configuration

[1293] 1. Input and registration of employee skill information

[1294] User: Enter their skill information (e.g., type and degree of disability, programming experience, etc.) through a dedicated application or website.

[1295] Device: Validate input and check skill information format.

[1296] Terminal: Sends skill information to the server.

[1297] 2. Activation of the generative AI model and generation of the assistive device

[1298] Server: Receives skill information sent from the device.

[1299] Server: Registers skill information in the database and launches the generative AI model.

[1300] Server: Generates personalized assistive devices based on the user's skill information and the type and degree of disability.

[1301] Server: Sends the generated assistive devices to the user's device.

[1302] 3. Provision and use of assistive devices

[1303] Terminal: Receives and stores the generated assistive devices.

[1304] Device: Provide the assistive device to the user in an appropriate format (e.g., audio description, text, visual aid).

[1305] User: Performs daily tasks using the assistive devices provided.

[1306] 4. Creation and provision of communication guidelines

[1307] Server: Generates communication guidelines appropriate for the user and their surrounding employees.

[1308] Server: Sends guidelines to the user's and nearby employees' devices.

[1309] Device: View and apply received guidelines.

[1310] 5. Supporting OJT and tracking progress

[1311] Server: Generates OJT training procedures based on the user's skill information and job requirements.

[1312] Server: Sends step-by-step instruction instructions to the user's device.

[1313] Terminal: Displays the received instruction procedure and provides it to the user.

[1314] User: Follow the provided instructional procedures and conduct OJT.

[1315] Terminal: Records the user's training progress in real time and sends it to the server.

[1316] Server: Records and analyzes progress in a database.

[1317] Server: Creates training progress reports and provides them to managers and companies.

[1318] Specific examples

[1319] New employee A with a visual impairment

[1320] 1. Enter and submit skill information

[1321] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[1322] Device: Checks the data and sends it to the server.

[1323] 2. Receiving and registering skill information

[1324] Server: Receives information sent from the device and registers it in a database.

[1325] 3. Generation of assistive devices

[1326] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[1327] Server: Sends tools to the device.

[1328] 4. Use of assistive devices

[1329] Terminal: Receive and save tools.

[1330] User: Launches the tool and performs programming tasks.

[1331] 5. Communication support

[1332] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[1333] Terminal: Receives the guidelines and displays them to employees.

[1334] Users: Communicate according to guidelines.

[1335] 6. Support for OJT

[1336] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[1337] Device: Save the guide and provide it to Person A.

[1338] User: Use the guide to progress through the task.

[1339] In this way, this system provides effective support for employees with disabilities and companies, creating an environment in which employees can function immediately as contributions to the workplace.

[1340] The processing flow will be explained below.

[1341] Step 1:

[1342] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, a screen for entering your skill information will appear.

[1343] Step 2:

[1344] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[1345] Step 3:

[1346] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[1347] Step 4:

[1348] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[1349] Step 5:

[1350] Server: Activates a generative AI model based on the registered skill information. The AI ​​model then begins the process of generating personalized assistive devices tailored to the user's needs.

[1351] Step 6:

[1352] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[1353] Step 7:

[1354] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[1355] Step 8:

[1356] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[1357] Step 9:

[1358] User: Activates the received assistive device and uses it for their own work. Any problems or feedback that arise while using the assistive device can be reported to the server via the device.

[1359] Step 10:

[1360] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[1361] Step 11:

[1362] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[1363] Step 12:

[1364] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[1365] Step 13:

[1366] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[1367] Step 14:

[1368] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[1369] Step 15:

[1370] Server: Sends the generated OJT instruction procedures to the user's terminal.

[1371] Step 16:

[1372] Terminal: Receives the instructional steps and displays them to the user.

[1373] Step 17:

[1374] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[1375] Step 18:

[1376] Terminal: Records the progress of work in real time and sends it to the server.

[1377] Step 19:

[1378] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[1379] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company.

[1380] Example 1

[1381] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1382] In conventional employment environments for employees with disabilities, there is often a lack of support to fully utilize the skills of the employees themselves, and appropriate communication methods and effective training methods are often lacking. As a result, it takes time for employees to become immediately effective, which can lead to a decrease in productivity throughout the organization. The purpose of this invention is to solve these problems and provide an environment in which employees with disabilities can perform their duties quickly and effectively.

[1383] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1384] In this invention, the server includes a means for recording the progress of the employee's training and transmitting it to the server in real time, a means for generating a training progress report and providing it to a manager or company, and a means for generating and providing guidelines for appropriate communication between the employee and other employees around him / her. This makes it possible to grasp the training progress in real time and provide appropriate communication and instruction methods so that the employee can quickly become a work-ready employee.

[1385] "Employee" refers to a working individual employed by a company or organization.

[1386] "Skill information" refers to detailed data such as an employee's abilities, experience, and knowledge.

[1387] A "generative artificial intelligence model" refers to a machine learning model that automatically generates specific outputs based on input data.

[1388] "Assistive devices" refers to tools, software, equipment, etc. that are useful to employees in carrying out their work.

[1389] "Communication guidelines" refer to guidelines and rules for employees and those around them to communicate appropriately.

[1390] "On-the-job training (OJT)" refers to practical training that takes place at the workplace.

[1391] "Training progress report" refers to a report that records and analyzes the progress of an employee's training.

[1392] "Real-time" refers to data and information being processed and provided almost immediately.

[1393] This invention is a support system for improving the employment environment for employees with disabilities, providing maximum support so that employees can work immediately. This system is composed of the following elements:

[1394] Input and registration of employee skill information

[1395] Users use a dedicated application or website to input their own skill information (e.g., type and degree of disability, programming experience, etc.) This data is received by the terminal, and after format confirmation and matching, it is sent to the server.

[1396] Activation of generative artificial intelligence models and generation of support tools

[1397] The server stores the received skill information in a database and activates a generative AI model. The AI ​​model generates personalized assistive devices (e.g., programming tools with voice feedback) based on the received skill information and the type and degree of disability. The generated assistive devices are sent from the server to the device.

[1398] Provision and use of assistive devices

[1399] The device receives the generated assistive devices, stores them, and provides them to the user in an appropriate format (e.g., audio guide, text, visual aids). The user uses the provided assistive devices to perform daily tasks.

[1400] Creation and provision of communication guidelines

[1401] The server generates communication guidelines suitable for the user and the surrounding employees, and transmits the guidelines to the terminals of the user and the surrounding employees. The terminals display and apply the received guidelines.

[1402] Supporting on-the-job training and tracking progress

[1403] The server generates OJT training procedures based on the user's skill information and job requirements, and sends the step-by-step training procedures to the user's device. The device displays the received training procedures and provides them to the user. The user follows the provided training procedures to carry out the OJT. The device records the training progress in real time and sends it to the server. The server records the progress in a database, creates a progress report, and provides it to managers and companies.

[1404] As a concrete example, consider the case of a new employee, Mr. A, who is visually impaired.

[1405] The user uses Mr. A's dedicated app to enter information about his visual impairment and programming experience.

[1406] The terminal checks the entered data and sends it to the server.

[1407] The server uses a generative AI model to generate programming tools with voice feedback for visually impaired people and sends them to the device.

[1408] The terminal receives the tool and provides it to the user. Person A uses the tool to perform his / her work.

[1409] The server generates appropriate communication methods for employees with visually impaired people and sends them to their terminals.

[1410] The terminal receives the guidelines and displays them to the employee. Mr. A and the employee communicate according to the guidelines.

[1411] The server generates a step-by-step audio guide for the programming task and sends it to the device. The device saves the guide and provides it to Person A. Person A uses the guide to progress through the task.

[1412] Example prompt sentence:

[1413] "Please create assistive tools to help new employees with visual impairments to smoothly carry out programming tasks."

[1414] "Please suggest effective ways for deaf people to communicate with their superiors."

[1415] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1416] Step 1:

[1417] Input and registration of employee skill information

[1418] The user opens a dedicated application or website.

[1419] The user fills out an input form with the type and degree of disability and skill information (e.g., programming experience, languages ​​fluent, etc.).

[1420] (Input) Type and degree of disability, skill information

[1421] (Processing) The terminal receives the input, performs format validation and matching, and checks that required fields are filled in.

[1422] (Output) Confirmed skill information

[1423] (Operation) The terminal sends the confirmed skill information to the server.

[1424] Step 2:

[1425] Activation of generative artificial intelligence models and generation of support tools

[1426] The server receives the skill information sent from the terminal.

[1427] (Input) Confirmed skill information

[1428] (Processing) The server stores the received information in a database and activates a generative AI model. Information about the user's skills and the type and degree of disability is obtained from the database, and a prompt is generated based on this information. The prompt is input into the generative AI model, which then generates an individualized assistive device.

[1429] (Output) Generated support equipment

[1430] (Operation) The server sends the generated support tool to the user's terminal.

[1431] Step 3:

[1432] Provision and use of assistive devices

[1433] The terminal receives the assistive tool transmitted from the server.

[1434] (Input) Generated support equipment

[1435] (Processing) The device stores the received assistive devices and provides them to the user in an appropriate format (e.g., audio guide, text, visual aid).

[1436] (Output) Assistive devices provided in a format that can be used by the user

[1437] (Action) The user uses the provided assistive device to perform daily tasks.

[1438] Step 4:

[1439] Creation and provision of communication guidelines

[1440] The server generates communication guidelines suitable for the user and the employees around him.

[1441] (Input) Information about the user and surrounding employees

[1442] (Processing) The server generates guidelines and transmits them to the terminals of the user and nearby employees.

[1443] (Output) Generated communication guidelines

[1444] (Operation) The terminal displays the received guidelines and applies them to the user and the employees around him / her.

[1445] Step 5:

[1446] Supporting on-the-job training and tracking progress

[1447] The server generates OJT training procedures based on the user's skill information and job requirements.

[1448] (Input) User skill information, job requirements

[1449] (Processing) The server generates step-by-step instruction procedures and sends them to the user's terminal.

[1450] (Output) Generated teaching steps

[1451] (Operation) The terminal displays the received training procedure and provides it to the user. The user follows the provided training procedure and performs OJT. The terminal records the user's training progress in real time and sends it to the server. The server records the progress in a database, creates a training progress report, and provides it to the manager or company.

[1452] (Application example 1)

[1453] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1454] In order for employees with disabilities to be immediately productive in the workplace, it is essential to provide them with individual assistive devices and appropriate communication support. However, conventional systems were unable to provide detailed support tailored to the type and degree of disability, and it was difficult to provide sufficient support in special situations such as driving assistance. This resulted in employees with disabilities being unable to fully utilize their abilities, and companies were ultimately unable to maximize their productivity. The purpose of this invention is to solve these problems.

[1455] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1456] In this invention, the server includes: means for a disabled employee to input and register his / her skill information; means for using a generative artificial intelligence model to generate personalized assistive devices and driving assistive devices based on the skill information; means for providing the generated assistive devices and driving assistive devices to the disabled employee; means for generating and providing guidelines for appropriate communication between the disabled employee and other employees around him / her; means for suggesting appropriate teaching methods to support on-the-job training and tracking the training progress; means for providing real-time assistance during driving using audio feedback and visual aids; and means for tracking the progress of driving tasks in real time and providing improvement suggestions and further assistance as needed. This enables disabled employees to work efficiently while receiving assistance tailored to their individual needs, and makes it possible to provide sufficient support, especially in special situations such as driving assistance.

[1457] "Employees with disabilities" refers to workers who have physical or sensory disabilities and require special assistance tailored to their individual needs.

[1458] "Skill information" refers to information about an employee's skills and experience, as well as the type and degree of disability.

[1459] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data and generates appropriate output for a specific purpose.

[1460] "Assistive devices" refers to assistive devices and software that employees with disabilities use to perform their daily work or specific tasks.

[1461] "Driver assistance devices" refers to devices and systems that are specifically designed to assist employees with disabilities while driving a motor vehicle.

[1462] The "Communication Guidelines" provide guidelines for smooth communication between employees with disabilities and those around them.

[1463] "On-the-job training (OJT)" refers to a training method in which employees acquire skills through actual work.

[1464] "Voice feedback" refers to a function that uses voice to convey information and instructions while working, etc.

[1465] "Visual aids" refers to devices or software that provide visual support.

[1466] "Real-time support" refers to support provided in response to the current situation.

[1467] "Driving task progress tracking" refers to monitoring and recording the progress of driving tasks in real time.

[1468] "Improvement suggestions and further support" refers to suggestions for improvements or problems found during the driving task or work, and providing additional support as needed.

[1469] The present invention is a system for supporting disabled employees in working efficiently and safely in a work environment, and particularly focuses on driving assistance for autonomous vehicles. Specific embodiments for implementing the present invention will be described below.

[1470] Hardware and software used

[1471] Hardware

[1472] Smartphone: Used by the employee to input skill information and receive feedback while driving.

[1473] Smart glasses: As a visual aid, they display driver assistance information while driving.

[1474] Head-mounted display: Displays assistance information and provides audio guidance while driving.

[1475] In-car robots: Providing real-time assistance while driving.

[1476] software

[1477] Generative AI model: Generates personalized assistive devices and driving assistance tools based on employee skill information.

[1478] System configuration

[1479] 1. Input and registration of employee skill information

[1480] The user uses a smartphone to input information about their skills and the type and degree of their disability.

[1481] The device checks the input, formats the skill information, and sends it to the server.

[1482] 2. Generation of assistive devices and driving assistive devices

[1483] The server receives the skill information sent from the terminal and registers it in a database.

[1484] A generative artificial intelligence model is activated to generate personalized assistive devices and driving assistive devices based on the employee's skill information and the type and degree of disability.

[1485] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot.

[1486] 3. Provision and use of assistive devices and driving aids

[1487] The terminal receives and stores the generated equipment.

[1488] The device provides the user with assistive devices in the form of audio guidance, visual aids, haptic feedback, etc.

[1489] The user uses the assistive device to perform work or drive.

[1490] 4. Creation and provision of communication guidelines

[1491] The server generates and transmits communication guidelines suitable for the user and the employees around him / her.

[1492] The terminal receives and displays the guidelines.

[1493] 5. Supporting OJT and tracking progress

[1494] The server generates OJT instruction procedures based on the skill information and job requirements and transmits them to the terminal.

[1495] The terminal displays the instruction procedure and provides it to the user.

[1496] The user follows the instruction procedure and performs OJT.

[1497] The device records the training progress in real time and transmits it to the server.

[1498] 6. Tracking driving task progress and suggesting improvements

[1499] The server tracks the progress of the driving task in real time and generates improvement suggestions or further assistance as needed.

[1500] The terminal provides the generated suggestions and assistance methods to the user.

[1501] Specific examples

[1502] In the case of visually impaired driver B

[1503] 1. Enter and submit skill information

[1504] User B uses a smartphone to enter information about his visual impairment and his level of vision.

[1505] Terminal: Checks the data and sends it to the server.

[1506] 2. Receiving and registering skill information

[1507] Server: Receives information sent from the terminal and registers it in a database.

[1508] 3. Generation of assistive devices and driving assistive devices

[1509] Server: Uses a generative AI model to generate a voice feedback system for the visually impaired.

[1510] Server: Sends the system to the smartphone.

[1511] 4. Use of assistive devices

[1512] Smartphone: Receives and stores voice feedback system.

[1513] User: Activates smartphone and uses voice feedback while driving.

[1514] 5. Communication support

[1515] Server: Generates and transmits appropriate communication methods for visually impaired people.

[1516] Device: Receives and applies guidelines, for example by alerting the driver to the sound of approaching vehicles.

[1517] 6. Support for OJT

[1518] Server: Generates step-by-step audio guidance for the driving task and sends it to the device.

[1519] Device: Save the guide and provide it to Person B.

[1520] User: Use the guide to safely complete the driving task.

[1521] Example prompts to input to the generative AI model

[1522] To the generative AI model:

[1523] Please create a driving assistance tool for a visually impaired user who needs to drive a car. The user has almost lost all of his / her sight. The tool needs to provide a proper audio feedback system and assist with obstacle detection and communication with other vehicles. Thank you in advance.

[1524] This system will help employees with disabilities to fully utilize their abilities, and will also enable companies to maximize their productivity.

[1525] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1526] Step 1:

[1527] Entering and submitting skill information

[1528] The user uses a smartphone to input their own skill information and the type and degree of disability. The device checks the input, formats the skill information, and sends it to the server. In this step, the input data includes the user's level of vision, programming experience, and required assistive functions. The device verifies the input data, formats it into the appropriate format, and transfers it to the server.

[1529] Step 2:

[1530] Receiving and registering skill information

[1531] The server receives the skill information sent from the terminal and registers it in the database. Specifically, it saves the received information in an SQL database and confirms that the skill information has been properly registered.

[1532] Step 3:

[1533] Generation of assistive devices and driving aids

[1534] The server launches a generative AI model to generate personalized assistive devices and driving aids based on the employee's skill information and the type and severity of their disability. At this stage, the AI ​​model inputs prompts and runs a process to generate assistive devices optimized for the employee's needs. Skill information and disability details are used as input data, and personalized feedback systems and visual aids are provided as output.

[1535] Step 4:

[1536] Providing the generated assistive devices

[1537] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot. Specifically, the generated digital feedback system, audio guide, etc. are transmitted to the terminal in file format and stored on the terminal.

[1538] Step 5:

[1539] Use of assistive devices

[1540] The device stores the generated assistive devices and converts them into a format to provide to the user. The user activates a smartphone or head-mounted display and uses audio guidance and visual aids to perform tasks or drive. For example, the user activates a voice feedback system stored on the smartphone and receives audio guidance while driving.

[1541] Step 6:

[1542] Creation and provision of communication guidelines

[1543] The server generates and transmits communication guidelines appropriate for the user and the employees around them. The data includes guidelines and examples of appropriate communication methods between employees. This is compiled into a document and sent to the terminal so that the employees around them can apply it.

[1544] Step 7:

[1545] Supporting on-the-job training and tracking progress

[1546] The server generates OJT training procedures based on skill information and job requirements and sends them to the terminal. It lists the step-by-step training procedures and provides them to the employee in the appropriate order. The terminal also records the user's progress in real time and sends it to the server. The server stores this progress in a database and generates progress reports as needed.

[1547] Step 8:

[1548] Tracking driving task progress and suggesting improvements

[1549] The server tracks the progress of the driving task in real time and generates suggestions for improvement or further assistance as needed. For example, feedback and suggestions for improvement from the obstacle detection system while driving are generated in real time and provided to the user's device, allowing the user to take immediate action.

[1550] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1551] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. This system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system provides more appropriate support tailored to the user's condition.

[1552] System configuration

[1553] 1. Input and registration of employee skill information

[1554] User: Enters their own skill information (e.g., type and degree of disability, past work experience, skills, etc.) through a dedicated application or website.

[1555] Device: Validate input and check skill information format.

[1556] Device: Converts skill information into JSON format and sends it to the server.

[1557] 2. Activation of the generative AI model and generation of the assistive device

[1558] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[1559] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[1560] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[1561] 3. Collecting emotional information using an emotion engine

[1562] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's tone of voice and facial expressions to determine their emotional state.

[1563] Terminal: Collects emotional information recognized by the emotion engine as data and sends it to the server.

[1564] 4. Provision and use of assistive devices

[1565] Server: Receives the emotion information sent from the emotion engine and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[1566] Terminal: Receives and stores the generated assistive devices. The assistive devices are displayed on the interface for easy access by the user.

[1567] User: Performs daily tasks using the assistive devices provided.

[1568] 5. Creation and provision of communication guidelines

[1569] Server: Generates communication guidelines appropriate for the user and their surrounding workers, including communication methods and considerations for visually impaired people.

[1570] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[1571] Device: View and apply received guidelines.

[1572] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[1573] 6. Supporting OJT and tracking progress

[1574] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[1575] Server: Sends the generated OJT instruction procedures to the user's terminal.

[1576] Terminal: Receives the instructional steps and displays them to the user.

[1577] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[1578] Terminal: Records the progress of work in real time and sends it to the server.

[1579] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[1580] Specific examples

[1581] New employee A, who has visual impairment and high stress

[1582] 1. Enter and submit skill information

[1583] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[1584] Device: Checks the data and sends it to the server.

[1585] 2. Receiving and registering skill information

[1586] Server: Receives information sent from the device and registers it in a database.

[1587] 3. Generation of assistive devices

[1588] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[1589] Server: Sends tools to the device.

[1590] 4. Stress assessment using an emotion engine

[1591] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[1592] Device: Sends data on high stress conditions to the server.

[1593] 5. Adjustment of assistive devices

[1594] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[1595] Server: Sends appropriately adjusted assistive devices to the device.

[1596] 6. Use of assistive devices

[1597] Terminal: Receives the adjusted assistive device and provides it to Person A.

[1598] User: Launches the tool and performs tasks.

[1599] 7. Communication Support

[1600] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[1601] Terminal: Receives the guidelines and displays them to employees.

[1602] Users: Communicate according to guidelines.

[1603] 8. Support for OJT

[1604] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[1605] Device: Save the guide and provide it to Person A.

[1606] User: Use the guide to progress through the task.

[1607] In this way, this system provides effective support for employees with disabilities and companies, creating an environment where employees can immediately function as a valuable asset. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[1608] The processing flow will be explained below.

[1609] Step 1:

[1610] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, the skill information input screen will be displayed.

[1611] Step 2:

[1612] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[1613] Step 3:

[1614] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[1615] Step 4:

[1616] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[1617] Step 5:

[1618] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[1619] Step 6:

[1620] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[1621] Step 7:

[1622] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[1623] Step 8:

[1624] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[1625] Step 9:

[1626] On the device: The emotion engine is activated to recognize the user's emotions in real time. The emotion engine analyzes voice tone and facial expressions to determine the user's emotional state (e.g., stress, excitement, concentration, etc.).

[1627] Step 10:

[1628] Terminal: Recognized emotion information is sent as data to the server.

[1629] Step 11:

[1630] Server: Receives emotional information and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[1631] Step 12:

[1632] Server: The adjusted assistive device is sent back to the user's device.

[1633] Step 13:

[1634] Device: Receives and saves the adjusted assistive devices. Notifies the user of the adjustments and provides feedback using icons and messages.

[1635] Step 14:

[1636] User: Uses the provided assistive devices to perform daily tasks. Any problems or feedback that arise during use are reported to the server via the device.

[1637] Step 15:

[1638] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[1639] Step 16:

[1640] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[1641] Step 17:

[1642] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[1643] Step 18:

[1644] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[1645] Step 19:

[1646] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[1647] Step 20:

[1648] Server: Sends the generated OJT instruction procedures to the user's terminal.

[1649] Step 21:

[1650] Terminal: Receives the instructional steps and displays them to the user.

[1651] Step 22:

[1652] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[1653] Step 23:

[1654] Terminal: Records the progress of work in real time and sends it to the server.

[1655] Step 24:

[1656] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[1657] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company. Furthermore, by combining it with an emotion engine, it is possible to dynamically adjust the support content according to the employee's emotional state, achieving more effective support.

[1658] Example 2

[1659] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1660] For many companies, providing appropriate assistive devices and a comfortable working environment for employees with disabilities is a difficult task. It is necessary to accurately grasp the employee's skill information, generate and provide customized assistive devices based on that information, and provide support that responds to fluctuations in their emotional state. However, current systems cannot effectively solve these issues.

[1661] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for an employee to input and register his / her own skill information, a means for generating an individualized assistive tool using a generative artificial intelligence model, and a means for providing the generated assistive tool to the employee. This makes it possible to generate and provide assistive tools tailored to the individual needs of employees with disabilities. In addition, by including a means for using a terminal equipped with an emotion engine that recognizes and collects the user's emotional information in real time and a means for adjusting the assistive tool based on the collected emotional information, it becomes possible to provide support that responds to changes in the employee's emotional state. This makes it possible to provide an environment in which employees can work comfortably and efficiently.

[1662] "Employee" refers to a person who is employed by a company or organization, and specifically includes people with disabilities.

[1663] "Skills information" refers to information including an employee's specific skills, past work experience, educational background, and the type and degree of disability.

[1664] A "dedicated application" is software designed to provide specific operations or functions, and provides an interface for users to input skill information.

[1665] "Generative AI model" refers to an AI technology for automatically generating assistive devices and other suggestions based on a worker's skill information and needs.

[1666] "Assistive devices" refers to the assistive devices and software tools that employees with disabilities need to perform their work efficiently and comfortably.

[1667] "Terminal" refers to an electronic device such as a computer, smartphone, or tablet, which is a device that allows a user to use this system.

[1668] "Server" refers to a central computer system that stores, processes data, and responds to various requests.

[1669] "Emotion engine" refers to artificial intelligence technology that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[1670] "Communication guidelines" refers to documents and tools that explain the guidelines and methods for employees and those around them to communicate appropriately.

[1671] "On-the-job training (OJT)" refers to a training method in which employees learn and improve their skills through actual work.

[1672] "Progress" refers to information that indicates the progress of an employee in completing a task or training.

[1673] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. The system uses a generative artificial intelligence model that generates personalized assistive devices based on the employee's skill information, which is input through a dedicated application or website. It also includes an emotion engine that collects the user's emotional information and adjusts the assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, supporting on-the-job training (OJT).

[1674] Hardware and software used

[1675] 1. Input and registration of employee skill information

[1676] The user enters skill information using a dedicated application (e.g., a smartphone app) or website.

[1677] The terminal verifies the entered skill information, converts the skill information into JSON format, and sends it to the server.

[1678] 2. Activation of the generative AI model and generation of the assistive device

[1679] The server receives the skill information in JSON format and registers it in a database. The technology used is a database management system (DBMS).

[1680] Based on the registered skill information, a generative artificial intelligence model (e.g., TensorFlow or PyTorch) is launched to generate personalized assistive devices that meet the user's needs.

[1681] The server transmits the generated assistive tool to the user's terminal.

[1682] 3. Collecting emotional information using an emotion engine

[1683] The device launches an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's voice tone and facial expressions.

[1684] The device transmits the collected emotion information to the server.

[1685] 4. Provision and use of assistive devices

[1686] The server adjusts the assistive device based on the emotion information and sends the adjusted assistive device to the user's terminal.

[1687] The terminal provides the user with the adjusted assistive device, which the user uses to perform the task.

[1688] 5. Creation and provision of communication guidelines

[1689] The server generates communication guidelines suitable for the user and surrounding employees and transmits them to the terminal.

[1690] The terminal displays guidelines, which employees follow when communicating.

[1691] 6. Supporting OJT and tracking progress

[1692] The server generates OJT training procedures based on the employee's skill information and job requirements.

[1693] The server sends the generated OJT procedure to the user, and the terminal displays it.

[1694] The user follows the instructional procedures to carry out OJT, records their progress through the terminal, and the server receives the data and tracks their progress.

[1695] Specific examples

[1696] New employee A, who has visual impairment and high stress

[1697] 1. Enter and submit your skill information:

[1698] User: Person A uses a dedicated app to enter that he or she is visually impaired and has programming experience.

[1699] Device: Checks this data and sends it to the server.

[1700] 2. Receiving and registering skill information:

[1701] Server: Receives information sent from the device and registers it in a database.

[1702] 3. Generating assistive devices:

[1703] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[1704] Server: Sends tools to the device.

[1705] 4. Stress assessment using emotion engine:

[1706] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[1707] Device: Sends data on high stress conditions to the server.

[1708] 5. Adjustment of assistive devices:

[1709] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[1710] Server: Sends appropriately adjusted assistive devices to the device.

[1711] 6. Use of assistive devices:

[1712] Terminal: Receives the adjusted assistive device and provides it to Person A.

[1713] User: Launches the tool and performs tasks.

[1714] Prompt Sentence Examples

[1715] "I am visually impaired and have three years of programming experience. Please suggest the best assistive device for me."

[1716] "Currently in a state of high stress. Please adjust your support equipment accordingly."

[1717] In this way, this system provides effective support for employees with disabilities and companies, and plays an important role in creating an environment where employees can function immediately as a workforce. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[1718] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1719] Step 1:

[1720] The user enters skill information. Using a dedicated application or website, employees enter their skill information (type of disability, skills, work experience, etc.). The entered information is saved in fields on the form. Specific rules apply to the format of the entered data.

[1721] Input: User's skills (e.g., "I am visually impaired and have 3 years of programming experience").

[1722] Output: Skill information saved in the input form.

[1723] Step 2:

[1724] The terminal validates the input data. The terminal validates the input data to ensure it conforms to the required format. Once validation is complete, it converts the data into JSON format.

[1725] Input: Skill information saved in the input form.

[1726] Data processing: Format check and validation, conversion to JSON format (e.g., {"Disability type": "Visual impairment", "Skills": "Programming", "Years of experience": 3}).

[1727] Output: Validated data in JSON format.

[1728] Step 3:

[1729] The device sends JSON format data to the server. The validated JSON data is sent to the server using an HTTP request.

[1730] Input: Validated data in JSON format.

[1731] Data operations: creating and sending HTTP requests.

[1732] Output: Skill information data sent to the server.

[1733] Step 4:

[1734] The server receives the skill information in JSON format and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[1735] Input: Submitted skill information in JSON format.

[1736] Data processing: INSERT operations into the database (e.g., registering data into the skill information table).

[1737] Output: Skill information registered in the database.

[1738] Step 5:

[1739] The server launches the generative AI model, which starts the process of generating appropriate assistive devices based on the skill information registered in the database.

[1740] Input: Skill information registered in the database.

[1741] Data computation: Initialization and execution of generative AI models.

[1742] Output: The process of creating a personalized assistive device begins.

[1743] Step 6:

[1744] The server generates assistive devices. Using generative AI models, it generates personalized assistive devices tailored to the user's needs, such as programming tools with voice feedback for the visually impaired.

[1745] Input: The parameters required for the generation process.

[1746] Data computation: Generation of assistive devices based on AI models.

[1747] Output: Individualized assistive devices.

[1748] Step 7:

[1749] The server sends the generated assistive device to the user's device, and returns the link or file of the generated assistive device to the device as an HTTP response.

[1750] Input: The generated assistive device.

[1751] Data operations: Creating and sending HTTP responses.

[1752] Output: Assistive devices sent to the user's device.

[1753] Step 8:

[1754] The device provides the assistive tools to the user and displays them on the interface for easy access by the user.

[1755] Input: Assistive device sent from the server.

[1756] Data calculation: Display settings for assistive devices.

[1757] Output: The assistive device provided to the user.

[1758] Step 9:

[1759] The user uses the assistive device. The user performs daily tasks using the provided assistive device.

[1760] Input: Assistive devices provided to the device.

[1761] Data calculation: Use of assistive devices.

[1762] Output: A deliverable that is useful for business.

[1763] Step 10:

[1764] The device activates an emotion engine to recognize the user's emotions, analyzing voice tone and facial expressions to determine the user's emotional state in real time.

[1765] Input: User audio and video data.

[1766] Data Computing: Sentiment Analysis.

[1767] Output: Determined emotion information.

[1768] Step 11:

[1769] The device sends emotional information to the server, which converts the recognized emotional information into JSON format and sends it to the server as an HTTP request.

[1770] Input: Determined emotion information.

[1771] Data operation: Creating and sending JSON format.

[1772] Output: Emotion information sent to the server.

[1773] Step 12:

[1774] The server receives the emotional information and adjusts the assistive device. For example, if the user is feeling stressed, the server generates an assistive device with relaxing music or additional guidance.

[1775] Input: The submitted emotion information.

[1776] Data calculation: Adjustment of assistive devices.

[1777] Output: Coordinated assistive devices.

[1778] Step 13:

[1779] The server sends the adjusted assistive device to the user's device, and then sends the appropriately adjusted assistive device again as an HTTP response.

[1780] Input: Adjusted assistive device.

[1781] Data operations: Creating and sending HTTP responses.

[1782] Output: The adjusted assistive devices are sent to the device.

[1783] Step 14:

[1784] Providing users with assistive devices that are adjusted to their devices, providing a relaxing environment for users to work comfortably.

[1785] Input: Calibration aid sent from server.

[1786] Data calculation: Display settings for assistive devices.

[1787] Output: The adjustment aid provided to the user.

[1788] Step 15:

[1789] The server generates and provides communication guidelines for the user and their surrounding employees, and sends them to the terminal.

[1790] Input: Information about the user and surrounding workers.

[1791] Data arithmetic: generating guidelines.

[1792] Output: The generated guidelines.

[1793] Step 16:

[1794] The terminal displays the communication guidelines. The guidelines are displayed on the terminal so that the user and nearby employees can easily refer to them.

[1795] Input: Communication guidelines sent by the server.

[1796] Data calculation: Guideline display settings.

[1797] Output: The displayed guidelines.

[1798] Step 17:

[1799] The server generates and transmits OJT training procedures based on the employee's skill information and job requirements.

[1800] Input: User skill information and job requirements.

[1801] Data calculation: Generation of OJT instruction procedures.

[1802] Output: Generated OJT instruction procedures.

[1803] Step 18:

[1804] The terminal displays the OJT instruction procedure. The procedure for the user to carry out the OJT is displayed on the interface.

[1805] Input: OJT training procedure sent from the server.

[1806] Data calculation: Display settings for instructional procedures.

[1807] Output: Displayed OJT instruction procedures.

[1808] Step 19:

[1809] The user performs OJT by following the provided instructional procedures to carry out the tasks and recording the progress in real time via the terminal.

[1810] Input: OJT training procedures.

[1811] Data calculation: Performing tasks and recording progress.

[1812] Output: Real-time recorded progress.

[1813] Step 20:

[1814] The device sends the progress to the server. The device sends the progress data recorded in real time to the server.

[1815] Input: Progress data recorded in real time.

[1816] Data calculation: Sending progress data.

[1817] Output: Progress data sent to the server.

[1818] Step 21:

[1819] The server records the training progress in a database and generates a report. Based on the progress data, the server stores the training progress in a database and generates a progress report.

[1820] Input: The progress data submitted.

[1821] Data calculation: recording to database and generating reports.

[1822] Output: The generated progress report.

[1823] (Application example 2)

[1824] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1825] It is important to create an environment in which employees with disabilities can immediately contribute to the workplace, but it is difficult to properly manage employees' skill and emotional information and provide them with personalized assistive devices. There is also a need for systems that improve communication between employees and those around them and enable employees to perform their work efficiently in a mentally stable state. Furthermore, there is a need to effectively support on-the-job training (OJT) and track the training process. The purpose of this invention is to solve these problems.

[1826] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1827] In this invention, the server includes: a means for an employee to input and register his / her own skill information; a means for using a generative artificial intelligence model to generate personalized assistive devices based on the skill information; a means for providing the generated assistive devices to the employee; a means for generating and providing guidelines for appropriate communication between the employee and other employees around him / her; a means for suggesting appropriate teaching methods to support on-the-job training and tracking the training progress; and a means for collecting employee emotion information using an emotion engine and adjusting the assistive devices. This makes it possible to create an environment in which employees can work immediately.

[1828] An "employee with a disability" is an employee who has a physical or mental disability and requires special assistance or consideration to accommodate that disability.

[1829] "Skill information" refers to information such as the skills, experience, and qualifications that employees possess.

[1830] "Individualized assistive devices" are devices created based on the skill information of a specific employee and the type and degree of disability, with the aim of assisting the employee in their work.

[1831] A "generative artificial intelligence model" is an artificial intelligence model that generates an output (here, an assistive device) according to a specific purpose based on input data.

[1832] The "Emotion Engine" is a technology that analyzes and recognizes emotional states in real time from voice tone and facial expressions.

[1833] "On-the-job training (OJT)" is workplace training conducted through actual work, and is an educational method in which knowledge and skills required for work are acquired through practice.

[1834] "Communication guidelines" are guidelines that compile specific instructions and advice for achieving appropriate communication.

[1835] This invention is a support system for improving the work efficiency of disabled employees in the workplace. The system uses a generative AI model that generates personalized assistive devices based on the employee's skill information. It also combines an emotion engine to provide support tailored to the employee's emotional state.

[1836] System configuration

[1837] 1. Input and registration of employee skill information

[1838] Users use a dedicated application or website to enter their skills information (e.g., type and degree of disability, past work experience, skills, etc.).

[1839] The terminal checks the entered skill information, converts it into JSON format, and sends it to the server.

[1840] 2. Activation of the generative AI model and generation of the assistive device

[1841] The server receives the JSON-formatted skill information sent from the device, analyzes it, and registers it in a database that includes information such as the user ID, type of disability, and skill content.

[1842] The server activates a generative AI model based on the registered skill information and generates personalized assistive tools that meet the user's needs.

[1843] The server generates and sends personalized assistive devices to the device, for example, a visually impaired user is provided with an assistive device with audio feedback.

[1844] 3. Collecting emotional information using an emotion engine

[1845] The device is equipped with an emotion engine that recognizes the user's emotions in real time by analyzing the user's tone of voice and facial expressions to determine their emotional state.

[1846] The terminal transmits the emotion information recognized by the emotion engine to the server.

[1847] 4. Provision and use of assistive devices

[1848] The server receives the emotional information and adjusts the assistive devices according to the user's current emotional state, for example, providing relaxation tools and guidance if the user is in a high-stress state.

[1849] The terminal displays the adjusted assistive tool to the user, allowing the user to easily access it.

[1850] The user performs daily tasks using the provided assistive devices.

[1851] 5. Creation and provision of communication guidelines

[1852] The server generates communication guidelines suitable for the user and the surrounding employees, and transmits them to the terminals of the user and the surrounding employees.

[1853] The terminal receives the guidelines and displays them to the user and nearby workers.

[1854] 6. Supporting OJT and tracking progress

[1855] The server generates OJT training procedures based on the employee's skill information and job requirements. These training procedures are provided in a step-by-step format.

[1856] The server transmits the generated OJT instruction procedure to the user's terminal.

[1857] The terminal receives the instruction procedure and displays it to the user.

[1858] The user follows the provided instructional procedures to carry out OJT, recording the progress of the work and any questions they may have in real time via the terminal.

[1859] The terminal records the progress of the work in real time and transmits it to the server.

[1860] The server records the progress in a database and generates reports to provide to administrators.

[1861] Specific examples

[1862] For example, if new employee A has a visual impairment but has programming experience, he or she can use a dedicated app to input his or her visual impairment and skills. The device then sends the data to a server, and an AI model generates a programming support tool with voice feedback. If the emotion engine determines that A is under high stress, relaxing music is added. The user and other employees perform their work according to appropriate communication guidelines, and on-the-job training is also supported step-by-step.

[1863] Example prompt sentence:

[1864] Mr. A is visually impaired and has programming experience. Please create an effective programming assistant tool with audio feedback that does not rely on vision for him. Also, please provide soothing music when he feels high stress.

[1865] This system allows employees with a variety of disabilities to function immediately and provides an environment in which they can work more effectively and safely in the workplace.

[1866] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1867] Step 1:

[1868] Users use a dedicated application or website to enter their own skill information (e.g., type and degree of disability, past work experience, skills, etc.) The entered data is converted into JSON format and sent to the server.

[1869] Input: User skill information

[1870] Output: JSON formatted data

[1871] Step 2:

[1872] The server receives the skill information in JSON format sent from the device, analyzes the received data, and registers the employee's ID, type of disability, skill information, etc. in a database.

[1873] Input: Skill information in JSON format

[1874] Output: Employee information stored in the database

[1875] Step 3:

[1876] The server activates a generative AI model based on the registered skill information, which generates personalized assistive devices tailored to the user's needs.

[1877] Input: Employee information stored in the database

[1878] Output: personalized assistive devices

[1879] Step 4:

[1880] The server transmits the generated assist tool to the terminal, which displays the received assist tool data to the user, making it available for use.

[1881] Input: personalized assistive devices

[1882] Output: Assistive device data displayed on the user's device

[1883] Step 5:

[1884] The device activates an emotion engine that recognizes the user's emotions in real time by analyzing voice tone and facial expressions to determine the user's emotional state.

[1885] Input: User's voice tone and facial expression data

[1886] Output: Perceived emotional state

[1887] Step 6:

[1888] The device sends the emotional information recognized by the emotion engine to the server, which then adjusts the assistive devices based on this emotional information. For example, if the user is in a high-stress state, it will add tools or guidance to help them relax.

[1889] Input: Perceived emotional state

[1890] Output: Adjusted assistive devices

[1891] Step 7:

[1892] The server generates communication guidelines suitable for the user and the employees around him / her and transmits them to the terminal, which then displays the received guidelines to the user and the employees around him / her.

[1893] Input: Database and emotion information

[1894] Output: Communication Guidelines

[1895] Step 8:

[1896] The server generates on-the-job training (OJT) instructions based on the employee's skill information and job requirements. The instructions are generated in a step-by-step format and sent to the terminal. The terminal displays the received instructions to the user and makes them available.

[1897] Input: Employee information and job requirements stored in the database

[1898] Output: OJT training procedure

[1899] Step 9:

[1900] Users follow the provided instructional procedures to carry out OJT. Any progress or questions they may have are recorded in real time on their devices and sent to the server. The server then records the progress in a database, generates a report, and provides it to the administrator.

[1901] Input: OJT progress and questions

[1902] Output: Progress report

[1903] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1904] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1905] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1906] [Fourth embodiment]

[1907] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1908] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1909] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1910] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1911] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1912] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1913] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1914] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1915] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1916] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1917] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1918] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1919] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1920] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective. The system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between employees and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process.

[1921] System configuration

[1922] 1. Input and registration of employee skill information

[1923] User: Enter their skill information (e.g., type and degree of disability, programming experience, etc.) through a dedicated application or website.

[1924] Device: Validate input and check skill information format.

[1925] Terminal: Sends skill information to the server.

[1926] 2. Activation of the generative AI model and generation of the assistive device

[1927] Server: Receives skill information sent from the device.

[1928] Server: Registers skill information in the database and launches the generative AI model.

[1929] Server: Generates personalized assistive devices based on the user's skill information and the type and degree of disability.

[1930] Server: Sends the generated assistive devices to the user's device.

[1931] 3. Provision and use of assistive devices

[1932] Terminal: Receives and stores the generated assistive devices.

[1933] Device: Provide the assistive device to the user in an appropriate format (e.g., audio description, text, visual aid).

[1934] User: Performs daily tasks using the assistive devices provided.

[1935] 4. Creation and provision of communication guidelines

[1936] Server: Generates communication guidelines appropriate for the user and their surrounding employees.

[1937] Server: Sends guidelines to the user's and nearby employees' devices.

[1938] Device: View and apply received guidelines.

[1939] 5. Supporting OJT and tracking progress

[1940] Server: Generates OJT training procedures based on the user's skill information and job requirements.

[1941] Server: Sends step-by-step instruction instructions to the user's device.

[1942] Terminal: Displays the received instruction procedure and provides it to the user.

[1943] User: Follow the provided instructional procedures and conduct OJT.

[1944] Terminal: Records the user's training progress in real time and sends it to the server.

[1945] Server: Records and analyzes progress in a database.

[1946] Server: Creates training progress reports and provides them to managers and companies.

[1947] Specific examples

[1948] New employee A with a visual impairment

[1949] 1. Enter and submit skill information

[1950] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[1951] Device: Checks the data and sends it to the server.

[1952] 2. Receiving and registering skill information

[1953] Server: Receives information sent from the device and registers it in a database.

[1954] 3. Generation of assistive devices

[1955] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[1956] Server: Sends tools to the device.

[1957] 4. Use of assistive devices

[1958] Terminal: Receive and save tools.

[1959] User: Launches the tool and performs programming tasks.

[1960] 5. Communication support

[1961] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[1962] Terminal: Receives the guidelines and displays them to employees.

[1963] Users: Communicate according to guidelines.

[1964] 6. Support for OJT

[1965] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[1966] Device: Save the guide and provide it to Person A.

[1967] User: Use the guide to progress through the task.

[1968] In this way, this system provides effective support for employees with disabilities and companies, creating an environment in which employees can function immediately as contributions to the workplace.

[1969] The processing flow will be explained below.

[1970] Step 1:

[1971] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, a screen for entering your skill information will appear.

[1972] Step 2:

[1973] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[1974] Step 3:

[1975] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[1976] Step 4:

[1977] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[1978] Step 5:

[1979] Server: Activates a generative AI model based on the registered skill information. The AI ​​model then begins the process of generating personalized assistive devices tailored to the user's needs.

[1980] Step 6:

[1981] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[1982] Step 7:

[1983] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[1984] Step 8:

[1985] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[1986] Step 9:

[1987] User: Activates the received assistive device and uses it for their own work. Any problems or feedback that arise while using the assistive device can be reported to the server via the device.

[1988] Step 10:

[1989] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[1990] Step 11:

[1991] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[1992] Step 12:

[1993] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[1994] Step 13:

[1995] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[1996] Step 14:

[1997] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[1998] Step 15:

[1999] Server: Sends the generated OJT instruction procedures to the user's terminal.

[2000] Step 16:

[2001] Terminal: Receives the instructional steps and displays them to the user.

[2002] Step 17:

[2003] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[2004] Step 18:

[2005] Terminal: Records the progress of work in real time and sends it to the server.

[2006] Step 19:

[2007] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[2008] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company.

[2009] Example 1

[2010] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2011] In conventional employment environments for employees with disabilities, there is often a lack of support to fully utilize the skills of the employees themselves, and appropriate communication methods and effective training methods are often lacking. As a result, it takes time for employees to become immediately effective, which can lead to a decrease in productivity throughout the organization. The purpose of this invention is to solve these problems and provide an environment in which employees with disabilities can perform their duties quickly and effectively.

[2012] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2013] In this invention, the server includes a means for recording the progress of the employee's training and transmitting it to the server in real time, a means for generating a training progress report and providing it to a manager or company, and a means for generating and providing guidelines for appropriate communication between the employee and other employees around him / her. This makes it possible to grasp the training progress in real time and provide appropriate communication and instruction methods so that the employee can quickly become a work-ready employee.

[2014] "Employee" refers to a working individual employed by a company or organization.

[2015] "Skill information" refers to detailed data such as an employee's abilities, experience, and knowledge.

[2016] A "generative artificial intelligence model" refers to a machine learning model that automatically generates specific outputs based on input data.

[2017] "Assistive devices" refers to tools, software, equipment, etc. that are useful to employees in carrying out their work.

[2018] "Communication guidelines" refer to guidelines and rules for employees and those around them to communicate appropriately.

[2019] "On-the-job training (OJT)" refers to practical training that takes place at the workplace.

[2020] "Training progress report" refers to a report that records and analyzes the progress of an employee's training.

[2021] "Real-time" refers to data and information being processed and provided almost immediately.

[2022] This invention is a support system for improving the employment environment for employees with disabilities, providing maximum support so that employees can work immediately. This system is composed of the following elements:

[2023] Input and registration of employee skill information

[2024] Users use a dedicated application or website to input their own skill information (e.g., type and degree of disability, programming experience, etc.) This data is received by the terminal, and after format confirmation and matching, it is sent to the server.

[2025] Activation of generative artificial intelligence models and generation of support tools

[2026] The server stores the received skill information in a database and activates a generative AI model. The AI ​​model generates personalized assistive devices (e.g., programming tools with voice feedback) based on the received skill information and the type and degree of disability. The generated assistive devices are sent from the server to the device.

[2027] Provision and use of assistive devices

[2028] The device receives the generated assistive devices, stores them, and provides them to the user in an appropriate format (e.g., audio guide, text, visual aids). The user uses the provided assistive devices to perform daily tasks.

[2029] Creation and provision of communication guidelines

[2030] The server generates communication guidelines suitable for the user and the surrounding employees, and transmits the guidelines to the terminals of the user and the surrounding employees. The terminals display and apply the received guidelines.

[2031] Supporting on-the-job training and tracking progress

[2032] The server generates OJT training procedures based on the user's skill information and job requirements, and sends the step-by-step training procedures to the user's device. The device displays the received training procedures and provides them to the user. The user follows the provided training procedures to carry out the OJT. The device records the training progress in real time and sends it to the server. The server records the progress in a database, creates a progress report, and provides it to managers and companies.

[2033] As a concrete example, consider the case of a new employee, Mr. A, who is visually impaired.

[2034] The user uses Mr. A's dedicated app to enter information about his visual impairment and programming experience.

[2035] The terminal checks the entered data and sends it to the server.

[2036] The server uses a generative AI model to generate programming tools with voice feedback for visually impaired people and sends them to the device.

[2037] The terminal receives the tool and provides it to the user. Person A uses the tool to perform his / her work.

[2038] The server generates appropriate communication methods for employees with visually impaired people and sends them to their terminals.

[2039] The terminal receives the guidelines and displays them to the employee. Mr. A and the employee communicate according to the guidelines.

[2040] The server generates a step-by-step audio guide for the programming task and sends it to the device. The device saves the guide and provides it to Person A. Person A uses the guide to progress through the task.

[2041] Example prompt sentence:

[2042] "Please create assistive tools to help new employees with visual impairments to smoothly carry out programming tasks."

[2043] "Please suggest effective ways for deaf people to communicate with their superiors."

[2044] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2045] Step 1:

[2046] Input and registration of employee skill information

[2047] The user opens a dedicated application or website.

[2048] The user fills out an input form with the type and degree of disability and skill information (e.g., programming experience, languages ​​fluent, etc.).

[2049] (Input) Type and degree of disability, skill information

[2050] (Processing) The terminal receives the input, performs format validation and matching, and checks that required fields are filled in.

[2051] (Output) Confirmed skill information

[2052] (Operation) The terminal sends the confirmed skill information to the server.

[2053] Step 2:

[2054] Activation of generative artificial intelligence models and generation of support tools

[2055] The server receives the skill information sent from the terminal.

[2056] (Input) Confirmed skill information

[2057] (Processing) The server stores the received information in a database and activates a generative AI model. Information about the user's skills and the type and degree of disability is obtained from the database, and a prompt is generated based on this information. The prompt is input into the generative AI model, which then generates an individualized assistive device.

[2058] (Output) Generated support equipment

[2059] (Operation) The server sends the generated support tool to the user's terminal.

[2060] Step 3:

[2061] Provision and use of assistive devices

[2062] The terminal receives the assistive tool transmitted from the server.

[2063] (Input) Generated support equipment

[2064] (Processing) The device stores the received assistive devices and provides them to the user in an appropriate format (e.g., audio guide, text, visual aid).

[2065] (Output) Assistive devices provided in a format that can be used by the user

[2066] (Action) The user uses the provided assistive device to perform daily tasks.

[2067] Step 4:

[2068] Creation and provision of communication guidelines

[2069] The server generates communication guidelines suitable for the user and the employees around him.

[2070] (Input) Information about the user and surrounding employees

[2071] (Processing) The server generates guidelines and transmits them to the terminals of the user and nearby employees.

[2072] (Output) Generated communication guidelines

[2073] (Operation) The terminal displays the received guidelines and applies them to the user and the employees around him / her.

[2074] Step 5:

[2075] Supporting on-the-job training and tracking progress

[2076] The server generates OJT training procedures based on the user's skill information and job requirements.

[2077] (Input) User skill information, job requirements

[2078] (Processing) The server generates step-by-step instruction procedures and sends them to the user's terminal.

[2079] (Output) Generated teaching steps

[2080] (Operation) The terminal displays the received training procedure and provides it to the user. The user follows the provided training procedure and performs OJT. The terminal records the user's training progress in real time and sends it to the server. The server records the progress in a database, creates a training progress report, and provides it to the manager or company.

[2081] (Application example 1)

[2082] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2083] In order for employees with disabilities to be immediately productive in the workplace, it is essential to provide them with individual assistive devices and appropriate communication support. However, conventional systems were unable to provide detailed support tailored to the type and degree of disability, and it was difficult to provide sufficient support in special situations such as driving assistance. This resulted in employees with disabilities being unable to fully utilize their abilities, and companies were ultimately unable to maximize their productivity. The purpose of this invention is to solve these problems.

[2084] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2085] In this invention, the server includes: means for a disabled employee to input and register his / her skill information; means for using a generative artificial intelligence model to generate personalized assistive devices and driving assistive devices based on the skill information; means for providing the generated assistive devices and driving assistive devices to the disabled employee; means for generating and providing guidelines for appropriate communication between the disabled employee and other employees around him / her; means for suggesting appropriate teaching methods to support on-the-job training and tracking the training progress; means for providing real-time assistance during driving using audio feedback and visual aids; and means for tracking the progress of driving tasks in real time and providing improvement suggestions and further assistance as needed. This enables disabled employees to work efficiently while receiving assistance tailored to their individual needs, and makes it possible to provide sufficient support, especially in special situations such as driving assistance.

[2086] "Employees with disabilities" refers to workers who have physical or sensory disabilities and require special assistance tailored to their individual needs.

[2087] "Skill information" refers to information about an employee's skills and experience, as well as the type and degree of disability.

[2088] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data and generates appropriate output for a specific purpose.

[2089] "Assistive devices" refers to assistive devices and software that employees with disabilities use to perform their daily work or specific tasks.

[2090] "Driver assistance devices" refers to devices and systems that are specifically designed to assist employees with disabilities while driving a motor vehicle.

[2091] The "Communication Guidelines" provide guidelines for smooth communication between employees with disabilities and those around them.

[2092] "On-the-job training (OJT)" refers to a training method in which employees acquire skills through actual work.

[2093] "Voice feedback" refers to a function that uses voice to convey information and instructions while working, etc.

[2094] "Visual aids" refers to devices or software that provide visual support.

[2095] "Real-time support" refers to support provided in response to the current situation.

[2096] "Driving task progress tracking" refers to monitoring and recording the progress of driving tasks in real time.

[2097] "Improvement suggestions and further support" refers to suggestions for improvements or problems found during the driving task or work, and providing additional support as needed.

[2098] The present invention is a system for supporting disabled employees in working efficiently and safely in a work environment, and particularly focuses on driving assistance for autonomous vehicles. Specific embodiments for implementing the present invention will be described below.

[2099] Hardware and software used

[2100] Hardware

[2101] Smartphone: Used by the employee to input skill information and receive feedback while driving.

[2102] Smart glasses: As a visual aid, they display driver assistance information while driving.

[2103] Head-mounted display: Displays assistance information and provides audio guidance while driving.

[2104] In-car robots: Providing real-time assistance while driving.

[2105] software

[2106] Generative AI model: Generates personalized assistive devices and driving assistance tools based on employee skill information.

[2107] System configuration

[2108] 1. Input and registration of employee skill information

[2109] The user uses a smartphone to input information about their skills and the type and degree of their disability.

[2110] The device checks the input, formats the skill information, and sends it to the server.

[2111] 2. Generation of assistive devices and driving assistive devices

[2112] The server receives the skill information sent from the terminal and registers it in a database.

[2113] A generative artificial intelligence model is activated to generate personalized assistive devices and driving assistive devices based on the employee's skill information and the type and degree of disability.

[2114] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot.

[2115] 3. Provision and use of assistive devices and driving aids

[2116] The terminal receives and stores the generated equipment.

[2117] The device provides the user with assistive devices in the form of audio guidance, visual aids, haptic feedback, etc.

[2118] The user uses the assistive device to perform work or drive.

[2119] 4. Creation and provision of communication guidelines

[2120] The server generates and transmits communication guidelines suitable for the user and the employees around him / her.

[2121] The terminal receives and displays the guidelines.

[2122] 5. Supporting OJT and tracking progress

[2123] The server generates OJT instruction procedures based on the skill information and job requirements and transmits them to the terminal.

[2124] The terminal displays the instruction procedure and provides it to the user.

[2125] The user follows the instruction procedure and performs OJT.

[2126] The device records the training progress in real time and transmits it to the server.

[2127] 6. Tracking driving task progress and suggesting improvements

[2128] The server tracks the progress of the driving task in real time and generates improvement suggestions or further assistance as needed.

[2129] The terminal provides the generated suggestions and assistance methods to the user.

[2130] Specific examples

[2131] In the case of visually impaired driver B

[2132] 1. Enter and submit skill information

[2133] User B uses a smartphone to enter information about his visual impairment and his level of vision.

[2134] Terminal: Checks the data and sends it to the server.

[2135] 2. Receiving and registering skill information

[2136] Server: Receives information sent from the terminal and registers it in a database.

[2137] 3. Generation of assistive devices and driving assistive devices

[2138] Server: Uses a generative AI model to generate a voice feedback system for the visually impaired.

[2139] Server: Sends the system to the smartphone.

[2140] 4. Use of assistive devices

[2141] Smartphone: Receives and stores voice feedback system.

[2142] User: Activates smartphone and uses voice feedback while driving.

[2143] 5. Communication support

[2144] Server: Generates and transmits appropriate communication methods for visually impaired people.

[2145] Device: Receives and applies guidelines, for example by alerting the driver to the sound of approaching vehicles.

[2146] 6. Support for OJT

[2147] Server: Generates step-by-step audio guidance for the driving task and sends it to the device.

[2148] Device: Save the guide and provide it to Person B.

[2149] User: Use the guide to safely complete the driving task.

[2150] Example prompts to input to the generative AI model

[2151] To the generative AI model:

[2152] Please create a driving assistance tool for a visually impaired user who needs to drive a car. The user has almost lost all of his / her sight. The tool needs to provide a proper audio feedback system and assist with obstacle detection and communication with other vehicles. Thank you in advance.

[2153] This system will help employees with disabilities to fully utilize their abilities, and will also enable companies to maximize their productivity.

[2154] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2155] Step 1:

[2156] Entering and submitting skill information

[2157] The user uses a smartphone to input their own skill information and the type and degree of disability. The device checks the input, formats the skill information, and sends it to the server. In this step, the input data includes the user's level of vision, programming experience, and required assistive functions. The device verifies the input data, formats it into the appropriate format, and transfers it to the server.

[2158] Step 2:

[2159] Receiving and registering skill information

[2160] The server receives the skill information sent from the terminal and registers it in the database. Specifically, it saves the received information in an SQL database and confirms that the skill information has been properly registered.

[2161] Step 3:

[2162] Generation of assistive devices and driving aids

[2163] The server launches a generative AI model to generate personalized assistive devices and driving aids based on the employee's skill information and the type and severity of their disability. At this stage, the AI ​​model inputs prompts and runs a process to generate assistive devices optimized for the employee's needs. Skill information and disability details are used as input data, and personalized feedback systems and visual aids are provided as output.

[2164] Step 4:

[2165] Providing the generated assistive devices

[2166] The server transmits the generated assistance tools and driving assistance tools to the user's smartphone, smart glasses, head-mounted display, or in-car robot. Specifically, the generated digital feedback system, audio guide, etc. are transmitted to the terminal in file format and stored on the terminal.

[2167] Step 5:

[2168] Use of assistive devices

[2169] The device stores the generated assistive devices and converts them into a format to provide to the user. The user activates a smartphone or head-mounted display and uses audio guidance and visual aids to perform tasks or drive. For example, the user activates a voice feedback system stored on the smartphone and receives audio guidance while driving.

[2170] Step 6:

[2171] Creation and provision of communication guidelines

[2172] The server generates and transmits communication guidelines appropriate for the user and the employees around them. The data includes guidelines and examples of appropriate communication methods between employees. This is compiled into a document and sent to the terminal so that the employees around them can apply it.

[2173] Step 7:

[2174] Supporting on-the-job training and tracking progress

[2175] The server generates OJT training procedures based on skill information and job requirements and sends them to the terminal. It lists the step-by-step training procedures and provides them to the employee in the appropriate order. The terminal also records the user's progress in real time and sends it to the server. The server stores this progress in a database and generates progress reports as needed.

[2176] Step 8:

[2177] Tracking driving task progress and suggesting improvements

[2178] The server tracks the progress of the driving task in real time and generates suggestions for improvement or further assistance as needed. For example, feedback and suggestions for improvement from the obstacle detection system while driving are generated in real time and provided to the user's device, allowing the user to take immediate action.

[2179] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2180] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. This system uses a generative artificial intelligence model in which employees input their own skill information and generate personalized assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, suggests appropriate teaching methods to support on-the-job training (OJT), and tracks the training process. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system provides more appropriate support tailored to the user's condition.

[2181] System configuration

[2182] 1. Input and registration of employee skill information

[2183] User: Enters their own skill information (e.g., type and degree of disability, past work experience, skills, etc.) through a dedicated application or website.

[2184] Device: Validate input and check skill information format.

[2185] Device: Converts skill information into JSON format and sends it to the server.

[2186] 2. Activation of the generative AI model and generation of the assistive device

[2187] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[2188] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[2189] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[2190] 3. Collecting emotional information using an emotion engine

[2191] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's tone of voice and facial expressions to determine their emotional state.

[2192] Terminal: Collects emotional information recognized by the emotion engine as data and sends it to the server.

[2193] 4. Provision and use of assistive devices

[2194] Server: Receives the emotion information sent from the emotion engine and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[2195] Terminal: Receives and stores the generated assistive devices. The assistive devices are displayed on the interface for easy access by the user.

[2196] User: Performs daily tasks using the assistive devices provided.

[2197] 5. Creation and provision of communication guidelines

[2198] Server: Generates communication guidelines appropriate for the user and their surrounding workers, including communication methods and considerations for visually impaired people.

[2199] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[2200] Device: View and apply received guidelines.

[2201] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[2202] 6. Supporting OJT and tracking progress

[2203] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[2204] Server: Sends the generated OJT instruction procedures to the user's terminal.

[2205] Terminal: Receives the instructional steps and displays them to the user.

[2206] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[2207] Terminal: Records the progress of work in real time and sends it to the server.

[2208] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[2209] Specific examples

[2210] New employee A, who has visual impairment and high stress

[2211] 1. Enter and submit skill information

[2212] User: Person A uses a dedicated app to enter information about his / her visual impairment and programming experience.

[2213] Device: Checks the data and sends it to the server.

[2214] 2. Receiving and registering skill information

[2215] Server: Receives information sent from the device and registers it in a database.

[2216] 3. Generation of assistive devices

[2217] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[2218] Server: Sends tools to the device.

[2219] 4. Stress assessment using an emotion engine

[2220] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[2221] Device: Sends data on high stress conditions to the server.

[2222] 5. Adjustment of assistive devices

[2223] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[2224] Server: Sends appropriately adjusted assistive devices to the device.

[2225] 6. Use of assistive devices

[2226] Terminal: Receives the adjusted assistive device and provides it to Person A.

[2227] User: Launches the tool and performs tasks.

[2228] 7. Communication Support

[2229] Server: Generates and sends appropriate communication methods for employees with visual impairments.

[2230] Terminal: Receives the guidelines and displays them to employees.

[2231] Users: Communicate according to guidelines.

[2232] 8. Support for OJT

[2233] Server: Generates step-by-step audio guides for programming tasks and sends them to the device.

[2234] Device: Save the guide and provide it to Person A.

[2235] User: Use the guide to progress through the task.

[2236] In this way, this system provides effective support for employees with disabilities and companies, creating an environment where employees can immediately function as a valuable asset. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[2237] The processing flow will be explained below.

[2238] Step 1:

[2239] User: Access the dedicated application or website and enter your user ID and password on the login screen. Once your login is approved, the skill information input screen will be displayed.

[2240] Step 2:

[2241] User: Enter the type and degree of their disability, past work experience, and skills (e.g., programming, data entry, etc.). Once you have completed the entry, press the "Submit" button.

[2242] Step 3:

[2243] Terminal: Validates the format of the entered skill information to ensure there are no omissions or errors. If there are no problems, converts the skill information into JSON format and sends it to the server.

[2244] Step 4:

[2245] Server: Receives skill information in JSON format sent from the device. Analyzes the received data and registers it in a database. This registration includes the user ID, type of disability, skill content, etc.

[2246] Step 5:

[2247] Server: Activates the generative AI model based on the registered skill information. The AI ​​model then starts the process of generating personalized assistive devices that meet the user's needs.

[2248] Step 6:

[2249] Server: Generates personalized assistive devices, such as programming tools with audio feedback for visually impaired users. The assistive devices are designed to help users perform their tasks efficiently.

[2250] Step 7:

[2251] Server: Sends the generated assistive devices to the user's device using a secure communication protocol to ensure data integrity and confidentiality.

[2252] Step 8:

[2253] Terminal: Receives the transmitted assistive devices and stores them in local storage. The assistive devices are displayed on the interface for easy access by the user.

[2254] Step 9:

[2255] On the device: The emotion engine is activated to recognize the user's emotions in real time. The emotion engine analyzes voice tone and facial expressions to determine the user's emotional state (e.g., stress, excitement, concentration, etc.).

[2256] Step 10:

[2257] Terminal: Recognized emotion information is sent as data to the server.

[2258] Step 11:

[2259] Server: Receives emotional information and adjusts the assistive devices according to the user's current emotional state. For example, if the user is feeling stressed, it provides tools and guidance to help them relax.

[2260] Step 12:

[2261] Server: The adjusted assistive device is sent back to the user's device.

[2262] Step 13:

[2263] Device: Receives and saves the adjusted assistive devices. Notifies the user of the adjustments and provides feedback using icons and messages.

[2264] Step 14:

[2265] User: Uses the provided assistive devices to perform daily tasks. Any problems or feedback that arise during use are reported to the server via the device.

[2266] Step 15:

[2267] Server: Generates appropriate communication guidelines for users and their surrounding workers, including communication methods and considerations for visually impaired people.

[2268] Step 16:

[2269] Server: Sends the generated guidelines to the user's and nearby employees' devices.

[2270] Step 17:

[2271] Terminal: Receives the guidelines and displays them as notifications or messages to the user and nearby employees.

[2272] Step 18:

[2273] Users: Make efforts to communicate appropriately with surrounding employees based on the guidelines. Refer to the guidelines as necessary.

[2274] Step 19:

[2275] Server: Generates OJT training procedures based on the user's skill information and job requirements. The training procedures are provided in a step-by-step format.

[2276] Step 20:

[2277] Server: Sends the generated OJT instruction procedures to the user's terminal.

[2278] Step 21:

[2279] Terminal: Receives the instructional steps and displays them to the user.

[2280] Step 22:

[2281] User: Follow the provided instructional procedures to carry out OJT. Record the progress of the work and any questions you may have in real time via the terminal.

[2282] Step 23:

[2283] Terminal: Records the progress of work in real time and sends it to the server.

[2284] Step 24:

[2285] Server: Records the training progress in a database and generates reports, which are provided to the administrator.

[2286] Through the above processing steps, this system can provide appropriate support and communication to employees with disabilities and those around them, improving the work efficiency of the entire company. Furthermore, by combining it with an emotion engine, it is possible to dynamically adjust the support content according to the employee's emotional state, achieving more effective support.

[2287] Example 2

[2288] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2289] For many companies, providing appropriate assistive devices and a comfortable working environment for employees with disabilities is a difficult task. It is necessary to accurately grasp the employee's skill information, generate and provide customized assistive devices based on that information, and provide support that responds to fluctuations in their emotional state. However, current systems cannot effectively solve these issues.

[2290] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for an employee to input and register his / her own skill information, a means for generating an individualized assistive tool using a generative artificial intelligence model, and a means for providing the generated assistive tool to the employee. This makes it possible to generate and provide assistive tools tailored to the individual needs of employees with disabilities. In addition, by including a means for using a terminal equipped with an emotion engine that recognizes and collects the user's emotional information in real time and a means for adjusting the assistive tool based on the collected emotional information, it becomes possible to provide support that responds to changes in the employee's emotional state. This makes it possible to provide an environment in which employees can work comfortably and efficiently.

[2291] "Employee" refers to a person who is employed by a company or organization, and specifically includes people with disabilities.

[2292] "Skills information" refers to information including an employee's specific skills, past work experience, educational background, and the type and degree of disability.

[2293] A "dedicated application" is software designed to provide specific operations or functions, and provides an interface for users to input skill information.

[2294] "Generative AI model" refers to an AI technology for automatically generating assistive devices and other suggestions based on a worker's skill information and needs.

[2295] "Assistive devices" refers to the assistive devices and software tools that employees with disabilities need to perform their work efficiently and comfortably.

[2296] "Terminal" refers to an electronic device such as a computer, smartphone, or tablet, which is a device that allows a user to use this system.

[2297] "Server" refers to a central computer system that stores, processes data, and responds to various requests.

[2298] "Emotion engine" refers to artificial intelligence technology that analyzes a user's tone of voice and facial expressions to recognize their emotional state in real time.

[2299] "Communication guidelines" refers to documents and tools that explain the guidelines and methods for employees and those around them to communicate appropriately.

[2300] "On-the-job training (OJT)" refers to a training method in which employees learn and improve their skills through actual work.

[2301] "Progress" refers to information that indicates the progress of an employee in completing a task or training.

[2302] This invention is a support system for improving the employment environment for employees with disabilities, helping them become immediately effective in the workplace. The system uses a generative artificial intelligence model that generates personalized assistive devices based on the employee's skill information, which is input through a dedicated application or website. It also includes an emotion engine that collects the user's emotional information and adjusts the assistive devices. It also provides guidelines for appropriate communication between the employee and those around them, supporting on-the-job training (OJT).

[2303] Hardware and software used

[2304] 1. Input and registration of employee skill information

[2305] The user enters skill information using a dedicated application (e.g., a smartphone app) or website.

[2306] The terminal verifies the entered skill information, converts the skill information into JSON format, and sends it to the server.

[2307] 2. Activation of the generative AI model and generation of the assistive device

[2308] The server receives the skill information in JSON format and registers it in a database. The technology used is a database management system (DBMS).

[2309] Based on the registered skill information, a generative artificial intelligence model (e.g., TensorFlow or PyTorch) is launched to generate personalized assistive devices that meet the user's needs.

[2310] The server transmits the generated assistive tool to the user's terminal.

[2311] 3. Collecting emotional information using an emotion engine

[2312] The device launches an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's voice tone and facial expressions.

[2313] The device transmits the collected emotion information to the server.

[2314] 4. Provision and use of assistive devices

[2315] The server adjusts the assistive device based on the emotion information and sends the adjusted assistive device to the user's terminal.

[2316] The terminal provides the user with the adjusted assistive device, which the user uses to perform the task.

[2317] 5. Creation and provision of communication guidelines

[2318] The server generates communication guidelines suitable for the user and surrounding employees and transmits them to the terminal.

[2319] The terminal displays guidelines, which employees follow when communicating.

[2320] 6. Supporting OJT and tracking progress

[2321] The server generates OJT training procedures based on the employee's skill information and job requirements.

[2322] The server sends the generated OJT procedure to the user, and the terminal displays it.

[2323] The user follows the instructional procedures to carry out OJT, records their progress through the terminal, and the server receives the data and tracks their progress.

[2324] Specific examples

[2325] New employee A, who has visual impairment and high stress

[2326] 1. Enter and submit your skill information:

[2327] User: Person A uses a dedicated app to enter that he or she is visually impaired and has programming experience.

[2328] Device: Checks this data and sends it to the server.

[2329] 2. Receiving and registering skill information:

[2330] Server: Receives information sent from the device and registers it in a database.

[2331] 3. Generating assistive devices:

[2332] Server: Uses generative AI models to generate programming tools with audio feedback for the visually impaired.

[2333] Server: Sends tools to the device.

[2334] 4. Stress assessment using emotion engine:

[2335] Device: The emotion engine determines whether Mr. A is in a high stress state based on his tone of voice and facial expression.

[2336] Device: Sends data on high stress conditions to the server.

[2337] 5. Adjustment of assistive devices:

[2338] Server: Receives emotional information and adjusts assistive devices such as relaxing music and guides.

[2339] Server: Sends appropriately adjusted assistive devices to the device.

[2340] 6. Use of assistive devices:

[2341] Terminal: Receives the adjusted assistive device and provides it to Person A.

[2342] User: Launches the tool and performs tasks.

[2343] Prompt Sentence Examples

[2344] "I am visually impaired and have three years of programming experience. Please suggest the best assistive device for me."

[2345] "Currently in a state of high stress. Please adjust your support equipment accordingly."

[2346] In this way, this system provides effective support for employees with disabilities and companies, and plays an important role in creating an environment where employees can function immediately as a workforce. The addition of an emotion engine enables more effective support by dynamically adjusting the support content taking into account the user's emotional state.

[2347] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2348] Step 1:

[2349] The user enters skill information. Using a dedicated application or website, employees enter their skill information (type of disability, skills, work experience, etc.). The entered information is saved in fields on the form. Specific rules apply to the format of the entered data.

[2350] Input: User's skills (e.g., "I am visually impaired and have 3 years of programming experience").

[2351] Output: Skill information saved in the input form.

[2352] Step 2:

[2353] The terminal validates the input data. The terminal validates the input data to ensure it conforms to the required format. Once validation is complete, it converts the data into JSON format.

[2354] Input: Skill information saved in the input form.

[2355] Data processing: Format check and validation, conversion to JSON format (e.g., {"Disability type": "Visual impairment", "Skills": "Programming", "Years of experience": 3}).

[2356] Output: Validated data in JSON format.

[2357] Step 3:

[2358] The device sends JSON format data to the server. The validated JSON data is sent to the server using an HTTP request.

[2359] Input: Validated data in JSON format.

[2360] Data operations: creating and sending HTTP requests.

[2361] Output: Skill information data sent to the server.

[2362] Step 4:

[2363] The server receives the skill information in JSON format and registers it in the database. The registration includes the user ID, type of disability, skill content, etc.

[2364] Input: Submitted skill information in JSON format.

[2365] Data processing: INSERT operations into the database (e.g., registering data into the skill information table).

[2366] Output: Skill information registered in the database.

[2367] Step 5:

[2368] The server launches the generative AI model, which starts the process of generating appropriate assistive devices based on the skill information registered in the database.

[2369] Input: Skill information registered in the database.

[2370] Data computation: Initialization and execution of generative AI models.

[2371] Output: The process of creating a personalized assistive device begins.

[2372] Step 6:

[2373] The server generates assistive devices. Using generative AI models, it generates personalized assistive devices tailored to the user's needs, such as programming tools with voice feedback for the visually impaired.

[2374] Input: The parameters required for the generation process.

[2375] Data computation: Generation of assistive devices based on AI models.

[2376] Output: Individualized assistive devices.

[2377] Step 7:

[2378] The server sends the generated assistive device to the user's device, and returns the link or file of the generated assistive device to the device as an HTTP response.

[2379] Input: The generated assistive device.

[2380] Data operations: Creating and sending HTTP responses.

[2381] Output: Assistive devices sent to the user's device.

[2382] Step 8:

[2383] The device provides the assistive tools to the user and displays them on the interface for easy access by the user.

[2384] Input: Assistive device sent from the server.

[2385] Data calculation: Display settings for assistive devices.

[2386] Output: The assistive device provided to the user.

[2387] Step 9:

[2388] The user uses the assistive device. The user performs daily tasks using the provided assis...

Claims

1. A means for employees with disabilities to input and register their skill information; a means for using a generative artificial intelligence model to generate an individualized assistive tool based on the skill information; means for providing the generated assistive tool to the worker; means for generating and providing guidelines for appropriate communication between the employee and employees around the employee; To support on-the-job training by suggesting appropriate teaching methods and tracking the training process. A system including:

2. The system according to claim 1, wherein the generative artificial intelligence model takes into account the skill information of the worker and the type and degree of disability when generating the individualized assistive device.

3. 2. The system according to claim 1, wherein the generative artificial intelligence model generates a step-by-step instruction procedure when proposing an instruction method for on-the-job training.

Images