System

Abstract

A system is provided.SOLUTION: A system for utilizing a generative AI model to automate and support workers' tasks and workflows, the system comprising means for providing an interface through which workers can input information to apply for a leave of absence, means for receiving and recording leave application information, means for automatically executing a necessary approval flow based on the recorded leave application information, and means for notifying the workers of the approval status of the leave application.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, internal processes such as vacation requests and expense reports have been performed manually, requiring a great deal of time and effort. This has reduced worker efficiency and hindered the prompt execution of work. Furthermore, the progress of procedures has been unclear, leading to occasional errors and other problems. The purpose of this invention is to solve these problems and enable workers to perform their work efficiently. [Means for solving the problem]

[0005] The present invention relates to a system for automating and supporting worker tasks and workflows using generative AI models. The system includes a means for providing an interface through which workers can input information for requesting vacation, a means for receiving and recording vacation request information, a means for automatically executing the necessary approval process based on the recorded vacation request information, and a means for notifying workers of the approval status of the vacation request. The system also includes a means for providing an interface through which workers can input information for expense reimbursement, a means for receiving and recording expense reimbursement information, a means for automatically generating expense request forms based on the recorded expense reimbursement information and executing the necessary approval process, and a means for notifying workers of the approval status of the expense reimbursement. Furthermore, the system includes a means for periodically checking the progress of the approval process and notifying the progress status, and a means for notifying workers of the final approval result. This streamlines previously manual processes and improves worker work efficiency.

[0006] A "generative AI model" refers to an algorithm or system that uses natural language processing and machine learning techniques to generate useful information from input data.

[0007] "Worker" refers to a person who belongs to a company or organization and performs work.

[0008] "Interface" refers to the screens and input forms that allow a user to interact with a system.

[0009] "Leave application information" refers to information such as the start date, end date, and reason for leave that workers enter into the system.

[0010] "Database" refers to a storage system that stores information in an organized manner and makes it available when needed.

[0011] "Approval flow" refers to a series of procedures in which a person with appropriate authority approves or rejects the requested information.

[0012] "Notification" refers to the system communicating a message or status change to a worker.

[0013] "Expense reimbursement information" refers to detailed information about work-related expenses incurred by a worker, including, for example, images of receipts, amounts, dates, and purposes.

[0014] An "expense application form" is a document automatically generated based on expense reimbursement information, and refers to a formal application document for obtaining approval. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system for automating and assisting worker tasks and workflows using generative AI models, designed to efficiently manage leave request and expense reimbursement processes.

[0037] An embodiment of the leave request process

[0038] server:

[0039] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[0040] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[0041] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[0042] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0043] Device:

[0044] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[0045] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[0046] User:

[0047] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[0048] 2. After submitting the information, the server automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved."

[0049] Expense reimbursement process embodiment

[0050] server:

[0051] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[0052] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[0053] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[0054] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0055] Device:

[0056] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0057] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[0058] User:

[0059] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[0060] 2. After submitting the information, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "the expense claim has been approved."

[0061] The system of this invention can automate and streamline leave applications and expense settlements, which were previously done manually, thereby reducing the workload on workers and improving work efficiency.

[0062] The processing flow will be explained below.

[0063] Time Off Request Process Steps

[0064] Step 1: Provide an interface

[0065] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[0066] Step 2: Enter your vacation request information

[0067] User: Enter the leave start date, end date, and reason into the interface.

[0068] Step 3: Submit your vacation request information

[0069] Terminal: Sends the vacation request information entered by the worker to the server.

[0070] Step 4: Receive and store leave request information

[0071] Server: Receives the submitted vacation request information and stores it in a database.

[0072] Step 5: Start the approval flow

[0073] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[0074] Step 6: Submitting an Approval Request

[0075] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0076] Step 7: Your manager's response

[0077] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0078] Step 8: Submit a secondary approval request

[0079] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[0080] Step 9: HR response

[0081] Server: Receives approval / rejection responses from the HR department and records them in a database.

[0082] Step 10: Notification of approval status

[0083] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[0084] Expense process steps

[0085] Step 1: Provide an interface

[0086] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0087] Step 2: Enter expense information

[0088] User: Enters receipt photo, amount, date, and description into the interface.

[0089] Step 3: Submit expense information

[0090] Terminal: Sends the expense reimbursement information entered by the worker to the server.

[0091] Step 4: Receive and store expense information

[0092] Server: Receives the expense report information and stores it in a database.

[0093] Step 5: Auto-generate application form

[0094] Server: Automatically generate expense reports based on the saved expense information.

[0095] Step 6: Start the approval flow

[0096] Server: Sends approval request to manager and accounting department.

[0097] Step 7: Submitting an Approval Request

[0098] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0099] Step 8: Your manager's response

[0100] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0101] Step 9: Submit a secondary approval request

[0102] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[0103] Step 10: Response from the accounting department

[0104] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[0105] Step 11: Notification of approval status

[0106] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Traditional leave application and expense reimbursement processes involve a lot of manual work, placing a heavy burden on workers. This manual work can also lead to application and approval information getting lost and delays in processing. This can lead to issues such as reduced work efficiency and increased risk of human error. Therefore, there is a need to automate these processes, reduce the burden on workers, and achieve an efficient workflow.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes a means for providing an interface that allows workers to input information for applying for leave, a means for receiving leave application information and recording it in a database, a means for automatically generating an approval request using a generative AI model based on the recorded leave application information and starting the necessary approval flow, a means for receiving approval or rejection responses from superiors and the human resources department and recording them in a database, and a means for notifying workers of the approval progress and the final approval result. This allows for a smoother application process for leave applications and expense reimbursement.

[0112] This makes it possible to automate processes, reduce the burden on workers, and achieve efficient, error-free work flows.

[0113] A "generative AI model" is an artificial intelligence model that is generated using natural language processing and machine learning to help automate specific tasks or operations.

[0114] "Worker" refers to an individual who is employed to perform work for a business or organization.

[0115] A "task" is a specific item or task that a worker performs as part of their job.

[0116] "Workflow" refers to a series of steps or processes that must be followed to perform a specific task.

[0117] A "leave request" is the formal process by which a worker seeks permission to take time off work for a specific period of time.

[0118] "Expense reimbursement" is the formal process by which workers are reimbursed by their organization for work-related expenses they incur.

[0119] "Interface" refers to the input and output means by which workers interact with a system.

[0120] "Means for providing an interface through which information can be entered" refers to technology that provides a screen or form through which workers can enter the necessary data or information into the system.

[0121] A "database" is a system for organizing and storing information electronically, and can take the form of SQL or NoSQL.

[0122] "Approval flow" refers to the process or steps by which a particular application or request is formally approved.

[0123] "Progress" refers to the state or status of how far a particular task or project has progressed.

[0124] "Notification" refers to the means by which the system sends messages to workers and other parties to inform them of status and results.

[0125] The present invention relates to a system that utilizes generative AI models to automate and assist worker tasks and workflows. The system aims to efficiently manage leave request and expense reimbursement processes. Specific embodiments are described in detail below.

[0126] An embodiment of the leave request process

[0127] server:

[0128] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information to apply for leave. This interface includes a form for entering the start date, end date, and reason for the leave. The leave application information entered by the worker (e.g., "I will be taking leave from December 1st to December 5th, 2023. The reason is travel.") is sent to the server, converted into JSON format, and recorded in a MySQL (registered trademark) or PostgreSQL database.

[0129] Next, a generative AI model (e.g., OpenAI's GPT-3®) is used to automatically generate approval request messages and send them to the supervisor and HR department. Slack APIs and email servers are used to send the messages. Approval or rejection responses from the supervisor and HR department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final result. Notifications are sent via email or in-app notifications.

[0130] Device:

[0131] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the form for requesting leave. The information entered in the form is sent to the server via an HTTP POST request.

[0132] User:

[0133] The worker enters the necessary information into the interface displayed on the terminal. For example, "I will take vacation from December 1st to December 5th, 2023. The reason is travel." and presses the submit button. The worker then checks the approval progress and final result (e.g., "Vacation request approved") via notifications from the server.

[0134] Expense reimbursement process embodiment

[0135] server:

[0136] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information for expense reporting. This interface includes a form for uploading photos of receipts and entering amounts, dates, and descriptions. The expense reporting information entered by the worker (e.g., "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses") is sent to the server, converted to JSON format, and recorded in a MySQL or PostgreSQL database.

[0137] Next, the generative AI model is used to automatically generate expense reports and send approval requests to superiors and the accounting department. Slack APIs and email servers are used to send the reports. Approval or rejection responses from superiors and the accounting department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final results. Notifications are sent via email and in-app notifications.

[0138] Device:

[0139] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form. The information entered in the form is sent to the server via an HTTP POST request.

[0140] User:

[0141] The worker enters the necessary information (photo of the receipt, amount, date, description) into the interface displayed on the terminal. For example, they enter "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses" and presses the submit button. The worker then checks the approval progress and final result (e.g., "Expense reimbursement has been approved") via notifications from the server.

[0142] The system of the present invention automates the previously manual processes of vacation application and expense settlement, reducing the workload of workers and enabling an efficient, error-free workflow.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Specific processing steps for the leave request process

[0145] Server-side processing steps

[0146] Step 1: Launch the interface

[0147] The server runs a web interface for workers to request time off, using software such as Flask or Express.

[0148] Input: None

[0149] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[0150] Step 2: Receiving leave request information

[0151] The server receives the vacation request information entered by the worker, specifically as JSON format data sent as an HTTP POST request.

[0152] Input: Leave start date, end date, reason

[0153] Output: Record the received JSON format data in the database.

[0154] Step 3: Generate and submit an approval request

[0155] The server generates an approval request message using a generative AI model based on the recorded vacation request information.

[0156] Input: Leave request information stored in the database

[0157] Data computation: Message generation by generative AI models (e.g., GPT-3)

[0158] Output: A message is created to send an approval request to your manager and HR.

[0159] Step 4: Receive and record approval and rejection responses

[0160] The server receives the approval or rejection response from the manager and HR department via the Slack API or a mail server.

[0161] Input: Approval or rejection response

[0162] Output: Record the received response in a database.

[0163] Step 5: Communicate progress and final results

[0164] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[0165] Input: Progress and final results in database

[0166] Output: Notification message to the worker

[0167] Terminal processing steps

[0168] Step 1: Interface display

[0169] The device renders the HTML, CSS, and JavaScript code sent from the server and displays a form for requesting leave.

[0170] Input: HTML / CSS / JavaScript code received from the server

[0171] Output: A vacation request form is displayed that the user can fill out.

[0172] Step 2: Send data

[0173] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[0174] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[0175] Output: HTTP POST request to the server

[0176] User processing steps

[0177] Step 1: Enter your information

[0178] Workers enter the necessary information into the interface displayed on the terminal.

[0179] Input: Leave start date, end date, reason

[0180] Output: The input information is saved in the form displayed on the device.

[0181] Step 2: Submit and check the results

[0182] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[0183] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[0184] Output: The data is sent to the server. Later, you will receive a notification that "your vacation request has been approved."

[0185] Specific processing steps in the expense reimbursement process

[0186] Server-side processing steps

[0187] Step 1: Launch the interface

[0188] The server runs a web interface for workers to submit expense reports, using software such as Flask or Express.

[0189] Input: None

[0190] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[0191] Step 2: Receiving expense information

[0192] The server receives the expense report information entered by the worker, specifically as JSON formatted data sent as an HTTP POST request.

[0193] Input: Receipt photo, amount, date, description

[0194] Output: Record the received JSON format data in the database.

[0195] Step 3: Generate and submit expense reports

[0196] The server generates expense claims using a generative AI model based on the recorded expense reimbursement information.

[0197] Input: Expense information stored in the database

[0198] Data Computing: Generative AI models (e.g., GPT-3) generate expense reports

[0199] Output: An expense report is created to be sent to the manager and accounting department for approval.

[0200] Step 4: Receive and record approval and rejection responses

[0201] The server receives approval or rejection responses from the manager and accounting department via the Slack API or email server.

[0202] Input: Approval or rejection response

[0203] Output: Record the received response in a database.

[0204] Step 5: Communicate progress and final results

[0205] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[0206] Input: Progress and final results in database

[0207] Output: Notification message to the worker

[0208] Terminal processing steps

[0209] Step 1: Interface display

[0210] The device renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form.

[0211] Input: HTML / CSS / JavaScript code received from the server

[0212] Output: A display of the expense form that the user can fill out.

[0213] Step 2: Send data

[0214] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[0215] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[0216] Output: HTTP POST request to the server

[0217] User processing steps

[0218] Step 1: Enter your information

[0219] Workers enter the necessary information into the interface displayed on the terminal.

[0220] Input: Receipt photo, amount, date, description

[0221] Output: The input information is saved in the form displayed on the device.

[0222] Step 2: Submit and check the results

[0223] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[0224] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[0225] Output: The data is sent to the server. Later, you will receive a notification that the expense claim has been approved.

[0226] (Application example 1)

[0227] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0228] Conventional worker task management and workflows require a lot of manual work, resulting in reduced work efficiency. In particular, procedures such as vacation requests and expense reimbursements require time and effort, and progress management of approval flows is cumbersome. Furthermore, real-time task management is difficult for on-site work such as in factories, so a means of efficiently managing worker progress is needed.

[0229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0230] In this invention, the server includes means for providing an interface through which workers can input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying workers of the approval status of the vacation application, means for providing an interface for adding and managing tasks, means for automatically executing the approval process for added tasks, means for monitoring and notifying task progress in real time, and means for processing and displaying task information in cooperation with smart devices. This enables the automation and efficiency of vacation application and expense settlement, which were previously done manually, and also realizes task management and real-time monitoring of work progress on factory floors.

[0231] A "generative AI model" is an artificial intelligence model that generates new information and content based on data.

[0232] "Worker" is a broad term that refers to people who work in companies or on-site.

[0233] A "task" refers to the work or activity a worker performs to achieve a specific goal.

[0234] "Workflow" refers to the sequence of steps or processes required to complete a particular task.

[0235] "Automation" refers to replacing manual human work with machines or systems and carrying out all procedures automatically.

[0236] "Interface" refers to the screen or method by which a user operates a system.

[0237] An "approval flow" refers to the series of steps that a request or task must go through in order to be approved.

[0238] A "smart device" refers to an electronic device that has an internet connection and is capable of processing and communicating data.

[0239] "Progress" indicates the state of how far a task or project has progressed.

[0240] "Real-time" refers to processing information in near-immediate response to real-world events.

[0241] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, particularly for streamlining vacation request and expense reimbursement processes. Specific embodiments of the system are described below.

[0242] An embodiment of the leave request process

[0243] server:

[0244] The server first launches an interface that allows workers to enter information to apply for leave, and provides it to the user. When the worker enters the leave application information (e.g., leave start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. Based on the saved leave application information, the server automatically initiates the necessary approval process. Specifically, it sends an approval request for the leave application to the supervisor and the human resources department. The server periodically checks the approval progress and notifies the worker of the progress and final approval result.

[0245] Device:

[0246] The terminal displays an interface for workers to request time off, including a form for entering the start and end dates and reason for the leave. Once the worker submits the information, the terminal sends it to the server, which then begins processing.

[0247] User:

[0248] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the server automatically starts the approval flow and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "The vacation request has been approved."

[0249] Expense reimbursement process embodiment

[0250] server:

[0251] The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user. When the worker enters expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in a database. The server then creates an automatically generated expense application form based on the stored expense reimbursement information and initiates the necessary approval process. Specifically, it sends an expense reimbursement approval request to the supervisor and accounting department. The server receives approval or rejection responses from the supervisor and accounting department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval results.

[0252] Device:

[0253] The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions. Once the worker submits the information, the terminal sends it to the server, which then processes it.

[0254] User:

[0255] The worker opens the expense settlement screen and enters the necessary information. For example, they upload a receipt stating "Business trip expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses." Once the information is submitted, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final results. For example, the worker receives a notification that "Expense settlement has been approved."

[0256] Task management and progress monitoring process implementation

[0257] server:

[0258] The server launches an interface for workers to add and manage tasks, providing it to users. When a worker enters task information (e.g., task content, deadline) and presses the submit button, the server receives this information and stores it in a database. The server automatically executes the approval flow for added tasks using a generative AI model. It also has the function of monitoring task progress in real time and notifying workers.

[0259] Device:

[0260] The terminal displays an interface for workers to add tasks and includes a form for entering task information. When the worker submits the information, the terminal sends it to the server, which starts processing.

[0261] User:

[0262] The worker opens the task management screen and enters new task information. For example, they might enter "Assembly of part A, deadline December 1, 2023." Once the information is submitted, the server uses the generative AI model to automatically execute the task approval flow, monitoring and notifying progress in real time. Specifically, the worker receives notifications stating, "Task approved," and "Part A assembly task completed."

[0263] Hardware and software used

[0264] Hardware: Smart devices (e.g., smart glasses, smartphones)

[0265] Software: Python server program and generative AI model (e.g., OpenAI GPT)

[0266] Example prompts to input to a generative AI model:

[0267] "I will be taking leave from December 1st to December 5th, 2023. The reason is travel."

[0268] "I've added a task to assemble part A. Please automate the approval of this task and report on its progress."

[0269] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0270] Step 1:

[0271] User:

[0272] The user opens the vacation application screen or task management screen. For vacation applications, the user enters information such as the start date, end date, and reason for the vacation, and then presses the send button. For task management, the user enters the task details, deadline, and other information, and then presses the send button. The input data is the vacation application information or task information (e.g., "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel.").

[0273] Step 2:

[0274] Device:

[0275] The terminal receives the vacation request information or task information entered by the user and sends it to the server. The input data is then sent to the server as structured data in JSON format, etc. The terminal displays a confirmation message if the transmission is successful.

[0276] Step 3:

[0277] server:

[0278] The server receives vacation application information or task information sent from the terminal and stores this information in a database. The stored data is referenced in later processing.

[0279] Step 4:

[0280] server:

[0281] The server executes the necessary approval process based on the saved vacation request or task information. Specifically, it uses a generative AI model to send an approval request to a superior or the HR department. The input data is the request or task information, and the output data is the approval request.

[0282] Step 5:

[0283] server:

[0284] The server receives a response of approval or rejection from the supervisor or HR department. The generative AI model analyzes this response data to determine the approval status. Specifically, it records whether the request is approved or not in a database. The input data is the approval response, and the output data is the updated approval status.

[0285] Step 6:

[0286] server:

[0287] The server periodically checks the progress of the approval flow and notifies the worker of the progress and final approval result. Specifically, it uses a notification system to send notifications to workers via email or smart device. The input data is the approval status, and the output data is the notification message.

[0288] Step 7:

[0289] server:

[0290] The server monitors the task progress in real time and displays it on the smart device as needed. The generative AI model analyzes the task progress and generates an alert if a problem occurs. The input data is the task progress data, and the output data is updated progress information and an alert message.

[0291] Step 8:

[0292] User:

[0293] Workers receive notifications of progress and approval results from their smart devices and take necessary actions, such as confirming approved leave or reporting completed tasks. The input data is the notification message, and the output data is the user action.

[0294] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0295] The present invention combines an emotion engine with a system for automating and assisting worker tasks and workflows using generative AI models. The system is designed to efficiently manage leave request and expense reimbursement processes, and can recognize worker emotions to adjust feedback and notifications to optimize the priority and urgency of workflows.

[0296] An embodiment of the leave request process

[0297] server:

[0298] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[0299] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[0300] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[0301] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0302] 5. The server uses an emotion engine to recognize the emotion of the worker when applying and records the emotion data, which is used to adjust the feedback and notification content.

[0303] Device:

[0304] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[0305] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[0306] User:

[0307] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[0308] 2. Once the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server then automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotion.

[0309] Expense reimbursement process embodiment

[0310] server:

[0311] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[0312] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[0313] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[0314] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0315] 5. The server uses an emotion engine to recognize the worker's emotions and record the emotion data. The emotion data is used to adjust the feedback and notification content, and also to adjust the priority and urgency of expense reimbursement approval flows.

[0316] Device:

[0317] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0318] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[0319] User:

[0320] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[0321] 2. When the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "expense reimbursement has been approved," along with feedback based on their emotion.

[0322] The system of this invention can automate and streamline tasks such as vacation requests and expense reimbursements, which were previously done manually. Furthermore, by utilizing an emotion engine, it is possible to provide feedback and notifications based on the worker's emotions and optimize the priority and urgency of the work flow. This reduces the workload of workers and improves work efficiency.

[0323] The processing flow will be explained below.

[0324] Time Off Request Process Steps

[0325] Step 1: Provide an interface

[0326] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[0327] Step 2: Enter your vacation request information

[0328] User: Enter the leave start date, end date, and reason into the interface.

[0329] Step 3: Submit your vacation request information

[0330] Terminal: The vacation application information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[0331] Step 4: Receive and store leave request information and emotion data

[0332] Server: Receives the submitted vacation request information and emotion data and stores them in a database.

[0333] Step 5: Start the approval flow

[0334] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[0335] Step 6: Submitting an Approval Request

[0336] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0337] Step 7: Your manager's response

[0338] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0339] Step 8: Submit a secondary approval request

[0340] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[0341] Step 9: HR response

[0342] Server: Receives approval / rejection responses from the HR department and records them in a database.

[0343] Step 10: Notification of approval status

[0344] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[0345] Expense process steps

[0346] Step 1: Provide an interface

[0347] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0348] Step 2: Enter expense information and emotion data

[0349] User: Enters receipt photo, amount, date, and description into the interface.

[0350] Step 3: Submit expense information and emotion data

[0351] Terminal: The expense report information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[0352] Step 4: Receiving and storing expense information and emotion data

[0353] Server: Receives the expense reimbursement information and emotion data and stores them in a database.

[0354] Step 5: Auto-generate application form

[0355] Server: Automatically generate expense reports based on the saved expense information.

[0356] Step 6: Start the approval flow

[0357] Server: Sends approval request to manager and accounting department.

[0358] Step 7: Submitting an Approval Request

[0359] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0360] Step 8: Your manager's response

[0361] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0362] Step 9: Submit a secondary approval request

[0363] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[0364] Step 10: Response from the accounting department

[0365] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[0366] Step 11: Notification of approval status

[0367] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[0368] Example 2

[0369] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0370] In conventional worker task and workflow management, tasks such as vacation requests and expense reports are often done manually, resulting in delays and inaccurate processing. Furthermore, flexible feedback and notifications based on worker emotions are not provided, affecting worker motivation and satisfaction. Therefore, the objective of this invention is to provide a system that solves these problems and efficiently and effectively manages worker tasks and workflows.

[0371] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0372] In this invention, the server includes means for providing an interface that allows workers to input information for applying for leave, means for receiving the leave application information and recording it in data storage means, means for automatically executing the necessary approval process based on the recorded leave application information, emotion recognition means for recognizing the worker's emotions and recording the emotion data, and means for adjusting feedback and notification content based on the emotion data. This makes it possible to streamline procedures for leave applications and expense settlement, and to provide feedback and notifications that reflect the worker's emotions.

[0373] An "interface" is a means for providing a display screen or input form for a user to input information.

[0374] "Data storage means" refers to a storage system such as a database for recording and saving received information.

[0375] A "generative AI model" is a model that uses artificial intelligence technology to generate new data and results based on user input information and past data.

[0376] "Emotion recognition means" refers to a technical means for analyzing and recognizing the emotions expressed by the user when inputting information, such as by using facial recognition technology or natural language processing.

[0377] An "approval flow" is a process in which information about leave or expense claims requested by workers is automatically sent to superiors or relevant departments, and a response of approval or rejection is received.

[0378] "Feedback" refers to the system returning a response or evaluation to the user, and includes content based on emotional data.

[0379] "Notification" is the means by which the system notifies users of necessary information and status updates, including emails and alert messages.

[0380] The present invention is a system for automating and supporting worker tasks and workflows using generative AI models, which combines an information input interface for workers to apply for leave, an information input interface for expense reimbursement, and emotion recognition functions. Specific embodiments are described in detail below.

[0381] System configuration

[0382] server:

[0383] The server runs Apache (registered trademark) Tomcat and provides an interface to users using JSP files. JDBC is used to receive information on leave requests and expense reports and save it in a database (MySQL). Python scripts and an SMTP server are used to execute approval flows and send notifications. Google (registered trademark)'s Emotion API is used to analyze the emotions felt by workers when they enter input. Feedback and notification content are adjusted based on the emotion recognition results, improving worker satisfaction.

[0384] Device:

[0385] The device uses React.js to display the user interface, including forms for inputting information such as vacation requests and expense reports. When the worker's input information and images are sent to the backend REST API, Google's Emotion API is called to perform emotion recognition.

[0386] User:

[0387] The user (worker) enters information for vacation applications and expense settlement through the terminal. For example, for vacation applications, the user enters "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel," and submits the application. For expense settlement, the user uploads a receipt stating "Business trip expenses on November 20th, 2023, amounting to 5,000 yen, transportation expenses."

[0388] Examples and prompts

[0389] Examples:

[0390] 1. Example of leave request:

[0391] The worker enters "I will be taking leave from December 1st to December 5th, 2023. The reason is travel." and submits the application.

[0392] 2. Expense reimbursement example:

[0393] The worker uploads a receipt stating "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" and applies for expense reimbursement.

[0394] Example prompt sentence:

[0395] Vacation request prompt:

[0396] Leave request prompt: "The leave start date is December 1, 2023, the end date is December 5, and the reason is travel. Please submit a leave request and generate feedback based on the emotion recognition results of the emotion engine."

[0397] Expense Prompt Statement:

[0398] Expense reimbursement prompt: "As expense reimbursement information, I have uploaded a travel expense receipt for November 20, 2023, amounting to 5,000 yen. Please generate an expense claim form and generate feedback based on the emotion recognition results of the emotion engine."

[0399] This reduces the workload of workers and improves work efficiency. In addition, by using emotion recognition technology, it is possible to provide feedback and notifications according to the worker's emotions and optimize the priority and urgency of work flows.

[0400] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0401] Time Off Request Process Flow

[0402] Step 1:

[0403] The server launches and provides a vacation request interface to the user.

[0404] Input: None

[0405] Output: Provides a leave request interface

[0406] What happens: The server starts Apache Tomcat and executes the JSP file to display the vacation request form to the user.

[0407] Step 2:

[0408] The user opens the vacation application screen and enters the necessary information.

[0409] Input: Leave start date, end date, reason

[0410] Output: Entered vacation request information

[0411] Specific behavior: The user enters "I will take vacation from December 1st to December 5th, 2023. The reason is travel" through an interface displayed by React.js.

[0412] Step 3:

[0413] When the user submits application information, the terminal transmits the information to the emotion recognition means and transmits emotion data to the server.

[0414] Input: The inputted leave request information

[0415] Output: Emotion recognition results and application information

[0416] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[0417] Step 4:

[0418] The server receives the application information and stores it in a database.

[0419] Input: Emotion recognition results and application information

[0420] Output: Application information and emotion data stored in a database

[0421] Specific operation: The server uses JDBC to store the received application information and emotion data in a MySQL database.

[0422] Step 5:

[0423] The server automatically executes the authorization flow based on the stored information.

[0424] Input: Application information stored in the database

[0425] Output: Send approval request

[0426] What it does: The server runs a Python script that uses an SMTP server to send email notifications to the manager and HR.

[0427] Step 6:

[0428] The supervisor and HR department will respond with approval or denial.

[0429] Input: Approval request email

[0430] Output: Approval / Rejection response

[0431] What happens: Your manager and HR can click on a link in the email to approve or reject the request.

[0432] Step 7:

[0433] The server receives the approval result, records it in a database, and notifies the worker.

[0434] Input: Approval / Rejection response

[0435] Output: Notification of approval status

[0436] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[0437] Step 8:

[0438] The server adjusts the feedback content based on the emotion recognition results.

[0439] Input: Emotion data

[0440] Output: Regulated Feedback

[0441] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[0442] Expense reimbursement process flow

[0443] Step 1:

[0444] The server launches an expense reimbursement interface and provides it to the user.

[0445] Input: None

[0446] Output: Provides expense reimbursement interface

[0447] What happens: The server starts Apache Tomcat and executes the JSP file to display the expense form to the user.

[0448] Step 2:

[0449] The user opens the expense settlement screen and enters the necessary information.

[0450] Input: Receipt photo, amount, date, description

[0451] Output: Expense information entered

[0452] Specific operation: The user uploads a receipt for "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" through an interface displayed in React.js.

[0453] Step 3:

[0454] When the user submits the payment information, the terminal transmits the information to the emotion recognition means and transmits the emotion data to the server.

[0455] Input: Expense information entered

[0456] Output: Emotion recognition results and payment information

[0457] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[0458] Step 4:

[0459] The server receives the payment information and stores it in a database.

[0460] Input: Emotion recognition results and payment information

[0461] Output: Payment information and emotion data stored in a database

[0462] Specific operation: The server uses JDBC to store the received payment information and emotion data in a MySQL database.

[0463] Step 5:

[0464] The server automatically generates expense reports based on the stored information and executes the approval process.

[0465] Input: Payment information stored in the database

[0466] Output: Expense report and approval request submission

[0467] What it does: The server runs a Python script and uses an SMTP server to send email notifications to the manager and accounting department.

[0468] Step 6:

[0469] The supervisor and accounting department respond with approval or rejection.

[0470] Input: Approval request email

[0471] Output: Approval / Rejection response

[0472] Specific behavior: The manager and accounting department click on the link in the email to approve or reject the request.

[0473] Step 7:

[0474] The server receives the approval result, records it in a database, and notifies the worker.

[0475] Input: Approval / Rejection response

[0476] Output: Notification of approval status

[0477] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[0478] Step 8:

[0479] The server adjusts the feedback content based on the emotion recognition results.

[0480] Input: Emotion data

[0481] Output: Regulated Feedback

[0482] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[0483] (Application example 2)

[0484] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0485] In modern industrial production sites, the number of tasks that workers must perform is increasing, resulting in a need for more efficient task management and the alleviation of psychological stress among workers. Furthermore, the complex and time-consuming processes involved in administrative procedures such as applying for vacation and reimbursing expenses present a challenge. Furthermore, while it is known that workers' emotions have a significant impact on work efficiency and satisfaction, there is a lack of mechanisms for appropriately managing and optimizing these.

[0486] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface that allows workers to input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying the worker of the approval status of the vacation application, and means for recognizing the worker's emotions and adjusting the feedback and notification content. This makes it possible to provide feedback according to the worker's emotions and optimize the priority and urgency of the work flow.

[0487] A "generative AI model" is an artificial intelligence technology that uses machine learning algorithms and deep learning to automatically generate and analyze data.

[0488] A "worker" is an individual who belongs to a company or organization and performs designated tasks or duties.

[0489] A "task" is a set of operations or instructions that a worker performs to achieve a specific goal.

[0490] "Workflow" refers to a series of work procedures or processes designed to efficiently accomplish a task.

[0491] An "emotion engine" is software that can recognize workers' emotions in real time and adjust feedback and notifications accordingly.

[0492] "Leave application" means a formal request made by a worker to be granted leave for a specified period of time.

[0493] "Expense reimbursement" is the process by which workers report work-related expenses to their company and request reimbursement.

[0494] An "interface" refers to the point of contact or means by which a system and a user exchange information.

[0495] An "approval flow" refers to the steps and procedures required for a process such as a vacation request or expense claim to be officially approved.

[0496] "Feedback" refers to the responses or information that a system provides to workers in response to their actions and emotions.

[0497] "Notification" refers to the act of the system sending information or messages to workers.

[0498] "Priority" is a criterion for determining the importance and urgency of a task or work.

[0499] "Optimization" refers to the techniques and processes for achieving goals to the fullest extent possible within limited resources and conditions.

[0500] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, combined with an emotion engine, designed to efficiently manage leave request and expense reimbursement processes, and capable of recognizing worker emotions to adjust feedback and notifications, optimizing the priority and urgency of workflows.

[0501] An embodiment of the leave request process

[0502] server:

[0503] The server launches an interface that allows workers to enter information to request time off, and provides it to the user. When a worker enters the information for their leave request (e.g., start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. The server automatically initiates the necessary approval process based on the saved leave request information. Specifically, it sends an approval request for the leave request to the supervisor and the human resources department. The server receives the approval or rejection responses from the supervisor and the human resources department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval result. Furthermore, the server uses an emotion engine to recognize the emotion expressed by the worker when they request the leave and records the emotion data. The emotion data is used to adjust the feedback and notification content.

[0504] Device:

[0505] The terminal displays an interface for workers to apply for leave, which includes a form for entering the start date, end date, and reason for the leave. Once the worker submits the information, the emotion engine recognizes the worker's emotions and sends the data to the server.

[0506] User:

[0507] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the emotion engine recognizes the worker's emotions and sends them to the server. The server automatically starts the approval flow and notifies the worker of the approval progress and final results. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotions.

[0508] Expense reimbursement process embodiment

[0509] server:

[0510] The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user. When a worker enters expense reimbursement information (e.g., receipt photos, amount, date, and description) and presses the submit button, the server receives this information and stores it in a database. The server then creates an automatically generated expense claim form based on the saved expense reimbursement information and initiates the necessary approval process. Specifically, it sends an expense reimbursement approval request to the supervisor and accounting department. The server receives approval or rejection responses from the supervisor and accounting department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval result. Furthermore, the server uses an emotion engine to recognize the worker's emotions and record the emotion data. The emotion data is used to adjust feedback and notification content, and also helps adjust the priority and urgency of the expense reimbursement approval process.

[0511] Device:

[0512] The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions. Once the worker submits the information, an emotion engine recognizes the worker's emotions and sends the data to the server.

[0513] User:

[0514] The worker opens the expense settlement screen and enters the necessary information. For example, they upload a receipt stating "Business trip expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses." Once the information is submitted, the emotion engine recognizes the worker's emotions and sends them to the server. The server automatically generates an expense claim form and starts the approval flow. The server notifies the worker of the approval progress and final results. For example, the worker receives a notification that "Expense settlement has been approved," and simultaneously receives feedback based on their emotions.

[0515] Contextual examples

[0516] For example, in a system where a factory robot is installed, when a worker uses a smartphone to input the task "pack parts A and B," the system receives the information and recognizes the worker's emotions using an emotion engine. If the emotion "stress" is recognized as a result, the system provides the worker with feedback such as "Don't worry, we will get to work right away."

[0517] Example prompts for generative AI models

[0518] You can improve the accuracy of the emotion engine by inputting the following prompt sentences into the generative AI model:

[0519] Prompt statement:

[0520] Recognize worker emotions based on the following user input: "Task: Pack parts A and B. Predict emotion: "

[0521] Examples:

[0522] User input: "Pack parts A and B", Emotion prediction: "Stress"

[0523] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0524] Step 1:

[0525] The terminal displays an interface for inputting any task.

[0526] Specifically, after the user logs in, the device displays a task input screen, and the user inputs the task details, such as "pack parts A and B." Once the input is complete, the user presses the send button.

[0527] Input: User task details (e.g. pack parts A and B)

[0528] Output: The input data is sent to the server

[0529] Step 2:

[0530] The server receives the task details entered by the user and recognizes the worker's emotions using an emotion engine.

[0531] Specifically, the server sends the received task content to the generative AI model, and uses the emotion engine to recognize emotions such as "stress." Example prompt sentences for emotion recognition are input to the model.

[0532] Input: Received task details (e.g., packing parts A and B)

[0533] Output: Worker emotion data (e.g., stress)

[0534] Step 3:

[0535] The server generates feedback based on the emotion data and notifies the worker.

[0536] Specifically, the server dynamically generates feedback content based on the output of the emotion engine (e.g., "Don't worry, we'll start working on it right away.") and sends that feedback to the device.

[0537] Input: Worker emotion data (e.g., stress)

[0538] Output: Feedback message (e.g. "Don't worry, we'll get to work on it soon.")

[0539] Step 4:

[0540] The terminal displays a feedback message to the worker.

[0541] Specifically, the terminal displays the feedback message received from the server as a pop-up notification or an in-app notification, which the worker can check.

[0542] Input: Feedback message from the server (e.g. "Don't worry, we'll get to work on it soon.")

[0543] Output: The worker confirms the feedback message

[0544] Step 5:

[0545] The server stores the worker's tasks in a database and initiates the task approval flow as needed.

[0546] Specifically, the server stores task records containing task details and emotion data in a database, and then sends approval requests to relevant departments if an approval flow is required for the task.

[0547] Input: Worker task details and emotion data (e.g., packing parts A and B, stress)

[0548] Output: Task record saved in database, approval request submitted

[0549] Step 6:

[0550] The server periodically monitors the progress of the approval flow and notifies the worker of the progress and final result.

[0551] Specifically, the server periodically checks the status of the approval flow and notifies the worker of the progress. When a final result such as "approving," "rejected," or "approved" is obtained, the server notifies the worker.

[0552] Input: Approval flow information in the database

[0553] Output: Notification of progress and final result

[0554] Step 7:

[0555] The server performs analysis based on the final processing results and emotion data and utilizes this data to optimize the system.

[0556] Specifically, the server analyzes the accumulated task content, emotional data, and feedback content, and uses this data to further optimize future feedback and approval flows.

[0557] Input: Task content, emotion data, feedback content

[0558] Output: Optimized system feedback and approval flow

[0559] The above is the flow of processing of the program of the system that realizes the application example.

[0560] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0561] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0562] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0563] [Second embodiment]

[0564] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0565] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0566] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0567] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0568] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0569] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0570] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0571] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0572] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0573] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0574] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0575] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0576] The present invention relates to a system for automating and assisting worker tasks and workflows using generative AI models, designed to efficiently manage leave request and expense reimbursement processes.

[0577] An embodiment of the leave request process

[0578] server:

[0579] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[0580] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[0581] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[0582] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0583] Device:

[0584] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[0585] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[0586] User:

[0587] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[0588] 2. After submitting the information, the server automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved."

[0589] Expense reimbursement process embodiment

[0590] server:

[0591] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[0592] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[0593] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[0594] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0595] Device:

[0596] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0597] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[0598] User:

[0599] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[0600] 2. After submitting the information, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "the expense claim has been approved."

[0601] The system of this invention can automate and streamline leave applications and expense settlements, which were previously done manually, thereby reducing the workload on workers and improving work efficiency.

[0602] The processing flow will be explained below.

[0603] Time Off Request Process Steps

[0604] Step 1: Provide an interface

[0605] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[0606] Step 2: Enter your vacation request information

[0607] User: Enter the leave start date, end date, and reason into the interface.

[0608] Step 3: Submit your vacation request information

[0609] Terminal: Sends the vacation request information entered by the worker to the server.

[0610] Step 4: Receive and store leave request information

[0611] Server: Receives the submitted vacation request information and stores it in a database.

[0612] Step 5: Start the approval flow

[0613] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[0614] Step 6: Submitting an Approval Request

[0615] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0616] Step 7: Your manager's response

[0617] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0618] Step 8: Submit a secondary approval request

[0619] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[0620] Step 9: HR response

[0621] Server: Receives approval / rejection responses from the HR department and records them in a database.

[0622] Step 10: Notification of approval status

[0623] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[0624] Expense process steps

[0625] Step 1: Provide an interface

[0626] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0627] Step 2: Enter expense information

[0628] User: Enters receipt photo, amount, date, and description into the interface.

[0629] Step 3: Submit expense information

[0630] Terminal: Sends the expense reimbursement information entered by the worker to the server.

[0631] Step 4: Receive and store expense information

[0632] Server: Receives the expense report information and stores it in a database.

[0633] Step 5: Auto-generate application form

[0634] Server: Automatically generate expense reports based on the saved expense information.

[0635] Step 6: Start the approval flow

[0636] Server: Sends approval request to manager and accounting department.

[0637] Step 7: Submitting an Approval Request

[0638] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0639] Step 8: Your manager's response

[0640] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0641] Step 9: Submit a secondary approval request

[0642] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[0643] Step 10: Response from the accounting department

[0644] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[0645] Step 11: Notification of approval status

[0646] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[0647] Example 1

[0648] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0649] Traditional leave application and expense reimbursement processes involve a lot of manual work, placing a heavy burden on workers. This manual work can also lead to application and approval information getting lost and delays in processing. This can lead to issues such as reduced work efficiency and increased risk of human error. Therefore, there is a need to automate these processes, reduce the burden on workers, and achieve an efficient workflow.

[0650] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0651] In this invention, the server includes a means for providing an interface that allows workers to input information for applying for leave, a means for receiving leave application information and recording it in a database, a means for automatically generating an approval request using a generative AI model based on the recorded leave application information and starting the necessary approval flow, a means for receiving approval or rejection responses from superiors and the human resources department and recording them in a database, and a means for notifying workers of the approval progress and the final approval result. This allows for a smoother application process for leave applications and expense reimbursement.

[0652] This makes it possible to automate processes, reduce the burden on workers, and achieve efficient, error-free work flows.

[0653] A "generative AI model" is an artificial intelligence model that is generated using natural language processing and machine learning to help automate specific tasks or operations.

[0654] "Worker" refers to an individual who is employed to perform work for a business or organization.

[0655] A "task" is a specific item or task that a worker performs as part of their job.

[0656] "Workflow" refers to a series of steps or processes that must be followed to perform a specific task.

[0657] A "leave request" is the formal process by which a worker seeks permission to take time off work for a specific period of time.

[0658] "Expense reimbursement" is the formal process by which workers are reimbursed by their organization for work-related expenses they incur.

[0659] "Interface" refers to the input and output means by which workers interact with a system.

[0660] "Means for providing an interface through which information can be entered" refers to technology that provides a screen or form through which workers can enter the necessary data or information into the system.

[0661] A "database" is a system for organizing and storing information electronically, and can take the form of SQL or NoSQL.

[0662] "Approval flow" refers to the process or steps by which a particular application or request is formally approved.

[0663] "Progress" refers to the state or status of how far a particular task or project has progressed.

[0664] "Notification" refers to the means by which the system sends messages to workers and other parties to inform them of status and results.

[0665] The present invention relates to a system that utilizes generative AI models to automate and assist worker tasks and workflows. The system aims to efficiently manage leave request and expense reimbursement processes. Specific embodiments are described in detail below.

[0666] An embodiment of the leave request process

[0667] server:

[0668] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information to apply for leave. This interface includes a form for entering the start date, end date, and reason for the leave. The leave request information entered by the worker (e.g., "I will be taking leave from December 1st to December 5th, 2023. The reason is travel.") is sent to the server, converted into JSON format, and recorded in a MySQL or PostgreSQL database.

[0669] Next, a generative AI model (e.g., OpenAI's GPT-3) is used to automatically generate approval request messages and send them to the supervisor and HR department. This is done using the Slack API or an email server. Approval or rejection responses from the supervisor and HR department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final result. Notifications are sent via email or in-app notifications.

[0670] Device:

[0671] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the vacation application form. The information entered in the form is sent to the server via an HTTP POST request.

[0672] User:

[0673] The worker enters the necessary information into the interface displayed on the terminal. For example, "I will take vacation from December 1st to December 5th, 2023. The reason is travel." and presses the submit button. The worker then checks the approval progress and final result (e.g., "Vacation request approved") via notifications from the server.

[0674] Expense reimbursement process embodiment

[0675] server:

[0676] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information for expense reporting. This interface includes a form for uploading photos of receipts and entering amounts, dates, and descriptions. The expense reporting information entered by the worker (e.g., "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses") is sent to the server, converted to JSON format, and recorded in a MySQL or PostgreSQL database.

[0677] Next, the generative AI model is used to automatically generate expense reports and send approval requests to superiors and the accounting department. Slack APIs and email servers are used to send the reports. Approval or rejection responses from superiors and the accounting department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final results. Notifications are sent via email and in-app notifications.

[0678] Device:

[0679] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form. The information entered in the form is sent to the server via an HTTP POST request.

[0680] User:

[0681] The worker enters the necessary information (photo of the receipt, amount, date, description) into the interface displayed on the terminal. For example, they enter "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses" and presses the submit button. The worker then checks the approval progress and final result (e.g., "Expense reimbursement has been approved") via notifications from the server.

[0682] The system of the present invention automates the previously manual processes of vacation application and expense settlement, reducing the workload of workers and enabling an efficient, error-free workflow.

[0683] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0684] Specific processing steps for the leave request process

[0685] Server-side processing steps

[0686] Step 1: Launch the interface

[0687] The server runs a web interface for workers to request time off, using software such as Flask or Express.

[0688] Input: None

[0689] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[0690] Step 2: Receiving leave request information

[0691] The server receives the vacation request information entered by the worker, specifically as JSON format data sent as an HTTP POST request.

[0692] Input: Leave start date, end date, reason

[0693] Output: Record the received JSON format data in the database.

[0694] Step 3: Generate and submit an approval request

[0695] The server generates an approval request message using a generative AI model based on the recorded vacation request information.

[0696] Input: Leave request information stored in the database

[0697] Data computation: Message generation by generative AI models (e.g., GPT-3)

[0698] Output: A message is created to send an approval request to your manager and HR.

[0699] Step 4: Receive and record approval and rejection responses

[0700] The server receives the approval or rejection response from the manager and HR department via the Slack API or a mail server.

[0701] Input: Approval or rejection response

[0702] Output: Record the received response in a database.

[0703] Step 5: Communicate progress and final results

[0704] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[0705] Input: Progress and final results in database

[0706] Output: Notification message to the worker

[0707] Terminal processing steps

[0708] Step 1: Interface display

[0709] The device renders the HTML, CSS, and JavaScript code sent from the server and displays a form for requesting leave.

[0710] Input: HTML / CSS / JavaScript code received from the server

[0711] Output: A vacation request form is displayed that the user can fill out.

[0712] Step 2: Send data

[0713] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[0714] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[0715] Output: HTTP POST request to the server

[0716] User processing steps

[0717] Step 1: Enter your information

[0718] Workers enter the necessary information into the interface displayed on the terminal.

[0719] Input: Leave start date, end date, reason

[0720] Output: The input information is saved in the form displayed on the device.

[0721] Step 2: Submit and check the results

[0722] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[0723] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[0724] Output: The data is sent to the server. Later, you will receive a notification that "your vacation request has been approved."

[0725] Specific processing steps in the expense reimbursement process

[0726] Server-side processing steps

[0727] Step 1: Launch the interface

[0728] The server runs a web interface for workers to submit expense reports, using software such as Flask or Express.

[0729] Input: None

[0730] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[0731] Step 2: Receiving expense information

[0732] The server receives the expense report information entered by the worker, specifically as JSON formatted data sent as an HTTP POST request.

[0733] Input: Receipt photo, amount, date, description

[0734] Output: Record the received JSON format data in the database.

[0735] Step 3: Generate and submit expense reports

[0736] The server generates expense claims using a generative AI model based on the recorded expense reimbursement information.

[0737] Input: Expense information stored in the database

[0738] Data Computing: Generative AI models (e.g., GPT-3) generate expense reports

[0739] Output: An expense report is created to be sent to the manager and accounting department for approval.

[0740] Step 4: Receive and record approval and rejection responses

[0741] The server receives approval or rejection responses from the manager and accounting department via the Slack API or email server.

[0742] Input: Approval or rejection response

[0743] Output: Record the received response in a database.

[0744] Step 5: Communicate progress and final results

[0745] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[0746] Input: Progress and final results in database

[0747] Output: Notification message to the worker

[0748] Terminal processing steps

[0749] Step 1: Interface display

[0750] The device renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form.

[0751] Input: HTML / CSS / JavaScript code received from the server

[0752] Output: A display of the expense form that the user can fill out.

[0753] Step 2: Send data

[0754] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[0755] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[0756] Output: HTTP POST request to the server

[0757] User processing steps

[0758] Step 1: Enter your information

[0759] Workers enter the necessary information into the interface displayed on the terminal.

[0760] Input: Receipt photo, amount, date, description

[0761] Output: The input information is saved in the form displayed on the device.

[0762] Step 2: Submit and check the results

[0763] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[0764] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[0765] Output: The data is sent to the server. Later, you will receive a notification that the expense claim has been approved.

[0766] (Application example 1)

[0767] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0768] Conventional worker task management and workflows require a lot of manual work, resulting in reduced work efficiency. In particular, procedures such as vacation requests and expense reimbursements require time and effort, and progress management of approval flows is cumbersome. Furthermore, real-time task management is difficult for on-site work such as in factories, so a means of efficiently managing worker progress is needed.

[0769] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0770] In this invention, the server includes means for providing an interface through which workers can input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying workers of the approval status of the vacation application, means for providing an interface for adding and managing tasks, means for automatically executing the approval process for added tasks, means for monitoring and notifying task progress in real time, and means for processing and displaying task information in cooperation with smart devices. This enables the automation and efficiency of vacation application and expense settlement, which were previously done manually, and also realizes task management and real-time monitoring of work progress on factory floors.

[0771] A "generative AI model" is an artificial intelligence model that generates new information and content based on data.

[0772] "Worker" is a broad term that refers to people who work in companies or on-site.

[0773] A "task" refers to the work or activity a worker performs to achieve a specific goal.

[0774] "Workflow" refers to the sequence of steps or processes required to complete a particular task.

[0775] "Automation" refers to replacing manual human work with machines or systems and carrying out all procedures automatically.

[0776] "Interface" refers to the screen or method by which a user operates a system.

[0777] An "approval flow" refers to the series of steps that a request or task must go through in order to be approved.

[0778] A "smart device" refers to an electronic device that has an internet connection and is capable of processing and communicating data.

[0779] "Progress" indicates the state of how far a task or project has progressed.

[0780] "Real-time" refers to processing information in near-immediate response to real-world events.

[0781] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, particularly for streamlining vacation request and expense reimbursement processes. Specific embodiments of the system are described below.

[0782] An embodiment of the leave request process

[0783] server:

[0784] The server first launches an interface that allows workers to enter information to apply for leave, and provides it to the user. When the worker enters the leave application information (e.g., leave start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. Based on the saved leave application information, the server automatically initiates the necessary approval process. Specifically, it sends an approval request for the leave application to the supervisor and the human resources department. The server periodically checks the approval progress and notifies the worker of the progress and final approval result.

[0785] Device:

[0786] The terminal displays an interface for workers to request time off, including a form for entering the start and end dates and reason for the leave. Once the worker submits the information, the terminal sends it to the server, which then begins processing.

[0787] User:

[0788] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the server automatically starts the approval flow and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "The vacation request has been approved."

[0789] Expense reimbursement process embodiment

[0790] server:

[0791] The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user. When the worker enters expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in a database. The server then creates an automatically generated expense application form based on the stored expense reimbursement information and initiates the necessary approval process. Specifically, it sends an expense reimbursement approval request to the supervisor and accounting department. The server receives approval or rejection responses from the supervisor and accounting department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval results.

[0792] Device:

[0793] The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions. Once the worker submits the information, the terminal sends it to the server, which then processes it.

[0794] User:

[0795] The worker opens the expense settlement screen and enters the necessary information. For example, they upload a receipt stating "Business trip expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses." Once the information is submitted, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final results. For example, the worker receives a notification that "Expense settlement has been approved."

[0796] Task management and progress monitoring process implementation

[0797] server:

[0798] The server launches an interface for workers to add and manage tasks, providing it to users. When a worker enters task information (e.g., task content, deadline) and presses the submit button, the server receives this information and stores it in a database. The server automatically executes the approval flow for added tasks using a generative AI model. It also has the function of monitoring task progress in real time and notifying workers.

[0799] Device:

[0800] The terminal displays an interface for workers to add tasks and includes a form for entering task information. When the worker submits the information, the terminal sends it to the server, which starts processing.

[0801] User:

[0802] The worker opens the task management screen and enters new task information. For example, they might enter "Assembly of part A, deadline December 1, 2023." Once the information is submitted, the server uses the generative AI model to automatically execute the task approval flow, monitoring and notifying progress in real time. Specifically, the worker receives notifications stating, "Task approved," and "Part A assembly task completed."

[0803] Hardware and software used

[0804] Hardware: Smart devices (e.g., smart glasses, smartphones)

[0805] Software: Python server program and generative AI model (e.g., OpenAI GPT)

[0806] Example prompts to input to a generative AI model:

[0807] "I will be taking leave from December 1st to December 5th, 2023. The reason is travel."

[0808] "I've added a task to assemble part A. Please automate the approval of this task and report on its progress."

[0809] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0810] Step 1:

[0811] User:

[0812] The user opens the vacation application screen or task management screen. For vacation applications, the user enters information such as the start date, end date, and reason for the vacation, and then presses the send button. For task management, the user enters the task details, deadline, and other information, and then presses the send button. The input data is the vacation application information or task information (e.g., "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel.").

[0813] Step 2:

[0814] Device:

[0815] The terminal receives the vacation request information or task information entered by the user and sends it to the server. The input data is then sent to the server as structured data in JSON format, etc. The terminal displays a confirmation message if the transmission is successful.

[0816] Step 3:

[0817] server:

[0818] The server receives vacation application information or task information sent from the terminal and stores this information in a database. The stored data is referenced in later processing.

[0819] Step 4:

[0820] server:

[0821] The server executes the necessary approval process based on the saved vacation request or task information. Specifically, it uses a generative AI model to send an approval request to a superior or the HR department. The input data is the request or task information, and the output data is the approval request.

[0822] Step 5:

[0823] server:

[0824] The server receives a response of approval or rejection from the supervisor or HR department. The generative AI model analyzes this response data to determine the approval status. Specifically, it records whether the request is approved or not in a database. The input data is the approval response, and the output data is the updated approval status.

[0825] Step 6:

[0826] server:

[0827] The server periodically checks the progress of the approval flow and notifies the worker of the progress and final approval result. Specifically, it uses a notification system to send notifications to workers via email or smart device. The input data is the approval status, and the output data is the notification message.

[0828] Step 7:

[0829] server:

[0830] The server monitors the task progress in real time and displays it on the smart device as needed. The generative AI model analyzes the task progress and generates an alert if a problem occurs. The input data is the task progress data, and the output data is updated progress information and an alert message.

[0831] Step 8:

[0832] User:

[0833] Workers receive notifications of progress and approval results from their smart devices and take necessary actions, such as confirming approved leave or reporting completed tasks. The input data is the notification message, and the output data is the user action.

[0834] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0835] The present invention combines an emotion engine with a system for automating and assisting worker tasks and workflows using generative AI models. The system is designed to efficiently manage leave request and expense reimbursement processes, and can recognize worker emotions to adjust feedback and notifications to optimize the priority and urgency of workflows.

[0836] An embodiment of the leave request process

[0837] server:

[0838] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[0839] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[0840] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[0841] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0842] 5. The server uses an emotion engine to recognize the emotion of the worker when applying and records the emotion data, which is used to adjust the feedback and notification content.

[0843] Device:

[0844] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[0845] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[0846] User:

[0847] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[0848] 2. Once the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server then automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotion.

[0849] Expense reimbursement process embodiment

[0850] server:

[0851] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[0852] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[0853] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[0854] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[0855] 5. The server uses an emotion engine to recognize the worker's emotions and record the emotion data. The emotion data is used to adjust the feedback and notification content, and also to adjust the priority and urgency of expense reimbursement approval flows.

[0856] Device:

[0857] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0858] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[0859] User:

[0860] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[0861] 2. When the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "expense reimbursement has been approved," along with feedback based on their emotion.

[0862] The system of this invention can automate and streamline tasks such as vacation requests and expense reimbursements, which were previously done manually. Furthermore, by utilizing an emotion engine, it is possible to provide feedback and notifications based on the worker's emotions and optimize the priority and urgency of the work flow. This reduces the workload of workers and improves work efficiency.

[0863] The processing flow will be explained below.

[0864] Time Off Request Process Steps

[0865] Step 1: Provide an interface

[0866] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[0867] Step 2: Enter your vacation request information

[0868] User: Enter the leave start date, end date, and reason into the interface.

[0869] Step 3: Submit your vacation request information

[0870] Terminal: The vacation application information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[0871] Step 4: Receive and store leave request information and emotion data

[0872] Server: Receives the submitted vacation request information and emotion data and stores them in a database.

[0873] Step 5: Start the approval flow

[0874] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[0875] Step 6: Submitting an Approval Request

[0876] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0877] Step 7: Your manager's response

[0878] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0879] Step 8: Submit a secondary approval request

[0880] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[0881] Step 9: HR response

[0882] Server: Receives approval / rejection responses from the HR department and records them in a database.

[0883] Step 10: Notification of approval status

[0884] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[0885] Expense process steps

[0886] Step 1: Provide an interface

[0887] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[0888] Step 2: Enter expense information and emotion data

[0889] User: Enters receipt photo, amount, date, and description into the interface.

[0890] Step 3: Submit expense information and emotion data

[0891] Terminal: The expense report information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[0892] Step 4: Receiving and storing expense information and emotion data

[0893] Server: Receives the expense reimbursement information and emotion data and stores them in a database.

[0894] Step 5: Auto-generate application form

[0895] Server: Automatically generate expense reports based on the saved expense information.

[0896] Step 6: Start the approval flow

[0897] Server: Sends approval request to manager and accounting department.

[0898] Step 7: Submitting an Approval Request

[0899] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[0900] Step 8: Your manager's response

[0901] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[0902] Step 9: Submit a secondary approval request

[0903] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[0904] Step 10: Response from the accounting department

[0905] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[0906] Step 11: Notification of approval status

[0907] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[0908] Example 2

[0909] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0910] In conventional worker task and workflow management, tasks such as vacation requests and expense reports are often done manually, resulting in delays and inaccurate processing. Furthermore, flexible feedback and notifications based on worker emotions are not provided, affecting worker motivation and satisfaction. Therefore, the objective of this invention is to provide a system that solves these problems and efficiently and effectively manages worker tasks and workflows.

[0911] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0912] In this invention, the server includes means for providing an interface that allows workers to input information for applying for leave, means for receiving the leave application information and recording it in data storage means, means for automatically executing the necessary approval process based on the recorded leave application information, emotion recognition means for recognizing the worker's emotions and recording the emotion data, and means for adjusting feedback and notification content based on the emotion data. This makes it possible to streamline procedures for leave applications and expense settlement, and to provide feedback and notifications that reflect the worker's emotions.

[0913] An "interface" is a means for providing a display screen or input form for a user to input information.

[0914] "Data storage means" refers to a storage system such as a database for recording and saving received information.

[0915] A "generative AI model" is a model that uses artificial intelligence technology to generate new data and results based on user input information and past data.

[0916] "Emotion recognition means" refers to a technical means for analyzing and recognizing the emotions expressed by the user when inputting information, such as by using facial recognition technology or natural language processing.

[0917] An "approval flow" is a process in which information about leave or expense claims requested by workers is automatically sent to superiors or relevant departments, and a response of approval or rejection is received.

[0918] "Feedback" refers to the system returning a response or evaluation to the user, and includes content based on emotional data.

[0919] "Notification" is the means by which the system notifies users of necessary information and status updates, including emails and alert messages.

[0920] The present invention is a system for automating and supporting worker tasks and workflows using generative AI models, which combines an information input interface for workers to apply for leave, an information input interface for expense reimbursement, and emotion recognition functions. Specific embodiments are described in detail below.

[0921] System configuration

[0922] server:

[0923] The server runs Apache Tomcat and uses JSP files to provide an interface to users. JDBC is used to receive information on leave requests and expense reports and store it in a database (MySQL). Python scripts and an SMTP server are used to execute approval flows and send notifications. Google's emotion API is used to analyze the emotions workers express when they enter information. Feedback and notification content are adjusted based on the emotion recognition results to improve worker satisfaction.

[0924] Device:

[0925] The device uses React.js to display the user interface, including forms for inputting information such as vacation requests and expense reports. When the worker's input information and images are sent to the backend REST API, Google's Emotion API is called to perform emotion recognition.

[0926] User:

[0927] The user (worker) enters information for vacation applications and expense settlement through the terminal. For example, for vacation applications, the user enters "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel," and submits the application. For expense settlement, the user uploads a receipt stating "Business trip expenses on November 20th, 2023, amounting to 5,000 yen, transportation expenses."

[0928] Examples and prompts

[0929] Examples:

[0930] 1. Example of leave request:

[0931] The worker enters "I will be taking leave from December 1st to December 5th, 2023. The reason is travel." and submits the application.

[0932] 2. Expense reimbursement example:

[0933] The worker uploads a receipt stating "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" and applies for expense reimbursement.

[0934] Example prompt sentence:

[0935] Vacation request prompt:

[0936] Leave request prompt: "The leave start date is December 1, 2023, the end date is December 5, and the reason is travel. Please submit a leave request and generate feedback based on the emotion recognition results of the emotion engine."

[0937] Expense Prompt Statement:

[0938] Expense reimbursement prompt: "As expense reimbursement information, I have uploaded a travel expense receipt for November 20, 2023, amounting to 5,000 yen. Please generate an expense claim form and generate feedback based on the emotion recognition results of the emotion engine."

[0939] This reduces the workload of workers and improves work efficiency. In addition, by using emotion recognition technology, it is possible to provide feedback and notifications according to the worker's emotions and optimize the priority and urgency of work flows.

[0940] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0941] Time Off Request Process Flow

[0942] Step 1:

[0943] The server launches and provides a vacation request interface to the user.

[0944] Input: None

[0945] Output: Provides a leave request interface

[0946] What happens: The server starts Apache Tomcat and executes the JSP file to display the vacation request form to the user.

[0947] Step 2:

[0948] The user opens the vacation application screen and enters the necessary information.

[0949] Input: Leave start date, end date, reason

[0950] Output: Entered vacation request information

[0951] Specific behavior: The user enters "I will take vacation from December 1st to December 5th, 2023. The reason is travel" through an interface displayed by React.js.

[0952] Step 3:

[0953] When the user submits application information, the terminal transmits the information to the emotion recognition means and transmits emotion data to the server.

[0954] Input: The inputted leave request information

[0955] Output: Emotion recognition results and application information

[0956] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[0957] Step 4:

[0958] The server receives the application information and stores it in a database.

[0959] Input: Emotion recognition results and application information

[0960] Output: Application information and emotion data stored in a database

[0961] Specific operation: The server uses JDBC to store the received application information and emotion data in a MySQL database.

[0962] Step 5:

[0963] The server automatically executes the authorization flow based on the stored information.

[0964] Input: Application information stored in the database

[0965] Output: Send approval request

[0966] What it does: The server runs a Python script that uses an SMTP server to send email notifications to the manager and HR.

[0967] Step 6:

[0968] The supervisor and HR department will respond with approval or denial.

[0969] Input: Approval request email

[0970] Output: Approval / Rejection response

[0971] What happens: Your manager and HR can click on a link in the email to approve or reject the request.

[0972] Step 7:

[0973] The server receives the approval result, records it in a database, and notifies the worker.

[0974] Input: Approval / Rejection response

[0975] Output: Notification of approval status

[0976] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[0977] Step 8:

[0978] The server adjusts the feedback content based on the emotion recognition results.

[0979] Input: Emotion data

[0980] Output: Regulated Feedback

[0981] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[0982] Expense reimbursement process flow

[0983] Step 1:

[0984] The server launches an expense reimbursement interface and provides it to the user.

[0985] Input: None

[0986] Output: Provides expense reimbursement interface

[0987] What happens: The server starts Apache Tomcat and executes the JSP file to display the expense form to the user.

[0988] Step 2:

[0989] The user opens the expense settlement screen and enters the necessary information.

[0990] Input: Receipt photo, amount, date, description

[0991] Output: Expense information entered

[0992] Specific operation: The user uploads a receipt for "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" through an interface displayed in React.js.

[0993] Step 3:

[0994] When the user submits the payment information, the terminal transmits the information to the emotion recognition means and transmits the emotion data to the server.

[0995] Input: Expense information entered

[0996] Output: Emotion recognition results and payment information

[0997] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[0998] Step 4:

[0999] The server receives the payment information and stores it in a database.

[1000] Input: Emotion recognition results and payment information

[1001] Output: Payment information and emotion data stored in a database

[1002] Specific operation: The server uses JDBC to store the received payment information and emotion data in a MySQL database.

[1003] Step 5:

[1004] The server automatically generates expense reports based on the stored information and executes the approval process.

[1005] Input: Payment information stored in the database

[1006] Output: Expense report and approval request submission

[1007] What it does: The server runs a Python script and uses an SMTP server to send email notifications to the manager and accounting department.

[1008] Step 6:

[1009] The supervisor and accounting department respond with approval or rejection.

[1010] Input: Approval request email

[1011] Output: Approval / Rejection response

[1012] Specific behavior: The manager and accounting department click on the link in the email to approve or reject the request.

[1013] Step 7:

[1014] The server receives the approval result, records it in a database, and notifies the worker.

[1015] Input: Approval / Rejection response

[1016] Output: Notification of approval status

[1017] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[1018] Step 8:

[1019] The server adjusts the feedback content based on the emotion recognition results.

[1020] Input: Emotion data

[1021] Output: Regulated Feedback

[1022] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[1023] (Application example 2)

[1024] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1025] In modern industrial production sites, the number of tasks that workers must perform is increasing, resulting in a need for more efficient task management and the alleviation of psychological stress among workers. Furthermore, the complex and time-consuming processes involved in administrative procedures such as applying for vacation and reimbursing expenses present a challenge. Furthermore, while it is known that workers' emotions have a significant impact on work efficiency and satisfaction, there is a lack of mechanisms for appropriately managing and optimizing these.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface that allows workers to input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying the worker of the approval status of the vacation application, and means for recognizing the worker's emotions and adjusting the feedback and notification content. This makes it possible to provide feedback according to the worker's emotions and optimize the priority and urgency of the work flow.

[1027] A "generative AI model" is an artificial intelligence technology that uses machine learning algorithms and deep learning to automatically generate and analyze data.

[1028] A "worker" is an individual who belongs to a company or organization and performs designated tasks or duties.

[1029] A "task" is a set of operations or instructions that a worker performs to achieve a specific goal.

[1030] "Workflow" refers to a series of work procedures or processes designed to efficiently accomplish a task.

[1031] An "emotion engine" is software that can recognize workers' emotions in real time and adjust feedback and notifications accordingly.

[1032] "Leave application" means a formal request made by a worker to be granted leave for a specified period of time.

[1033] "Expense reimbursement" is the process by which workers report work-related expenses to their company and request reimbursement.

[1034] An "interface" refers to the point of contact or means by which a system and a user exchange information.

[1035] An "approval flow" refers to the steps and procedures required for a process such as a vacation request or expense claim to be officially approved.

[1036] "Feedback" refers to the responses or information that a system provides to workers in response to their actions and emotions.

[1037] "Notification" refers to the act of the system sending information or messages to workers.

[1038] "Priority" is a criterion for determining the importance and urgency of a task or work.

[1039] "Optimization" refers to the techniques and processes for achieving goals to the fullest extent possible within limited resources and conditions.

[1040] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, combined with an emotion engine, designed to efficiently manage leave request and expense reimbursement processes, and capable of recognizing worker emotions to adjust feedback and notifications, optimizing the priority and urgency of workflows.

[1041] An embodiment of the leave request process

[1042] server:

[1043] The server launches an interface that allows workers to enter information to request time off, and provides it to the user. When a worker enters the information for their leave request (e.g., start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. The server automatically initiates the necessary approval process based on the saved leave request information. Specifically, it sends an approval request for the leave request to the supervisor and the human resources department. The server receives the approval or rejection responses from the supervisor and the human resources department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval result. Furthermore, the server uses an emotion engine to recognize the emotion expressed by the worker when they request the leave and records the emotion data. The emotion data is used to adjust the feedback and notification content.

[1044] Device:

[1045] The terminal displays an interface for workers to apply for leave, which includes a form for entering the start date, end date, and reason for the leave. Once the worker submits the information, the emotion engine recognizes the worker's emotions and sends the data to the server.

[1046] User:

[1047] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the emotion engine recognizes the worker's emotions and sends them to the server. The server automatically starts the approval flow and notifies the worker of the approval progress and final results. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotions.

[1048] Expense reimbursement process embodiment

[1049] server:

[1050] The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user. When a worker enters expense reimbursement information (e.g., receipt photos, amount, date, and description) and presses the submit button, the server receives this information and stores it in a database. The server then creates an automatically generated expense claim form based on the saved expense reimbursement information and initiates the necessary approval process. Specifically, it sends an expense reimbursement approval request to the supervisor and accounting department. The server receives approval or rejection responses from the supervisor and accounting department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval result. Furthermore, the server uses an emotion engine to recognize the worker's emotions and record the emotion data. The emotion data is used to adjust feedback and notification content, and also helps adjust the priority and urgency of the expense reimbursement approval process.

[1051] Device:

[1052] The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions. Once the worker submits the information, an emotion engine recognizes the worker's emotions and sends the data to the server.

[1053] User:

[1054] The worker opens the expense settlement screen and enters the necessary information. For example, they upload a receipt stating "Business trip expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses." Once the information is submitted, the emotion engine recognizes the worker's emotions and sends them to the server. The server automatically generates an expense claim form and starts the approval flow. The server notifies the worker of the approval progress and final results. For example, the worker receives a notification that "Expense settlement has been approved," and simultaneously receives feedback based on their emotions.

[1055] Contextual examples

[1056] For example, in a system where a factory robot is installed, when a worker uses a smartphone to input the task "pack parts A and B," the system receives the information and recognizes the worker's emotions using an emotion engine. If the emotion "stress" is recognized as a result, the system provides the worker with feedback such as "Don't worry, we will get to work right away."

[1057] Example prompts for generative AI models

[1058] You can improve the accuracy of the emotion engine by inputting the following prompt sentences into the generative AI model:

[1059] Prompt statement:

[1060] Recognize worker emotions based on the following user input: "Task: Pack parts A and B. Predict emotion: "

[1061] Examples:

[1062] User input: "Pack parts A and B", Emotion prediction: "Stress"

[1063] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1064] Step 1:

[1065] The terminal displays an interface for inputting any task.

[1066] Specifically, after the user logs in, the device displays a task input screen, and the user inputs the task details, such as "pack parts A and B." Once the input is complete, the user presses the send button.

[1067] Input: User task details (e.g. pack parts A and B)

[1068] Output: The input data is sent to the server

[1069] Step 2:

[1070] The server receives the task details entered by the user and recognizes the worker's emotions using an emotion engine.

[1071] Specifically, the server sends the received task content to the generative AI model, and uses the emotion engine to recognize emotions such as "stress." Example prompt sentences for emotion recognition are input to the model.

[1072] Input: Received task details (e.g., packing parts A and B)

[1073] Output: Worker emotion data (e.g., stress)

[1074] Step 3:

[1075] The server generates feedback based on the emotion data and notifies the worker.

[1076] Specifically, the server dynamically generates feedback content based on the output of the emotion engine (e.g., "Don't worry, we'll start working on it right away.") and sends that feedback to the device.

[1077] Input: Worker emotion data (e.g., stress)

[1078] Output: Feedback message (e.g. "Don't worry, we'll get to work on it soon.")

[1079] Step 4:

[1080] The terminal displays a feedback message to the worker.

[1081] Specifically, the terminal displays the feedback message received from the server as a pop-up notification or an in-app notification, which the worker can check.

[1082] Input: Feedback message from the server (e.g. "Don't worry, we'll get to work on it soon.")

[1083] Output: The worker confirms the feedback message

[1084] Step 5:

[1085] The server stores the worker's tasks in a database and initiates the task approval flow as needed.

[1086] Specifically, the server stores task records containing task details and emotion data in a database, and then sends approval requests to relevant departments if an approval flow is required for the task.

[1087] Input: Worker task details and emotion data (e.g., packing parts A and B, stress)

[1088] Output: Task record saved in database, approval request submitted

[1089] Step 6:

[1090] The server periodically monitors the progress of the approval flow and notifies the worker of the progress and final result.

[1091] Specifically, the server periodically checks the status of the approval flow and notifies the worker of the progress. When a final result such as "approving," "rejected," or "approved" is obtained, the server notifies the worker.

[1092] Input: Approval flow information in the database

[1093] Output: Notification of progress and final result

[1094] Step 7:

[1095] The server performs analysis based on the final processing results and emotion data and utilizes this data to optimize the system.

[1096] Specifically, the server analyzes the accumulated task content, emotional data, and feedback content, and uses this data to further optimize future feedback and approval flows.

[1097] Input: Task content, emotion data, feedback content

[1098] Output: Optimized system feedback and approval flow

[1099] The above is the flow of processing of the program of the system that realizes the application example.

[1100] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1101] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1102] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1103] [Third embodiment]

[1104] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1105] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1106] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1107] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1108] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1109] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1110] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1111] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1112] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1113] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1114] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1115] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1116] The present invention relates to a system for automating and assisting worker tasks and workflows using generative AI models, designed to efficiently manage leave request and expense reimbursement processes.

[1117] An embodiment of the leave request process

[1118] server:

[1119] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[1120] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[1121] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[1122] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1123] Device:

[1124] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[1125] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[1126] User:

[1127] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[1128] 2. After submitting the information, the server automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved."

[1129] Expense reimbursement process embodiment

[1130] server:

[1131] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[1132] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[1133] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[1134] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1135] Device:

[1136] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1137] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[1138] User:

[1139] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[1140] 2. After submitting the information, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "the expense claim has been approved."

[1141] The system of this invention can automate and streamline leave applications and expense settlements, which were previously done manually, thereby reducing the workload on workers and improving work efficiency.

[1142] The processing flow will be explained below.

[1143] Time Off Request Process Steps

[1144] Step 1: Provide an interface

[1145] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[1146] Step 2: Enter your vacation request information

[1147] User: Enter the leave start date, end date, and reason into the interface.

[1148] Step 3: Submit your vacation request information

[1149] Terminal: Sends the vacation request information entered by the worker to the server.

[1150] Step 4: Receive and store leave request information

[1151] Server: Receives the submitted vacation request information and stores it in a database.

[1152] Step 5: Start the approval flow

[1153] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[1154] Step 6: Submitting an Approval Request

[1155] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1156] Step 7: Your manager's response

[1157] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1158] Step 8: Submit a secondary approval request

[1159] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[1160] Step 9: HR response

[1161] Server: Receives approval / rejection responses from the HR department and records them in a database.

[1162] Step 10: Notification of approval status

[1163] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[1164] Expense process steps

[1165] Step 1: Provide an interface

[1166] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1167] Step 2: Enter expense information

[1168] User: Enters receipt photo, amount, date, and description into the interface.

[1169] Step 3: Submit expense information

[1170] Terminal: Sends the expense reimbursement information entered by the worker to the server.

[1171] Step 4: Receive and store expense information

[1172] Server: Receives the expense report information and stores it in a database.

[1173] Step 5: Auto-generate application form

[1174] Server: Automatically generate expense reports based on the saved expense information.

[1175] Step 6: Start the approval flow

[1176] Server: Sends approval request to manager and accounting department.

[1177] Step 7: Submitting an Approval Request

[1178] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1179] Step 8: Your manager's response

[1180] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1181] Step 9: Submit a secondary approval request

[1182] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[1183] Step 10: Response from the accounting department

[1184] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[1185] Step 11: Notification of approval status

[1186] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[1187] Example 1

[1188] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1189] Traditional leave application and expense reimbursement processes involve a lot of manual work, placing a heavy burden on workers. This manual work can also lead to application and approval information getting lost and delays in processing. This can lead to issues such as reduced work efficiency and increased risk of human error. Therefore, there is a need to automate these processes, reduce the burden on workers, and achieve an efficient workflow.

[1190] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1191] In this invention, the server includes a means for providing an interface that allows workers to input information for applying for leave, a means for receiving leave application information and recording it in a database, a means for automatically generating an approval request using a generative AI model based on the recorded leave application information and starting the necessary approval flow, a means for receiving approval or rejection responses from superiors and the human resources department and recording them in a database, and a means for notifying workers of the approval progress and the final approval result. This allows for a smoother application process for leave applications and expense reimbursement.

[1192] This makes it possible to automate processes, reduce the burden on workers, and achieve efficient, error-free work flows.

[1193] A "generative AI model" is an artificial intelligence model that is generated using natural language processing and machine learning to help automate specific tasks or operations.

[1194] "Worker" refers to an individual who is employed to perform work for a business or organization.

[1195] A "task" is a specific item or task that a worker performs as part of their job.

[1196] "Workflow" refers to a series of steps or processes that must be followed to perform a specific task.

[1197] A "leave request" is the formal process by which a worker seeks permission to take time off work for a specific period of time.

[1198] "Expense reimbursement" is the formal process by which workers are reimbursed by their organization for work-related expenses they incur.

[1199] "Interface" refers to the input and output means by which workers interact with a system.

[1200] "Means for providing an interface through which information can be entered" refers to technology that provides a screen or form through which workers can enter the necessary data or information into the system.

[1201] A "database" is a system for organizing and storing information electronically, and can take the form of SQL or NoSQL.

[1202] "Approval flow" refers to the process or steps by which a particular application or request is formally approved.

[1203] "Progress" refers to the state or status of how far a particular task or project has progressed.

[1204] "Notification" refers to the means by which the system sends messages to workers and other parties to inform them of status and results.

[1205] The present invention relates to a system that utilizes generative AI models to automate and assist worker tasks and workflows. The system aims to efficiently manage leave request and expense reimbursement processes. Specific embodiments are described in detail below.

[1206] An embodiment of the leave request process

[1207] server:

[1208] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information to apply for leave. This interface includes a form for entering the start date, end date, and reason for the leave. The leave request information entered by the worker (e.g., "I will be taking leave from December 1st to December 5th, 2023. The reason is travel.") is sent to the server, converted into JSON format, and recorded in a MySQL or PostgreSQL database.

[1209] Next, a generative AI model (e.g., OpenAI's GPT-3) is used to automatically generate approval request messages and send them to the supervisor and HR department. This is done using the Slack API or an email server. Approval or rejection responses from the supervisor and HR department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final result. Notifications are sent via email or in-app notifications.

[1210] Device:

[1211] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the vacation application form. The information entered in the form is sent to the server via an HTTP POST request.

[1212] User:

[1213] The worker enters the necessary information into the interface displayed on the terminal. For example, "I will take vacation from December 1st to December 5th, 2023. The reason is travel." and presses the submit button. The worker then checks the approval progress and final result (e.g., "Vacation request approved") via notifications from the server.

[1214] Expense reimbursement process embodiment

[1215] server:

[1216] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information for expense reporting. This interface includes a form for uploading photos of receipts and entering amounts, dates, and descriptions. The expense reporting information entered by the worker (e.g., "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses") is sent to the server, converted to JSON format, and recorded in a MySQL or PostgreSQL database.

[1217] Next, the generative AI model is used to automatically generate expense reports and send approval requests to superiors and the accounting department. Slack APIs and email servers are used to send the reports. Approval or rejection responses from superiors and the accounting department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final results. Notifications are sent via email and in-app notifications.

[1218] Device:

[1219] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form. The information entered in the form is sent to the server via an HTTP POST request.

[1220] User:

[1221] The worker enters the necessary information (photo of the receipt, amount, date, description) into the interface displayed on the terminal. For example, they enter "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses" and presses the submit button. The worker then checks the approval progress and final result (e.g., "Expense reimbursement has been approved") via notifications from the server.

[1222] The system of the present invention automates the previously manual processes of vacation application and expense settlement, reducing the workload of workers and enabling an efficient, error-free workflow.

[1223] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1224] Specific processing steps for the leave request process

[1225] Server-side processing steps

[1226] Step 1: Launch the interface

[1227] The server runs a web interface for workers to request time off, using software such as Flask or Express.

[1228] Input: None

[1229] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[1230] Step 2: Receiving leave request information

[1231] The server receives the vacation request information entered by the worker, specifically as JSON format data sent as an HTTP POST request.

[1232] Input: Leave start date, end date, reason

[1233] Output: Record the received JSON format data in the database.

[1234] Step 3: Generate and submit an approval request

[1235] The server generates an approval request message using a generative AI model based on the recorded vacation request information.

[1236] Input: Leave request information stored in the database

[1237] Data computation: Message generation by generative AI models (e.g., GPT-3)

[1238] Output: A message is created to send an approval request to your manager and HR.

[1239] Step 4: Receive and record approval and rejection responses

[1240] The server receives the approval or rejection response from the manager and HR department via the Slack API or a mail server.

[1241] Input: Approval or rejection response

[1242] Output: Record the received response in a database.

[1243] Step 5: Communicate progress and final results

[1244] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[1245] Input: Progress and final results in database

[1246] Output: Notification message to the worker

[1247] Terminal processing steps

[1248] Step 1: Interface display

[1249] The device renders the HTML, CSS, and JavaScript code sent from the server and displays a form for requesting leave.

[1250] Input: HTML / CSS / JavaScript code received from the server

[1251] Output: A vacation request form is displayed that the user can fill out.

[1252] Step 2: Send data

[1253] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[1254] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[1255] Output: HTTP POST request to the server

[1256] User processing steps

[1257] Step 1: Enter your information

[1258] Workers enter the necessary information into the interface displayed on the terminal.

[1259] Input: Leave start date, end date, reason

[1260] Output: The input information is saved in the form displayed on the device.

[1261] Step 2: Submit and check the results

[1262] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[1263] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[1264] Output: The data is sent to the server. Later, you will receive a notification that "your vacation request has been approved."

[1265] Specific processing steps in the expense reimbursement process

[1266] Server-side processing steps

[1267] Step 1: Launch the interface

[1268] The server runs a web interface for workers to submit expense reports, using software such as Flask or Express.

[1269] Input: None

[1270] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[1271] Step 2: Receiving expense information

[1272] The server receives the expense report information entered by the worker, specifically as JSON formatted data sent as an HTTP POST request.

[1273] Input: Receipt photo, amount, date, description

[1274] Output: Record the received JSON format data in the database.

[1275] Step 3: Generate and submit expense reports

[1276] The server generates expense claims using a generative AI model based on the recorded expense reimbursement information.

[1277] Input: Expense information stored in the database

[1278] Data Computing: Generative AI models (e.g., GPT-3) generate expense reports

[1279] Output: An expense report is created to be sent to the manager and accounting department for approval.

[1280] Step 4: Receive and record approval and rejection responses

[1281] The server receives approval or rejection responses from the manager and accounting department via the Slack API or email server.

[1282] Input: Approval or rejection response

[1283] Output: Record the received response in a database.

[1284] Step 5: Communicate progress and final results

[1285] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[1286] Input: Progress and final results in database

[1287] Output: Notification message to the worker

[1288] Terminal processing steps

[1289] Step 1: Interface display

[1290] The device renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form.

[1291] Input: HTML / CSS / JavaScript code received from the server

[1292] Output: A display of the expense form that the user can fill out.

[1293] Step 2: Send data

[1294] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[1295] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[1296] Output: HTTP POST request to the server

[1297] User processing steps

[1298] Step 1: Enter your information

[1299] Workers enter the necessary information into the interface displayed on the terminal.

[1300] Input: Receipt photo, amount, date, description

[1301] Output: The input information is saved in the form displayed on the device.

[1302] Step 2: Submit and check the results

[1303] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[1304] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[1305] Output: The data is sent to the server. Later, you will receive a notification that the expense claim has been approved.

[1306] (Application example 1)

[1307] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1308] Conventional worker task management and workflows require a lot of manual work, resulting in reduced work efficiency. In particular, procedures such as vacation requests and expense reimbursements require time and effort, and progress management of approval flows is cumbersome. Furthermore, real-time task management is difficult for on-site work such as in factories, so a means of efficiently managing worker progress is needed.

[1309] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1310] In this invention, the server includes means for providing an interface through which workers can input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying workers of the approval status of the vacation application, means for providing an interface for adding and managing tasks, means for automatically executing the approval process for added tasks, means for monitoring and notifying task progress in real time, and means for processing and displaying task information in cooperation with smart devices. This enables the automation and efficiency of vacation application and expense settlement, which were previously done manually, and also realizes task management and real-time monitoring of work progress on factory floors.

[1311] A "generative AI model" is an artificial intelligence model that generates new information and content based on data.

[1312] "Worker" is a broad term that refers to people who work in companies or on-site.

[1313] A "task" refers to the work or activity a worker performs to achieve a specific goal.

[1314] "Workflow" refers to the sequence of steps or processes required to complete a particular task.

[1315] "Automation" refers to replacing manual human work with machines or systems and carrying out all procedures automatically.

[1316] "Interface" refers to the screen or method by which a user operates a system.

[1317] An "approval flow" refers to the series of steps that a request or task must go through in order to be approved.

[1318] A "smart device" refers to an electronic device that has an internet connection and is capable of processing and communicating data.

[1319] "Progress" indicates the state of how far a task or project has progressed.

[1320] "Real-time" refers to processing information in near-immediate response to real-world events.

[1321] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, particularly for streamlining vacation request and expense reimbursement processes. Specific embodiments of the system are described below.

[1322] An embodiment of the leave request process

[1323] server:

[1324] The server first launches an interface that allows workers to enter information to apply for leave, and provides it to the user. When the worker enters the leave application information (e.g., leave start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. Based on the saved leave application information, the server automatically initiates the necessary approval process. Specifically, it sends an approval request for the leave application to the supervisor and the human resources department. The server periodically checks the approval progress and notifies the worker of the progress and final approval result.

[1325] Device:

[1326] The terminal displays an interface for workers to request time off, including a form for entering the start and end dates and reason for the leave. Once the worker submits the information, the terminal sends it to the server, which then begins processing.

[1327] User:

[1328] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the server automatically starts the approval flow and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "The vacation request has been approved."

[1329] Expense reimbursement process embodiment

[1330] server:

[1331] The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user. When the worker enters expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in a database. The server then creates an automatically generated expense application form based on the stored expense reimbursement information and initiates the necessary approval process. Specifically, it sends an expense reimbursement approval request to the supervisor and accounting department. The server receives approval or rejection responses from the supervisor and accounting department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval results.

[1332] Device:

[1333] The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions. Once the worker submits the information, the terminal sends it to the server, which then processes it.

[1334] User:

[1335] The worker opens the expense settlement screen and enters the necessary information. For example, they upload a receipt stating "Business trip expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses." Once the information is submitted, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final results. For example, the worker receives a notification that "Expense settlement has been approved."

[1336] Task management and progress monitoring process implementation

[1337] server:

[1338] The server launches an interface for workers to add and manage tasks, providing it to users. When a worker enters task information (e.g., task content, deadline) and presses the submit button, the server receives this information and stores it in a database. The server automatically executes the approval flow for added tasks using a generative AI model. It also has the function of monitoring task progress in real time and notifying workers.

[1339] Device:

[1340] The terminal displays an interface for workers to add tasks and includes a form for entering task information. When the worker submits the information, the terminal sends it to the server, which starts processing.

[1341] User:

[1342] The worker opens the task management screen and enters new task information. For example, they might enter "Assembly of part A, deadline December 1, 2023." Once the information is submitted, the server uses the generative AI model to automatically execute the task approval flow, monitoring and notifying progress in real time. Specifically, the worker receives notifications stating, "Task approved," and "Part A assembly task completed."

[1343] Hardware and software used

[1344] Hardware: Smart devices (e.g., smart glasses, smartphones)

[1345] Software: Python server program and generative AI model (e.g., OpenAI GPT)

[1346] Example prompts to input to a generative AI model:

[1347] "I will be taking leave from December 1st to December 5th, 2023. The reason is travel."

[1348] "I've added a task to assemble part A. Please automate the approval of this task and report on its progress."

[1349] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1350] Step 1:

[1351] User:

[1352] The user opens the vacation application screen or task management screen. For vacation applications, the user enters information such as the start date, end date, and reason for the vacation, and then presses the send button. For task management, the user enters the task details, deadline, and other information, and then presses the send button. The input data is the vacation application information or task information (e.g., "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel.").

[1353] Step 2:

[1354] Device:

[1355] The terminal receives the vacation request information or task information entered by the user and sends it to the server. The input data is then sent to the server as structured data in JSON format, etc. The terminal displays a confirmation message if the transmission is successful.

[1356] Step 3:

[1357] server:

[1358] The server receives vacation application information or task information sent from the terminal and stores this information in a database. The stored data is referenced in later processing.

[1359] Step 4:

[1360] server:

[1361] The server executes the necessary approval process based on the saved vacation request or task information. Specifically, it uses a generative AI model to send an approval request to a superior or the HR department. The input data is the request or task information, and the output data is the approval request.

[1362] Step 5:

[1363] server:

[1364] The server receives a response of approval or rejection from the supervisor or HR department. The generative AI model analyzes this response data to determine the approval status. Specifically, it records whether the request is approved or not in a database. The input data is the approval response, and the output data is the updated approval status.

[1365] Step 6:

[1366] server:

[1367] The server periodically checks the progress of the approval flow and notifies the worker of the progress and final approval result. Specifically, it uses a notification system to send notifications to workers via email or smart device. The input data is the approval status, and the output data is the notification message.

[1368] Step 7:

[1369] server:

[1370] The server monitors the task progress in real time and displays it on the smart device as needed. The generative AI model analyzes the task progress and generates an alert if a problem occurs. The input data is the task progress data, and the output data is updated progress information and an alert message.

[1371] Step 8:

[1372] User:

[1373] Workers receive notifications of progress and approval results from their smart devices and take necessary actions, such as confirming approved leave or reporting completed tasks. The input data is the notification message, and the output data is the user action.

[1374] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1375] The present invention combines an emotion engine with a system for automating and assisting worker tasks and workflows using generative AI models. The system is designed to efficiently manage leave request and expense reimbursement processes, and can recognize worker emotions to adjust feedback and notifications to optimize the priority and urgency of workflows.

[1376] An embodiment of the leave request process

[1377] server:

[1378] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[1379] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[1380] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[1381] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1382] 5. The server uses an emotion engine to recognize the emotion of the worker when applying and records the emotion data, which is used to adjust the feedback and notification content.

[1383] Device:

[1384] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[1385] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[1386] User:

[1387] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[1388] 2. Once the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server then automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotion.

[1389] Expense reimbursement process embodiment

[1390] server:

[1391] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[1392] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[1393] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[1394] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1395] 5. The server uses an emotion engine to recognize the worker's emotions and record the emotion data. The emotion data is used to adjust the feedback and notification content, and also to adjust the priority and urgency of expense reimbursement approval flows.

[1396] Device:

[1397] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1398] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[1399] User:

[1400] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[1401] 2. When the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "expense reimbursement has been approved," along with feedback based on their emotion.

[1402] The system of this invention can automate and streamline tasks such as vacation requests and expense reimbursements, which were previously done manually. Furthermore, by utilizing an emotion engine, it is possible to provide feedback and notifications based on the worker's emotions and optimize the priority and urgency of the work flow. This reduces the workload of workers and improves work efficiency.

[1403] The processing flow will be explained below.

[1404] Time Off Request Process Steps

[1405] Step 1: Provide an interface

[1406] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[1407] Step 2: Enter your vacation request information

[1408] User: Enter the leave start date, end date, and reason into the interface.

[1409] Step 3: Submit your vacation request information

[1410] Terminal: The vacation application information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[1411] Step 4: Receive and store leave request information and emotion data

[1412] Server: Receives the submitted vacation request information and emotion data and stores them in a database.

[1413] Step 5: Start the approval flow

[1414] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[1415] Step 6: Submitting an Approval Request

[1416] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1417] Step 7: Your manager's response

[1418] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1419] Step 8: Submit a secondary approval request

[1420] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[1421] Step 9: HR response

[1422] Server: Receives approval / rejection responses from the HR department and records them in a database.

[1423] Step 10: Notification of approval status

[1424] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[1425] Expense process steps

[1426] Step 1: Provide an interface

[1427] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1428] Step 2: Enter expense information and emotion data

[1429] User: Enters receipt photo, amount, date, and description into the interface.

[1430] Step 3: Submit expense information and emotion data

[1431] Terminal: The expense report information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[1432] Step 4: Receiving and storing expense information and emotion data

[1433] Server: Receives the expense reimbursement information and emotion data and stores them in a database.

[1434] Step 5: Auto-generate application form

[1435] Server: Automatically generate expense reports based on the saved expense information.

[1436] Step 6: Start the approval flow

[1437] Server: Sends approval request to manager and accounting department.

[1438] Step 7: Submitting an Approval Request

[1439] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1440] Step 8: Your manager's response

[1441] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1442] Step 9: Submit a secondary approval request

[1443] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[1444] Step 10: Response from the accounting department

[1445] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[1446] Step 11: Notification of approval status

[1447] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[1448] Example 2

[1449] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1450] In conventional worker task and workflow management, tasks such as vacation requests and expense reports are often done manually, resulting in delays and inaccurate processing. Furthermore, flexible feedback and notifications based on worker emotions are not provided, affecting worker motivation and satisfaction. Therefore, the objective of this invention is to provide a system that solves these problems and efficiently and effectively manages worker tasks and workflows.

[1451] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1452] In this invention, the server includes means for providing an interface that allows workers to input information for applying for leave, means for receiving the leave application information and recording it in data storage means, means for automatically executing the necessary approval process based on the recorded leave application information, emotion recognition means for recognizing the worker's emotions and recording the emotion data, and means for adjusting feedback and notification content based on the emotion data. This makes it possible to streamline procedures for leave applications and expense settlement, and to provide feedback and notifications that reflect the worker's emotions.

[1453] An "interface" is a means for providing a display screen or input form for a user to input information.

[1454] "Data storage means" refers to a storage system such as a database for recording and saving received information.

[1455] A "generative AI model" is a model that uses artificial intelligence technology to generate new data and results based on user input information and past data.

[1456] "Emotion recognition means" refers to a technical means for analyzing and recognizing the emotions expressed by the user when inputting information, such as by using facial recognition technology or natural language processing.

[1457] An "approval flow" is a process in which information about leave or expense claims requested by workers is automatically sent to superiors or relevant departments, and a response of approval or rejection is received.

[1458] "Feedback" refers to the system returning a response or evaluation to the user, and includes content based on emotional data.

[1459] "Notification" is the means by which the system notifies users of necessary information and status updates, including emails and alert messages.

[1460] The present invention is a system for automating and supporting worker tasks and workflows using generative AI models, which combines an information input interface for workers to apply for leave, an information input interface for expense reimbursement, and emotion recognition functions. Specific embodiments are described in detail below.

[1461] System configuration

[1462] server:

[1463] The server runs Apache Tomcat and uses JSP files to provide an interface to users. JDBC is used to receive information on leave requests and expense reports and store it in a database (MySQL). Python scripts and an SMTP server are used to execute approval flows and send notifications. Google's emotion API is used to analyze the emotions workers express when they enter information. Feedback and notification content are adjusted based on the emotion recognition results to improve worker satisfaction.

[1464] Device:

[1465] The device uses React.js to display the user interface, including forms for inputting information such as vacation requests and expense reports. When the worker's input information and images are sent to the backend REST API, Google's Emotion API is called to perform emotion recognition.

[1466] User:

[1467] The user (worker) enters information for vacation applications and expense settlement through the terminal. For example, for vacation applications, the user enters "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel," and submits the application. For expense settlement, the user uploads a receipt stating "Business trip expenses on November 20th, 2023, amounting to 5,000 yen, transportation expenses."

[1468] Examples and prompts

[1469] Examples:

[1470] 1. Example of leave request:

[1471] The worker enters "I will be taking leave from December 1st to December 5th, 2023. The reason is travel." and submits the application.

[1472] 2. Expense reimbursement example:

[1473] The worker uploads a receipt stating "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" and applies for expense reimbursement.

[1474] Example prompt sentence:

[1475] Vacation request prompt:

[1476] Leave request prompt: "The leave start date is December 1, 2023, the end date is December 5, and the reason is travel. Please submit a leave request and generate feedback based on the emotion recognition results of the emotion engine."

[1477] Expense Prompt Statement:

[1478] Expense reimbursement prompt: "As expense reimbursement information, I have uploaded a travel expense receipt for November 20, 2023, amounting to 5,000 yen. Please generate an expense claim form and generate feedback based on the emotion recognition results of the emotion engine."

[1479] This reduces the workload of workers and improves work efficiency. In addition, by using emotion recognition technology, it is possible to provide feedback and notifications according to the worker's emotions and optimize the priority and urgency of work flows.

[1480] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1481] Time Off Request Process Flow

[1482] Step 1:

[1483] The server launches and provides a vacation request interface to the user.

[1484] Input: None

[1485] Output: Provides a leave request interface

[1486] What happens: The server starts Apache Tomcat and executes the JSP file to display the vacation request form to the user.

[1487] Step 2:

[1488] The user opens the vacation application screen and enters the necessary information.

[1489] Input: Leave start date, end date, reason

[1490] Output: Entered vacation request information

[1491] Specific behavior: The user enters "I will take vacation from December 1st to December 5th, 2023. The reason is travel" through an interface displayed by React.js.

[1492] Step 3:

[1493] When the user submits application information, the terminal transmits the information to the emotion recognition means and transmits emotion data to the server.

[1494] Input: The inputted leave request information

[1495] Output: Emotion recognition results and application information

[1496] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[1497] Step 4:

[1498] The server receives the application information and stores it in a database.

[1499] Input: Emotion recognition results and application information

[1500] Output: Application information and emotion data stored in a database

[1501] Specific operation: The server uses JDBC to store the received application information and emotion data in a MySQL database.

[1502] Step 5:

[1503] The server automatically executes the authorization flow based on the stored information.

[1504] Input: Application information stored in the database

[1505] Output: Send approval request

[1506] What it does: The server runs a Python script that uses an SMTP server to send email notifications to the manager and HR.

[1507] Step 6:

[1508] The supervisor and HR department will respond with approval or denial.

[1509] Input: Approval request email

[1510] Output: Approval / Rejection response

[1511] What happens: Your manager and HR can click on a link in the email to approve or reject the request.

[1512] Step 7:

[1513] The server receives the approval result, records it in a database, and notifies the worker.

[1514] Input: Approval / Rejection response

[1515] Output: Notification of approval status

[1516] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[1517] Step 8:

[1518] The server adjusts the feedback content based on the emotion recognition results.

[1519] Input: Emotion data

[1520] Output: Regulated Feedback

[1521] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[1522] Expense reimbursement process flow

[1523] Step 1:

[1524] The server launches an expense reimbursement interface and provides it to the user.

[1525] Input: None

[1526] Output: Provides expense reimbursement interface

[1527] What happens: The server starts Apache Tomcat and executes the JSP file to display the expense form to the user.

[1528] Step 2:

[1529] The user opens the expense settlement screen and enters the necessary information.

[1530] Input: Receipt photo, amount, date, description

[1531] Output: Expense information entered

[1532] Specific operation: The user uploads a receipt for "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" through an interface displayed in React.js.

[1533] Step 3:

[1534] When the user submits the payment information, the terminal transmits the information to the emotion recognition means and transmits the emotion data to the server.

[1535] Input: Expense information entered

[1536] Output: Emotion recognition results and payment information

[1537] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[1538] Step 4:

[1539] The server receives the payment information and stores it in a database.

[1540] Input: Emotion recognition results and payment information

[1541] Output: Payment information and emotion data stored in a database

[1542] Specific operation: The server uses JDBC to store the received payment information and emotion data in a MySQL database.

[1543] Step 5:

[1544] The server automatically generates expense reports based on the stored information and executes the approval process.

[1545] Input: Payment information stored in the database

[1546] Output: Expense report and approval request submission

[1547] What it does: The server runs a Python script and uses an SMTP server to send email notifications to the manager and accounting department.

[1548] Step 6:

[1549] The supervisor and accounting department respond with approval or rejection.

[1550] Input: Approval request email

[1551] Output: Approval / Rejection response

[1552] Specific behavior: The manager and accounting department click on the link in the email to approve or reject the request.

[1553] Step 7:

[1554] The server receives the approval result, records it in a database, and notifies the worker.

[1555] Input: Approval / Rejection response

[1556] Output: Notification of approval status

[1557] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[1558] Step 8:

[1559] The server adjusts the feedback content based on the emotion recognition results.

[1560] Input: Emotion data

[1561] Output: Regulated Feedback

[1562] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[1563] (Application example 2)

[1564] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1565] In modern industrial production sites, the number of tasks that workers must perform is increasing, resulting in a need for more efficient task management and the alleviation of psychological stress among workers. Furthermore, the complex and time-consuming processes involved in administrative procedures such as applying for vacation and reimbursing expenses present a challenge. Furthermore, while it is known that workers' emotions have a significant impact on work efficiency and satisfaction, there is a lack of mechanisms for appropriately managing and optimizing these.

[1566] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface that allows workers to input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying the worker of the approval status of the vacation application, and means for recognizing the worker's emotions and adjusting the feedback and notification content. This makes it possible to provide feedback according to the worker's emotions and optimize the priority and urgency of the work flow.

[1567] A "generative AI model" is an artificial intelligence technology that uses machine learning algorithms and deep learning to automatically generate and analyze data.

[1568] A "worker" is an individual who belongs to a company or organization and performs designated tasks or duties.

[1569] A "task" is a set of operations or instructions that a worker performs to achieve a specific goal.

[1570] "Workflow" refers to a series of work procedures or processes designed to efficiently accomplish a task.

[1571] An "emotion engine" is software that can recognize workers' emotions in real time and adjust feedback and notifications accordingly.

[1572] "Leave application" means a formal request made by a worker to be granted leave for a specified period of time.

[1573] "Expense reimbursement" is the process by which workers report work-related expenses to their company and request reimbursement.

[1574] An "interface" refers to the point of contact or means by which a system and a user exchange information.

[1575] An "approval flow" refers to the steps and procedures required for a process such as a vacation request or expense claim to be officially approved.

[1576] "Feedback" refers to the responses or information that a system provides to workers in response to their actions and emotions.

[1577] "Notification" refers to the act of the system sending information or messages to workers.

[1578] "Priority" is a criterion for determining the importance and urgency of a task or work.

[1579] "Optimization" refers to the techniques and processes for achieving goals to the fullest extent possible within limited resources and conditions.

[1580] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, combined with an emotion engine, designed to efficiently manage leave request and expense reimbursement processes, and capable of recognizing worker emotions to adjust feedback and notifications, optimizing the priority and urgency of workflows.

[1581] An embodiment of the leave request process

[1582] server:

[1583] The server launches an interface that allows workers to enter information to request time off, and provides it to the user. When a worker enters the information for their leave request (e.g., start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. The server automatically initiates the necessary approval process based on the saved leave request information. Specifically, it sends an approval request for the leave request to the supervisor and the human resources department. The server receives the approval or rejection responses from the supervisor and the human resources department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval result. Furthermore, the server uses an emotion engine to recognize the emotion expressed by the worker when they request the leave and records the emotion data. The emotion data is used to adjust the feedback and notification content.

[1584] Device:

[1585] The terminal displays an interface for workers to apply for leave, which includes a form for entering the start date, end date, and reason for the leave. Once the worker submits the information, the emotion engine recognizes the worker's emotions and sends the data to the server.

[1586] User:

[1587] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the emotion engine recognizes the worker's emotions and sends them to the server. The server automatically starts the approval flow and notifies the worker of the approval progress and final results. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotions.

[1588] Expense reimbursement process embodiment

[1589] server:

[1590] The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user. When a worker enters expense reimbursement information (e.g., receipt photos, amount, date, and description) and presses the submit button, the server receives this information and stores it in a database. The server then creates an automatically generated expense claim form based on the saved expense reimbursement information and initiates the necessary approval process. Specifically, it sends an expense reimbursement approval request to the supervisor and accounting department. The server receives approval or rejection responses from the supervisor and accounting department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval result. Furthermore, the server uses an emotion engine to recognize the worker's emotions and record the emotion data. The emotion data is used to adjust feedback and notification content, and also helps adjust the priority and urgency of the expense reimbursement approval process.

[1591] Device:

[1592] The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions. Once the worker submits the information, an emotion engine recognizes the worker's emotions and sends the data to the server.

[1593] User:

[1594] The worker opens the expense settlement screen and enters the necessary information. For example, they upload a receipt stating "Business trip expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses." Once the information is submitted, the emotion engine recognizes the worker's emotions and sends them to the server. The server automatically generates an expense claim form and starts the approval flow. The server notifies the worker of the approval progress and final results. For example, the worker receives a notification that "Expense settlement has been approved," and simultaneously receives feedback based on their emotions.

[1595] Contextual examples

[1596] For example, in a system where a factory robot is installed, when a worker uses a smartphone to input the task "pack parts A and B," the system receives the information and recognizes the worker's emotions using an emotion engine. If the emotion "stress" is recognized as a result, the system provides the worker with feedback such as "Don't worry, we will get to work right away."

[1597] Example prompts for generative AI models

[1598] You can improve the accuracy of the emotion engine by inputting the following prompt sentences into the generative AI model:

[1599] Prompt statement:

[1600] Recognize worker emotions based on the following user input: "Task: Pack parts A and B. Predict emotion: "

[1601] Examples:

[1602] User input: "Pack parts A and B", Emotion prediction: "Stress"

[1603] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1604] Step 1:

[1605] The terminal displays an interface for inputting any task.

[1606] Specifically, after the user logs in, the device displays a task input screen, and the user inputs the task details, such as "pack parts A and B." Once the input is complete, the user presses the send button.

[1607] Input: User task details (e.g. pack parts A and B)

[1608] Output: The input data is sent to the server

[1609] Step 2:

[1610] The server receives the task details entered by the user and recognizes the worker's emotions using an emotion engine.

[1611] Specifically, the server sends the received task content to the generative AI model, and uses the emotion engine to recognize emotions such as "stress." Example prompt sentences for emotion recognition are input to the model.

[1612] Input: Received task details (e.g., packing parts A and B)

[1613] Output: Worker emotion data (e.g., stress)

[1614] Step 3:

[1615] The server generates feedback based on the emotion data and notifies the worker.

[1616] Specifically, the server dynamically generates feedback content based on the output of the emotion engine (e.g., "Don't worry, we'll start working on it right away.") and sends that feedback to the device.

[1617] Input: Worker emotion data (e.g., stress)

[1618] Output: Feedback message (e.g. "Don't worry, we'll get to work on it soon.")

[1619] Step 4:

[1620] The terminal displays a feedback message to the worker.

[1621] Specifically, the terminal displays the feedback message received from the server as a pop-up notification or an in-app notification, which the worker can check.

[1622] Input: Feedback message from the server (e.g. "Don't worry, we'll get to work on it soon.")

[1623] Output: The worker confirms the feedback message

[1624] Step 5:

[1625] The server stores the worker's tasks in a database and initiates the task approval flow as needed.

[1626] Specifically, the server stores task records containing task details and emotion data in a database, and then sends approval requests to relevant departments if an approval flow is required for the task.

[1627] Input: Worker task details and emotion data (e.g., packing parts A and B, stress)

[1628] Output: Task record saved in database, approval request submitted

[1629] Step 6:

[1630] The server periodically monitors the progress of the approval flow and notifies the worker of the progress and final result.

[1631] Specifically, the server periodically checks the status of the approval flow and notifies the worker of the progress. When a final result such as "approving," "rejected," or "approved" is obtained, the server notifies the worker.

[1632] Input: Approval flow information in the database

[1633] Output: Notification of progress and final result

[1634] Step 7:

[1635] The server performs analysis based on the final processing results and emotion data and utilizes this data to optimize the system.

[1636] Specifically, the server analyzes the accumulated task content, emotional data, and feedback content, and uses this data to further optimize future feedback and approval flows.

[1637] Input: Task content, emotion data, feedback content

[1638] Output: Optimized system feedback and approval flow

[1639] The above is the flow of processing of the program of the system that realizes the application example.

[1640] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1641] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1642] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1643] [Fourth embodiment]

[1644] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1645] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1646] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1647] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1648] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1649] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1650] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1651] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1652] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1653] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1654] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1655] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1656] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1657] The present invention relates to a system for automating and assisting worker tasks and workflows using generative AI models, designed to efficiently manage leave request and expense reimbursement processes.

[1658] An embodiment of the leave request process

[1659] server:

[1660] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[1661] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[1662] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[1663] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1664] Device:

[1665] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[1666] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[1667] User:

[1668] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[1669] 2. After submitting the information, the server automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved."

[1670] Expense reimbursement process embodiment

[1671] server:

[1672] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[1673] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[1674] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[1675] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1676] Device:

[1677] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1678] 2. When the worker submits the information they entered, the terminal sends it to the server, which then begins processing it.

[1679] User:

[1680] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[1681] 2. After submitting the information, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "the expense claim has been approved."

[1682] The system of this invention can automate and streamline leave applications and expense settlements, which were previously done manually, thereby reducing the workload on workers and improving work efficiency.

[1683] The processing flow will be explained below.

[1684] Time Off Request Process Steps

[1685] Step 1: Provide an interface

[1686] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[1687] Step 2: Enter your vacation request information

[1688] User: Enter the leave start date, end date, and reason into the interface.

[1689] Step 3: Submit your vacation request information

[1690] Terminal: Sends the vacation request information entered by the worker to the server.

[1691] Step 4: Receive and store leave request information

[1692] Server: Receives the submitted vacation request information and stores it in a database.

[1693] Step 5: Start the approval flow

[1694] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[1695] Step 6: Submitting an Approval Request

[1696] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1697] Step 7: Your manager's response

[1698] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1699] Step 8: Submit a secondary approval request

[1700] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[1701] Step 9: HR response

[1702] Server: Receives approval / rejection responses from the HR department and records them in a database.

[1703] Step 10: Notification of approval status

[1704] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[1705] Expense process steps

[1706] Step 1: Provide an interface

[1707] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1708] Step 2: Enter expense information

[1709] User: Enters receipt photo, amount, date, and description into the interface.

[1710] Step 3: Submit expense information

[1711] Terminal: Sends the expense reimbursement information entered by the worker to the server.

[1712] Step 4: Receive and store expense information

[1713] Server: Receives the expense report information and stores it in a database.

[1714] Step 5: Auto-generate application form

[1715] Server: Automatically generate expense reports based on the saved expense information.

[1716] Step 6: Start the approval flow

[1717] Server: Sends approval request to manager and accounting department.

[1718] Step 7: Submitting an Approval Request

[1719] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1720] Step 8: Your manager's response

[1721] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1722] Step 9: Submit a secondary approval request

[1723] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[1724] Step 10: Response from the accounting department

[1725] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[1726] Step 11: Notification of approval status

[1727] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result.

[1728] Example 1

[1729] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1730] Traditional leave application and expense reimbursement processes involve a lot of manual work, placing a heavy burden on workers. This manual work can also lead to application and approval information getting lost and delays in processing. This can lead to issues such as reduced work efficiency and increased risk of human error. Therefore, there is a need to automate these processes, reduce the burden on workers, and achieve an efficient workflow.

[1731] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1732] In this invention, the server includes a means for providing an interface that allows workers to input information for applying for leave, a means for receiving leave application information and recording it in a database, a means for automatically generating an approval request using a generative AI model based on the recorded leave application information and starting the necessary approval flow, a means for receiving approval or rejection responses from superiors and the human resources department and recording them in a database, and a means for notifying workers of the approval progress and the final approval result. This allows for a smoother application process for leave applications and expense reimbursement.

[1733] This makes it possible to automate processes, reduce the burden on workers, and achieve efficient, error-free work flows.

[1734] A "generative AI model" is an artificial intelligence model that is generated using natural language processing and machine learning to help automate specific tasks or operations.

[1735] "Worker" refers to an individual who is employed to perform work for a business or organization.

[1736] A "task" is a specific item or task that a worker performs as part of their job.

[1737] "Workflow" refers to a series of steps or processes that must be followed to perform a specific task.

[1738] A "leave request" is the formal process by which a worker seeks permission to take time off work for a specific period of time.

[1739] "Expense reimbursement" is the formal process by which workers are reimbursed by their organization for work-related expenses they incur.

[1740] "Interface" refers to the input and output means by which workers interact with a system.

[1741] "Means for providing an interface through which information can be entered" refers to technology that provides a screen or form through which workers can enter the necessary data or information into the system.

[1742] A "database" is a system for organizing and storing information electronically, and can take the form of SQL or NoSQL.

[1743] "Approval flow" refers to the process or steps by which a particular application or request is formally approved.

[1744] "Progress" refers to the state or status of how far a particular task or project has progressed.

[1745] "Notification" refers to the means by which the system sends messages to workers and other parties to inform them of status and results.

[1746] The present invention relates to a system that utilizes generative AI models to automate and assist worker tasks and workflows. The system aims to efficiently manage leave request and expense reimbursement processes. Specific embodiments are described in detail below.

[1747] An embodiment of the leave request process

[1748] server:

[1749] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information to apply for leave. This interface includes a form for entering the start date, end date, and reason for the leave. The leave request information entered by the worker (e.g., "I will be taking leave from December 1st to December 5th, 2023. The reason is travel.") is sent to the server, converted into JSON format, and recorded in a MySQL or PostgreSQL database.

[1750] Next, a generative AI model (e.g., OpenAI's GPT-3) is used to automatically generate approval request messages and send them to the supervisor and HR department. This is done using the Slack API or an email server. Approval or rejection responses from the supervisor and HR department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final result. Notifications are sent via email or in-app notifications.

[1751] Device:

[1752] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the vacation application form. The information entered in the form is sent to the server via an HTTP POST request.

[1753] User:

[1754] The worker enters the necessary information into the interface displayed on the terminal. For example, "I will take vacation from December 1st to December 5th, 2023. The reason is travel." and presses the submit button. The worker then checks the approval progress and final result (e.g., "Vacation request approved") via notifications from the server.

[1755] Expense reimbursement process embodiment

[1756] server:

[1757] The server uses a web framework such as Flask or Express to launch a web interface where workers can enter information for expense reporting. This interface includes a form for uploading photos of receipts and entering amounts, dates, and descriptions. The expense reporting information entered by the worker (e.g., "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses") is sent to the server, converted to JSON format, and recorded in a MySQL or PostgreSQL database.

[1758] Next, the generative AI model is used to automatically generate expense reports and send approval requests to superiors and the accounting department. Slack APIs and email servers are used to send the reports. Approval or rejection responses from superiors and the accounting department are received in the same way and recorded in a database. The server periodically checks the progress and notifies the worker of the progress and final results. Notifications are sent via email and in-app notifications.

[1759] Device:

[1760] The device (PC or smartphone) renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form. The information entered in the form is sent to the server via an HTTP POST request.

[1761] User:

[1762] The worker enters the necessary information (photo of the receipt, amount, date, description) into the interface displayed on the terminal. For example, they enter "Upload receipt for travel expenses on November 20, 2023, amounting to 5,000 yen, as transportation expenses" and presses the submit button. The worker then checks the approval progress and final result (e.g., "Expense reimbursement has been approved") via notifications from the server.

[1763] The system of the present invention automates the previously manual processes of vacation application and expense settlement, reducing the workload of workers and enabling an efficient, error-free workflow.

[1764] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1765] Specific processing steps for the leave request process

[1766] Server-side processing steps

[1767] Step 1: Launch the interface

[1768] The server runs a web interface for workers to request time off, using software such as Flask or Express.

[1769] Input: None

[1770] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[1771] Step 2: Receiving leave request information

[1772] The server receives the vacation request information entered by the worker, specifically as JSON format data sent as an HTTP POST request.

[1773] Input: Leave start date, end date, reason

[1774] Output: Record the received JSON format data in the database.

[1775] Step 3: Generate and submit an approval request

[1776] The server generates an approval request message using a generative AI model based on the recorded vacation request information.

[1777] Input: Leave request information stored in the database

[1778] Data computation: Message generation by generative AI models (e.g., GPT-3)

[1779] Output: A message is created to send an approval request to your manager and HR.

[1780] Step 4: Receive and record approval and rejection responses

[1781] The server receives the approval or rejection response from the manager and HR department via the Slack API or a mail server.

[1782] Input: Approval or rejection response

[1783] Output: Record the received response in a database.

[1784] Step 5: Communicate progress and final results

[1785] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[1786] Input: Progress and final results in database

[1787] Output: Notification message to the worker

[1788] Terminal processing steps

[1789] Step 1: Interface display

[1790] The device renders the HTML, CSS, and JavaScript code sent from the server and displays a form for requesting leave.

[1791] Input: HTML / CSS / JavaScript code received from the server

[1792] Output: A vacation request form is displayed that the user can fill out.

[1793] Step 2: Send data

[1794] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[1795] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[1796] Output: HTTP POST request to the server

[1797] User processing steps

[1798] Step 1: Enter your information

[1799] Workers enter the necessary information into the interface displayed on the terminal.

[1800] Input: Leave start date, end date, reason

[1801] Output: The input information is saved in the form displayed on the device.

[1802] Step 2: Submit and check the results

[1803] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[1804] Input: Leave request information entered by the worker (e.g., leave start date, end date, reason)

[1805] Output: The data is sent to the server. Later, you will receive a notification that "your vacation request has been approved."

[1806] Specific processing steps in the expense reimbursement process

[1807] Server-side processing steps

[1808] Step 1: Launch the interface

[1809] The server runs a web interface for workers to submit expense reports, using software such as Flask or Express.

[1810] Input: None

[1811] Output: A web page composed of HTML / CSS / JavaScript is generated and served to the user.

[1812] Step 2: Receiving expense information

[1813] The server receives the expense report information entered by the worker, specifically as JSON formatted data sent as an HTTP POST request.

[1814] Input: Receipt photo, amount, date, description

[1815] Output: Record the received JSON format data in the database.

[1816] Step 3: Generate and submit expense reports

[1817] The server generates expense claims using a generative AI model based on the recorded expense reimbursement information.

[1818] Input: Expense information stored in the database

[1819] Data Computing: Generative AI models (e.g., GPT-3) generate expense reports

[1820] Output: An expense report is created to be sent to the manager and accounting department for approval.

[1821] Step 4: Receive and record approval and rejection responses

[1822] The server receives approval or rejection responses from the manager and accounting department via the Slack API or email server.

[1823] Input: Approval or rejection response

[1824] Output: Record the received response in a database.

[1825] Step 5: Communicate progress and final results

[1826] The server periodically checks the progress and notifies the worker of the progress and final results via email and in-app notifications.

[1827] Input: Progress and final results in database

[1828] Output: Notification message to the worker

[1829] Terminal processing steps

[1830] Step 1: Interface display

[1831] The device renders the HTML, CSS, and JavaScript code sent from the server and displays the expense reimbursement form.

[1832] Input: HTML / CSS / JavaScript code received from the server

[1833] Output: A display of the expense form that the user can fill out.

[1834] Step 2: Send data

[1835] The worker enters information into the form and presses the submit button, after which the terminal sends an HTTP POST request to the server.

[1836] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[1837] Output: HTTP POST request to the server

[1838] User processing steps

[1839] Step 1: Enter your information

[1840] Workers enter the necessary information into the interface displayed on the terminal.

[1841] Input: Receipt photo, amount, date, description

[1842] Output: The input information is saved in the form displayed on the device.

[1843] Step 2: Submit and check the results

[1844] The worker enters the information and presses the "Submit" button, which sends the data to the server and starts the approval process.

[1845] Input: Expense information entered by the worker (e.g., photo of receipt, amount, date, description)

[1846] Output: The data is sent to the server. Later, you will receive a notification that the expense claim has been approved.

[1847] (Application example 1)

[1848] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1849] Conventional worker task management and workflows require a lot of manual work, resulting in reduced work efficiency. In particular, procedures such as vacation requests and expense reimbursements require time and effort, and progress management of approval flows is cumbersome. Furthermore, real-time task management is difficult for on-site work such as in factories, so a means of efficiently managing worker progress is needed.

[1850] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1851] In this invention, the server includes means for providing an interface through which workers can input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying workers of the approval status of the vacation application, means for providing an interface for adding and managing tasks, means for automatically executing the approval process for added tasks, means for monitoring and notifying task progress in real time, and means for processing and displaying task information in cooperation with smart devices. This enables the automation and efficiency of vacation application and expense settlement, which were previously done manually, and also realizes task management and real-time monitoring of work progress on factory floors.

[1852] A "generative AI model" is an artificial intelligence model that generates new information and content based on data.

[1853] "Worker" is a broad term that refers to people who work in companies or on-site.

[1854] A "task" refers to the work or activity a worker performs to achieve a specific goal.

[1855] "Workflow" refers to the sequence of steps or processes required to complete a particular task.

[1856] "Automation" refers to replacing manual human work with machines or systems and carrying out all procedures automatically.

[1857] "Interface" refers to the screen or method by which a user operates a system.

[1858] An "approval flow" refers to the series of steps that a request or task must go through in order to be approved.

[1859] A "smart device" refers to an electronic device that has an internet connection and is capable of processing and communicating data.

[1860] "Progress" indicates the state of how far a task or project has progressed.

[1861] "Real-time" refers to processing information in near-immediate response to real-world events.

[1862] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, particularly for streamlining vacation request and expense reimbursement processes. Specific embodiments of the system are described below.

[1863] An embodiment of the leave request process

[1864] server:

[1865] The server first launches an interface that allows workers to enter information to apply for leave, and provides it to the user. When the worker enters the leave application information (e.g., leave start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. Based on the saved leave application information, the server automatically initiates the necessary approval process. Specifically, it sends an approval request for the leave application to the supervisor and the human resources department. The server periodically checks the approval progress and notifies the worker of the progress and final approval result.

[1866] Device:

[1867] The terminal displays an interface for workers to request time off, including a form for entering the start and end dates and reason for the leave. Once the worker submits the information, the terminal sends it to the server, which then begins processing.

[1868] User:

[1869] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the server automatically starts the approval flow and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "The vacation request has been approved."

[1870] Expense reimbursement process embodiment

[1871] server:

[1872] The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user. When the worker enters expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in a database. The server then creates an automatically generated expense application form based on the stored expense reimbursement information and initiates the necessary approval process. Specifically, it sends an expense reimbursement approval request to the supervisor and accounting department. The server receives approval or rejection responses from the supervisor and accounting department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval results.

[1873] Device:

[1874] The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions. Once the worker submits the information, the terminal sends it to the server, which then processes it.

[1875] User:

[1876] The worker opens the expense settlement screen and enters the necessary information. For example, they upload a receipt stating "Business trip expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses." Once the information is submitted, the server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final results. For example, the worker receives a notification that "Expense settlement has been approved."

[1877] Task management and progress monitoring process implementation

[1878] server:

[1879] The server launches an interface for workers to add and manage tasks, providing it to users. When a worker enters task information (e.g., task content, deadline) and presses the submit button, the server receives this information and stores it in a database. The server automatically executes the approval flow for added tasks using a generative AI model. It also has the function of monitoring task progress in real time and notifying workers.

[1880] Device:

[1881] The terminal displays an interface for workers to add tasks and includes a form for entering task information. When the worker submits the information, the terminal sends it to the server, which starts processing.

[1882] User:

[1883] The worker opens the task management screen and enters new task information. For example, they might enter "Assembly of part A, deadline December 1, 2023." Once the information is submitted, the server uses the generative AI model to automatically execute the task approval flow, monitoring and notifying progress in real time. Specifically, the worker receives notifications stating, "Task approved," and "Part A assembly task completed."

[1884] Hardware and software used

[1885] Hardware: Smart devices (e.g., smart glasses, smartphones)

[1886] Software: Python server program and generative AI model (e.g., OpenAI GPT)

[1887] Example prompts to input to a generative AI model:

[1888] "I will be taking leave from December 1st to December 5th, 2023. The reason is travel."

[1889] "I've added a task to assemble part A. Please automate the approval of this task and report on its progress."

[1890] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1891] Step 1:

[1892] User:

[1893] The user opens the vacation application screen or task management screen. For vacation applications, the user enters information such as the start date, end date, and reason for the vacation, and then presses the send button. For task management, the user enters the task details, deadline, and other information, and then presses the send button. The input data is the vacation application information or task information (e.g., "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel.").

[1894] Step 2:

[1895] Device:

[1896] The terminal receives the vacation request information or task information entered by the user and sends it to the server. The input data is then sent to the server as structured data in JSON format, etc. The terminal displays a confirmation message if the transmission is successful.

[1897] Step 3:

[1898] server:

[1899] The server receives vacation application information or task information sent from the terminal and stores this information in a database. The stored data is referenced in later processing.

[1900] Step 4:

[1901] server:

[1902] The server executes the necessary approval process based on the saved vacation request or task information. Specifically, it uses a generative AI model to send an approval request to a superior or the HR department. The input data is the request or task information, and the output data is the approval request.

[1903] Step 5:

[1904] server:

[1905] The server receives a response of approval or rejection from the supervisor or HR department. The generative AI model analyzes this response data to determine the approval status. Specifically, it records whether the request is approved or not in a database. The input data is the approval response, and the output data is the updated approval status.

[1906] Step 6:

[1907] server:

[1908] The server periodically checks the progress of the approval flow and notifies the worker of the progress and final approval result. Specifically, it uses a notification system to send notifications to workers via email or smart device. The input data is the approval status, and the output data is the notification message.

[1909] Step 7:

[1910] server:

[1911] The server monitors the task progress in real time and displays it on the smart device as needed. The generative AI model analyzes the task progress and generates an alert if a problem occurs. The input data is the task progress data, and the output data is updated progress information and an alert message.

[1912] Step 8:

[1913] User:

[1914] Workers receive notifications of progress and approval results from their smart devices and take necessary actions, such as confirming approved leave or reporting completed tasks. The input data is the notification message, and the output data is the user action.

[1915] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1916] The present invention combines an emotion engine with a system for automating and assisting worker tasks and workflows using generative AI models. The system is designed to efficiently manage leave request and expense reimbursement processes, and can recognize worker emotions to adjust feedback and notifications to optimize the priority and urgency of workflows.

[1917] An embodiment of the leave request process

[1918] server:

[1919] 1. The server launches and provides the user with an interface where workers can enter information to request time off.

[1920] 2. When the worker enters the vacation request information (e.g., vacation start date, end date, reason) and presses the submit button, the server receives this information and stores it in the database.

[1921] 3. The server automatically initiates the necessary approval process based on the saved vacation request information. Specifically, it sends an approval request for the vacation request to the supervisor and the HR department.

[1922] 4. The server receives approval or rejection responses from the supervisor and the HR department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1923] 5. The server uses an emotion engine to recognize the emotion of the worker when applying and records the emotion data, which is used to adjust the feedback and notification content.

[1924] Device:

[1925] 1. The terminal displays an interface for the worker to request leave, which includes a form for entering the start date, end date, and reason for the leave.

[1926] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[1927] User:

[1928] 1. The worker opens the vacation application screen and enters the necessary information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel."

[1929] 2. Once the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server then automatically starts the approval process and notifies the worker of the approval progress and final result. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotion.

[1930] Expense reimbursement process embodiment

[1931] server:

[1932] 1. The server launches an interface that allows workers to enter information for expense reimbursement and provides it to the user.

[1933] 2. When the worker enters the expense reimbursement information (e.g., photo of receipt, amount, date, description) and presses the submit button, the server receives this information and stores it in the database.

[1934] 3. The server automatically creates an expense claim form based on the saved expense information and initiates the necessary approval process, specifically sending an expense claim approval request to the superior and accounting department.

[1935] 4. The server receives approval or rejection responses from the supervisor and accounting department, records them in the database, periodically checks the approval progress, and notifies the worker of the progress and final approval result.

[1936] 5. The server uses an emotion engine to recognize the worker's emotions and record the emotion data. The emotion data is used to adjust the feedback and notification content, and also to adjust the priority and urgency of expense reimbursement approval flows.

[1937] Device:

[1938] 1. The terminal displays an interface for workers to submit expenses, including a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1939] 2. When the worker submits the input information, the emotion engine recognizes the worker's emotion and also sends that data to the server.

[1940] User:

[1941] 1. The worker opens the expense settlement screen and enters the necessary information. For example, upload a receipt as "Travel expenses for November 20, 2023, amounting to 5,000 yen, transportation expenses."

[1942] 2. When the information is submitted, the emotion engine recognizes the worker's emotion and sends it to the server. The server automatically generates an expense claim form and starts the approval process. The server notifies the worker of the approval progress and final result. For example, the worker receives a notification that "expense reimbursement has been approved," along with feedback based on their emotion.

[1943] The system of this invention can automate and streamline tasks such as vacation requests and expense reimbursements, which were previously done manually. Furthermore, by utilizing an emotion engine, it is possible to provide feedback and notifications based on the worker's emotions and optimize the priority and urgency of the work flow. This reduces the workload of workers and improves work efficiency.

[1944] The processing flow will be explained below.

[1945] Time Off Request Process Steps

[1946] Step 1: Provide an interface

[1947] Terminal: Displays an interface for workers to request time off, providing a form for entering the start date, end date, and reason for the leave.

[1948] Step 2: Enter your vacation request information

[1949] User: Enter the leave start date, end date, and reason into the interface.

[1950] Step 3: Submit your vacation request information

[1951] Terminal: The vacation application information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[1952] Step 4: Receive and store leave request information and emotion data

[1953] Server: Receives the submitted vacation request information and emotion data and stores them in a database.

[1954] Step 5: Start the approval flow

[1955] Server: Retrieves vacation request information from the database and sends approval requests to the manager and HR department.

[1956] Step 6: Submitting an Approval Request

[1957] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1958] Step 7: Your manager's response

[1959] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1960] Step 8: Submit a secondary approval request

[1961] Server: If approval is given from the supervisor, send an approval request to the HR department, wait for the HR department's response and record it in the database.

[1962] Step 9: HR response

[1963] Server: Receives approval / rejection responses from the HR department and records them in a database.

[1964] Step 10: Notification of approval status

[1965] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[1966] Expense process steps

[1967] Step 1: Provide an interface

[1968] Terminal: Displays an interface for workers to submit expenses, providing a form for uploading photos of receipts and entering amounts, dates, and descriptions.

[1969] Step 2: Enter expense information and emotion data

[1970] User: Enters receipt photo, amount, date, and description into the interface.

[1971] Step 3: Submit expense information and emotion data

[1972] Terminal: The expense report information entered by the worker is sent to the server. The emotion engine also recognizes the worker's emotions and sends the data to the server.

[1973] Step 4: Receiving and storing expense information and emotion data

[1974] Server: Receives the expense reimbursement information and emotion data and stores them in a database.

[1975] Step 5: Auto-generate application form

[1976] Server: Automatically generate expense reports based on the saved expense information.

[1977] Step 6: Start the approval flow

[1978] Server: Sends approval request to manager and accounting department.

[1979] Step 7: Submitting an Approval Request

[1980] Server: Sends approval request email to the manager, waits for the manager's response, and records it in the database.

[1981] Step 8: Your manager's response

[1982] Server: Receives approval or rejection responses from the supervisor and records them in a database.

[1983] Step 9: Submit a secondary approval request

[1984] Server: If approval is received from the supervisor, it sends an approval request to the accounting department, waits for the accounting department's response, and records it in the database.

[1985] Step 10: Response from the accounting department

[1986] Server: Receives approval / rejection responses from the accounting department and records them in the database.

[1987] Step 11: Notification of approval status

[1988] Server: Regularly checks the progress of the approval flow and notifies the worker of the progress and final approval result. Adjusts the notification content based on the worker's sentiment.

[1989] Example 2

[1990] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1991] In conventional worker task and workflow management, tasks such as vacation requests and expense reports are often done manually, resulting in delays and inaccurate processing. Furthermore, flexible feedback and notifications based on worker emotions are not provided, affecting worker motivation and satisfaction. Therefore, the objective of this invention is to provide a system that solves these problems and efficiently and effectively manages worker tasks and workflows.

[1992] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1993] In this invention, the server includes means for providing an interface that allows workers to input information for applying for leave, means for receiving the leave application information and recording it in data storage means, means for automatically executing the necessary approval process based on the recorded leave application information, emotion recognition means for recognizing the worker's emotions and recording the emotion data, and means for adjusting feedback and notification content based on the emotion data. This makes it possible to streamline procedures for leave applications and expense settlement, and to provide feedback and notifications that reflect the worker's emotions.

[1994] An "interface" is a means for providing a display screen or input form for a user to input information.

[1995] "Data storage means" refers to a storage system such as a database for recording and saving received information.

[1996] A "generative AI model" is a model that uses artificial intelligence technology to generate new data and results based on user input information and past data.

[1997] "Emotion recognition means" refers to a technical means for analyzing and recognizing the emotions expressed by the user when inputting information, such as by using facial recognition technology or natural language processing.

[1998] An "approval flow" is a process in which information about leave or expense claims requested by workers is automatically sent to superiors or relevant departments, and a response of approval or rejection is received.

[1999] "Feedback" refers to the system returning a response or evaluation to the user, and includes content based on emotional data.

[2000] "Notification" is the means by which the system notifies users of necessary information and status updates, including emails and alert messages.

[2001] The present invention is a system for automating and supporting worker tasks and workflows using generative AI models, which combines an information input interface for workers to apply for leave, an information input interface for expense reimbursement, and emotion recognition functions. Specific embodiments are described in detail below.

[2002] System configuration

[2003] server:

[2004] The server runs Apache Tomcat and uses JSP files to provide an interface to users. JDBC is used to receive information on leave requests and expense reports and store it in a database (MySQL). Python scripts and an SMTP server are used to execute approval flows and send notifications. Google's emotion API is used to analyze the emotions workers express when they enter information. Feedback and notification content are adjusted based on the emotion recognition results to improve worker satisfaction.

[2005] Device:

[2006] The device uses React.js to display the user interface, including forms for inputting information such as vacation requests and expense reports. When the worker's input information and images are sent to the backend REST API, Google's Emotion API is called to perform emotion recognition.

[2007] User:

[2008] The user (worker) enters information for vacation applications and expense settlement through the terminal. For example, for vacation applications, the user enters "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel," and submits the application. For expense settlement, the user uploads a receipt stating "Business trip expenses on November 20th, 2023, amounting to 5,000 yen, transportation expenses."

[2009] Examples and prompts

[2010] Examples:

[2011] 1. Example of leave request:

[2012] The worker enters "I will be taking leave from December 1st to December 5th, 2023. The reason is travel." and submits the application.

[2013] 2. Expense reimbursement example:

[2014] The worker uploads a receipt stating "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" and applies for expense reimbursement.

[2015] Example prompt sentence:

[2016] Vacation request prompt:

[2017] Leave request prompt: "The leave start date is December 1, 2023, the end date is December 5, and the reason is travel. Please submit a leave request and generate feedback based on the emotion recognition results of the emotion engine."

[2018] Expense Prompt Statement:

[2019] Expense reimbursement prompt: "As expense reimbursement information, I have uploaded a travel expense receipt for November 20, 2023, amounting to 5,000 yen. Please generate an expense claim form and generate feedback based on the emotion recognition results of the emotion engine."

[2020] This reduces the workload of workers and improves work efficiency. In addition, by using emotion recognition technology, it is possible to provide feedback and notifications according to the worker's emotions and optimize the priority and urgency of work flows.

[2021] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2022] Time Off Request Process Flow

[2023] Step 1:

[2024] The server launches and provides a vacation request interface to the user.

[2025] Input: None

[2026] Output: Provides a leave request interface

[2027] What happens: The server starts Apache Tomcat and executes the JSP file to display the vacation request form to the user.

[2028] Step 2:

[2029] The user opens the vacation application screen and enters the necessary information.

[2030] Input: Leave start date, end date, reason

[2031] Output: Entered vacation request information

[2032] Specific behavior: The user enters "I will take vacation from December 1st to December 5th, 2023. The reason is travel" through an interface displayed by React.js.

[2033] Step 3:

[2034] When the user submits application information, the terminal transmits the information to the emotion recognition means and transmits emotion data to the server.

[2035] Input: The inputted leave request information

[2036] Output: Emotion recognition results and application information

[2037] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[2038] Step 4:

[2039] The server receives the application information and stores it in a database.

[2040] Input: Emotion recognition results and application information

[2041] Output: Application information and emotion data stored in a database

[2042] Specific operation: The server uses JDBC to store the received application information and emotion data in a MySQL database.

[2043] Step 5:

[2044] The server automatically executes the authorization flow based on the stored information.

[2045] Input: Application information stored in the database

[2046] Output: Send approval request

[2047] What it does: The server runs a Python script that uses an SMTP server to send email notifications to the manager and HR.

[2048] Step 6:

[2049] The supervisor and HR department will respond with approval or denial.

[2050] Input: Approval request email

[2051] Output: Approval / Rejection response

[2052] What happens: Your manager and HR can click on a link in the email to approve or reject the request.

[2053] Step 7:

[2054] The server receives the approval result, records it in a database, and notifies the worker.

[2055] Input: Approval / Rejection response

[2056] Output: Notification of approval status

[2057] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[2058] Step 8:

[2059] The server adjusts the feedback content based on the emotion recognition results.

[2060] Input: Emotion data

[2061] Output: Regulated Feedback

[2062] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[2063] Expense reimbursement process flow

[2064] Step 1:

[2065] The server launches an expense reimbursement interface and provides it to the user.

[2066] Input: None

[2067] Output: Provides expense reimbursement interface

[2068] What happens: The server starts Apache Tomcat and executes the JSP file to display the expense form to the user.

[2069] Step 2:

[2070] The user opens the expense settlement screen and enters the necessary information.

[2071] Input: Receipt photo, amount, date, description

[2072] Output: Expense information entered

[2073] Specific operation: The user uploads a receipt for "Travel expenses on November 20, 2023, amounting to 5,000 yen, transportation expenses" through an interface displayed in React.js.

[2074] Step 3:

[2075] When the user submits the payment information, the terminal transmits the information to the emotion recognition means and transmits the emotion data to the server.

[2076] Input: Expense information entered

[2077] Output: Emotion recognition results and payment information

[2078] Specific operation: The device sends information to the Google Emotion API, receives the emotion result, and then sends the information to the server through the REST API.

[2079] Step 4:

[2080] The server receives the payment information and stores it in a database.

[2081] Input: Emotion recognition results and payment information

[2082] Output: Payment information and emotion data stored in a database

[2083] Specific operation: The server uses JDBC to store the received payment information and emotion data in a MySQL database.

[2084] Step 5:

[2085] The server automatically generates expense reports based on the stored information and executes the approval process.

[2086] Input: Payment information stored in the database

[2087] Output: Expense report and approval request submission

[2088] What it does: The server runs a Python script and uses an SMTP server to send email notifications to the manager and accounting department.

[2089] Step 6:

[2090] The supervisor and accounting department respond with approval or rejection.

[2091] Input: Approval request email

[2092] Output: Approval / Rejection response

[2093] Specific behavior: The manager and accounting department click on the link in the email to approve or reject the request.

[2094] Step 7:

[2095] The server receives the approval result, records it in a database, and notifies the worker.

[2096] Input: Approval / Rejection response

[2097] Output: Notification of approval status

[2098] Specific operation: The server uses JDBC to record the approval result in MySQL and sends a notification email to the worker through the Nginx server.

[2099] Step 8:

[2100] The server adjusts the feedback content based on the emotion recognition results.

[2101] Input: Emotion data

[2102] Output: Regulated Feedback

[2103] What it does: The server generates customized feedback based on the emotion data and sends it to the worker.

[2104] (Application example 2)

[2105] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2106] In modern industrial production sites, the number of tasks that workers must perform is increasing, resulting in a need for more efficient task management and the alleviation of psychological stress among workers. Furthermore, the complex and time-consuming processes involved in administrative procedures such as applying for vacation and reimbursing expenses present a challenge. Furthermore, while it is known that workers' emotions have a significant impact on work efficiency and satisfaction, there is a lack of mechanisms for appropriately managing and optimizing these.

[2107] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface that allows workers to input information for applying for vacation, means for receiving and recording the vacation application information, means for automatically executing the necessary approval process based on the recorded vacation application information, means for notifying the worker of the approval status of the vacation application, and means for recognizing the worker's emotions and adjusting the feedback and notification content. This makes it possible to provide feedback according to the worker's emotions and optimize the priority and urgency of the work flow.

[2108] A "generative AI model" is an artificial intelligence technology that uses machine learning algorithms and deep learning to automatically generate and analyze data.

[2109] A "worker" is an individual who belongs to a company or organization and performs designated tasks or duties.

[2110] A "task" is a set of operations or instructions that a worker performs to achieve a specific goal.

[2111] "Workflow" refers to a series of work procedures or processes designed to efficiently accomplish a task.

[2112] An "emotion engine" is software that can recognize workers' emotions in real time and adjust feedback and notifications accordingly.

[2113] "Leave application" means a formal request made by a worker to be granted leave for a specified period of time.

[2114] "Expense reimbursement" is the process by which workers report work-related expenses to their company and request reimbursement.

[2115] An "interface" refers to the point of contact or means by which a system and a user exchange information.

[2116] An "approval flow" refers to the steps and procedures required for a process such as a vacation request or expense claim to be officially approved.

[2117] "Feedback" refers to the responses or information that a system provides to workers in response to their actions and emotions.

[2118] "Notification" refers to the act of the system sending information or messages to workers.

[2119] "Priority" is a criterion for determining the importance and urgency of a task or work.

[2120] "Optimization" refers to the techniques and processes for achieving goals to the fullest extent possible within limited resources and conditions.

[2121] The present invention is a system for automating and assisting worker tasks and workflows using generative AI models, combined with an emotion engine, designed to efficiently manage leave request and expense reimbursement processes, and capable of recognizing worker emotions to adjust feedback and notifications, optimizing the priority and urgency of workflows.

[2122] An embodiment of the leave request process

[2123] server:

[2124] The server launches an interface that allows workers to enter information to request time off, and provides it to the user. When a worker enters the information for their leave request (e.g., start date, end date, reason) and presses the submit button, the server receives this information and stores it in a database. The server automatically initiates the necessary approval process based on the saved leave request information. Specifically, it sends an approval request for the leave request to the supervisor and the human resources department. The server receives the approval or rejection responses from the supervisor and the human resources department and records them in the database. It periodically checks the approval progress and notifies the worker of the progress and final approval result. Furthermore, the server uses an emotion engine to recognize the emotion expressed by the worker when they request the leave and records the emotion data. The emotion data is used to adjust the feedback and notification content.

[2125] Device:

[2126] The terminal displays an interface for workers to apply for leave, which includes a form for entering the start date, end date, and reason for the leave. Once the worker submits the information, the emotion engine recognizes the worker's emotions and sends the data to the server.

[2127] User:

[2128] The worker opens the vacation request screen and enters the required information. For example, they might enter, "I will be taking vacation from December 1st to December 5th, 2023. The reason is travel." Once the information is submitted, the emotion engine recognizes the worker's emotions and sends them to the server. The server automatically starts the approval flow and notifies the worker of the approval progress and final results. Specifically, the worker receives a notification that "the vacation request has been approved," along with feedback based on their emotions.

[2129] Expense reimbursement proce...

Claims

1. 1. A system for automating and assisting worker tasks and workflows utilizing generative AI models, comprising: a means for providing an interface through which a worker can input information for requesting leave; means for receiving and recording leave request information; A means for automatically executing the necessary approval flow based on the recorded leave request information; a means of informing workers of the approval status of their leave requests; A system including:

2. A means for providing an interface that enables workers to input information for expense reimbursement; means for receiving and recording expense reimbursement information; A means for automatically generating expense claims based on recorded expense reimbursement information and executing the necessary approval process; A means of notifying workers of the approval status of expense reimbursements; The system of claim 1 further comprising:

3. A means to periodically check the progress of the approval flow and notify the progress status, A means of notifying workers of the final approval result; The system of claim 1 or 2, further comprising:

Images