system

Abstract

We provide the system. [Solution] A natural language processing tool that automatically analyzes work instructions and breaks them down into specific tasks, Assignment means for appropriately distributing the aforementioned tasks to human users and artificial intelligence agents, A progress management system that monitors task progress in real time, adjusts priorities, and reallocates resources, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a workplace environment where employees and artificial intelligence agents coexist, business instructions are ambiguous and difficult to understand, and task allocation tends to be unbalanced. As a result, the load may be biased towards specific employees, leading to a problem that the overall productivity decreases. In order to solve such problems, it is necessary to efficiently and autonomously decompose business instructions into specific tasks and allocate them to optimal resources. In addition, it is necessary to monitor the progress of tasks in real time and dynamically adjust priorities and resource allocations to improve the efficiency of the entire business.

Means for Solving the Problems

[0005] This invention provides a system that analyzes high-level instructions from managers using natural language processing technology and automatically breaks them down into specific tasks. This system includes an assignment mechanism for distributing the decomposed tasks to appropriate human users and artificial intelligence agents. Furthermore, the system monitors task progress in real time and dynamically adjusts priorities and resource allocation as needed, enabling efficient task execution. In addition, the system allows the artificial intelligence agent to accumulate completed tasks as process automations and utilize them for similar tasks in the future, thereby continuously improving operational efficiency. It also provides an intuitive interface for users to easily input work instructions and includes features to optimize the entire user experience, from receiving instructions to receiving progress notifications.

[0006] "Work instructions" are instructions given within a company or organization that outline high-level goals and objectives, and serve as the starting point for concretizing work processes.

[0007] "Natural language processing means" are technical methods for analyzing human language, extracting information, and processing it, enabling computers to understand and manipulate human language.

[0008] A "task" is a specific activity or unit of work performed to achieve a particular objective, and refers to a manageable task that has been broken down from a work instruction.

[0009] An "artificial intelligence agent" is a computer program or system designed to operate autonomously and perform specific tasks, possessing the ability to learn and adapt in order to assist in decision-making and problem-solving.

[0010] "Assignment methods" refer to the methods and processes for distributing tasks to appropriate resources, aiming to optimize resource allocation for efficient task execution.

[0011] "Progress management means" refers to methods or processes for monitoring and continuously evaluating the progress of a task, enabling timely information provision and plan adjustments.

[0012] Process automation is a technique or method that standardizes tasks and stores them in a reusable format to efficiently handle similar tasks in the future.

[0013] An "interface" is an element that provides a means or screen display for a user to interact with a system and input or output information, and its purpose is to improve the convenience of system operation. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention provides a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. Specifically, the server utilizes natural language processing technology to automatically understand complex work instructions and convert them into structured tasks. The server analyzes high-level instructions received from managers and assigns the extracted tasks to artificial intelligence agents or human users.

[0036] The server executes algorithms for the artificial intelligence agent and generates operational guidelines and notifications required by humans. During this process, the server monitors task progress, checking progress in real time and reallocating or prioritizing resources as needed to ensure efficient execution.

[0037] The terminal provides an interface that allows users to easily input high-level work instructions and displays progress and related notifications. Through this interface, users can understand the current status of their work and perform intuitive operations when necessary.

[0038] As a concrete example, if the server receives the instruction to "conduct market research for a new project and report the analysis results," it analyzes the instruction and breaks it down into two main tasks: "collecting market data" and "creating a report." The AI ​​agent is responsible for data collection, and the user creates a report based on the results. Throughout the process, the server constantly monitors the progress of these tasks and updates the results in real time.

[0039] Furthermore, specific tasks completed by the AI ​​agent are recorded as process automations, allowing for reuse in similar tasks in the future. In this way, the system improves productivity and enables efficient work with fewer people. This invention dramatically improves the operational efficiency of companies and allows managers to have more free time to make strategic decisions.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user inputs high-level work instructions using the terminal's interface. The terminal then converts the input instructions into a data format and prepares to send them to the server.

[0043] Step 2:

[0044] The server inputs the work instructions received from the terminal into a natural language processing module, which then analyzes the instructions. This process involves extracting specific tasks through keyword and contextual analysis.

[0045] Step 3:

[0046] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide a market research instruction into specific tasks such as "data collection" and "analysis report."

[0047] Step 4:

[0048] The server assigns the broken-down tasks to human users and artificial intelligence agents. Here, the optimal assignment is made based on the nature of each task and the capacity of the resources.

[0049] Step 5:

[0050] The device displays the details of the task assigned to the user. This includes the task content, deadline, and required resources, allowing the user to see the actions required.

[0051] Step 6:

[0052] The server monitors task progress in real time and collects progress data. The server analyzes the status of each task and reallocates resources and adjusts task priorities as needed.

[0053] Step 7:

[0054] When a task is completed, the server checks the results and records the tasks handled by the artificial intelligence agent for process automation. This contributes to improving efficiency in future operations.

[0055] Step 8:

[0056] The terminal notifies the user of completed tasks and progress information, supporting performance evaluation and transition to the next step. This increases transparency and efficiency throughout the entire workflow.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] In business operations, it is difficult for managers and users to efficiently break down high-level work instructions into tasks and appropriately allocate them to humans and artificial intelligence. Furthermore, there is a lack of mechanisms for monitoring task progress in real time and for appropriately reallocating resources and adjusting priorities. As a result, operational efficiency declines, hindering productivity improvements.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes natural language processing means for automatically analyzing work instructions and breaking them down into generalized tasks; assignment means for appropriately allocating the tasks to human users and artificial intelligence agents; progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and means for providing an interface for users to input work instructions and optimizing the overall user experience. This enables efficient analysis of work instructions, appropriate task allocation, and real-time monitoring of progress, thereby maximizing productivity.

[0062] "Natural language processing means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into generalized tasks.

[0063] "Assignment means" refers to technology that has the function of appropriately distributing extracted tasks to human users and artificial intelligence agents.

[0064] A "progress management system" is a technology that has the function of monitoring the progress of tasks in real time and adjusting priorities and reallocating resources as needed.

[0065] "Means of providing an interface" refers to technologies that provide input and display methods for users to input work instructions and optimize the overall user experience.

[0066] A "storage method" is a technology that has the function of recording tasks completed by an artificial intelligence agent as process automation and saving data for use in similar tasks in the future.

[0067] "Means for generating feedback" refers to technologies for creating information that contributes to optimizing business processes based on generated data.

[0068] This invention is a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. The server analyzes work instructions using natural language processing technology. Specifically, the server uses natural language processing software such as "spaCy" or "BERT" to understand the content of the work instructions and convert them into structured tasks. Through this process, the instructions are broken down into more specific tasks.

[0069] The server can use task management tools such as "JIRA" and "Trello" for task distribution. This allows the server to assign extracted tasks to artificial intelligence agents and human users. Furthermore, the server utilizes data analysis tools such as "pandas" and "NumPy" to support the AI ​​agents in efficiently executing specified data processing tasks.

[0070] The terminal provides an intuitive interface for users to input work instructions. The terminal utilizes front-end frameworks such as "React" and "Vue.js," allowing users to easily input information. Furthermore, task progress and related notifications are presented to the user in real time through this terminal.

[0071] As a concrete example, the server receives a task instruction to "conduct market research for a new project and report the analysis results," and then performs the analysis. This instruction is broken down into two tasks: "collecting market data" and "creating a report," which are then assigned to an AI agent and a user, respectively. The AI ​​agent collects market data using "pandas," and the user creates the report using "Google Docs" or similar tools.

[0072] Examples of prompts to input into a generative AI model include the following:

[0073] "Conduct market research for the new project and report the analysis results. Break down the instructions into appropriate tasks and assign them to the appropriate individuals."

[0074] As described above, this system is optimized to improve operational efficiency and has a structure that enables the execution of tasks under specific conditions.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The server receives high-level work instructions from users through terminals. The input consists of work instructions entered by the user using an interface. The server acquires these instructions as input data. Next, the server analyzes the content of the instructions using "spaCy" or "BERT" and breaks them down into main tasks. Through this analysis, the instructions are transformed into generalized tasks, and task identification is obtained as output.

[0078] Step 2:

[0079] The server appropriately structures the tasks obtained from the analysis. These structured tasks are then assigned to human users or artificial intelligence agents. The input is the identification of tasks obtained in step 1, and the output is the assignment of specific tasks to users and agents. The server uses tools such as "JIRA" or "Trello" to consider the resources and skills required to perform each task.

[0080] Step 3:

[0081] The artificial intelligence agent receives tasks assigned to it from a server. The input is task assignment information from the server, and the agent uses tools such as "pandas" and "NumPy" to collect and process the data. This processing yields specific data and results as output.

[0082] Step 4:

[0083] The terminal provides the user with progress updates and related notifications for tasks assigned to them. The input is task progress information from the server. The terminal uses frameworks such as React and Vue.js to visually present this information to the user through an intuitive and user-friendly interface. The user then performs tasks such as creating reports based on the provided information.

[0084] Step 5:

[0085] The server monitors progress information from users and agents in real time, and readjusts resource allocation and priorities. Input is real-time data on the status of each task, and output is an optimized process. At this stage, the server optimizes tasks and ensures improved overall system productivity.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] With technological advancements, there is a growing need for equipment and transportation systems to process increasingly complex instructions in real time and to respond dynamically to crew demands. However, currently, understanding operational orders and executing tasks flexibly based on them is difficult, often hindering efficient operation. In particular, optimizing operational routes and dynamically adjusting transport equipment presents challenges in automatically responding to increasingly diverse instructions. Therefore, there is a need for a system that can analyze operational orders entered in natural language, automatically break down tasks, and make appropriate decisions.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes a natural language processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human users and artificial intelligence agents; a progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; an analysis means capable of understanding operation orders in natural language and automatically breaking them down into multiple tasks to be performed by transport equipment; and a decision means for understanding crew requests and dynamically optimizing the operation route. This enables the equipment in operation to respond flexibly to crew requests and complex instructions in real time, achieving efficient operation.

[0091] "Work instructions" are instructions that specify the actions and procedures necessary to achieve a particular objective or task.

[0092] "Natural language processing methods" refer to technologies and techniques for enabling computers to understand and analyze natural language used by humans in everyday life.

[0093] An "assignment mechanism" is a system that distributes analyzed tasks to the most suitable agent or user and issues instructions to them.

[0094] A "progress management tool" is a function that allows for real-time monitoring of task progress and, as necessary, reprioritization of tasks and adjustment of resources.

[0095] An "operation order" is a set of instructions for transport equipment to operate along a specific route or in a particular manner.

[0096] An "analysis tool" is a mechanism for breaking down input information or instructions into structured tasks.

[0097] "Decision-making tools" refer to functions that enable the selection or execution of the optimal course of action depending on the situation.

[0098] This invention is a system for transporting equipment and means of transport that analyzes instructions input in natural language, efficiently breaks them down into multiple tasks, and executes them. The system consists of three main components: a server, a terminal, and a user.

[0099] The server utilizes natural language processing technologies such as Google Cloud Natural Language API and AWS® Comprehend to automatically analyze work instructions and break them down into specific tasks. This enables advanced natural language analysis and task structuring. The analyzed tasks are then assigned to the most suitable artificial intelligence agent or human. Throughout this process, the server monitors progress in real time, adjusting priorities and reallocating resources as needed.

[0100] The terminal provides an interface that allows users to input operational commands in natural language using a smartphone or head-mounted display. Based on the input commands, users can intuitively check various progress statuses and notifications during operation. In addition, the operational route is automatically optimized and suggested based on the user's requests. In this process, past data is utilized to provide efficient suggestions, allowing users to consistently select the optimal route.

[0101] For example, if a user riding in an autonomous vehicle enters a request through the interface such as, "I want to go to my destination next Thursday at 9:00 AM and stop at a cafe along the way," this request is analyzed by the server, and the optimal route is calculated in real time. This calculation takes into account the user's preferences and past route data.

[0102] Examples of prompt statements to input into a generative AI model are as follows:

[0103] "The user wishes to visit their destination at 9:00 AM on Thursday, November 2023. They also wish to have breakfast at a cafe along the way. Please suggest the best route."

[0104] This system enables transport equipment and means of transportation to operate automatically and efficiently in response to real-time user requests.

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] Users input operational commands in natural language through a smartphone or head-mounted display interface. These commands include operational requests such as specific destinations or points of interest along the route. This input serves as the starting point for the system's processing.

[0108] Step 2:

[0109] The server receives this natural language command. Next, using the Google Cloud Natural Language API, it interprets the command and breaks it down into several specific tasks. The input is the command statement from the user, and the output is each of the decomposed tasks. At this stage, the tasks are stored as text data.

[0110] Step 3:

[0111] The server assigns each task to an executable process based on the individual tasks it receives. It assigns AI programs to tasks that can be handled by artificial intelligence agents, and notifies the user of tasks requiring human intervention. The input is a list of broken-down tasks, and the output is the assignment to specific personnel or agents.

[0112] Step 4:

[0113] The server uses GOOGLE FI® rebase to monitor task progress in real time. It continuously collects progress data and updates the database as needed. The input is event data related to task progress, and the output is a dataset of the latest progress information.

[0114] Step 5:

[0115] The server automatically prioritizes and reallocates resources as needed based on data from Firebase. It optimizes the entire system in response to task progress and new requests. Inputs are progress information and resource usage, and outputs are updated priorities and resource allocations.

[0116] Step 6:

[0117] The terminal visually displays the route and progress information obtained from the server to the user. This allows the user to check real-time information about the operation and provide feedback as needed. The input is visualized progress data from the server, and the output is displayed information in a format that is easy for the user to understand.

[0118] This series of processes makes it possible to execute user operation requests efficiently and optimally.

[0119] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0120] This invention relates to a system that automatically analyzes work instructions, breaks them down into specific tasks, and effectively assigns them to users and artificial intelligence agents. In particular, by incorporating an emotion engine, this system recognizes the user's emotional state in real time, thereby improving work efficiency. The embodiments thereof are described below.

[0121] The server receives work instructions provided by the user and analyzes them using a natural language processing module. The analyzed instructions are broken down into individual tasks, and each task is assigned to the most suitable resource, namely a human user or an artificial intelligence agent.

[0122] In this process, the emotion engine plays a crucial role. Specifically, an emotion recognition function installed on the device analyzes the user's facial expressions and voice, and evaluates their emotional state in real time. The server can use the data obtained from the emotion engine to adjust task priorities based on the user's stress and fatigue levels.

[0123] For example, when a user inputs the instruction "Create a market research report for a new product by next week" via a terminal, the server uses natural language processing to break down this instruction into tasks such as "collect market data," "design the report," and "present the results." If the emotion engine detects a high-stress state at this time, the server can re-evaluate the order of the tasks and adjust the allocation to reduce the burden on the user.

[0124] Furthermore, tasks completed by the artificial intelligence agent are recorded in a database as process automation and efficiently reused for future tasks. This record is fed back to the user via the terminal, improving the overall transparency of the work.

[0125] The server also analyzes users' past emotional data and suggests optimal strategies for task allocation and progress, thereby helping to continuously improve work efficiency. This approach makes it possible to simultaneously achieve increased productivity and reduced psychological burden in the work environment.

[0126] The following describes the processing flow.

[0127] Step 1:

[0128] The user uses a terminal to input high-level work instructions. The terminal prepares to send the user's instructions as digital data to the server.

[0129] Step 2:

[0130] The server inputs the work instructions received from the terminal into a natural language processing module and analyzes their content. Specifically, it analyzes the instruction text to extract task keywords and related context.

[0131] Step 3:

[0132] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide "market research" into specific tasks such as "data collection," "competitor analysis," and "report creation of analysis results."

[0133] Step 4:

[0134] The device uses an emotion engine to detect the user's emotional state. It analyzes data such as voice tone, facial expressions, and behavioral patterns to evaluate the current emotional state.

[0135] Step 5:

[0136] The server uses data obtained from the emotion engine to optimize task allocation. For example, if a user is experiencing high stress levels, it adjusts the assignment to prioritize lighter tasks.

[0137] Step 6:

[0138] The server notifies the AI ​​agent and human users of the assigned tasks and updates progress information on each terminal in real time.

[0139] Step 7:

[0140] The device provides the user with feedback on task completion and progress. This includes the next steps to take and other helpful information for the user.

[0141] Step 8:

[0142] The server records completed tasks as process automations and saves them to a database. This record can be reused and utilized for similar tasks in the future.

[0143] Step 9:

[0144] The server analyzes the user's work patterns using past sentiment data and proposes an optimal task strategy based on this analysis. The terminal then presents the user with an action plan based on the new strategy.

[0145] (Example 2)

[0146] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0147] In today's work environment, effectively analyzing work instructions and breaking them down into specific tasks is difficult. Furthermore, there is a need to allocate tasks to the optimal resources while effectively managing task progress, taking into account the user's emotional state. A system is needed to streamline this process, reduce the psychological burden on users, and improve work efficiency.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0149] In this invention, the server includes a language analysis means for automatically analyzing work instructions and breaking them down into specific tasks, an allocation means for appropriately assigning the tasks to human users and machine intelligence agents, and an emotion recognition means for analyzing the user's emotional state in real time and adjusting the priority of tasks. This makes it possible not only to effectively break down work instructions into tasks and allocate them to the optimal resources, but also to adjust the priority of tasks while taking into account the user's emotional state, thereby improving work efficiency and reducing psychological burden.

[0150] "Language analysis means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0151] "Distribution means" refers to technology that provides the function of appropriately allocating the analyzed tasks to human users and machine intelligence agents.

[0152] "Emotion recognition means" refers to technology that analyzes a user's emotional state in real time and provides functions to influence the prioritization and allocation of tasks.

[0153] A "progress management system" is a technology that provides the functionality to monitor the progress of work and adjust priorities or reallocate resources as needed.

[0154] "Storage means" refers to technology that automates the procedures of completed tasks by a machine intelligence agent, stores them, and utilizes them for similar tasks in the future.

[0155] "Structure" refers to the interface that allows users to efficiently perform a series of operations, from inputting work instructions to receiving those instructions and receiving progress notifications.

[0156] In an embodiment of this invention, the server receives a work instruction and analyzes the instruction using a language analysis means. For this purpose, the server can utilize common APIs and frameworks as natural language processing technologies (for example, OpenAI®'s language model API or other natural language processing tools). Through this analysis, the work instruction is broken down into specific tasks.

[0157] The server then uses allocation mechanisms to distribute the analyzed work to the most suitable resources. These resources include human users and machine intelligence agents. Resource selection is determined based on the nature of the work and the user's capabilities and schedule.

[0158] The device is equipped with emotion recognition capabilities, which allow it to analyze the user's facial expressions and voice. This could involve using input devices such as cameras and microphones. For example, by utilizing facial recognition and voice analysis technologies, the device could monitor the user's stress and fatigue levels in real time.

[0159] When a user enters a high-level task instruction such as "Create a market research report for a new product," the server receives it and uses its natural language processing tools to break down the instruction into specific tasks (e.g., "Collect market data," "Design the report," "Present the results").

[0160] If emotion recognition reveals that a user is in a high-stress state, the server is designed to automatically adjust allocation and task priorities to reduce the burden. For example, specific tasks can be assigned to machine intelligence agents to alleviate the burden.

[0161] Furthermore, the server uses storage methods to save a history of completed tasks and assignments to a database. This historical data is used as a knowledge base to efficiently perform similar tasks in the future.

[0162] This system provides support to organizations and individuals to efficiently carry out their work through a user interface. A possible example of a specific prompt might be, "Break down the work instructions into tasks and allocate them to the most suitable resources based on the user's emotional state." This prompt prompts the system to perform actions in accordance with the user's needs.

[0163] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0164] Step 1:

[0165] The server receives work instructions from the user. The server receives work instructions in natural language entered by the user on a terminal as input. The server stores these work instructions as data and prepares them for analysis. The output is work instruction data that can be analyzed.

[0166] Step 2:

[0167] The server analyzes the received work instructions using language analysis tools. The input is the work instruction data saved in step 1. This analysis includes a process of converting the instruction content into a structured data format using natural language processing tools. The output is structured data broken down into specific tasks.

[0168] Step 3:

[0169] The server allocates the analyzed tasks to appropriate resources using allocation mechanisms. It receives structured data generated in step 2 as input. For each task, the server selects the most suitable human user or machine intelligence agent and assigns the task. The output is resource allocation information for each task.

[0170] Step 4:

[0171] The device analyzes the user's emotional state using emotion recognition technology. It uses data from the user's facial expressions and voice, obtained from a camera and microphone connected to the device, as input. The device then runs emotion analysis software and outputs the user's emotional state as numerical data.

[0172] Step 5:

[0173] The server adjusts task priorities based on emotion recognition data. The input is the user's emotional state data obtained from step 4. The server reconfigures task priorities and reallocates resources according to the user's stress and fatigue levels. The output is the adjusted priority list and updated resource allocation information.

[0174] Step 6:

[0175] The server records completed work and its results in a database using storage methods. The input is the work history data obtained as a result of steps 3 and 5. This updates the database as a knowledge base that can be used in the future. The output is the recorded work history and performance data.

[0176] Step 7:

[0177] The user receives overall progress and feedback through the terminal. Inputs include progress data and adjustment information based on emotional changes transmitted from the server. The terminal displays this data clearly to the user, providing transparency to the work progress. Output is visual feedback information provided to the user.

[0178] (Application Example 2)

[0179] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0180] In operations, the emotions and psychological state of workers often affect the efficiency and productivity of their work. However, it is difficult to allocate tasks appropriately while taking these factors into account, which can result in decreased productivity and increased workload for workers. Furthermore, there is a lack of efficient systems for effectively utilizing data from past tasks and reflecting it in future operations.

[0181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0182] In this invention, the server includes a text processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human operators and artificial intelligence agents; a management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and a recognition means for sensing the psychological state of workers and optimizing task assignment based on that data. This enables efficient task assignment that takes into account the emotional state of workers and improves the transparency of operations.

[0183] A "text processing means" is a module that has the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0184] An "assignment mechanism" is a system that performs the process of appropriately assigning analyzed tasks to human operators or artificial intelligence agents.

[0185] A "management tool" is a system that has the function of monitoring task progress in real time, adjusting priorities as needed, and reallocating resources.

[0186] A "recognition means" is a module that senses the psychological state of workers from their facial expressions and voices, and optimizes task assignment based on that data.

[0187] An "artificial intelligence agent" is an automation system or program designed to efficiently perform a specific task.

[0188] This invention is a system that utilizes a combination of technologies to maximize operational efficiency. Specifically, the server receives work instructions, analyzes them using natural language processing, and breaks them down into multiple tasks. Furthermore, it is possible to evaluate the user's psychological state using emotion recognition functionality installed on the terminal. Using this data, the server performs the optimal task allocation.

[0189] The appropriate assignment of tasks to human operators and artificial intelligence agents is carried out through assignment mechanisms. In this process, the system manages task progress in real time and reallocates priorities and resources as needed. This reduces stress and psychological burden on users and improves the overall efficiency of the system.

[0190] A concrete application example is a system that improves the work efficiency of factory workers. For instance, if a recognition system detects that a worker is in a high-stress state, the server optimizes that worker's tasks and reallocates them to other resources. In this way, the overall workflow of the factory proceeds smoothly.

[0191] The following prompt statements can be considered as examples of how generative AI models can be used.

[0192] "Please optimize task assignments, taking into account the psychological state of the workers. High stress levels have been detected."

[0193] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0194] Step 1:

[0195] The server receives work instructions from the user as input. Using a natural language processing module, it analyzes the instructions, breaks them down into specific tasks, and generates that information as output. Through text analysis, the instructions are classified into meaningful tasks such as "collect market data" or "design reports."

[0196] Step 2:

[0197] The device captures the user's facial expressions and voice as real-time input and uses emotion recognition to evaluate their psychological state. This data is analyzed by an emotion engine, which quantifies and outputs stress and fatigue levels. For example, it detects changes in the user's voice tone and facial expressions and provides a numerical value indicating a high-stress state.

[0198] Step 3:

[0199] The server receives the acquired emotional data as input and optimizes tasks through its assignment mechanism. Specifically, it adjusts task priorities based on the emotional data and re-evaluates the assignment to human operators or artificial intelligence agents. This process seeks task placement that reduces the user's psychological burden and improves productivity.

[0200] Step 4:

[0201] The server monitors task progress in real time and provides users with feedback on progress and any issues. It collects progress data as input and uses it to adjust priorities and reallocate resources. This monitoring ensures continuous, real-time resource optimization.

[0202] Step 5:

[0203] The artificial intelligence agent executes assigned tasks and reports completion data to the server. Task completion status is recorded as process automation and used to efficiently handle similar tasks in the future. This data storage process enhances the overall operational efficiency of the system.

[0204] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0205] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0206] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0207] [Second Embodiment]

[0208] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0209] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0210] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0211] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0212] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0213] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0214] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0215] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0216] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0217] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0218] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0219] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0220] This invention provides a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. Specifically, the server utilizes natural language processing technology to automatically understand complex work instructions and convert them into structured tasks. The server analyzes high-level instructions received from managers and assigns the extracted tasks to artificial intelligence agents or human users.

[0221] The server executes algorithms for the artificial intelligence agent and generates operational guidelines and notifications required by humans. During this process, the server monitors task progress, checking progress in real time and reallocating or prioritizing resources as needed to ensure efficient execution.

[0222] The terminal provides an interface that allows users to easily input high-level work instructions and displays progress and related notifications. Through this interface, users can understand the current status of their work and perform intuitive operations when necessary.

[0223] As a concrete example, if the server receives the instruction to "conduct market research for a new project and report the analysis results," it analyzes the instruction and breaks it down into two main tasks: "collecting market data" and "creating a report." The AI ​​agent is responsible for data collection, and the user creates a report based on the results. Throughout the process, the server constantly monitors the progress of these tasks and updates the results in real time.

[0224] Furthermore, specific tasks completed by the AI ​​agent are recorded as process automations, allowing for reuse in similar tasks in the future. In this way, the system improves productivity and enables efficient work with fewer people. This invention dramatically improves the operational efficiency of companies and allows managers to have more free time to make strategic decisions.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] The user inputs high-level work instructions using the terminal's interface. The terminal then converts the input instructions into a data format and prepares to send them to the server.

[0228] Step 2:

[0229] The server inputs the work instructions received from the terminal into a natural language processing module, which then analyzes the instructions. This process involves extracting specific tasks through keyword and contextual analysis.

[0230] Step 3:

[0231] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide a market research instruction into specific tasks such as "data collection" and "analysis report."

[0232] Step 4:

[0233] The server assigns the broken-down tasks to human users and artificial intelligence agents. Here, the optimal assignment is made based on the nature of each task and the capacity of the resources.

[0234] Step 5:

[0235] The device displays the details of the task assigned to the user. This includes the task content, deadline, and required resources, allowing the user to see the actions required.

[0236] Step 6:

[0237] The server monitors task progress in real time and collects progress data. The server analyzes the status of each task and reallocates resources and adjusts task priorities as needed.

[0238] Step 7:

[0239] When a task is completed, the server checks the results and records the tasks handled by the artificial intelligence agent for process automation. This contributes to improving efficiency in future operations.

[0240] Step 8:

[0241] The terminal notifies the user of completed tasks and progress information, supporting performance evaluation and transition to the next step. This increases transparency and efficiency throughout the entire workflow.

[0242] (Example 1)

[0243] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0244] In business operations, it is difficult for managers and users to efficiently break down high-level work instructions into tasks and appropriately allocate them to humans and artificial intelligence. Furthermore, there is a lack of mechanisms for monitoring task progress in real time and for appropriately reallocating resources and adjusting priorities. As a result, operational efficiency declines, hindering productivity improvements.

[0245] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0246] In this invention, the server includes natural language processing means for automatically analyzing work instructions and breaking them down into generalized tasks; assignment means for appropriately allocating the tasks to human users and artificial intelligence agents; progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and means for providing an interface for users to input work instructions and optimizing the overall user experience. This enables efficient analysis of work instructions, appropriate task allocation, and real-time monitoring of progress, thereby maximizing productivity.

[0247] "Natural language processing means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into generalized tasks.

[0248] "Assignment means" refers to technology that has the function of appropriately distributing extracted tasks to human users and artificial intelligence agents.

[0249] A "progress management system" is a technology that has the function of monitoring the progress of tasks in real time and adjusting priorities and reallocating resources as needed.

[0250] "Means of providing an interface" refers to technologies that provide input and display methods for users to input work instructions and optimize the overall user experience.

[0251] A "storage method" is a technology that has the function of recording tasks completed by an artificial intelligence agent as process automation and saving data for use in similar tasks in the future.

[0252] "Means for generating feedback" refers to technologies for creating information that contributes to optimizing business processes based on generated data.

[0253] This invention is a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. The server analyzes work instructions using natural language processing technology. Specifically, the server uses natural language processing software such as "spaCy" or "BERT" to understand the content of the work instructions and convert them into structured tasks. Through this process, the instructions are broken down into more specific tasks.

[0254] The server can use task management tools such as "JIRA" and "Trello" for task distribution. This allows the server to assign extracted tasks to artificial intelligence agents and human users. Furthermore, the server utilizes data analysis tools such as "pandas" and "NumPy" to support the AI ​​agents in efficiently executing specified data processing tasks.

[0255] The terminal provides an intuitive interface for users to input work instructions. The terminal utilizes front-end frameworks such as "React" and "Vue.js," allowing users to easily input information. Furthermore, task progress and related notifications are presented to the user in real time through this terminal.

[0256] As a concrete example, the server receives a task instruction to "conduct market research for a new project and report the analysis results," and then performs the analysis. This instruction is broken down into two tasks: "collecting market data" and "creating a report," which are then assigned to an AI agent and a user, respectively. The AI ​​agent collects market data using "pandas," and the user creates the report using "Google Docs" or similar tools.

[0257] Examples of prompts to input into a generative AI model include the following:

[0258] "Conduct market research for the new project and report the analysis results. Break down the instructions into appropriate tasks and assign them to the appropriate individuals."

[0259] As described above, this system is optimized to improve operational efficiency and has a structure that enables the execution of tasks under specific conditions.

[0260] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0261] Step 1:

[0262] The server receives high-level work instructions from users through terminals. The input consists of work instructions entered by the user using an interface. The server acquires these instructions as input data. Next, the server analyzes the content of the instructions using "spaCy" or "BERT" and breaks them down into main tasks. Through this analysis, the instructions are transformed into generalized tasks, and task identification is obtained as output.

[0263] Step 2:

[0264] The server appropriately structures the tasks obtained from the analysis. These structured tasks are then assigned to human users or artificial intelligence agents. The input is the identification of tasks obtained in step 1, and the output is the assignment of specific tasks to users and agents. The server uses tools such as "JIRA" or "Trello" to consider the resources and skills required to perform each task.

[0265] Step 3:

[0266] The artificial intelligence agent receives tasks assigned to it from a server. The input is task assignment information from the server, and the agent uses tools such as "pandas" and "NumPy" to collect and process the data. This processing yields specific data and results as output.

[0267] Step 4:

[0268] The terminal provides the user with progress updates and related notifications for tasks assigned to them. The input is task progress information from the server. The terminal uses frameworks such as React and Vue.js to visually present this information to the user through an intuitive and user-friendly interface. The user then performs tasks such as creating reports based on the provided information.

[0269] Step 5:

[0270] The server monitors progress information from users and agents in real time, and readjusts resource allocation and priorities. Input is real-time data on the status of each task, and output is an optimized process. At this stage, the server optimizes tasks and ensures improved overall system productivity.

[0271] (Application Example 1)

[0272] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0273] With technological advancements, there is a growing need for equipment and transportation systems to process increasingly complex instructions in real time and to respond dynamically to crew demands. However, currently, understanding operational orders and executing tasks flexibly based on them is difficult, often hindering efficient operation. In particular, optimizing operational routes and dynamically adjusting transport equipment presents challenges in automatically responding to increasingly diverse instructions. Therefore, there is a need for a system that can analyze operational orders entered in natural language, automatically break down tasks, and make appropriate decisions.

[0274] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0275] In this invention, the server includes a natural language processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human users and artificial intelligence agents; a progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; an analysis means capable of understanding operation orders in natural language and automatically breaking them down into multiple tasks to be performed by transport equipment; and a decision means for understanding crew requests and dynamically optimizing the operation route. This enables the equipment in operation to respond flexibly to crew requests and complex instructions in real time, achieving efficient operation.

[0276] "Work instructions" are instructions that specify the actions and procedures necessary to achieve a particular objective or task.

[0277] "Natural language processing methods" refer to technologies and techniques for enabling computers to understand and analyze natural language used by humans in everyday life.

[0278] The "assignment means" is a mechanism that distributes and instructs the analyzed tasks to the optimal agents or users.

[0279] The "progress management means" is a function for monitoring the progress of tasks in real time and, if necessary, reviewing priorities and adjusting resources.

[0280] The "operation order" is an instruction for the transport equipment to operate in a specific route or method.

[0281] The "analysis means" is a mechanism for decomposing the input information or instructions into structured tasks based on them.

[0282] The "judgment means" refers to a function for making an optimal selection or processing according to the situation.

[0283] This invention is a system that analyzes instructions input in natural language in transport equipment or means of transportation and efficiently decomposes and executes them into multiple operations. The system consists of three main components: a server, a terminal, and a user.

[0284] The server makes full use of natural language processing technologies such as Google Cloud Natural Language API and AWS Comprehend to automatically analyze business instructions and decompose them into specific tasks. As a result, advanced natural language analysis and task structuring are performed. The analyzed tasks are then allocated to the optimal artificial intelligence agents or humans. In this process, the server monitors the progress in real time and adjusts priorities and redistributes resources.

[0285] The terminal provides an interface that allows the user to input operation commands in natural language using a smartphone or a head-mounted display. The user can intuitively check various progress situations and notifications during operation based on the input commands. Also, based on the user's requirements, the operation route is automatically optimized and proposed. At this time, by utilizing past data to make an efficient proposal, the user can consistently select the optimal route.

[0286] As a specific example, when a user riding in an autonomous vehicle inputs an operation request such as "I want to go to the destination at 9 o'clock next Thursday and stop by a café on the way" from the interface, this request is analyzed by the server, and the optimal route is calculated in real time. The user's preferences and past route data are also taken into consideration.

[0287] Examples of prompt sentences to be input into the generative AI model are as follows.

[0288] "The user hopes to visit the destination at 9 am on Thursday, November 2023. There is also a hope to have breakfast at a café on the way. Please propose the optimal route."

[0289] With this system, transportation equipment and means of transportation can operate automatically and efficiently in response to real-time user requests.

[0290] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0291] Step 1:

[0292] The user inputs an operation command in natural language through the interface of a smartphone or a head-mounted display. What is input is an operation request such as a specific destination or a place to stop by on the way. This input becomes the starting point of the system's processing.

[0293] Step 2:

[0294] The server receives this natural language command. Next, using the Google Cloud Natural Language API, it interprets the command and breaks it down into several specific tasks. The input is the command statement from the user, and the output is each of the decomposed tasks. At this stage, the tasks are stored as text data.

[0295] Step 3:

[0296] The server assigns each task to an executable process based on the individual tasks it receives. It assigns AI programs to tasks that can be handled by artificial intelligence agents, and notifies the user of tasks requiring human intervention. The input is a list of broken-down tasks, and the output is the assignment to specific personnel or agents.

[0297] Step 4:

[0298] The server uses Google Firebase to monitor task progress in real time. It continuously collects progress data and updates the database as needed. The input is event data related to the task's progress, and the output is a dataset of the latest progress information.

[0299] Step 5:

[0300] The server automatically prioritizes and reallocates resources as needed based on data from Firebase. It optimizes the entire system in response to task progress and new requests. Inputs are progress information and resource usage, and outputs are updated priorities and resource allocations.

[0301] Step 6:

[0302] The terminal visually presents the operation route and progress information obtained from the server to the user. As a result, the user can check the information during operation in real time and provide feedback as needed. The input is visualizable progress data from the server, and the output is display information in a format that is easy for the user to understand.

[0303] Through this series of processes, it becomes possible to execute the user's operation requests efficiently and optimally.

[0304] Furthermore, an emotion engine that estimates the user's emotions may be combined. That is, the specific processing unit 290 may estimate the user's emotions using the emotion recognition model 59 and perform specific processing using the user's emotions.

[0305] The present invention is a system that automatically analyzes business instructions, decomposes them into specific tasks, and effectively allocates them to users and artificial intelligence agents. By combining an emotion engine in particular, this system can recognize the user's emotional state in real time and improve business efficiency. Embodiments thereof will be described below.

[0306] The server receives business instructions provided by the user and analyzes them using a natural language processing module. The analyzed instructions are decomposed into individual tasks, and each task is allocated to an optimal resource, that is, a human user or an artificial intelligence agent.

[0307] In this process, the emotion engine plays an important role. Specifically, the emotion recognition function installed in the terminal analyzes the user's facial expressions, voice, etc., and evaluates the emotional state in real time. The server can utilize the data obtained from the emotion engine and adjust the task priorities based on the stress and fatigue levels of the user.

[0308] For example, when a user inputs the instruction "Create a market research report for a new product by next week" via a terminal, the server uses natural language processing to break down this instruction into tasks such as "collect market data," "design the report," and "present the results." If the emotion engine detects a high-stress state at this time, the server can re-evaluate the order of the tasks and adjust the allocation to reduce the burden on the user.

[0309] Furthermore, tasks completed by the artificial intelligence agent are recorded in a database as process automation and efficiently reused for future tasks. This record is fed back to the user via the terminal, improving the overall transparency of the work.

[0310] The server also analyzes users' past emotional data and suggests optimal strategies for task allocation and progress, thereby helping to continuously improve work efficiency. This approach makes it possible to simultaneously achieve increased productivity and reduced psychological burden in the work environment.

[0311] The following describes the processing flow.

[0312] Step 1:

[0313] The user uses a terminal to input high-level work instructions. The terminal prepares to send the user's instructions as digital data to the server.

[0314] Step 2:

[0315] The server inputs the work instructions received from the terminal into a natural language processing module and analyzes their content. Specifically, it analyzes the instruction text to extract task keywords and related context.

[0316] Step 3:

[0317] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide "market research" into specific tasks such as "data collection," "competitor analysis," and "report creation of analysis results."

[0318] Step 4:

[0319] The device uses an emotion engine to detect the user's emotional state. It analyzes data such as voice tone, facial expressions, and behavioral patterns to evaluate the current emotional state.

[0320] Step 5:

[0321] The server uses data obtained from the emotion engine to optimize task allocation. For example, if a user is experiencing high stress levels, it adjusts the assignment to prioritize lighter tasks.

[0322] Step 6:

[0323] The server notifies the AI ​​agent and human users of the assigned tasks and updates progress information on each terminal in real time.

[0324] Step 7:

[0325] The device provides the user with feedback on task completion and progress. This includes the next steps to take and other helpful information for the user.

[0326] Step 8:

[0327] The server records completed tasks as process automations and saves them to a database. This record can be reused and utilized for similar tasks in the future.

[0328] Step 9:

[0329] The server analyzes the user's work patterns using past sentiment data and proposes an optimal task strategy based on this analysis. The terminal then presents the user with an action plan based on the new strategy.

[0330] (Example 2)

[0331] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0332] In today's work environment, effectively analyzing work instructions and breaking them down into specific tasks is difficult. Furthermore, there is a need to allocate tasks to the optimal resources while effectively managing task progress, taking into account the user's emotional state. A system is needed to streamline this process, reduce the psychological burden on users, and improve work efficiency.

[0333] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0334] In this invention, the server includes a language analysis means for automatically analyzing work instructions and breaking them down into specific tasks, an allocation means for appropriately assigning the tasks to human users and machine intelligence agents, and an emotion recognition means for analyzing the user's emotional state in real time and adjusting the priority of tasks. This makes it possible not only to effectively break down work instructions into tasks and allocate them to the optimal resources, but also to adjust the priority of tasks while taking into account the user's emotional state, thereby improving work efficiency and reducing psychological burden.

[0335] "Language analysis means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0336] "Distribution means" refers to technology that provides the function of appropriately allocating the analyzed tasks to human users and machine intelligence agents.

[0337] "Emotion recognition means" refers to technology that analyzes a user's emotional state in real time and provides functions to influence the prioritization and allocation of tasks.

[0338] A "progress management system" is a technology that provides the functionality to monitor the progress of work and adjust priorities or reallocate resources as needed.

[0339] "Storage means" refers to technology that automates the procedures of completed tasks by a machine intelligence agent, stores them, and utilizes them for similar tasks in the future.

[0340] "Structure" refers to the interface that allows users to efficiently perform a series of operations, from inputting work instructions to receiving those instructions and receiving progress notifications.

[0341] In an embodiment of this invention, the server receives a work instruction and analyzes the instruction using a language analysis means. For this purpose, the server can utilize common APIs and frameworks as natural language processing technologies (for example, OpenAI's language model API or other natural language processing tools). Through this analysis, the work instruction is broken down into specific tasks.

[0342] The server then uses allocation mechanisms to distribute the analyzed work to the most suitable resources. These resources include human users and machine intelligence agents. Resource selection is determined based on the nature of the work and the user's capabilities and schedule.

[0343] The device is equipped with emotion recognition capabilities, which allow it to analyze the user's facial expressions and voice. This could involve using input devices such as cameras and microphones. For example, by utilizing facial recognition and voice analysis technologies, the device could monitor the user's stress and fatigue levels in real time.

[0344] When a user enters a high-level task instruction such as "Create a market research report for a new product," the server receives it and uses its natural language processing tools to break down the instruction into specific tasks (e.g., "Collect market data," "Design the report," "Present the results").

[0345] If emotion recognition reveals that a user is in a high-stress state, the server is designed to automatically adjust allocation and task priorities to reduce the burden. For example, specific tasks can be assigned to machine intelligence agents to alleviate the burden.

[0346] Furthermore, the server uses storage methods to save a history of completed tasks and assignments to a database. This historical data is used as a knowledge base to efficiently perform similar tasks in the future.

[0347] This system provides support to organizations and individuals to efficiently carry out their work through a user interface. A possible example of a specific prompt might be, "Break down the work instructions into tasks and allocate them to the most suitable resources based on the user's emotional state." This prompt prompts the system to perform actions in accordance with the user's needs.

[0348] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0349] Step 1:

[0350] The server receives work instructions from the user. The server receives work instructions in natural language entered by the user on a terminal as input. The server stores these work instructions as data and prepares them for analysis. The output is work instruction data that can be analyzed.

[0351] Step 2:

[0352] The server analyzes the received work instructions using language analysis tools. The input is the work instruction data saved in step 1. This analysis includes a process of converting the instruction content into a structured data format using natural language processing tools. The output is structured data broken down into specific tasks.

[0353] Step 3:

[0354] The server allocates the analyzed tasks to appropriate resources using allocation mechanisms. It receives structured data generated in step 2 as input. For each task, the server selects the most suitable human user or machine intelligence agent and assigns the task. The output is resource allocation information for each task.

[0355] Step 4:

[0356] The device analyzes the user's emotional state using emotion recognition technology. It uses data from the user's facial expressions and voice, obtained from a camera and microphone connected to the device, as input. The device then runs emotion analysis software and outputs the user's emotional state as numerical data.

[0357] Step 5:

[0358] The server adjusts task priorities based on emotion recognition data. The input is the user's emotional state data obtained from step 4. The server reconfigures task priorities and reallocates resources according to the user's stress and fatigue levels. The output is the adjusted priority list and updated resource allocation information.

[0359] Step 6:

[0360] The server records completed work and its results in a database using storage methods. The input is the work history data obtained as a result of steps 3 and 5. This updates the database as a knowledge base that can be used in the future. The output is the recorded work history and performance data.

[0361] Step 7:

[0362] The user receives overall progress and feedback through the terminal. Inputs include progress data and adjustment information based on emotional changes transmitted from the server. The terminal displays this data clearly to the user, providing transparency to the work progress. Output is visual feedback information provided to the user.

[0363] (Application Example 2)

[0364] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0365] In operations, the emotions and psychological state of workers often affect the efficiency and productivity of their work. However, it is difficult to allocate tasks appropriately while taking these factors into account, which can result in decreased productivity and increased workload for workers. Furthermore, there is a lack of efficient systems for effectively utilizing data from past tasks and reflecting it in future operations.

[0366] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0367] In this invention, the server includes a text processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human operators and artificial intelligence agents; a management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and a recognition means for sensing the psychological state of workers and optimizing task assignment based on that data. This enables efficient task assignment that takes into account the emotional state of workers and improves the transparency of operations.

[0368] A "text processing means" is a module that has the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0369] An "assignment mechanism" is a system that performs the process of appropriately assigning analyzed tasks to human operators or artificial intelligence agents.

[0370] A "management tool" is a system that has the function of monitoring task progress in real time, adjusting priorities as needed, and reallocating resources.

[0371] A "recognition means" is a module that senses the psychological state of workers from their facial expressions and voices, and optimizes task assignment based on that data.

[0372] An "artificial intelligence agent" is an automation system or program designed to efficiently perform a specific task.

[0373] This invention is a system that utilizes a combination of technologies to maximize operational efficiency. Specifically, the server receives work instructions, analyzes them using natural language processing, and breaks them down into multiple tasks. Furthermore, it is possible to evaluate the user's psychological state using emotion recognition functionality installed on the terminal. Using this data, the server performs the optimal task allocation.

[0374] The appropriate assignment of tasks to human operators and artificial intelligence agents is carried out through assignment mechanisms. In this process, the system manages task progress in real time and reallocates priorities and resources as needed. This reduces stress and psychological burden on users and improves the overall efficiency of the system.

[0375] A concrete application example is a system that improves the work efficiency of factory workers. For instance, if a recognition system detects that a worker is in a high-stress state, the server optimizes that worker's tasks and reallocates them to other resources. In this way, the overall workflow of the factory proceeds smoothly.

[0376] The following prompt statements can be considered as examples of how generative AI models can be used.

[0377] "Please optimize task assignments, taking into account the psychological state of the workers. High stress levels have been detected."

[0378] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0379] Step 1:

[0380] The server receives work instructions from the user as input. Using a natural language processing module, it analyzes the instructions, breaks them down into specific tasks, and generates that information as output. Through text analysis, the instructions are classified into meaningful tasks such as "collect market data" or "design reports."

[0381] Step 2:

[0382] The device captures the user's facial expressions and voice as real-time input and uses emotion recognition to evaluate their psychological state. This data is analyzed by an emotion engine, which quantifies and outputs stress and fatigue levels. For example, it detects changes in the user's voice tone and facial expressions and provides a numerical value indicating a high-stress state.

[0383] Step 3:

[0384] The server receives the acquired emotional data as input and optimizes tasks through its assignment mechanism. Specifically, it adjusts task priorities based on the emotional data and re-evaluates the assignment to human operators or artificial intelligence agents. This process seeks task placement that reduces the user's psychological burden and improves productivity.

[0385] Step 4:

[0386] The server monitors task progress in real time and provides users with feedback on progress and any issues. It collects progress data as input and uses it to adjust priorities and reallocate resources. This monitoring ensures continuous, real-time resource optimization.

[0387] Step 5:

[0388] The artificial intelligence agent executes assigned tasks and reports completion data to the server. Task completion status is recorded as process automation and used to efficiently handle similar tasks in the future. This data storage process enhances the overall operational efficiency of the system.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0390] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0392] [Third Embodiment]

[0393] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0394] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0395] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0397] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0399] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0400] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0401] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0404] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0405] This invention provides a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. Specifically, the server utilizes natural language processing technology to automatically understand complex work instructions and convert them into structured tasks. The server analyzes high-level instructions received from managers and assigns the extracted tasks to artificial intelligence agents or human users.

[0406] The server executes algorithms for the artificial intelligence agent and generates operational guidelines and notifications required by humans. During this process, the server monitors task progress, checking progress in real time and reallocating or prioritizing resources as needed to ensure efficient execution.

[0407] The terminal provides an interface that allows users to easily input high-level work instructions and displays progress and related notifications. Through this interface, users can understand the current status of their work and perform intuitive operations when necessary.

[0408] As a concrete example, if the server receives the instruction to "conduct market research for a new project and report the analysis results," it analyzes the instruction and breaks it down into two main tasks: "collecting market data" and "creating a report." The AI ​​agent is responsible for data collection, and the user creates a report based on the results. Throughout the process, the server constantly monitors the progress of these tasks and updates the results in real time.

[0409] Furthermore, specific tasks completed by the AI ​​agent are recorded as process automations, allowing for reuse in similar tasks in the future. In this way, the system improves productivity and enables efficient work with fewer people. This invention dramatically improves the operational efficiency of companies and allows managers to have more free time to make strategic decisions.

[0410] The following describes the processing flow.

[0411] Step 1:

[0412] The user inputs high-level work instructions using the terminal's interface. The terminal then converts the input instructions into a data format and prepares to send them to the server.

[0413] Step 2:

[0414] The server inputs the work instructions received from the terminal into a natural language processing module, which then analyzes the instructions. This process involves extracting specific tasks through keyword and contextual analysis.

[0415] Step 3:

[0416] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide a market research instruction into specific tasks such as "data collection" and "analysis report."

[0417] Step 4:

[0418] The server assigns the broken-down tasks to human users and artificial intelligence agents. Here, the optimal assignment is made based on the nature of each task and the capacity of the resources.

[0419] Step 5:

[0420] The device displays the details of the task assigned to the user. This includes the task content, deadline, and required resources, allowing the user to see the actions required.

[0421] Step 6:

[0422] The server monitors task progress in real time and collects progress data. The server analyzes the status of each task and reallocates resources and adjusts task priorities as needed.

[0423] Step 7:

[0424] When a task is completed, the server checks the results and records the tasks handled by the artificial intelligence agent for process automation. This contributes to improving efficiency in future operations.

[0425] Step 8:

[0426] The terminal notifies the user of completed tasks and progress information, supporting performance evaluation and transition to the next step. This increases transparency and efficiency throughout the entire workflow.

[0427] (Example 1)

[0428] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0429] In business operations, it is difficult for managers and users to efficiently break down high-level work instructions into tasks and appropriately allocate them to humans and artificial intelligence. Furthermore, there is a lack of mechanisms for monitoring task progress in real time and for appropriately reallocating resources and adjusting priorities. As a result, operational efficiency declines, hindering productivity improvements.

[0430] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0431] In this invention, the server includes natural language processing means for automatically analyzing work instructions and breaking them down into generalized tasks; assignment means for appropriately allocating the tasks to human users and artificial intelligence agents; progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and means for providing an interface for users to input work instructions and optimizing the overall user experience. This enables efficient analysis of work instructions, appropriate task allocation, and real-time monitoring of progress, thereby maximizing productivity.

[0432] "Natural language processing means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into generalized tasks.

[0433] "Assignment means" refers to technology that has the function of appropriately distributing extracted tasks to human users and artificial intelligence agents.

[0434] A "progress management system" is a technology that has the function of monitoring the progress of tasks in real time and adjusting priorities and reallocating resources as needed.

[0435] "Means of providing an interface" refers to technologies that provide input and display methods for users to input work instructions and optimize the overall user experience.

[0436] A "storage method" is a technology that has the function of recording tasks completed by an artificial intelligence agent as process automation and saving data for use in similar tasks in the future.

[0437] "Means for generating feedback" refers to technologies for creating information that contributes to optimizing business processes based on generated data.

[0438] This invention is a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. The server analyzes work instructions using natural language processing technology. Specifically, the server uses natural language processing software such as "spaCy" or "BERT" to understand the content of the work instructions and convert them into structured tasks. Through this process, the instructions are broken down into more specific tasks.

[0439] The server can use task management tools such as "JIRA" and "Trello" for task distribution. This allows the server to assign extracted tasks to artificial intelligence agents and human users. Furthermore, the server utilizes data analysis tools such as "pandas" and "NumPy" to support the AI ​​agents in efficiently executing specified data processing tasks.

[0440] The terminal provides an intuitive interface for users to input work instructions. The terminal utilizes front-end frameworks such as "React" and "Vue.js," allowing users to easily input information. Furthermore, task progress and related notifications are presented to the user in real time through this terminal.

[0441] As a concrete example, the server receives a task instruction to "conduct market research for a new project and report the analysis results," and then performs the analysis. This instruction is broken down into two tasks: "collecting market data" and "creating a report," which are then assigned to an AI agent and a user, respectively. The AI ​​agent collects market data using "pandas," and the user creates the report using "Google Docs" or similar tools.

[0442] Examples of prompts to input into a generative AI model include the following:

[0443] "Conduct market research for the new project and report the analysis results. Break down the instructions into appropriate tasks and assign them to the appropriate individuals."

[0444] As described above, this system is optimized to improve operational efficiency and has a structure that enables the execution of tasks under specific conditions.

[0445] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0446] Step 1:

[0447] The server receives high-level work instructions from users through terminals. The input consists of work instructions entered by the user using an interface. The server acquires these instructions as input data. Next, the server analyzes the content of the instructions using "spaCy" or "BERT" and breaks them down into main tasks. Through this analysis, the instructions are transformed into generalized tasks, and task identification is obtained as output.

[0448] Step 2:

[0449] The server appropriately structures the tasks obtained from the analysis. These structured tasks are then assigned to human users or artificial intelligence agents. The input is the identification of tasks obtained in step 1, and the output is the assignment of specific tasks to users and agents. The server uses tools such as "JIRA" or "Trello" to consider the resources and skills required to perform each task.

[0450] Step 3:

[0451] The artificial intelligence agent receives tasks assigned to it from a server. The input is task assignment information from the server, and the agent uses tools such as "pandas" and "NumPy" to collect and process the data. This processing yields specific data and results as output.

[0452] Step 4:

[0453] The terminal provides the user with progress updates and related notifications for tasks assigned to them. The input is task progress information from the server. The terminal uses frameworks such as React and Vue.js to visually present this information to the user through an intuitive and user-friendly interface. The user then performs tasks such as creating reports based on the provided information.

[0454] Step 5:

[0455] The server monitors progress information from users and agents in real time, and readjusts resource allocation and priorities. Input is real-time data on the status of each task, and output is an optimized process. At this stage, the server optimizes tasks and ensures improved overall system productivity.

[0456] (Application Example 1)

[0457] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0458] With technological advancements, there is a growing need for equipment and transportation systems to process increasingly complex instructions in real time and to respond dynamically to crew demands. However, currently, understanding operational orders and executing tasks flexibly based on them is difficult, often hindering efficient operation. In particular, optimizing operational routes and dynamically adjusting transport equipment presents challenges in automatically responding to increasingly diverse instructions. Therefore, there is a need for a system that can analyze operational orders entered in natural language, automatically break down tasks, and make appropriate decisions.

[0459] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0460] In this invention, the server includes a natural language processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human users and artificial intelligence agents; a progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; an analysis means capable of understanding operation orders in natural language and automatically breaking them down into multiple tasks to be performed by transport equipment; and a decision means for understanding crew requests and dynamically optimizing the operation route. This enables the equipment in operation to respond flexibly to crew requests and complex instructions in real time, achieving efficient operation.

[0461] "Work instructions" are instructions that specify the actions and procedures necessary to achieve a particular objective or task.

[0462] "Natural language processing methods" refer to technologies and techniques for enabling computers to understand and analyze natural language used by humans in everyday life.

[0463] An "assignment mechanism" is a system that distributes analyzed tasks to the most suitable agent or user and issues instructions to them.

[0464] A "progress management tool" is a function that allows for real-time monitoring of task progress and, as necessary, reprioritization of tasks and adjustment of resources.

[0465] An "operation order" is a set of instructions for transport equipment to operate along a specific route or in a particular manner.

[0466] An "analysis tool" is a mechanism for breaking down input information or instructions into structured tasks.

[0467] "Decision-making tools" refer to functions that enable the selection or execution of the optimal course of action depending on the situation.

[0468] This invention is a system for transporting equipment and means of transport that analyzes instructions input in natural language, efficiently breaks them down into multiple tasks, and executes them. The system consists of three main components: a server, a terminal, and a user.

[0469] The server utilizes natural language processing technologies such as Google Cloud Natural Language API and AWS Comprehend to automatically analyze work instructions and break them down into specific tasks. This enables advanced natural language analysis and task structuring. The analyzed tasks are then assigned to the most suitable artificial intelligence agent or human. Throughout this process, the server monitors progress in real time, adjusting priorities and reallocating resources as needed.

[0470] The terminal provides an interface that allows users to input operational commands in natural language using a smartphone or head-mounted display. Based on the input commands, users can intuitively check various progress statuses and notifications during operation. In addition, the operational route is automatically optimized and suggested based on the user's requests. In this process, past data is utilized to provide efficient suggestions, allowing users to consistently select the optimal route.

[0471] For example, if a user riding in an autonomous vehicle enters a request through the interface such as, "I want to go to my destination next Thursday at 9:00 AM and stop at a cafe along the way," this request is analyzed by the server, and the optimal route is calculated in real time. This calculation takes into account the user's preferences and past route data.

[0472] Examples of prompt statements to input into a generative AI model are as follows:

[0473] "The user wishes to visit their destination at 9:00 AM on Thursday, November 2023. They also wish to have breakfast at a cafe along the way. Please suggest the best route."

[0474] This system enables transport equipment and means of transportation to operate automatically and efficiently in response to real-time user requests.

[0475] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0476] Step 1:

[0477] Users input operational commands in natural language through a smartphone or head-mounted display interface. These commands include operational requests such as specific destinations or points of interest along the route. This input serves as the starting point for the system's processing.

[0478] Step 2:

[0479] The server receives this natural language command. Next, using the Google Cloud Natural Language API, it interprets the command and breaks it down into several specific tasks. The input is the command statement from the user, and the output is each of the decomposed tasks. At this stage, the tasks are stored as text data.

[0480] Step 3:

[0481] The server assigns each task to an executable process based on the individual tasks it receives. It assigns AI programs to tasks that can be handled by artificial intelligence agents, and notifies the user of tasks requiring human intervention. The input is a list of broken-down tasks, and the output is the assignment to specific personnel or agents.

[0482] Step 4:

[0483] The server uses Google Firebase to monitor task progress in real time. It continuously collects progress data and updates the database as needed. The input is event data related to the task's progress, and the output is a dataset of the latest progress information.

[0484] Step 5:

[0485] The server automatically prioritizes and reallocates resources as needed based on data from Firebase. It optimizes the entire system in response to task progress and new requests. Inputs are progress information and resource usage, and outputs are updated priorities and resource allocations.

[0486] Step 6:

[0487] The terminal visually displays the route and progress information obtained from the server to the user. This allows the user to check real-time information about the operation and provide feedback as needed. The input is visualized progress data from the server, and the output is displayed information in a format that is easy for the user to understand.

[0488] This series of processes makes it possible to execute user operation requests efficiently and optimally.

[0489] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0490] This invention relates to a system that automatically analyzes work instructions, breaks them down into specific tasks, and effectively assigns them to users and artificial intelligence agents. In particular, by incorporating an emotion engine, this system recognizes the user's emotional state in real time, thereby improving work efficiency. The embodiments thereof are described below.

[0491] The server receives work instructions provided by the user and analyzes them using a natural language processing module. The analyzed instructions are broken down into individual tasks, and each task is assigned to the most suitable resource, namely a human user or an artificial intelligence agent.

[0492] In this process, the emotion engine plays a crucial role. Specifically, an emotion recognition function installed on the device analyzes the user's facial expressions and voice, and evaluates their emotional state in real time. The server can use the data obtained from the emotion engine to adjust task priorities based on the user's stress and fatigue levels.

[0493] For example, when a user inputs the instruction "Create a market research report for a new product by next week" via a terminal, the server uses natural language processing to break down this instruction into tasks such as "collect market data," "design the report," and "present the results." If the emotion engine detects a high-stress state at this time, the server can re-evaluate the order of the tasks and adjust the allocation to reduce the burden on the user.

[0494] Furthermore, tasks completed by the artificial intelligence agent are recorded in a database as process automation and efficiently reused for future tasks. This record is fed back to the user via the terminal, improving the overall transparency of the work.

[0495] The server also analyzes users' past emotional data and suggests optimal strategies for task allocation and progress, thereby helping to continuously improve work efficiency. This approach makes it possible to simultaneously achieve increased productivity and reduced psychological burden in the work environment.

[0496] The following describes the processing flow.

[0497] Step 1:

[0498] The user uses a terminal to input high-level work instructions. The terminal prepares to send the user's instructions as digital data to the server.

[0499] Step 2:

[0500] The server inputs the work instructions received from the terminal into a natural language processing module and analyzes their content. Specifically, it analyzes the instruction text to extract task keywords and related context.

[0501] Step 3:

[0502] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide "market research" into specific tasks such as "data collection," "competitor analysis," and "report creation of analysis results."

[0503] Step 4:

[0504] The device uses an emotion engine to detect the user's emotional state. It analyzes data such as voice tone, facial expressions, and behavioral patterns to evaluate the current emotional state.

[0505] Step 5:

[0506] The server uses data obtained from the emotion engine to optimize task allocation. For example, if a user is experiencing high stress levels, it adjusts the assignment to prioritize lighter tasks.

[0507] Step 6:

[0508] The server notifies the AI ​​agent and human users of the assigned tasks and updates progress information on each terminal in real time.

[0509] Step 7:

[0510] The device provides the user with feedback on task completion and progress. This includes the next steps to take and other helpful information for the user.

[0511] Step 8:

[0512] The server records completed tasks as process automations and saves them to a database. This record can be reused and utilized for similar tasks in the future.

[0513] Step 9:

[0514] The server analyzes the user's work patterns using past sentiment data and proposes an optimal task strategy based on this analysis. The terminal then presents the user with an action plan based on the new strategy.

[0515] (Example 2)

[0516] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0517] In today's work environment, effectively analyzing work instructions and breaking them down into specific tasks is difficult. Furthermore, there is a need to allocate tasks to the optimal resources while effectively managing task progress, taking into account the user's emotional state. A system is needed to streamline this process, reduce the psychological burden on users, and improve work efficiency.

[0518] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0519] In this invention, the server includes a language analysis means for automatically analyzing work instructions and breaking them down into specific tasks, an allocation means for appropriately assigning the tasks to human users and machine intelligence agents, and an emotion recognition means for analyzing the user's emotional state in real time and adjusting the priority of tasks. This makes it possible not only to effectively break down work instructions into tasks and allocate them to the optimal resources, but also to adjust the priority of tasks while taking into account the user's emotional state, thereby improving work efficiency and reducing psychological burden.

[0520] "Language analysis means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0521] "Distribution means" refers to technology that provides the function of appropriately allocating the analyzed tasks to human users and machine intelligence agents.

[0522] "Emotion recognition means" refers to technology that analyzes a user's emotional state in real time and provides functions to influence the prioritization and allocation of tasks.

[0523] A "progress management system" is a technology that provides the functionality to monitor the progress of work and adjust priorities or reallocate resources as needed.

[0524] "Storage means" refers to technology that automates the procedures of completed tasks by a machine intelligence agent, stores them, and utilizes them for similar tasks in the future.

[0525] "Structure" refers to the interface that allows users to efficiently perform a series of operations, from inputting work instructions to receiving those instructions and receiving progress notifications.

[0526] In an embodiment of this invention, the server receives a work instruction and analyzes the instruction using a language analysis means. For this purpose, the server can utilize common APIs and frameworks as natural language processing technologies (for example, OpenAI's language model API or other natural language processing tools). Through this analysis, the work instruction is broken down into specific tasks.

[0527] The server then uses allocation mechanisms to distribute the analyzed work to the most suitable resources. These resources include human users and machine intelligence agents. Resource selection is determined based on the nature of the work and the user's capabilities and schedule.

[0528] The device is equipped with emotion recognition capabilities, which allow it to analyze the user's facial expressions and voice. This could involve using input devices such as cameras and microphones. For example, by utilizing facial recognition and voice analysis technologies, the device could monitor the user's stress and fatigue levels in real time.

[0529] When a user enters a high-level task instruction such as "Create a market research report for a new product," the server receives it and uses its natural language processing tools to break down the instruction into specific tasks (e.g., "Collect market data," "Design the report," "Present the results").

[0530] If emotion recognition reveals that a user is in a high-stress state, the server is designed to automatically adjust allocation and task priorities to reduce the burden. For example, specific tasks can be assigned to machine intelligence agents to alleviate the burden.

[0531] Furthermore, the server uses storage methods to save a history of completed tasks and assignments to a database. This historical data is used as a knowledge base to efficiently perform similar tasks in the future.

[0532] This system provides support to organizations and individuals to efficiently carry out their work through a user interface. A possible example of a specific prompt might be, "Break down the work instructions into tasks and allocate them to the most suitable resources based on the user's emotional state." This prompt prompts the system to perform actions in accordance with the user's needs.

[0533] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0534] Step 1:

[0535] The server receives work instructions from the user. The server receives work instructions in natural language entered by the user on a terminal as input. The server stores these work instructions as data and prepares them for analysis. The output is work instruction data that can be analyzed.

[0536] Step 2:

[0537] The server analyzes the received work instructions using language analysis tools. The input is the work instruction data saved in step 1. This analysis includes a process of converting the instruction content into a structured data format using natural language processing tools. The output is structured data broken down into specific tasks.

[0538] Step 3:

[0539] The server allocates the analyzed tasks to appropriate resources using allocation mechanisms. It receives structured data generated in step 2 as input. For each task, the server selects the most suitable human user or machine intelligence agent and assigns the task. The output is resource allocation information for each task.

[0540] Step 4:

[0541] The device analyzes the user's emotional state using emotion recognition technology. It uses data from the user's facial expressions and voice, obtained from a camera and microphone connected to the device, as input. The device then runs emotion analysis software and outputs the user's emotional state as numerical data.

[0542] Step 5:

[0543] The server adjusts task priorities based on emotion recognition data. The input is the user's emotional state data obtained from step 4. The server reconfigures task priorities and reallocates resources according to the user's stress and fatigue levels. The output is the adjusted priority list and updated resource allocation information.

[0544] Step 6:

[0545] The server records completed work and its results in a database using storage methods. The input is the work history data obtained as a result of steps 3 and 5. This updates the database as a knowledge base that can be used in the future. The output is the recorded work history and performance data.

[0546] Step 7:

[0547] The user receives overall progress and feedback through the terminal. Inputs include progress data and adjustment information based on emotional changes transmitted from the server. The terminal displays this data clearly to the user, providing transparency to the work progress. Output is visual feedback information provided to the user.

[0548] (Application Example 2)

[0549] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0550] In operations, the emotions and psychological state of workers often affect the efficiency and productivity of their work. However, it is difficult to allocate tasks appropriately while taking these factors into account, which can result in decreased productivity and increased workload for workers. Furthermore, there is a lack of efficient systems for effectively utilizing data from past tasks and reflecting it in future operations.

[0551] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0552] In this invention, the server includes a text processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human operators and artificial intelligence agents; a management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and a recognition means for sensing the psychological state of workers and optimizing task assignment based on that data. This enables efficient task assignment that takes into account the emotional state of workers and improves the transparency of operations.

[0553] A "text processing means" is a module that has the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0554] An "assignment mechanism" is a system that performs the process of appropriately assigning analyzed tasks to human operators or artificial intelligence agents.

[0555] A "management tool" is a system that has the function of monitoring task progress in real time, adjusting priorities as needed, and reallocating resources.

[0556] A "recognition means" is a module that senses the psychological state of workers from their facial expressions and voices, and optimizes task assignment based on that data.

[0557] An "artificial intelligence agent" is an automation system or program designed to efficiently perform a specific task.

[0558] This invention is a system that utilizes a combination of technologies to maximize operational efficiency. Specifically, the server receives work instructions, analyzes them using natural language processing, and breaks them down into multiple tasks. Furthermore, it is possible to evaluate the user's psychological state using emotion recognition functionality installed on the terminal. Using this data, the server performs the optimal task allocation.

[0559] The appropriate assignment of tasks to human operators and artificial intelligence agents is carried out through assignment mechanisms. In this process, the system manages task progress in real time and reallocates priorities and resources as needed. This reduces stress and psychological burden on users and improves the overall efficiency of the system.

[0560] A concrete application example is a system that improves the work efficiency of factory workers. For instance, if a recognition system detects that a worker is in a high-stress state, the server optimizes that worker's tasks and reallocates them to other resources. In this way, the overall workflow of the factory proceeds smoothly.

[0561] The following prompt statements can be considered as examples of how generative AI models can be used.

[0562] "Please optimize task assignments, taking into account the psychological state of the workers. High stress levels have been detected."

[0563] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0564] Step 1:

[0565] The server receives work instructions from the user as input. Using a natural language processing module, it analyzes the instructions, breaks them down into specific tasks, and generates that information as output. Through text analysis, the instructions are classified into meaningful tasks such as "collect market data" or "design reports."

[0566] Step 2:

[0567] The device captures the user's facial expressions and voice as real-time input and uses emotion recognition to evaluate their psychological state. This data is analyzed by an emotion engine, which quantifies and outputs stress and fatigue levels. For example, it detects changes in the user's voice tone and facial expressions and provides a numerical value indicating a high-stress state.

[0568] Step 3:

[0569] The server receives the acquired emotional data as input and optimizes tasks through its assignment mechanism. Specifically, it adjusts task priorities based on the emotional data and re-evaluates the assignment to human operators or artificial intelligence agents. This process seeks task placement that reduces the user's psychological burden and improves productivity.

[0570] Step 4:

[0571] The server monitors task progress in real time and provides users with feedback on progress and any issues. It collects progress data as input and uses it to adjust priorities and reallocate resources. This monitoring ensures continuous, real-time resource optimization.

[0572] Step 5:

[0573] The artificial intelligence agent executes assigned tasks and reports completion data to the server. Task completion status is recorded as process automation and used to efficiently handle similar tasks in the future. This data storage process enhances the overall operational efficiency of the system.

[0574] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0575] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0576] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0577] [Fourth Embodiment]

[0578] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0579] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0580] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0581] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0582] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0583] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0584] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0585] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0586] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0587] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0588] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0589] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0590] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0591] This invention provides a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. Specifically, the server utilizes natural language processing technology to automatically understand complex work instructions and convert them into structured tasks. The server analyzes high-level instructions received from managers and assigns the extracted tasks to artificial intelligence agents or human users.

[0592] The server executes algorithms for the artificial intelligence agent and generates operational guidelines and notifications required by humans. During this process, the server monitors task progress, checking progress in real time and reallocating or prioritizing resources as needed to ensure efficient execution.

[0593] The terminal provides an interface that allows users to easily input high-level work instructions and displays progress and related notifications. Through this interface, users can understand the current status of their work and perform intuitive operations when necessary.

[0594] As a concrete example, if the server receives the instruction to "conduct market research for a new project and report the analysis results," it analyzes the instruction and breaks it down into two main tasks: "collecting market data" and "creating a report." The AI ​​agent is responsible for data collection, and the user creates a report based on the results. Throughout the process, the server constantly monitors the progress of these tasks and updates the results in real time.

[0595] Furthermore, specific tasks completed by the AI ​​agent are recorded as process automations, allowing for reuse in similar tasks in the future. In this way, the system improves productivity and enables efficient work with fewer people. This invention dramatically improves the operational efficiency of companies and allows managers to have more free time to make strategic decisions.

[0596] The following describes the processing flow.

[0597] Step 1:

[0598] The user inputs high-level work instructions using the terminal's interface. The terminal then converts the input instructions into a data format and prepares to send them to the server.

[0599] Step 2:

[0600] The server inputs the work instructions received from the terminal into a natural language processing module, which then analyzes the instructions. This process involves extracting specific tasks through keyword and contextual analysis.

[0601] Step 3:

[0602] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide a market research instruction into specific tasks such as "data collection" and "analysis report."

[0603] Step 4:

[0604] The server assigns the broken-down tasks to human users and artificial intelligence agents. Here, the optimal assignment is made based on the nature of each task and the capacity of the resources.

[0605] Step 5:

[0606] The device displays the details of the task assigned to the user. This includes the task content, deadline, and required resources, allowing the user to see the actions required.

[0607] Step 6:

[0608] The server monitors task progress in real time and collects progress data. The server analyzes the status of each task and reallocates resources and adjusts task priorities as needed.

[0609] Step 7:

[0610] When a task is completed, the server checks the results and records the tasks handled by the artificial intelligence agent for process automation. This contributes to improving efficiency in future operations.

[0611] Step 8:

[0612] The terminal notifies the user of completed tasks and progress information, supporting performance evaluation and transition to the next step. This increases transparency and efficiency throughout the entire workflow.

[0613] (Example 1)

[0614] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0615] In business operations, it is difficult for managers and users to efficiently break down high-level work instructions into tasks and appropriately allocate them to humans and artificial intelligence. Furthermore, there is a lack of mechanisms for monitoring task progress in real time and for appropriately reallocating resources and adjusting priorities. As a result, operational efficiency declines, hindering productivity improvements.

[0616] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0617] In this invention, the server includes natural language processing means for automatically analyzing work instructions and breaking them down into generalized tasks; assignment means for appropriately allocating the tasks to human users and artificial intelligence agents; progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and means for providing an interface for users to input work instructions and optimizing the overall user experience. This enables efficient analysis of work instructions, appropriate task allocation, and real-time monitoring of progress, thereby maximizing productivity.

[0618] "Natural language processing means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into generalized tasks.

[0619] "Assignment means" refers to technology that has the function of appropriately distributing extracted tasks to human users and artificial intelligence agents.

[0620] A "progress management system" is a technology that has the function of monitoring the progress of tasks in real time and adjusting priorities and reallocating resources as needed.

[0621] "Means of providing an interface" refers to technologies that provide input and display methods for users to input work instructions and optimize the overall user experience.

[0622] A "storage method" is a technology that has the function of recording tasks completed by an artificial intelligence agent as process automation and saving data for use in similar tasks in the future.

[0623] "Means for generating feedback" refers to technologies for creating information that contributes to optimizing business processes based on generated data.

[0624] This invention is a system that maximizes business productivity by efficiently analyzing work instructions, breaking them down into tasks, and allocating them to appropriate resources. The server analyzes work instructions using natural language processing technology. Specifically, the server uses natural language processing software such as "spaCy" or "BERT" to understand the content of the work instructions and convert them into structured tasks. Through this process, the instructions are broken down into more specific tasks.

[0625] The server can use task management tools such as "JIRA" and "Trello" for task distribution. This allows the server to assign extracted tasks to artificial intelligence agents and human users. Furthermore, the server utilizes data analysis tools such as "pandas" and "NumPy" to support the AI ​​agents in efficiently executing specified data processing tasks.

[0626] The terminal provides an intuitive interface for users to input work instructions. The terminal utilizes front-end frameworks such as "React" and "Vue.js," allowing users to easily input information. Furthermore, task progress and related notifications are presented to the user in real time through this terminal.

[0627] As a concrete example, the server receives a task instruction to "conduct market research for a new project and report the analysis results," and then performs the analysis. This instruction is broken down into two tasks: "collecting market data" and "creating a report," which are then assigned to an AI agent and a user, respectively. The AI ​​agent collects market data using "pandas," and the user creates the report using "Google Docs" or similar tools.

[0628] Examples of prompts to input into a generative AI model include the following:

[0629] "Conduct market research for the new project and report the analysis results. Break down the instructions into appropriate tasks and assign them to the appropriate individuals."

[0630] As described above, this system is optimized to improve operational efficiency and has a structure that enables the execution of tasks under specific conditions.

[0631] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0632] Step 1:

[0633] The server receives high-level work instructions from users through terminals. The input consists of work instructions entered by the user using an interface. The server acquires these instructions as input data. Next, the server analyzes the content of the instructions using "spaCy" or "BERT" and breaks them down into main tasks. Through this analysis, the instructions are transformed into generalized tasks, and task identification is obtained as output.

[0634] Step 2:

[0635] The server appropriately structures the tasks obtained from the analysis. These structured tasks are then assigned to human users or artificial intelligence agents. The input is the identification of tasks obtained in step 1, and the output is the assignment of specific tasks to users and agents. The server uses tools such as "JIRA" or "Trello" to consider the resources and skills required to perform each task.

[0636] Step 3:

[0637] The artificial intelligence agent receives tasks assigned to it from a server. The input is task assignment information from the server, and the agent uses tools such as "pandas" and "NumPy" to collect and process the data. This processing yields specific data and results as output.

[0638] Step 4:

[0639] The terminal provides the user with progress updates and related notifications for tasks assigned to them. The input is task progress information from the server. The terminal uses frameworks such as React and Vue.js to visually present this information to the user through an intuitive and user-friendly interface. The user then performs tasks such as creating reports based on the provided information.

[0640] Step 5:

[0641] The server monitors progress information from users and agents in real time, and readjusts resource allocation and priorities. Input is real-time data on the status of each task, and output is an optimized process. At this stage, the server optimizes tasks and ensures improved overall system productivity.

[0642] (Application Example 1)

[0643] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] With technological advancements, there is a growing need for equipment and transportation systems to process increasingly complex instructions in real time and to respond dynamically to crew demands. However, currently, understanding operational orders and executing tasks flexibly based on them is difficult, often hindering efficient operation. In particular, optimizing operational routes and dynamically adjusting transport equipment presents challenges in automatically responding to increasingly diverse instructions. Therefore, there is a need for a system that can analyze operational orders entered in natural language, automatically break down tasks, and make appropriate decisions.

[0645] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0646] In this invention, the server includes a natural language processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human users and artificial intelligence agents; a progress management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; an analysis means capable of understanding operation orders in natural language and automatically breaking them down into multiple tasks to be performed by transport equipment; and a decision means for understanding crew requests and dynamically optimizing the operation route. This enables the equipment in operation to respond flexibly to crew requests and complex instructions in real time, achieving efficient operation.

[0647] "Work instructions" are instructions that specify the actions and procedures necessary to achieve a particular objective or task.

[0648] "Natural language processing methods" refer to technologies and techniques for enabling computers to understand and analyze natural language used by humans in everyday life.

[0649] An "assignment mechanism" is a system that distributes analyzed tasks to the most suitable agent or user and issues instructions to them.

[0650] A "progress management tool" is a function that allows for real-time monitoring of task progress and, as necessary, reprioritization of tasks and adjustment of resources.

[0651] An "operation order" is a set of instructions for transport equipment to operate along a specific route or in a particular manner.

[0652] An "analysis tool" is a mechanism for breaking down input information or instructions into structured tasks.

[0653] "Decision-making tools" refer to functions that enable the selection or execution of the optimal course of action depending on the situation.

[0654] This invention is a system for transporting equipment and means of transport that analyzes instructions input in natural language, efficiently breaks them down into multiple tasks, and executes them. The system consists of three main components: a server, a terminal, and a user.

[0655] The server utilizes natural language processing technologies such as Google Cloud Natural Language API and AWS Comprehend to automatically analyze work instructions and break them down into specific tasks. This enables advanced natural language analysis and task structuring. The analyzed tasks are then assigned to the most suitable artificial intelligence agent or human. Throughout this process, the server monitors progress in real time, adjusting priorities and reallocating resources as needed.

[0656] The terminal provides an interface that allows users to input operational commands in natural language using a smartphone or head-mounted display. Based on the input commands, users can intuitively check various progress statuses and notifications during operation. In addition, the operational route is automatically optimized and suggested based on the user's requests. In this process, past data is utilized to provide efficient suggestions, allowing users to consistently select the optimal route.

[0657] For example, if a user riding in an autonomous vehicle enters a request through the interface such as, "I want to go to my destination next Thursday at 9:00 AM and stop at a cafe along the way," this request is analyzed by the server, and the optimal route is calculated in real time. This calculation takes into account the user's preferences and past route data.

[0658] Examples of prompt statements to input into a generative AI model are as follows:

[0659] "The user wishes to visit their destination at 9:00 AM on Thursday, November 2023. They also wish to have breakfast at a cafe along the way. Please suggest the best route."

[0660] This system enables transport equipment and means of transportation to operate automatically and efficiently in response to real-time user requests.

[0661] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0662] Step 1:

[0663] Users input operational commands in natural language through a smartphone or head-mounted display interface. These commands include operational requests such as specific destinations or points of interest along the route. This input serves as the starting point for the system's processing.

[0664] Step 2:

[0665] The server receives this natural language command. Next, using the Google Cloud Natural Language API, it interprets the command and breaks it down into several specific tasks. The input is the command statement from the user, and the output is each of the decomposed tasks. At this stage, the tasks are stored as text data.

[0666] Step 3:

[0667] The server assigns each task to an executable process based on the individual tasks it receives. It assigns AI programs to tasks that can be handled by artificial intelligence agents, and notifies the user of tasks requiring human intervention. The input is a list of broken-down tasks, and the output is the assignment to specific personnel or agents.

[0668] Step 4:

[0669] The server uses Google Firebase to monitor task progress in real time. It continuously collects progress data and updates the database as needed. The input is event data related to the task's progress, and the output is a dataset of the latest progress information.

[0670] Step 5:

[0671] The server automatically prioritizes and reallocates resources as needed based on data from Firebase. It optimizes the entire system in response to task progress and new requests. Inputs are progress information and resource usage, and outputs are updated priorities and resource allocations.

[0672] Step 6:

[0673] The terminal visually displays the route and progress information obtained from the server to the user. This allows the user to check real-time information about the operation and provide feedback as needed. The input is visualized progress data from the server, and the output is displayed information in a format that is easy for the user to understand.

[0674] This series of processes makes it possible to execute user operation requests efficiently and optimally.

[0675] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0676] This invention relates to a system that automatically analyzes work instructions, breaks them down into specific tasks, and effectively assigns them to users and artificial intelligence agents. In particular, by incorporating an emotion engine, this system recognizes the user's emotional state in real time, thereby improving work efficiency. The embodiments thereof are described below.

[0677] The server receives work instructions provided by the user and analyzes them using a natural language processing module. The analyzed instructions are broken down into individual tasks, and each task is assigned to the most suitable resource, namely a human user or an artificial intelligence agent.

[0678] In this process, the emotion engine plays a crucial role. Specifically, an emotion recognition function installed on the device analyzes the user's facial expressions and voice, and evaluates their emotional state in real time. The server can use the data obtained from the emotion engine to adjust task priorities based on the user's stress and fatigue levels.

[0679] For example, when a user inputs the instruction "Create a market research report for a new product by next week" via a terminal, the server uses natural language processing to break down this instruction into tasks such as "collect market data," "design the report," and "present the results." If the emotion engine detects a high-stress state at this time, the server can re-evaluate the order of the tasks and adjust the allocation to reduce the burden on the user.

[0680] Furthermore, tasks completed by the artificial intelligence agent are recorded in a database as process automation and efficiently reused for future tasks. This record is fed back to the user via the terminal, improving the overall transparency of the work.

[0681] The server also analyzes users' past emotional data and suggests optimal strategies for task allocation and progress, thereby helping to continuously improve work efficiency. This approach makes it possible to simultaneously achieve increased productivity and reduced psychological burden in the work environment.

[0682] The following describes the processing flow.

[0683] Step 1:

[0684] The user uses a terminal to input high-level work instructions. The terminal prepares to send the user's instructions as digital data to the server.

[0685] Step 2:

[0686] The server inputs the work instructions received from the terminal into a natural language processing module and analyzes their content. Specifically, it analyzes the instruction text to extract task keywords and related context.

[0687] Step 3:

[0688] The server breaks down the work instructions into specific tasks based on the analysis results. For example, it might divide "market research" into specific tasks such as "data collection," "competitor analysis," and "report creation of analysis results."

[0689] Step 4:

[0690] The device uses an emotion engine to detect the user's emotional state. It analyzes data such as voice tone, facial expressions, and behavioral patterns to evaluate the current emotional state.

[0691] Step 5:

[0692] The server uses data obtained from the emotion engine to optimize task allocation. For example, if a user is experiencing high stress levels, it adjusts the assignment to prioritize lighter tasks.

[0693] Step 6:

[0694] The server notifies the AI ​​agent and human users of the assigned tasks and updates progress information on each terminal in real time.

[0695] Step 7:

[0696] The device provides the user with feedback on task completion and progress. This includes the next steps to take and other helpful information for the user.

[0697] Step 8:

[0698] The server records completed tasks as process automations and saves them to a database. This record can be reused and utilized for similar tasks in the future.

[0699] Step 9:

[0700] The server analyzes the user's work patterns using past sentiment data and proposes an optimal task strategy based on this analysis. The terminal then presents the user with an action plan based on the new strategy.

[0701] (Example 2)

[0702] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0703] In today's work environment, effectively analyzing work instructions and breaking them down into specific tasks is difficult. Furthermore, there is a need to allocate tasks to the optimal resources while effectively managing task progress, taking into account the user's emotional state. A system is needed to streamline this process, reduce the psychological burden on users, and improve work efficiency.

[0704] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0705] In this invention, the server includes a language analysis means for automatically analyzing work instructions and breaking them down into specific tasks, an allocation means for appropriately assigning the tasks to human users and machine intelligence agents, and an emotion recognition means for analyzing the user's emotional state in real time and adjusting the priority of tasks. This makes it possible not only to effectively break down work instructions into tasks and allocate them to the optimal resources, but also to adjust the priority of tasks while taking into account the user's emotional state, thereby improving work efficiency and reducing psychological burden.

[0706] "Language analysis means" refers to technology that provides the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0707] "Distribution means" refers to technology that provides the function of appropriately allocating the analyzed tasks to human users and machine intelligence agents.

[0708] "Emotion recognition means" refers to technology that analyzes a user's emotional state in real time and provides functions to influence the prioritization and allocation of tasks.

[0709] A "progress management system" is a technology that provides the functionality to monitor the progress of work and adjust priorities or reallocate resources as needed.

[0710] "Storage means" refers to technology that automates the procedures of completed tasks by a machine intelligence agent, stores them, and utilizes them for similar tasks in the future.

[0711] "Structure" refers to the interface that allows users to efficiently perform a series of operations, from inputting work instructions to receiving those instructions and receiving progress notifications.

[0712] In an embodiment of this invention, the server receives a work instruction and analyzes the instruction using a language analysis means. For this purpose, the server can utilize common APIs and frameworks as natural language processing technologies (for example, OpenAI's language model API or other natural language processing tools). Through this analysis, the work instruction is broken down into specific tasks.

[0713] The server then uses allocation mechanisms to distribute the analyzed work to the most suitable resources. These resources include human users and machine intelligence agents. Resource selection is determined based on the nature of the work and the user's capabilities and schedule.

[0714] The device is equipped with emotion recognition capabilities, which allow it to analyze the user's facial expressions and voice. This could involve using input devices such as cameras and microphones. For example, by utilizing facial recognition and voice analysis technologies, the device could monitor the user's stress and fatigue levels in real time.

[0715] When a user enters a high-level task instruction such as "Create a market research report for a new product," the server receives it and uses its natural language processing tools to break down the instruction into specific tasks (e.g., "Collect market data," "Design the report," "Present the results").

[0716] If emotion recognition reveals that a user is in a high-stress state, the server is designed to automatically adjust allocation and task priorities to reduce the burden. For example, specific tasks can be assigned to machine intelligence agents to alleviate the burden.

[0717] Furthermore, the server uses storage methods to save a history of completed tasks and assignments to a database. This historical data is used as a knowledge base to efficiently perform similar tasks in the future.

[0718] This system provides support to organizations and individuals to efficiently carry out their work through a user interface. A possible example of a specific prompt might be, "Break down the work instructions into tasks and allocate them to the most suitable resources based on the user's emotional state." This prompt prompts the system to perform actions in accordance with the user's needs.

[0719] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0720] Step 1:

[0721] The server receives work instructions from the user. The server receives work instructions in natural language entered by the user on a terminal as input. The server stores these work instructions as data and prepares them for analysis. The output is work instruction data that can be analyzed.

[0722] Step 2:

[0723] The server analyzes the received work instructions using language analysis tools. The input is the work instruction data saved in step 1. This analysis includes a process of converting the instruction content into a structured data format using natural language processing tools. The output is structured data broken down into specific tasks.

[0724] Step 3:

[0725] The server allocates the analyzed tasks to appropriate resources using allocation mechanisms. It receives structured data generated in step 2 as input. For each task, the server selects the most suitable human user or machine intelligence agent and assigns the task. The output is resource allocation information for each task.

[0726] Step 4:

[0727] The device analyzes the user's emotional state using emotion recognition technology. It uses data from the user's facial expressions and voice, obtained from a camera and microphone connected to the device, as input. The device then runs emotion analysis software and outputs the user's emotional state as numerical data.

[0728] Step 5:

[0729] The server adjusts task priorities based on emotion recognition data. The input is the user's emotional state data obtained from step 4. The server reconfigures task priorities and reallocates resources according to the user's stress and fatigue levels. The output is the adjusted priority list and updated resource allocation information.

[0730] Step 6:

[0731] The server records completed work and its results in a database using storage methods. The input is the work history data obtained as a result of steps 3 and 5. This updates the database as a knowledge base that can be used in the future. The output is the recorded work history and performance data.

[0732] Step 7:

[0733] The user receives overall progress and feedback through the terminal. Inputs include progress data and adjustment information based on emotional changes transmitted from the server. The terminal displays this data clearly to the user, providing transparency to the work progress. Output is visual feedback information provided to the user.

[0734] (Application Example 2)

[0735] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] In operations, the emotions and psychological state of workers often affect the efficiency and productivity of their work. However, it is difficult to allocate tasks appropriately while taking these factors into account, which can result in decreased productivity and increased workload for workers. Furthermore, there is a lack of efficient systems for effectively utilizing data from past tasks and reflecting it in future operations.

[0737] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0738] In this invention, the server includes a text processing means for automatically analyzing work instructions and breaking them down into specific tasks; an assignment means for appropriately distributing the tasks to human operators and artificial intelligence agents; a management means for monitoring task progress in real time, adjusting priorities, and reallocating resources; and a recognition means for sensing the psychological state of workers and optimizing task assignment based on that data. This enables efficient task assignment that takes into account the emotional state of workers and improves the transparency of operations.

[0739] A "text processing means" is a module that has the function of automatically analyzing work instructions and breaking them down into specific tasks.

[0740] An "assignment mechanism" is a system that performs the process of appropriately assigning analyzed tasks to human operators or artificial intelligence agents.

[0741] A "management tool" is a system that has the function of monitoring task progress in real time, adjusting priorities as needed, and reallocating resources.

[0742] A "recognition means" is a module that senses the psychological state of workers from their facial expressions and voices, and optimizes task assignment based on that data.

[0743] An "artificial intelligence agent" is an automation system or program designed to efficiently perform a specific task.

[0744] This invention is a system that utilizes a combination of technologies to maximize operational efficiency. Specifically, the server receives work instructions, analyzes them using natural language processing, and breaks them down into multiple tasks. Furthermore, it is possible to evaluate the user's psychological state using emotion recognition functionality installed on the terminal. Using this data, the server performs the optimal task allocation.

[0745] The appropriate assignment of tasks to human operators and artificial intelligence agents is carried out through assignment mechanisms. In this process, the system manages task progress in real time and reallocates priorities and resources as needed. This reduces stress and psychological burden on users and improves the overall efficiency of the system.

[0746] A concrete application example is a system that improves the work efficiency of factory workers. For instance, if a recognition system detects that a worker is in a high-stress state, the server optimizes that worker's tasks and reallocates them to other resources. In this way, the overall workflow of the factory proceeds smoothly.

[0747] The following prompt statements can be considered as examples of how generative AI models can be used.

[0748] "Please optimize task assignments, taking into account the psychological state of the workers. High stress levels have been detected."

[0749] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0750] Step 1:

[0751] The server receives work instructions from the user as input. Using a natural language processing module, it analyzes the instructions, breaks them down into specific tasks, and generates that information as output. Through text analysis, the instructions are classified into meaningful tasks such as "collect market data" or "design reports."

[0752] Step 2:

[0753] The device captures the user's facial expressions and voice as real-time input and uses emotion recognition to evaluate their psychological state. This data is analyzed by an emotion engine, which quantifies and outputs stress and fatigue levels. For example, it detects changes in the user's voice tone and facial expressions and provides a numerical value indicating a high-stress state.

[0754] Step 3:

[0755] The server receives the acquired emotional data as input and optimizes tasks through its assignment mechanism. Specifically, it adjusts task priorities based on the emotional data and re-evaluates the assignment to human operators or artificial intelligence agents. This process seeks task placement that reduces the user's psychological burden and improves productivity.

[0756] Step 4:

[0757] The server monitors task progress in real time and provides users with feedback on progress and any issues. It collects progress data as input and uses it to adjust priorities and reallocate resources. This monitoring ensures continuous, real-time resource optimization.

[0758] Step 5:

[0759] The artificial intelligence agent executes assigned tasks and reports completion data to the server. Task completion status is recorded as process automation and used to efficiently handle similar tasks in the future. This data storage process enhances the overall operational efficiency of the system.

[0760] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0761] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0762] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0763] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0764] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0765] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0766] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0767] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0768] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0769] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0770] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0771] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0772] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0773] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0774] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0775] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0776] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0777] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0778] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0779] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0780] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0781] The following is further disclosed regarding the embodiments described above.

[0782] (Claim 1)

[0783] A natural language processing tool that automatically analyzes work instructions and breaks them down into specific tasks,

[0784] Assignment means for appropriately distributing the aforementioned tasks to human users and artificial intelligence agents,

[0785] A progress management system that monitors task progress in real time, adjusts priorities, and reallocates resources,

[0786] A system that includes this.

[0787] (Claim 2)

[0788] The system according to claim 1, comprising a storage means for automating and accumulating tasks completed by an artificial intelligence agent and utilizing them for similar tasks in the future.

[0789] (Claim 3)

[0790] The system according to claim 1, comprising means for providing an interface for users to input high-level work instructions and for optimizing the overall user experience from receiving instructions to receiving progress notifications.

[0791] "Example 1"

[0792] (Claim 1)

[0793] A natural language processing tool that automatically analyzes work instructions and breaks them down into generalized tasks,

[0794] Assignment means for appropriately distributing the aforementioned tasks to human users and artificial intelligence agents,

[0795] A progress management system that monitors task progress in real time, adjusts priorities, and reallocates resources,

[0796] A means of providing an interface for users to input work instructions and optimizing the overall user experience,

[0797] A system that includes this.

[0798] (Claim 2)

[0799] The system according to claim 1, comprising a means for recording tasks completed by an artificial intelligence agent as process automation and for effectively utilizing them for similar tasks in the future.

[0800] (Claim 3)

[0801] The system according to claim 1, comprising means for generating feedback that contributes to the optimization of business processes based on generated data.

[0802] "Application Example 1"

[0803] (Claim 1)

[0804] A natural language processing tool that automatically analyzes work instructions and breaks them down into specific tasks,

[0805] Assignment means for appropriately distributing the aforementioned tasks to human users and artificial intelligence agents,

[0806] A progress management system that monitors task progress in real time, adjusts priorities, and reallocates resources,

[0807] An analysis means equipped with the ability to understand operational instructions in natural language and automatically break them down into multiple tasks to be performed by transport equipment,

[0808] A decision-making mechanism to understand crew requirements and dynamically optimize the route,

[0809] A system that includes this.

[0810] (Claim 2)

[0811] The system according to claim 1, comprising a storage means for automating and accumulating tasks completed by an artificial intelligence agent and utilizing them for similar tasks in the future.

[0812] (Claim 3)

[0813] The system according to claim 1, which includes a means to provide an interface for users to input high-level work instructions, optimize the overall user experience from receiving instructions to receiving progress notifications, and include a function to suggest the optimal transport route based on the input transport request.

[0814] "Example 2 of combining an emotion engine"

[0815] (Claim 1)

[0816] A language analysis tool that automatically analyzes work instructions and breaks them down into specific tasks,

[0817] Distribution means for appropriately assigning the aforementioned tasks to human users and machine intelligence agents,

[0818] An emotion recognition means that analyzes the user's emotional state in real time and adjusts the priority of tasks,

[0819] A progress management system that monitors the progress of work, adjusts priorities, and reallocates resources,

[0820] A system that includes this.

[0821] (Claim 2)

[0822] The system according to claim 1, comprising a means for accumulating data for automating the procedures of a machine intelligence agent and utilizing it for similar tasks in the future.

[0823] (Claim 3)

[0824] The system according to claim 1, comprising a structure for users to input higher-level work instructions and means for optimizing the overall user experience from receiving instructions to receiving progress notifications.

[0825] "Application example 2 when combining with an emotional engine"

[0826] (Claim 1)

[0827] A text processing method that automatically analyzes work instructions and breaks them down into specific tasks,

[0828] Assignment means for appropriately distributing the aforementioned tasks to human operators and artificial intelligence agents,

[0829] A management system that monitors task progress in real time, adjusts priorities, and reallocates resources,

[0830] A recognition means that senses the psychological state of workers and optimizes task assignment based on that data,

[0831] A system that includes this.

[0832] (Claim 2)

[0833] The system according to claim 1, comprising a storage means for accumulating and storing completed tasks by an artificial intelligence agent through process automation, and utilizing them for similar tasks in the future.

[0834] (Claim 3)

[0835] The system according to claim 1, which provides a platform for workers to input high-level work instructions and includes means for optimizing the overall user experience from receiving instructions to receiving progress notifications. [Explanation of symbols]

[0836] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A natural language processing tool that automatically analyzes work instructions and breaks them down into specific tasks, Assignment means for appropriately distributing the aforementioned tasks to human users and artificial intelligence agents, A progress management system that monitors task progress in real time, adjusts priorities, and reallocates resources, A system that includes this.

2. The system according to claim 1, comprising storage means for automating and accumulating tasks completed by an artificial intelligence agent and utilizing them for similar tasks in the future.

3. The system according to claim 1, comprising means for providing an interface for users to input high-level work instructions and optimizing the overall user experience from receiving instructions to receiving progress notifications.

Images