System

Abstract

To provide a system.SOLUTION: A system comprises: means for receiving an instruction from a user; means for analyzing the instruction from the user by natural language processing and determining whether the instruction from the user is any one of an office work proxy of meeting adjustment, meeting attendance, business trip arrangement, expense settlement, attendance, and material creation; and means for executing the office work proxy indicated by a determination result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While conventional business proxy systems can perform administrative tasks based on user instructions, they have difficulty performing more complex tasks such as attending meetings on behalf of others or sharing summaries after the event. They also lack the ability to properly analyze user instructions and select and execute the most appropriate tasks. [Means for solving the problem]

[0005] Based on user instructions, the system of the present invention handles administrative tasks such as meeting coordination, travel arrangements, expense settlement, attendance, and document preparation, as well as attending meetings on behalf of users and sharing summaries afterward. Furthermore, the system analyzes user instructions using natural language processing and selects and executes appropriate tasks. This reduces the user's workload and supports efficient work execution. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15]FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention has an interface for receiving instructions from a user. This interface supports formats that allow users to give instructions naturally, such as text input and voice input. For example, if a user instructs, "Please make travel arrangements for a conference in Tokyo next Monday," the system analyzes this instruction and selects the task of making travel arrangements.

[0029] "Example 2"

[0030] The system of the present invention includes modules for executing selected tasks. These modules have functions corresponding to various tasks. For example, if a user selects the task of arranging a business trip, the system will launch the business trip arrangement module and make the necessary arrangements for the business trip, such as reserving an airline ticket, booking a hotel, and arranging transportation.

[0031] "Example 3"

[0032] The system of the present invention can also attend meetings on behalf of users and share the summary after the meeting. For example, if a user requests, "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting," the system will launch the meeting proxy attendance module and join the meeting at the specified time. After the meeting ends, the system will summarize the contents of the meeting and share it with the user.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Receive instructions from the user through the interface. For example, receive an instruction such as "Please make travel arrangements for a meeting in Tokyo next Monday."

[0036] Step 2: The received instruction is analyzed using natural language processing. Through this analysis, the task of "business trip arrangements" is selected.

[0037] Step 3: The module corresponding to the selected task is started. In this case, the business trip arrangement module is started.

[0038] Step 4: The travel arrangements module makes the necessary arrangements for the business trip, such as booking air tickets, hotel reservations, and transportation.

[0039] "Example 2"

[0040] Step 1: Receive instructions from the user through the interface. For example, receive an instruction such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting."

[0041] Step 2: The received instruction is analyzed using natural language processing. This analysis results in the selection of the task "attend a meeting on behalf of another person."

[0042] Step 3: Start the module corresponding to the selected task. In this case, the conference proxy attendance module is started.

[0043] Step 4: The meeting proxy attendance module joins the meeting at the specified time.

[0044] Step 5: After the meeting is over, the meeting proxy attendance module summarizes the contents of the meeting.

[0045] Step 6: Share the summarized meeting content with users.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] Conventional administrative services require users to manually input instructions for various tasks, which does not improve work efficiency. Furthermore, the difficulty of properly analyzing user instructions and responding quickly results in a lack of reduction in workload. Furthermore, the lack of an interface that can accept instructions in natural language poses a problem of low user convenience.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes means having an interface for receiving instructions from a user via text input or voice input, means for analyzing the user's instructions using natural language processing technology, means for selecting an appropriate task based on the analysis results, and means for executing the selected task. This makes it possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute an appropriate task.

[0051] "User" means any person or entity that uses the System to issue instructions.

[0052] "Instructions" refer to requests or commands given by a user to a system.

[0053] "Interface" refers to the means by which a user inputs instructions into a system, including text input and voice input.

[0054] "Natural language processing technology" refers to technology for analyzing user instructions and understanding their meaning.

[0055] "Parsing" refers to the process of breaking down a user's instructions and extracting meaning.

[0056] "Business" refers to the tasks or work that a system performs based on user instructions.

[0057] "Selection" refers to the process of determining the appropriate task based on the analysis results.

[0058] "Execution" refers to the process of specifically carrying out the selected task.

[0059] "Administrative agency" refers to tasks such as meeting coordination, travel arrangements, expense settlement, attendance management, and document preparation that the system performs on behalf of the user.

[0060] "Meeting coordination" refers to the task of arranging the date and location of a meeting.

[0061] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[0062] "Expense settlement" refers to the work of calculating and settling users' expenses.

[0063] "Time attendance management" refers to the task of managing users' attendance and departure records.

[0064] "Document creation" refers to the task of creating documents and presentation materials based on user instructions.

[0065] "Proxy attendance" refers to the task of attending a meeting on behalf of a user.

[0066] "Summary sharing" refers to the task of summarizing the contents of a meeting and sharing it with users.

[0067] MODE FOR CARRYING OUT THE INVENTION

[0068] The present invention is a system that receives instructions from a user, analyzes them, and selects and executes an appropriate task. A specific embodiment of this system will be described below.

[0069] 1. Receive instructions from the user

[0070] Users input instructions using the device's interface. This interface supports text input and voice input, allowing users to give instructions naturally. For example, a user opens a smartphone app and enters text, such as "Please arrange travel for a meeting in Tokyo next Monday." In the case of voice input, the user speaks the same instruction into the microphone.

[0071] 2. Parsing the instructions

[0072] The server receives instructions sent from the device. The server analyzes the text of the instructions using natural language processing technology such as Google® Cloud Natural Language API or IBM Watson® Natural Language Understanding. Through this analysis, the content of the instructions (e.g., "next Monday," "Tokyo," "meeting," "business trip arrangements") is extracted.

[0073] 3. Selecting a task

[0074] The server selects an appropriate task based on the analysis results. For example, if the keyword "business trip arrangement" is included, the server selects the task of business trip arrangement. This selection can be performed using a predefined rule-based system or a machine learning model (for example, a model using TENSORFLOW (registered trademark) or PyTorch).

[0075] 4. Execute the task

[0076] The server executes the selected task. Specifically, it calls the travel agency's API (for example, Expedia API or Amadeus API) to make business trip arrangements. The server sends the necessary information (date, time, location, purpose, etc.) to the API to complete the reservation. Once the reservation is complete, the server notifies the user of the result. For example, the app may display a message saying, "Business trip arrangements completed."

[0077] Specific examples

[0078] User instructions

[0079] The user instructs the system, "Please arrange travel for a meeting in Tokyo next Monday."

[0080] Prompt Sentence Examples

[0081] Examples of prompts to input to a generative AI model might include:

[0082] The user has instructed us to "make travel arrangements for a conference in Tokyo next Monday." Analyze this instruction and select the task of making travel arrangements.

[0083] This prompt sentence can be used to have a generative AI model parse the instructions and select a task.

[0084] In this way, it becomes possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute the appropriate tasks. This system reduces the user's workload and supports efficient work execution.

[0085] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0086] Step 1:

[0087] The user inputs instructions using the device's interface. The interface supports text input and voice input. For example, a user opens a smartphone app and inputs text such as, "Please arrange a business trip for a meeting in Tokyo next Monday." In the case of voice input, the user speaks the same instruction into the microphone. The input instruction is sent from the device to the server.

[0088] Step 2:

[0089] The server receives instructions sent from the device and analyzes them using natural language processing technology. Specifically, it uses software such as Google Cloud Natural Language API and IBM Watson Natural Language Understanding. The input for analysis is the user's instruction text, and the output is keywords and phrases resulting from the analysis (such as "next Monday," "Tokyo," "meeting," and "business trip arrangements").

[0090] Step 3:

[0091] The server selects the appropriate task based on the analysis results. It selects the task using a predefined rule-based system or machine learning model (for example, a model using TensorFlow or PyTorch) based on keywords and phrases extracted from the analysis results. The input is the analysis results, and the output is the selected task (in this case, "business trip arrangements").

[0092] Step 4:

[0093] The server executes the selected task. Specifically, it calls the travel agency's API (for example, Expedia API or Amadeus API) to make business trip arrangements. The server sends the necessary information (date, time, location, purpose, etc.) to the API to complete the reservation. The input is the selected task and the necessary information, and the output is a notification that the reservation has been completed.

[0094] Step 5:

[0095] The server notifies the user that the reservation is complete. For example, the app displays "Business trip arrangements completed." The input is the reservation completion notification, and the output is a notification message to the user.

[0096] In this way, it becomes possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute the appropriate tasks. This system reduces the user's workload and supports efficient work execution.

[0097] (Application example 1)

[0098] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Conventional administrative services systems often have limited interfaces for receiving user instructions, and in particular do not support voice input. Furthermore, they lack the functionality to analyze user instructions and generate optimal suggestions, making it impossible to fully reduce the workload of users. Furthermore, in certain operations, such as food delivery, it is necessary to efficiently process user instructions and provide optimal suggestions.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0101] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, business trip arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for converting the user's voice input into text, means for analyzing the user's instructions using a generative AI model and generating optimal suggestions, and means for completing orders based on the generated suggestions. This makes it possible to efficiently process the user's voice instructions and provide optimal suggestions.

[0102] "User instructions" refer to requests or commands given by a user to a system.

[0103] "Meeting coordination" refers to the task of coordinating the date, location, and participants of a meeting.

[0104] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[0105] "Expense reimbursement" refers to the process of calculating and reimbursing expenses related to work.

[0106] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[0107] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[0108] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[0109] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and understanding the agenda.

[0110] "Post-meeting summary sharing" refers to the task of summarizing the minutes after the meeting and sharing them with relevant parties.

[0111] "Voice input" refers to a method in which a user inputs instructions by voice through a microphone.

[0112] "Convert to text" refers to the process of converting voice-input instructions into text information.

[0113] A "generative AI model" is an algorithm that uses artificial intelligence to analyze user instructions and generate appropriate suggestions.

[0114] An "optimal suggestion" is the most appropriate solution or option derived by the generative AI model based on the user's instructions.

[0115] "Completing an order" refers to the task of carrying out the necessary procedures based on the user's instructions and finalizing the order.

[0116] A system for implementing this invention has an interface for receiving instructions from a user and supports voice and text input. When a user inputs instructions by voice, the system converts the speech into text and analyzes it using a generative AI model. Based on the analysis results, an optimal proposal is generated and presented to the user. Once the user confirms and approves the proposal, the system completes the order.

[0117] Hardware and software used

[0118] Hardware: Smartphone, microphone

[0119] Software: Python, SpeechRecognition library, OpenAI® API

[0120] Data processing and calculation

[0121] 1. Speech Recognition: Uses the smartphone's microphone to capture the user's voice input and converts this speech to text using the SpeechRecognition library.

[0122] 2. Generative AI model: Using the OpenAI API, the generative AI model analyzes the textual user instructions and generates optimal suggestions based on the user instructions.

[0123] 3. Present proposal and complete order: The generated proposal is presented to the user, and once the user approves it, the system completes the order.

[0124] Specific examples

[0125] If a user says, "I'd like to order pizza from a nearby Italian restaurant," the system converts the speech to text and analyzes it using a generative AI model. The analysis results suggest the best Italian restaurant and pizza menu. If the user accepts the suggestion, the system sends the order to the restaurant and completes the order.

[0126] Prompt Sentence Examples

[0127] A user says: 'I want to order pizza from an Italian restaurant near me.' Based on this instruction, suggest the best restaurant and menu and complete the order.

[0128] In this way, the user's voice instructions can be processed efficiently and optimal suggestions can be made.

[0129] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0130] Step 1:

[0131] The user uses the smartphone's microphone to input voice.

[0132] Input: User's spoken command (e.g., "I'd like to order pizza from a nearby Italian restaurant.")

[0133] Output: Audio data

[0134] Specific operation: The user activates the voice input function on their smartphone and inputs instructions by voice.

[0135] Step 2:

[0136] The device captures the voice data and converts it to text using the SpeechRecognition library.

[0137] Input: Audio data

[0138] Output: Text data (e.g., "I would like to order pizza from a nearby Italian restaurant")

[0139] Specific operation: The device passes the voice data to the SpeechRecognition library, performs voice recognition, and generates text data.

[0140] Step 3:

[0141] The device sends the text data to the generative AI model for analysis.

[0142] Input: Text data

[0143] Output: Analysis results (e.g., recommendations for the best restaurants and menus)

[0144] Specific operation: The device calls the OpenAI API, sends text data as a prompt, and has it analyzed by the generative AI model.

[0145] Step 4:

[0146] The server receives the analysis results of the generated AI model and presents optimal suggestions to the user.

[0147] Input: Analysis results

[0148] Output: Suggestions (e.g., best Italian restaurants and pizza menus)

[0149] Specific operation: The server sends the analysis results to the user's smartphone and displays suggestions to the user.

[0150] Step 5:

[0151] The user reviews and approves the proposal.

[0152] Input: Proposal

[0153] Output: Approval data

[0154] Specific operation: The user checks the proposal on the smartphone screen and presses the approval button.

[0155] Step 6:

[0156] The server receives the authorization data and completes the order.

[0157] Input: Approval data

[0158] Output: Order completion notification

[0159] What happens: The server uses the authorization data to send the order to the suggested restaurant and notifies the user that the order is complete.

[0160] Example 2

[0161] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] Conventional administrative services systems have the problem that it is time-consuming for users to select tasks and input the necessary information, making it difficult to carry out tasks efficiently. Furthermore, while it is required that the information entered by users be processed appropriately and results be provided quickly, there has been a lack of effective means to achieve this. Furthermore, a system that can reduce the workload on users and support efficient task execution is needed.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0164] In this invention, the server includes means for the user to select a task and input the necessary information, means for the terminal to receive the information input by the user and send it to the server, means for the server to start and execute an appropriate module based on the received information, and means for the server to send the arrangement results to the terminal and for the terminal to display the arrangement results to the user. This allows the user to easily select a task and input the necessary information, and the server to quickly start the appropriate module to process and provide the results to the user.

[0165] "User" means an individual or organization that uses the system to select a task and enter the required information.

[0166] "Instructions" refer to the operations that users perform on the system, such as selecting tasks and inputting information.

[0167] "Administrative agency" refers to the performance of tasks such as meeting coordination, travel arrangements, expense settlement, attendance management, and document preparation.

[0168] "Meeting coordination" refers to the task of arranging the date, time, and location of a meeting and notifying participants.

[0169] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[0170] "Expense settlement" refers to the task of organizing and settling expenses incurred for business trips and other work.

[0171] "Attendance management" refers to the task of managing employee attendance information such as attendance, departure, and vacation.

[0172] "Document creation" refers to the task of creating various documents necessary for business operations.

[0173] "Proxy attendance" refers to the task of attending a meeting on behalf of a user and understanding the content of the meeting.

[0174] "Summary sharing" refers to the task of summarizing the contents of a meeting and sharing it with users.

[0175] "Natural language processing" refers to technology for analyzing instructions from users and selecting appropriate tasks.

[0176] "Workload" refers to the effort and time required by a user to carry out their work.

[0177] "Efficient business execution" means performing business operations quickly and accurately.

[0178] "Terminal" refers to a device that allows a user to select a task and enter information.

[0179] "Server" refers to a computer system that receives information from a user, activates the appropriate module, and performs processing.

[0180] A "module" refers to a part of a program for performing a specific task.

[0181] "Arrangement result" refers to the result obtained after the server executes the task.

[0182] The present invention is a system in which a user selects a task and inputs the necessary information, and the server starts up an appropriate module to execute the task. A specific embodiment of this system will be described below.

[0183] Hardware and software used

[0184] server

[0185] The server is a high-performance computer system that manages and executes business modules. Specifically, it uses general server hardware (e.g., a high-performance server computer), and business modules are developed in Python and run on the server using the Django framework. MySQL (registered trademark) is used as the database.

[0186] Terminal

[0187] The terminal is a device that allows the user to select a task and enter the necessary information. Specifically, the terminal is a PC or smartphone (e.g., Windows PC or smartphone) that the user uses.

[0188] System Operation

[0189] User Actions

[0190] The user uses the terminal interface to select the task they want to perform. For example, to select the task of arranging a business trip, the user clicks the "Business Trip Arrangements" button. The user then enters information such as the departure point, destination, departure date, and return date.

[0191] Terminal handling

[0192] The device receives the information entered by the user and sends it to the server using an HTTP request, which communicates the user's instructions to the server.

[0193] Server Processing

[0194] The server analyzes the received information and selects and launches the appropriate business module. For example, it launches a business trip arrangement module. This module makes the arrangements necessary for the business trip, such as booking an airline ticket, booking a hotel, and arranging transportation. To do this, it uses external APIs (e.g., airline APIs, hotel booking site APIs).

[0195] Displaying the results

[0196] Once the arrangements are complete, the server sends the results to the terminal. Specifically, it generates JSON data including detailed arrangement information (e.g., reservation confirmation number, hotel address, transportation details) and sends it as an HTTP response to the terminal. The terminal then displays the received arrangement results to the user. Specifically, it displays the arrangement results on the screen in an easy-to-read format. For example, it could display a list of the airline ticket reservation confirmation number, hotel address, and transportation details.

[0197] Specific examples

[0198] Specific examples of business trip arrangements

[0199] 1. The user clicks the "Business Trip Arrangements" button on the device.

[0200] 2. The user enters the departure location as "Tokyo," the destination as "New York," the departure date as "December 1, 2023," and the return date as "December 5, 2023."

[0201] 3. The device sends the input information to the server.

[0202] 4. The server launches the business trip arrangement module.

[0203] 5. The server will book your flight, hotel and transportation.

[0204] 6. The server sends the arrangement results to the terminal.

[0205] 7. The terminal displays the arrangement results to the user.

[0206] Example prompts for generative AI models

[0207] "Please select the business trip arrangement task, enter the departure point as Tokyo, the destination as New York, the departure date as December 1, 2023, and the return date as December 5, 2023."

[0208] In this way, users can easily make business trip arrangements.

[0209] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0210] Step 1:

[0211] The user selects a task.

[0212] The user uses the terminal interface to select the task they wish to perform. For example, to select the task of arranging a business trip, the user clicks the "Business Trip Arrangements" button. The input is an instruction to select the task, and the output is information about the selected task.

[0213] Step 2:

[0214] The terminal receives the user's input information.

[0215] The user inputs information such as the departure point, destination, departure date, and return date. The terminal receives this information and stores it as input information. The input includes various pieces of information entered by the user, and the output is stored in the terminal.

[0216] Step 3:

[0217] The terminal sends the input information to the server.

[0218] The device sends the received information to the server using an HTTP request. Specifically, it converts the input information into JSON format and sends it to the server's API endpoint. The input is the information entered by the user, and the output is the information sent to the server.

[0219] Step 4:

[0220] The server launches the appropriate module based on the information received.

[0221] The server analyzes the received information and selects and launches the appropriate business module. For example, it launches a business trip arrangement module. The input is the information sent from the terminal, and the output is the launched module.

[0222] Step 5:

[0223] The server executes the module and makes the necessary arrangements.

[0224] The server executes the launched module and makes the necessary arrangements. In the case of the business trip arrangement module, it makes flight reservations, hotel reservations, and transportation arrangements. For this, it uses external APIs (e.g., airline APIs, hotel reservation site APIs). The input is the user's business trip information, and the output is the arrangement results.

[0225] Step 6:

[0226] The server sends the arrangement result to the terminal.

[0227] Once the arrangements are complete, the server sends the results to the terminal. Specifically, it generates JSON data containing detailed arrangement information (e.g., reservation confirmation number, hotel address, transportation details) and sends it as an HTTP response to the terminal. The input is the arrangement results, and the output is the arrangement results sent to the terminal.

[0228] Step 7:

[0229] The terminal displays the results of the arrangement to the user.

[0230] The terminal displays the received arrangement results to the user. Specifically, the arrangement results are displayed on the screen in an easy-to-read format. For example, the airline ticket reservation confirmation number, hotel address, and transportation details are displayed in a list. The input is the arrangement results sent from the server, and the output is the arrangement results displayed to the user.

[0231] (Application example 2)

[0232] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0233] While conventional administrative services can efficiently handle tasks such as coordinating meetings, arranging business trips, and expense settlements, they do not address other needs in users' daily lives, such as arranging food delivery. This requires users to use a separate food delivery app, which creates a disconnect between managing work and daily life. Furthermore, there is no system that automatically selects the optimal restaurant based on the user's location and food preferences, arranges delivery methods, or adjusts delivery times, reducing user convenience.

[0234] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0235] In this invention, the server includes means for handling administrative tasks such as arranging meetings, arranging business trips, settling expenses, managing attendance, and preparing documents based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries afterward, and means for searching for the most suitable restaurant based on the user's location information and food preferences, ordering food, arranging delivery means, and adjusting delivery times. This allows users to efficiently manage both their work and daily lives using a single system.

[0236] "Instructions from the user" refers to the operations or inputs that the user makes to the system, and the system then performs a specific task based on these instructions.

[0237] "Meeting coordination" refers to the task of coordinating the date, time, location, and participants of a meeting, and finalizing the schedule.

[0238] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[0239] "Expense reimbursement" refers to the process of applying for, approving, and settling work-related expenses.

[0240] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and vacations.

[0241] "Document creation" refers to the task of creating documents and presentation materials needed for meetings, reports, etc.

[0242] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and preparing minutes and summaries.

[0243] "Post-event summary sharing" refers to the task of summarizing the contents of meetings and discussions and sharing them with relevant parties.

[0244] "Location information" refers to information about the user's current location or a specified location, and the system uses this information to perform its tasks.

[0245] "Food preferences" refers to the types of food and cuisine that a user prefers, and the system uses this information to select the most suitable restaurant.

[0246] "Restaurant search" refers to the task of finding suitable restaurants based on the user's location information and food preferences.

[0247] "Ordering food" refers to the task of ordering specific food from a selected restaurant.

[0248] "Arranging delivery means" refers to the task of arranging means to deliver the ordered food to the user's designated location.

[0249] "Delivery time adjustment" refers to the task of adjusting the delivery time so that the food arrives at the user's designated location at the appropriate time.

[0250] A system for implementing this invention includes means for carrying out administrative tasks such as coordinating meetings, arranging business trips, settling expenses, keeping attendance records, and preparing documents based on instructions from a user, means for attending meetings on behalf of a user and sharing summaries afterward, and means for searching for the most suitable restaurant based on the user's location information and food preferences, ordering food, arranging for delivery, and adjusting delivery times.

[0251] 1. System Program

[0252] The system is implemented using Python programs and REST APIs. The interface for receiving user instructions is provided as a smartphone application. When a user enters instructions through the application, the instructions are sent to the server, which then performs the appropriate tasks based on the instructions.

[0253] 2. Program Processing

[0254] The server processes and calculates data using the following hardware and software.

[0255] Hardware: Servers, smartphones

[0256] Software: Python, REST API, natural language processing libraries (e.g. spaCy, NLTK)

[0257] Processing flow

[0258] 1. Receiving user instructions: The user inputs instructions through a smartphone application. For example, "I want to eat sushi in Tokyo."

[0259] 2. Natural Language Processing: The server uses a natural language processing library to analyze the user's instructions and select the appropriate task.

[0260] 3. Execution of tasks: The server executes the following tasks based on the analysis results.

[0261] Restaurant search: Find the best restaurants based on your location and food preferences.

[0262] Food Ordering: Order specific food from selected restaurants.

[0263] Arranging delivery: Arranging delivery of the ordered food to the user's designated location.

[0264] Delivery time adjustment: Adjust delivery times to ensure food arrives at the user's desired location at the appropriate time.

[0265] 3. Specific Examples

[0266] As a concrete example, the prompt sentence when the user inputs "I live in Tokyo and I want to eat sushi" is as follows:

[0267] Prompt Sentence Examples

[0268] User location: Tokyo

[0269] Food preference: Sushi

[0270] Based on this prompt, the system operates as follows:

[0271] 1. Restaurant Search: Search for sushi restaurants in Tokyo and choose the best one.

[0272] 2. Order food: Order a "Sushi Set" at the selected restaurant.

[0273] 3. Arrange delivery: Once the order is completed, arrange delivery and schedule delivery based on the user's location.

[0274] In this way, users can easily use food delivery services.

[0275] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0276] Step 1:

[0277] A user inputs a command through a smartphone application, for example, "I want to eat sushi in Tokyo." This command is then sent from the application to the server.

[0278] Input: User instructions (e.g., "I want to eat sushi in Tokyo")

[0279] Output: User instructions sent to the server

[0280] Step 2:

[0281] The server uses a natural language processing library (e.g., spaCy, NLTK) to analyze the user's instructions, extracting their location (Tokyo) and food preference (sushi).

[0282] Input: User instructions

[0283] Output: Analysis results (location information and food preferences)

[0284] What it does: The server uses a natural language processing library to analyze the text and extract location information and food preferences.

[0285] Step 3:

[0286] Based on the analysis results, the server calls a restaurant search API and searches for restaurants that match the user's location information and food preferences.

[0287] Input: Analysis results (location information and food preferences)

[0288] Output: List of restaurants

[0289] Specific operation: The server passes location information and food preferences as parameters to the restaurant search API and obtains a list of the most suitable restaurants.

[0290] Step 4:

[0291] The server selects the most suitable restaurant from the acquired list of restaurants and places an order for food.

[0292] Input: A list of restaurants

[0293] Output: Order confirmation information

[0294] Specific operation: The server selects the most suitable restaurant from the list of restaurants based on criteria such as rating and distance, and calls the ordering API to order food.

[0295] Step 5:

[0296] After the order is completed, the server calls the delivery means arrangement API and arranges for the means to deliver the food to the location specified by the user.

[0297] Input: Order confirmation information

[0298] Output: Delivery arrangement confirmation information

[0299] Specific operation: The server calls the delivery means arrangement API based on the order confirmation information and arranges for delivery means.

[0300] Step 6:

[0301] After arranging the delivery means, the server adjusts the delivery time and notifies the user.

[0302] Input: Delivery means arrangement confirmation information

[0303] Output: Delivery time notification

[0304] Specific operation: The server adjusts the delivery time based on the delivery means arrangement confirmation information and notifies the user.

[0305] In this way, users can easily use food delivery services.

[0306] Example 3

[0307] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0308] Conventional administrative proxy systems did not efficiently handle proxy attendance at meetings or the subsequent sharing of summaries, and were unable to sufficiently reduce the user's workload. Furthermore, the process of recording, transcribing, and generating summaries of meeting content was not automated, requiring users to perform the tasks manually, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to quickly share the generated summaries, which sometimes resulted in delays in the transmission of information.

[0309] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0310] In this invention, the server includes means for performing administrative tasks such as meeting coordination, travel arrangements, expense settlement, time attendance, and document preparation based on instructions from the user, means for attending the meeting on behalf of the user and sharing a summary afterwards, means for recording the audio data of the meeting and transcribing it using speech recognition software, means for summarizing the transcript using a generative AI model, and means for sharing the summary with the user. This automates the entire process from attending the meeting on behalf of the user to generating and sharing the summary, significantly reducing the user's workload and enabling more efficient work performance.

[0311] "User" means any person or entity that uses the System to issue instructions.

[0312] An "instruction" is a request or command given by a user to a system.

[0313] "Meeting coordination" refers to the task of coordinating the date, location, and participants of a meeting.

[0314] "Business trip arrangements" refers to the work of arranging transportation and accommodation necessary for business trips.

[0315] "Expense settlement" refers to the process of calculating and settling expenses incurred for business trips and other work.

[0316] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[0317] "Document creation" refers to the task of creating documents and presentation materials required for meetings and business.

[0318] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[0319] "Meeting proxy attendance" refers to attending a meeting on behalf of a user.

[0320] "Summary sharing" refers to summarizing the contents of a meeting and communicating that summary to users.

[0321] "Audio data" refers to data that records the audio of a conference.

[0322] "Speech recognition software" means software for converting voice data into text.

[0323] "Transcription" is the process of converting audio data into text.

[0324] A "generative AI model" is a model that uses artificial intelligence to process data and perform summarization or other generative tasks.

[0325] A "summary" is a text that briefly summarizes the contents of a meeting.

[0326] "Sharing" refers to communicating the generated summary to a user.

[0327] MODE FOR CARRYING OUT THE INVENTION

[0328] This invention is a system that handles administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on user instructions. It also has the functionality to attend meetings on behalf of users and share summaries afterward. This system operates in cooperation with the server, the terminal, and the user.

[0329] Hardware and software used

[0330] server:

[0331] The server plays a central role in receiving instructions from users and carrying out various administrative tasks.

[0332] The server uses video conferencing software (e.g., Zoom, MICROSOFT® TEAMS®) to participate in the conference.

[0333] The server transcribes the meeting audio data using speech recognition software (e.g., Google Cloud Speech-to-Text API).

[0334] The server summarizes the transcription data using a generative AI model (e.g., OpenAI's GPT-4®).

[0335] Device:

[0336] A terminal is a device that allows a user to input instructions, such as a PC or smartphone.

[0337] The terminal provides an interface for transmitting user instructions to the server.

[0338] user:

[0339] A user is an individual or corporation that uses the system to instruct administrative services.

[0340] Users use terminals to input instructions into the system and receive results.

[0341] Program processing

[0342] Upon receiving instructions from the user, the server launches a meeting proxy attendance module, which joins the meeting using video conferencing software at the specified time. Audio data from the meeting is recorded in real time and transcribed using speech recognition software. The transcribed data is then summarized using a generative AI model. The generated summary is shared with the user via email or a chat app.

[0343] Specific examples

[0344] When a user instructs the server to "attend the meeting tomorrow at 10:00 on his behalf and share the summary of the meeting," the server performs the following process.

[0345] 1. The server starts the conference proxy attendance module and joins the designated conference using the video conferencing software.

[0346] 2. The server records the audio of the meeting and transcribes it using speech recognition software.

[0347] 3. The server inputs the transcription data into a generative AI model to generate a summary of the meeting.

[0348] 4. The server sends the generated summary to the user's email address.

[0349] Example prompt sentence:

[0350] "Please summarize the transcript of the following meeting. Please include key agenda items, decisions made, and upcoming action items."

[0351] In this way, the server attends the conference on behalf of the user based on the user's instructions, and generates and shares a summary of the conference. This significantly reduces the user's workload and enables efficient work execution. The flow of the specific processing in the third embodiment will be described with reference to FIG. 15.

[0352] Step 1:

[0353] The user inputs instructions into the system using a terminal. Specifically, the user inputs an instruction such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting," and presses the send button. This instruction is then sent to the server.

[0354] Input: User instructions (natural language)

[0355] Output: Instruction data sent to the server

[0356] Step 2:

[0357] The server receives instructions from the user, analyzes the instructions using natural language processing, and extracts information (date and time, conference URL, etc.) from the analyzed instructions to launch the conference proxy attendance module.

[0358] Input: User-specified data

[0359] Output: Parsed instruction information (date and time, meeting URL, etc.)

[0360] Step 3:

[0361] The server launches the meeting proxy attendance module at the specified time. Specifically, the server joins the meeting using video conferencing software (e.g., Zoom, Microsoft Teams). The server automatically connects to the meeting using the meeting URL.

[0362] Input: Parsed instruction information (date and time, meeting URL, etc.)

[0363] Output: Meeting participation status

[0364] Step 4:

[0365] The server records the audio data of the meeting in real time, and the recorded audio data is transcribed using speech recognition software (e.g., Google Cloud Speech-to-Text API).

[0366] Input: Meeting audio data

[0367] Output: Transcribed text data

[0368] Step 5:

[0369] The server uses a generative AI model (e.g., OpenAI's GPT-4) to summarize the transcription data. The server inputs the prompt "Please summarize the transcription data of the following meeting. Please include important agenda items, decisions, and next action items" into the generative AI model and generates a summary.

[0370] Input: Transcribed text data, prompt sentence

[0371] Output: Summarized text data

[0372] Step 6:

[0373] The server then shares the generated summary with the user. Specifically, it sends the summary to the user's email address or chat app (e.g., Slack, Microsoft Teams). The user can then check the summary and understand the content of the meeting.

[0374] Input: Summarized text data

[0375] Output: Summary data sent to the user

[0376] In this way, the server attends the meeting on behalf of the user and generates and shares a summary of the meeting based on the user's instructions, significantly reducing the user's workload and enabling them to carry out their work more efficiently.

[0377] (Application example 3)

[0378] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0379] In today's business environment, meetings are becoming more frequent, making it difficult for field workers and managers to attend every meeting. It is also important to accurately understand the content of meetings and efficiently summarize and share them, but this requires a great deal of time and effort. This increases the workload and hinders efficient business execution, creating a challenge.

[0380] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0381] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance management, and document preparation based on instructions from the user, means for attending the meeting on behalf of the user and sharing a summary afterwards, means for recording the audio of the meeting and converting it into text using speech recognition technology, means for summarizing the meeting content using a generative AI model, and means for sharing the summary results with the user. This enables the user to efficiently understand the meeting content without attending the meeting and reduces the workload.

[0382] "User" refers to any individual or legal entity that uses the System.

[0383] "Instructions" refer to commands or requests made by a user to a system.

[0384] "Meeting coordination" refers to the task of setting and coordinating the date, time, location, and participants of a meeting.

[0385] "Business trip arrangements" refers to the work of arranging transportation, accommodation, etc. necessary for business trips.

[0386] "Expense reimbursement" refers to the work of calculating and reimbursing expenses related to work.

[0387] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and vacations.

[0388] "Document creation" refers to the task of creating documents and presentation materials required for meetings and business.

[0389] "Administrative agency" refers to the business of performing administrative tasks on behalf of users.

[0390] "Meeting proxy attendance" refers to the task of attending a meeting on behalf of a user and recording the meeting contents.

[0391] "Post-meeting summary sharing" refers to the task of summarizing the meeting contents after the meeting and sharing them with users.

[0392] "Speech recognition technology" refers to technology that converts speech into text.

[0393] "Generative AI model" refers to a model that uses artificial intelligence to generate or summarize text.

[0394] "Summary result" refers to a text summarizing the contents of the meeting.

[0395] "Sharing" refers to providing the summary results to the user.

[0396] A system for implementing this invention includes a means for performing administrative tasks such as coordinating meetings, arranging travel, settling expenses, keeping attendance records, and preparing documents based on instructions from a user, a means for attending meetings on behalf of a user and sharing summaries afterward, a means for recording the audio of the meeting and converting it into text using speech recognition technology, a means for summarizing the meeting content using a generative AI model, and a means for sharing the summary results with the user.

[0397] When the server receives instructions from a user, it first analyzes them using natural language processing technology. Based on the analyzed instructions, it selects and executes the appropriate task. For example, if a user instructs the server to "attend the meeting tomorrow at 10:00 on my behalf and share the meeting summary," the server will launch the meeting proxy attendance module and join the meeting at the specified time.

[0398] When a meeting starts, a device (e.g., a microphone mounted on a factory robot) records the audio of the meeting. The recorded audio data is converted into text using speech recognition technology (e.g., Google Speech Recognition API). The converted text is summarized using a generative AI model (e.g., Hugging Face's transformers library).

[0399] The summary results are shared with users via the server, allowing users to efficiently understand the content of the meeting without attending the meeting.

[0400] As a concrete example, consider the case where a factory manager instructs, "Please attend the production meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." When this prompt sentence is entered into the system, the system will attend the meeting at the specified time, and after it is over, generate a summary and share it with the manager.

[0401] Example prompt sentence:

[0402] "I would like you to attend the production meeting tomorrow at 10:00 on my behalf and share a summary of the meeting."

[0403] In this way, it becomes possible to reduce the workload of the user and support efficient business execution.

[0404] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0405] Step 1:

[0406] The user enters instructions into the system.

[0407] Input: User instructions (e.g., "Please attend the production meeting tomorrow at 10:00 on my behalf and share the summary of the meeting.")

[0408] Output: Parsed instructions

[0409] Specific operation: The user inputs instructions in natural language, and the server receives the instructions.

[0410] Step 2:

[0411] The server analyzes the user's instructions using natural language processing technology.

[0412] Input: User instructions

[0413] Output: Parsed instructions (e.g., request for meeting date, time, content, and summary)

[0414] Specific operation: The server uses natural language processing technology (e.g., NLTK or spaCy) to parse the instructions and extract the necessary information.

[0415] Step 3:

[0416] The server starts the conference proxy attendance module and participates in the conference at the designated time.

[0417] Input: Parsed instructions (meeting date and time, content)

[0418] Output: Ready to join the meeting

[0419] Specific operation: The server starts the conference proxy attendance module and prepares to participate in the conference at the specified date and time.

[0420] Step 4:

[0421] The device records the audio of the meeting.

[0422] Input: Meeting audio

[0423] Output: Recorded audio data

[0424] Specific operation: A terminal (for example, a microphone mounted on a factory robot) records the audio of the meeting in real time.

[0425] Step 5:

[0426] The server uses voice recognition technology to convert the recorded voice data into text.

[0427] Input: Recorded audio data

[0428] Output: Text data

[0429] Specific operation: The server uses speech recognition technology (e.g., Google Speech Recognition API) to convert the voice data into text.

[0430] Step 6:

[0431] The server summarizes the text data using a generative AI model.

[0432] Input: Text data

[0433] Output: Summarized text

[0434] What it does: The server uses a generative AI model (e.g., Hugging Face's transformers library) to summarize the text data.

[0435] Step 7:

[0436] The server shares the summary results with the user.

[0437] Input: Summarized text

[0438] Output: Summary results shared with users

[0439] Specific operation: The server shares the summary results with the user by email, notification, etc.

[0440] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0441] "Example 1"

[0442] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system receives instructions from a user and performs tasks based on those instructions. The emotion engine recognizes the user's emotions and adjusts the priority and execution method of tasks according to those emotions. For example, if the emotion engine recognizes that the user is feeling stressed, it will prioritize tasks that have the potential to reduce stress. Also, if the emotion engine recognizes that the user is feeling joy, it will select tasks that will maintain that joy. In this way, the emotion engine performs tasks taking the user's emotions into consideration, thereby achieving more efficient task execution that provides greater user satisfaction.

[0443] "Example 2"

[0444] In another embodiment of the present invention, a system is provided in which an emotion engine adjusts a task execution method in accordance with a user's emotion. For example, if the emotion engine recognizes that the user is feeling angry, it selects a task to calm the anger and executes the task in a gentle tone. Alternatively, if the emotion engine recognizes that the user is feeling sad, it selects a task to ease the sadness and executes the task with encouraging words. In this way, the emotion engine adjusts the task execution method taking the user's emotion into consideration, thereby realizing task execution adapted to the user's emotional state.

[0445] "Example 3"

[0446] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system receives instructions from a user and performs tasks based on those instructions. The emotion engine recognizes the user's emotions and adjusts the priority and execution method of tasks according to those emotions. For example, if the emotion engine recognizes that the user is feeling stressed, it will prioritize tasks that have the potential to reduce stress. Also, if the emotion engine recognizes that the user is feeling joy, it will select tasks that will maintain that joy. In this way, the emotion engine performs tasks taking the user's emotions into consideration, thereby achieving more efficient task execution that provides greater user satisfaction.

[0447] The processing flow of each embodiment will be described below.

[0448] "Example 1"

[0449] Step 1: The system receives instructions from the user.

[0450] Step 2: The emotion engine recognizes the user's emotions.

[0451] Step 3: The emotion engine adjusts task priorities based on the emotions it recognizes.

[0452] Step 4: The system executes its work based on the adjusted priorities.

[0453] "Example 2"

[0454] Step 1: The system receives instructions from the user.

[0455] Step 2: The emotion engine recognizes the user's emotions.

[0456] Step 3: The emotion engine adjusts how work is performed depending on the emotions it recognizes.

[0457] Step 4: The system performs the task based on the adjusted execution method.

[0458] Example 1

[0459] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0460] Conventional administrative services systems are unable to consider the user's emotions or stress levels when carrying out tasks based on the user's instructions, making it difficult to carry out tasks efficiently and with high satisfaction. In addition, they are unable to properly prioritize tasks, making it difficult to reduce the user's workload.

[0461] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0462] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for analyzing the user's instructions using natural language processing and selecting and executing appropriate tasks, means for recognizing the user's emotions and adjusting the priority and execution method of tasks according to those emotions, and means for notifying the user when tasks are completed. This allows tasks to be performed efficiently and with a high level of satisfaction by taking the user's emotions into consideration.

[0463] "User" means an individual or corporation that uses the system to issue instructions and request work.

[0464] An "instruction" is a request or command given by a user to a system, and may take the form of text input or voice input.

[0465] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0466] "Emotion recognition" is a technology that analyzes and recognizes a user's emotional state.

[0467] "Task priority" refers to the order in which a task is determined to be performed first among multiple tasks.

[0468] "Administrative agency" refers to performing tasks such as coordinating meetings, arranging business trips, settling expenses, managing attendance, and preparing documents on behalf of users.

[0469] "Attending a meeting on your behalf" means attending a meeting on your behalf and recording the contents of the meeting.

[0470] "Summary sharing" refers to providing users with a concise summary of the contents of a meeting or work.

[0471] "Notification of task completion" is the act of notifying the user that the system has completed a task.

[0472] An "emotion engine" is an engine that has the function of analyzing the user's emotions and adjusting the priorities and execution methods of work based on those emotions.

[0473] The system of the present invention has an interface for receiving instructions from a user. This interface supports formats that allow users to give instructions naturally, such as text input and voice input. For example, if a user instructs, "Please make travel arrangements for a conference in Tokyo next Monday," the system analyzes this instruction and selects the task of making travel arrangements.

[0474] Hardware and software used

[0475] Hardware: Servers, terminals (PCs, smartphones)

[0476] Software: Natural language processing engines (e.g., Google Cloud Natural Language API), emotion recognition engines (e.g., IBM Watson Tone Analyzer), generative AI models (e.g., OpenAI GPT-3®)

[0477] Specific operation of the system

[0478] 1. User input instructions:

[0479] The user uses the device to input instructions using text or voice.

[0480] Specific operation: The user opens the app on their smartphone and gives a voice command such as, "Please make travel arrangements for a meeting in Tokyo next Monday."

[0481] 2. The device sends instructions to the server:

[0482] The terminal receives the user's input and sends it to the server.

[0483] Specific operation: The smartphone converts the voice data into text and sends the text data to the server.

[0484] 3. The server parses the instructions:

[0485] The server uses a natural language processing engine to parse the user's instructions.

[0486] Specific operation: The server analyzes the text "Please make travel arrangements for a meeting in Tokyo next Monday" and identifies the task of making travel arrangements.

[0487] 4. The server recognizes emotions:

[0488] The server uses an emotion recognition engine to analyze the user's emotions.

[0489] Specific behavior: The server recognizes that it is experiencing stress from the user's instructions.

[0490] 5. The server selects and prioritizes tasks:

[0491] The server uses an emotion engine to prioritize tasks based on the user's emotions.

[0492] Specific action: The server decides to prioritize business trip arrangements to reduce stress.

[0493] 6. The server performs the task:

[0494] The server collects the necessary data and arranges the trip.

[0495] What happens: The server accesses an airline booking website and reserves the best flight for the user, as well as a hotel reservation.

[0496] 7. The server sends the results to the device:

[0497] The server notifies the terminal of the completion of the task.

[0498] Specific operation: The server generates a message saying "Business trip arrangements have been completed. Details have been sent by email." and sends it to the terminal.

[0499] 8. The device notifies the user of the results:

[0500] The device receives the notification from the server and displays it to the user.

[0501] What happens: The smartphone displays a notification to let the user know that their travel arrangements have been completed.

[0502] Examples and prompts

[0503] Examples:

[0504] The user instructs, "Please arrange travel for a meeting in Tokyo next Monday."

[0505] The terminal sends this instruction to the server.

[0506] The server analyzes the instructions and identifies the business trip arrangement task.

[0507] The server uses an emotion recognition engine to determine that the user is feeling stressed.

[0508] The server decides to give top priority to business trip arrangements.

[0509] The server makes flight and hotel reservations.

[0510] The server notifies the terminal of the completion of the task.

[0511] The terminal notifies the user that the travel arrangements have been completed.

[0512] Example prompt sentence:

[0513] "A user has requested travel arrangements for a meeting in Tokyo next Monday. Analyze the user's emotions and make arrangements quickly if they are stressed."

[0514] In this way, the system of the present invention analyzes the user's instructions and performs tasks taking into account the user's emotions, thereby providing efficient and satisfying services.

[0515] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0516] Step 1:

[0517] The user inputs instructions.

[0518] Specific operation: The user uses a device (PC or smartphone) to input instructions via text or voice.

[0519] Input: User instructions (e.g., "Please make travel arrangements for a meeting in Tokyo next Monday.")

[0520] Output: User's instruction data (text or audio data)

[0521] Step 2:

[0522] The terminal sends an instruction to the server.

[0523] Specific operation: The device converts the voice data into text and sends the text data to the server.

[0524] Input: User's instruction data (text or voice data)

[0525] Output: Instruction data in text format (sent to the server)

[0526] Step 3:

[0527] The server parses the instructions.

[0528] What happens: The server uses a natural language processing engine to parse the user's instructions.

[0529] Input: Instruction data in text format

[0530] Output: Analysis results (identification of work, e.g., business trip arrangements)

[0531] Step 4:

[0532] The server recognizes emotions.

[0533] Specific operation: The server uses an emotion recognition engine to analyze the user's emotions.

[0534] Input: Instruction data in text format

[0535] Output: Sentiment analysis result (e.g., feeling stressed)

[0536] Step 5:

[0537] The server selects and prioritizes the work.

[0538] Specific operation: The server uses the emotion engine to prioritize tasks based on the user's emotions.

[0539] Input: Analysis results (job identification) and sentiment analysis results

[0540] Output: Priority-set tasks (e.g., business trip arrangements are given top priority)

[0541] Step 6:

[0542] The server performs the work.

[0543] Specific operation: The server collects the necessary data and arranges the trip.

[0544] Input: Prioritized tasks

[0545] Output: The result of the task (e.g., flight and hotel reservations completed)

[0546] Step 7:

[0547] The server sends the results to the terminal.

[0548] Specific operation: The server notifies the terminal that the task has been completed.

[0549] Input: Work execution results

[0550] Output: Completion notification message (e.g. "The business trip arrangements have been completed. Details have been sent by email.")

[0551] Step 8:

[0552] The terminal notifies the user of the result.

[0553] Specific behavior: The device receives the notification from the server and displays it to the user.

[0554] Input: Completion notification message

[0555] Output: Notification to the user (e.g., notification displayed on smartphone)

[0556] (Application example 1)

[0557] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0558] Conventional administrative outsourcing systems perform tasks without considering the user's emotions, which can result in reduced user satisfaction and operational efficiency. Food delivery services also face the problem of not being able to fully meet user needs because they do not suggest optimal menus based on the user's emotions. There is a need to solve these issues and realize optimal task execution and suggestions based on the user's emotions.

[0559] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0560] In this invention, the server includes means for performing administrative tasks such as arranging meetings, arranging business trips, settling expenses, managing attendance, and preparing documents based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries afterward, means for recognizing the user's emotions and adjusting the priority and execution method of tasks according to those emotions, means for making optimal suggestions based on the user's emotions, means for receiving the user's instructions through voice input or text input, and means for suggesting appropriate menus and options based on emotion analysis. This enables optimal task execution and suggestions according to the user's emotions.

[0561] "User instructions" refer to requests or commands given by a user to a system.

[0562] "Meeting coordination" refers to the task of coordinating the date, time, location, and participants of a meeting.

[0563] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[0564] "Expense settlement" refers to the work of calculating and settling expenses related to work.

[0565] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[0566] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[0567] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[0568] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and preparing minutes and summaries.

[0569] "Post-meeting summary sharing" refers to the task of sharing minutes and summaries with relevant parties after the meeting has ended.

[0570] "Emotion recognition" means analyzing emotions from a user's voice or text and understanding those emotions.

[0571] "Adjusting the priority and execution method of work" means changing the importance of work and execution procedures according to the user's feelings.

[0572] "Optimal suggestions" means presenting the most appropriate options based on the user's emotions and situation.

[0573] "Voice input and text input" refers to the way in which a user gives instructions to a system by voice or text.

[0574] "Emotion analysis" is a technology that analyzes emotions from a user's voice or text.

[0575] "Menu and option suggestions" refers to presenting optimal meal menus and service options based on the user's emotions.

[0576] The system for implementing this invention includes an interface for receiving instructions from a user, an emotion engine, a natural language processing engine, a task execution engine, and a proposal engine. The specific operation of each engine and the hardware and software used will be described below.

[0577] Interface

[0578] Users can communicate with the system through voice input or text input. Voice input uses a microphone and speech recognition software (e.g., Google Speech Recognition API), while text input uses a keyboard or touchscreen.

[0579] Emotion Engine

[0580] The emotion engine analyzes the emotion from the user's voice or text. It uses a natural language processing library (e.g., TextBlob) for emotion analysis. The emotion engine calculates the user's emotion score and classifies it as positive, negative, or neutral.

[0581] Natural Language Processing Engine

[0582] The natural language processing engine analyzes the user's instructions and selects the appropriate task. A generative AI model (e.g., GPT-3) is used for natural language processing. The generative AI model understands the user's instructions and generates the appropriate prompt.

[0583] Business execution engine

[0584] The business execution engine executes the selected business operations, such as meeting arrangements, business trip arrangements, expense settlement, attendance management, and document preparation, using various APIs and databases (e.g., Google Calendar API, Expedia API).

[0585] Suggestion Engine

[0586] The recommendation engine suggests the most suitable menus and options based on the user's emotions. It makes appropriate suggestions based on the results of emotion analysis. For example, if the user is feeling stressed, it will suggest menus that have a relaxing effect.

[0587] Specific examples

[0588] If a user says, "I'm tired today, so I want to have a relaxing meal," the emotion engine will recognize the negative emotion, and the suggestion engine will suggest a menu that has a relaxing effect.

[0589] Prompt Sentence Examples

[0590] Example prompts to input to a generative AI model:

[0591] If a user says, "I'm tired today, so I want a relaxing meal," design a system that judges the emotion score as negative and suggests a menu that has a relaxing effect.

[0592] In this way, it becomes possible to carry out tasks and make suggestions optimally according to the user's emotions.

[0593] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0594] Step 1:

[0595] Users give instructions to the system through voice input or text input. In the case of voice input, voice data is picked up using the device's microphone and converted into text data using speech recognition software (e.g., Google Speech Recognition API). In the case of text input, text data is entered using the device's keyboard or touch screen. The input data is sent to the server as user instructions.

[0596] Input: User voice or text data

[0597] Output: Text data (user instructions)

[0598] Step 2:

[0599] The server passes the received text data to the emotion engine, which analyzes the user's emotions. A natural language processing library (e.g., TextBlob) is used for emotion analysis. The emotion engine calculates an emotion score from the text data and classifies it as positive, negative, or neutral.

[0600] Input: Text data (user instructions)

[0601] Output: Sentiment score (positive, negative, neutral)

[0602] Step 3:

[0603] The server passes instructions to a natural language processing engine based on the emotion score and selects the appropriate task. A generative AI model (e.g., GPT-3) is used for natural language processing. The generative AI model understands the user's instructions and generates an appropriate prompt.

[0604] Input: Emotion score, text data (user instructions)

[0605] Output: prompt statement

[0606] Step 4:

[0607] The server passes the generated prompt to the business execution engine and executes the selected business. For example, it handles administrative tasks such as meeting arrangements, business trip arrangements, expense settlement, attendance management, and document preparation. This is done using various APIs and databases (e.g., Google Calendar API, Expedia API).

[0608] Input: prompt statement

[0609] Output: Business execution results

[0610] Step 5:

[0611] The server passes instructions to the recommendation engine based on the emotion score and the results of the task execution, and suggests the most appropriate menus and options based on the user's emotions. Appropriate suggestions are made based on the results of emotion analysis. For example, if the user is feeling stressed, a menu with a relaxing effect is suggested.

[0612] Input: Emotion score, business execution results

[0613] Output: Proposal content (menus and options)

[0614] Step 6:

[0615] The server notifies the user of the proposed content. The notification is made via the device's display or audio output. The user can then review the proposed content and make selections or changes as necessary.

[0616] Input: Proposal content (menu and options)

[0617] Output: User notification

[0618] Example 2

[0619] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0620] Conventional administrative services systems perform tasks without considering the user's emotional state, which makes it difficult to reduce the user's stress and frustration. Furthermore, there was a lack of efficient means to obtain the data necessary to perform tasks, which led to problems with reduced work efficiency. Furthermore, there was an insufficient means to properly analyze user instructions and select the most appropriate tasks, making it difficult to reduce the user's workload.

[0621] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0622] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance management, and document preparation based on user instructions; means for proxy attendance at meetings and sharing post-event summaries; means for analyzing the user's emotions and adjusting the way tasks are performed based on those emotions; means for using emotion analysis software to analyze the user's emotions; and means for acquiring data necessary for task execution from an external data provision service. This enables tasks to be performed in accordance with the user's emotional state, thereby reducing user stress and frustration. Furthermore, efficiently acquiring necessary data can improve task efficiency. Furthermore, by appropriately analyzing user instructions and selecting the optimal task, it is possible to reduce the user's workload.

[0623] "Instructions from the user" refers to the operations and inputs that the user makes to the system.

[0624] "Meeting coordination" refers to the task of setting the date, time, and location of a meeting and notifying participants.

[0625] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[0626] "Expense reimbursement" refers to the process of applying for, approving, and settling work-related expenses.

[0627] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and vacations.

[0628] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[0629] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and understanding the content of the meeting.

[0630] "Post-meeting summary sharing" refers to the task of summarizing the meeting contents after the meeting and sharing them with relevant parties.

[0631] "Emotion analysis software" refers to software that analyzes a user's facial expressions and voice to recognize their emotions.

[0632] "External data provision services" refers to external services and APIs that provide data necessary for business operations.

[0633] "Natural language processing" refers to technology for analyzing instructions from users and selecting appropriate tasks.

[0634] "Workload" refers to the burden and stress that users feel when performing their work.

[0635] "Efficient business execution" means performing business operations quickly and accurately.

[0636] This invention is a system that performs administrative tasks such as meeting arrangements, business trip arrangements, expense settlement, attendance management, and document preparation based on user instructions. The system has a function to analyze the user's emotions and adjust the way tasks are performed based on those emotions. It also includes a function to obtain data required for performing tasks from external data providing services.

[0637] Hardware and software used

[0638] Hardware:

[0639] Terminal: A device operated by a user (e.g., a computer, smartphone, tablet)

[0640] Server: A server that manages the entire system and executes various tasks.

[0641] software:

[0642] Emotion analysis software: Software for analyzing a user's facial expressions and voice (e.g., Microsoft Azure (registered trademark) Emotion API)

[0643] Natural language processing software: Software that analyzes user instructions and selects the appropriate task.

[0644] External data services: External services and APIs that provide data necessary for business operations (e.g., airline APIs, hotel booking site APIs)

[0645] Program processing

[0646] The server accepts the user's terminal and selects the task they wish to perform from the system interface. For example, if the user selects business trip arrangements, the server launches the business trip arrangement module and makes flight reservations, hotel reservations, and transportation arrangements. In this case, the server uses the airline's API and the hotel reservation site's API to obtain the necessary data and complete the reservation.

[0647] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice. It uses emotion analysis software, such as Microsoft Azure's Emotion API, to analyze the emotion. Based on the analysis results, the emotion engine adjusts how tasks are performed. For example, if the emotion engine recognizes that the user is angry, it will instruct the task to be performed in a calmer tone.

[0648] Specific examples

[0649] As a concrete example, consider the case where a user selects a business trip and the emotion engine recognizes the user's emotion as "anger." In this case, the server launches the business trip arrangement module and makes flight reservations, hotel reservations, and transportation arrangements. At the same time, the emotion engine performs the tasks in a calm tone to calm the user's anger.

[0650] Prompt Sentence Examples

[0651] If a user selects a business trip and the emotion engine identifies the user's emotion as "anger," how would the system perform its task?

[0652] If you feed this prompt to a generative AI model, you'll get the following response:

[0653] If a user selects a business trip and the emotion engine recognizes the user's emotion as "anger," the system will launch the business trip booking module to book flights, hotels, and transportation. At the same time, the emotion engine will perform the task in a gentle tone to calm the user's anger. For example, it will display a reservation confirmation message in gentle language to reduce the user's stress.

[0654] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0655] Step 1:

[0656] The user selects a task.

[0657] The user selects "Business Trip Arrangements" from the menu displayed on the terminal screen. As input, the user's selection is sent to the system. As output, the server receives the user's selection and proceeds to the next processing step.

[0658] Step 2:

[0659] The server starts a module corresponding to the selected task.

[0660] The server receives the click event of the "Business Trip Arrangements" button and starts the business trip arrangements module. The user's selection information is passed to the server as input. As output, the business trip arrangements module is initialized and proceeds to the next processing step.

[0661] Step 3:

[0662] The server retrieves the necessary data and executes the task.

[0663] The server sends a request to the airline's API based on the user's business trip itinerary to obtain the best flight information. It then sends a request to the hotel reservation site's API to reserve the accommodation. The user's business trip itinerary information is passed to the server as input. The obtained flight information and hotel reservation information are presented to the user as output.

[0664] Step 4:

[0665] The emotion engine analyzes the user's emotions.

[0666] The emotion engine captures the user's facial expressions with the device's camera and analyzes their emotions using emotion analysis software. As input, the user's facial image is sent to the emotion analysis software. As output, the analysis results are returned to the server.

[0667] Step 5:

[0668] The emotion engine adjusts how work is performed based on the analysis results.

[0669] Based on the analysis results, the emotion engine determines that the user is feeling angry and instructs the server to execute the task in a calmer tone. The emotion analysis results are passed to the emotion engine as input. As output, instructions for adjusting the task execution method are sent to the server.

[0670] Step 6:

[0671] The servers complete the work in a coordinated manner.

[0672] The server follows the instructions of the emotion engine to display a message confirming the reservation in a calm tone. As input, instructions for adjusting how the task is performed are passed to the server. As output, a message in a calm tone is displayed to the user.

[0673] (Application example 2)

[0674] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0675] Conventional administrative services systems perform tasks without considering the user's emotional state, making it impossible to reduce the user's psychological burden. Furthermore, they are unable to provide optimal suggestions based on the user's emotions, preventing an improved user experience. This leads to problems such as reduced work efficiency and increased stress for users.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0677] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for recognizing the user's emotional state and adjusting the method of performing work accordingly, and means for making optimal suggestions based on the user's emotional state. This makes it possible to perform work according to the user's emotional state, reducing the user's psychological burden and improving work efficiency.

[0678] "User" means an individual or corporation that uses the system to direct business operations and receive services.

[0679] An "instruction" is a command or request that a user makes to a system to request the system to carry out a task.

[0680] "Meeting coordination" involves coordinating the date, location, and participants of a meeting, and finalizing the schedule.

[0681] "Business trip arrangements" refers to the task of booking and arranging the air tickets, hotels, transportation, etc. required for a business trip.

[0682] "Expense settlement" is the process of calculating and settling expenses related to work.

[0683] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[0684] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[0685] "Administrative agency" is a service that performs various administrative tasks on behalf of users.

[0686] "Proxy attendance" refers to the task of attending a meeting on behalf of a user and understanding the agenda.

[0687] "Summary sharing" is the task of summarizing the contents of a meeting or business and sharing it with the relevant parties.

[0688] "Emotional state" refers to the user's psychological state or mood, and is what the system recognizes.

[0689] "Adjusting how work is performed" means changing the way work is carried out or the way people respond depending on the user's emotional state.

[0690] "Optimal suggestions" means suggesting the most appropriate options or actions based on the user's emotional state and situation.

[0691] The system for implementing this invention performs various administrative tasks based on user instructions, and has the ability to recognize the user's emotional state and adjust the way tasks are performed accordingly. Specifically, the following hardware and software are used:

[0692] Hardware

[0693] Server: A central processing unit that receives and processes business instructions.

[0694] Smartphone: A device that allows users to input instructions and receive results.

[0695] Emotion Recognition Device: A device used to recognize a user's emotional state (e.g., a smartphone camera or microphone).

[0696] software

[0697] Natural language processing library: Software for analyzing user instructions and selecting appropriate tasks.

[0698] Emotion Recognizer: Software for recognizing a user's emotional state.

[0699] Food Recommender: Software for making optimal recommendations based on the user's emotional state.

[0700] Processing flow

[0701] 1. User instruction input: The user inputs instructions to the system using a smartphone. For example, the user inputs instructions that include emotions, such as "I'm very tired today."

[0702] 2. Emotion Recognition: The emotion recognition device recognizes the user's emotional state and analyzes it using an emotion recognition library.

[0703] 3. Job selection: A natural language processing library analyzes the user's instructions and selects the appropriate job.

[0704] 4. Adjusting the way work is performed: Adjusting the way work is performed based on the emotional state recognized by the emotion recognition library.

[0705] 5. Best Suggestion: The food recommendation library makes best suggestions based on the user's emotional state.

[0706] Specific examples

[0707] For example, if a user types, "I'm very tired today," the emotion recognition library will recognize "fatigue," and the food recommendation library will recommend "relaxing herbal tea and snacks."

[0708] Prompt Sentence Examples

[0709] "Tell me how you're feeling: I'm very tired today."

[0710] "Tell me how you feel: I'm very happy today."

[0711] In this way, it becomes possible to carry out work and make optimal suggestions according to the user's emotional state, thereby reducing the user's psychological burden and improving work efficiency.

[0712] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0713] Step 1:

[0714] A user inputs instructions into the system using a smartphone. The input instructions are sent to the server in text format. For example, an instruction including emotions such as "I'm very tired today" is input.

[0715] Step 2:

[0716] The server recognizes the user's emotional state using an emotion recognition device (smartphone camera or microphone). It uses an emotion recognition library (EmotionRecognizer) to analyze the input text data and identify the user's emotional state. The input is text data, and the output is an emotional state such as "fatigue" or "joy."

[0717] Step 3:

[0718] The server analyzes the user's instructions using a natural language processing library. Based on the analyzed instructions, it selects an appropriate task. For example, based on the emotional state of "fatigue," it selects a task with a relaxing effect. The input is the user's instruction text and emotional state, and the output is the selected task.

[0719] Step 4:

[0720] The server adjusts the way the task is performed based on the emotional state recognized by the emotion recognition library. For example, if the user feels tired, the server adjusts the tone of the task to be performed calmly. The input is the emotional state and the selected task, and the output is the adjusted way of performing the task.

[0721] Step 5:

[0722] The server uses a food recommendation library (FoodRecommender) to make optimal suggestions based on the user's emotional state. For example, based on the emotional state of "fatigue," it recommends "relaxing herbal tea and light meals." The input is the emotional state, and the output is the recommended menu.

[0723] Step 6:

[0724] The server sends the optimal proposal to the user's smartphone and displays it to the user. The user can check the proposed menu and task execution method, and make selections or changes as necessary. The input is the recommended menu, and the output is the display to the user.

[0725] Example 3

[0726] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0727] Conventional administrative proxy systems can perform tasks such as coordinating meetings and arranging business trips based on user instructions, but they have issues with not being able to attend meetings on behalf of others, sharing summaries of the meeting content, or adjusting tasks based on the user's emotions. Furthermore, there was a lack of a way to efficiently summarize the content of meetings and share it with users, making it difficult to reduce the user's workload.

[0728] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0729] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for recognizing the user's emotions and adjusting the priority and execution method of tasks according to the emotions, means for using a generative AI model to summarize the content of the meeting, and means for sharing the generated summary with the user. This makes it possible to attend meetings on behalf of the user based on instructions from the user, efficiently summarize and share the content, and also adjust tasks according to the user's emotions.

[0730] A "user" is an entity that uses the system and issues instructions.

[0731] An "instruction" is a request or command given by a user to a system.

[0732] "Meeting coordination" refers to the task of setting and coordinating the date, time, location, and participants of a meeting.

[0733] "Business trip arrangements" refers to the task of arranging transportation, accommodation, and other items necessary for a business trip.

[0734] "Expense settlement" refers to the process of calculating and settling expenses related to work.

[0735] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[0736] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[0737] "Administrative agency" refers to carrying out administrative tasks on behalf of the user.

[0738] "Meeting proxy attendance" refers to attending a meeting on behalf of a user.

[0739] "Summary sharing" refers to providing users with a concise summary of the meeting contents.

[0740] "Emotion recognition" refers to analyzing a user's emotions and understanding those emotions.

[0741] "Task priority adjustment" refers to changing the order in which tasks are performed depending on the user's emotions and situation.

[0742] A "generative AI model" is a model that uses artificial intelligence to analyze data and perform specific tasks.

[0743] "Summary generation" refers to analyzing the content of a meeting, extracting the important points, and summarizing them concisely.

[0744] "Natural language processing" is a technology that uses computers to analyze and understand human language.

[0745] The system of the present invention performs administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance management, and document preparation based on user instructions. It also has the ability to attend meetings on behalf of users and share summaries afterward. It also has the ability to recognize users' emotions and adjust the priorities and execution methods of tasks accordingly.

[0746] Hardware and software used

[0747] Hardware: Servers, terminals (PCs, smartphones)

[0748] Software: Meeting proxy attendance module, emotion engine, generative AI model

[0749] Specific operation of the system

[0750] Receiving user instructions

[0751] The user uses the device to instruct, "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." The device receives this instruction and sends it to the server. Specifically, the user opens the smartphone app and inputs instructions using voice input or text input. The device then sends this instruction to the server as text data.

[0752] Launching the meeting proxy attendance module

[0753] The server analyzes the received instructions and launches the meeting proxy attendance module. The server then prepares to join the meeting at the specified time. Specifically, the server checks the meeting schedule using the calendar API and automatically logs in to the online meeting tool (e.g., Zoom, Microsoft Teams) at the specified time.

[0754] Summary of the meeting

[0755] When the meeting ends, the server uses a generative AI model to summarize the meeting content. The server collects the meeting's audio and text data and inputs it into the generative AI model. Specifically, the server converts the meeting recording data into text using speech recognition software (e.g., Google Speech-to-Text), and then inputs the text data into a generative AI model (e.g., GPT-4) to generate a summary.

[0756] Share a summary

[0757] The server sends the generated summary to the user, who can then view it on their device. Specifically, the server sends the generated summary to the user via email or a chat app (e.g., Slack, Microsoft Teams). The user can then view the summary on their smartphone or PC.

[0758] Emotion Engine Operation

[0759] The user expresses their emotions to the system (e.g., feeling stressed, feeling happy). The server uses an emotion engine to recognize the user's emotions. The server then uses the emotion engine to adjust the priority and execution of tasks according to the user's emotions. Specifically, when a user enters "feeling stressed" in the app, the server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the emotion and prioritize reminders and tasks to reduce stress.

[0760] Specific examples

[0761] User instruction: "Please delegate to the meeting tomorrow at 10am and share the meeting summary."

[0762] Example prompt for a generative AI model: "Summarize the meeting. Extract the key points and summarize them concisely."

[0763] In this way, the system attends meetings on behalf of the user based on the user's instructions, generates and shares a summary of the meeting, and also adjusts tasks according to the user's emotions using an emotion engine. The flow of the specific processing in the third embodiment will be described with reference to FIG. 21.

[0764] Step 1:

[0765] The user uses the device to input instructions. The user opens an app on their smartphone or PC and inputs instructions by voice or text, such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." The device then sends this instruction to the server as text data.

[0766] Input: User's voice or text instructions

[0767] Output: Text data sent to the server

[0768] Step 2:

[0769] The server analyzes the received instructions and launches the meeting proxy attendance module. The server analyzes the instructions using natural language processing and extracts the meeting date, time, and content. The server checks the meeting schedule using a calendar API and prepares to automatically log in to the online meeting tool at the specified time.

[0770] Input: User's instruction text data

[0771] Output: Launch of meeting proxy attendance module

[0772] Step 3:

[0773] The server automatically logs in to the online conference tool at the specified time and participates in the conference. The server collects the conference audio and text data in real time.

[0774] Input: Meeting schedule information

[0775] Output: Meeting audio and text data

[0776] Step 4:

[0777] After the meeting ends, the server uses the generative AI model to summarize the meeting content. The server converts the meeting recording into text using speech recognition software, and then inputs the text data into the generative AI model to generate a summary.

[0778] Input: Meeting audio and text data

[0779] Output: Summary text

[0780] Step 5:

[0781] The server sends the generated summary to the user. The server sends the generated summary to the user via email or a chat app. The user can then check the summary on their smartphone or PC.

[0782] Input: Summary text

[0783] Output: Summary text sent to the user

[0784] Step 6:

[0785] Users express their emotions to the system. When a user enters "I'm feeling stressed" in the app, the server uses an emotion engine to analyze the emotion. The server then uses the emotion engine to adjust the priority and execution of tasks according to the user's emotions.

[0786] Input: User emote text

[0787] Output: Coordinated work priorities and execution methods

[0788] In this way, the system attends meetings on behalf of the user, summarizes the content efficiently, and shares it based on the user's instructions. It also adjusts work according to the user's emotions using an emotion engine.

[0789] (Application example 3)

[0790] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0791] Conventional administrative proxy systems perform tasks without considering the user's feelings, which can affect the user's stress and satisfaction. Furthermore, there was a lack of means for proxy attendance or sharing summaries during meetings and discussions within the factory, making it difficult to carry out tasks efficiently. This increased the user's workload and reduced work efficiency.

[0792] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0793] In this invention, the server includes means for performing administrative tasks such as arranging meetings, arranging business trips, settling expenses, keeping attendance records, and preparing documents based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries afterward, means including an emotion engine that recognizes the user's emotions and adjusts the priority and execution method of work in accordance with those emotions, and means including a robot assistant that attends meetings and conferences in a factory on behalf of the user and summarizes and shares the contents of the meetings. This enables efficient work execution that takes the user's emotions into consideration, and makes it possible to attend meetings and conferences in a factory on behalf of the user and share summaries.

[0794] "User" refers to an individual or organization that uses the system to issue instructions.

[0795] "Instructions" refer to specific requests or commands that a user gives to a system.

[0796] "Meeting coordination" refers to the task of coordinating the date, time, location, and participants of a meeting.

[0797] "Business trip arrangements" refers to the work of arranging transportation, accommodation, etc. necessary for business trips.

[0798] "Expense reimbursement" refers to the work of calculating and reimbursing expenses related to work.

[0799] "Time attendance" refers to the task of managing employee attendance information such as attendance, departure, and vacation.

[0800] "Document creation" refers to the task of creating materials necessary for meetings and business.

[0801] "Administrative agency" refers to carrying out administrative tasks on behalf of the user.

[0802] "Proxy attendance" refers to attending a meeting or conference on behalf of a user.

[0803] "Summary sharing" refers to summarizing the contents of a meeting or conference and sharing it with users.

[0804] An "emotion engine" refers to the part of the system that recognizes the user's emotions and adjusts the priorities and execution methods of work based on those emotions.

[0805] A "robot assistant" is a robot that attends meetings and discussions within a factory on behalf of employees and summarizes and shares the contents of the meetings.

[0806] A system for implementing this invention has functions to handle administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on user instructions. It also includes functions to attend meetings on behalf of users and share summaries afterward, and an emotion engine that recognizes users' emotions and adjusts task priorities and execution methods accordingly. It also includes a robot assistant that attends meetings and conferences in factories on behalf of users and summarizes and shares the contents of those meetings.

[0807] System Program Overview

[0808] The server implements the system using the following hardware and software.

[0809] Hardware: Robots that can move around the factory (e.g. Pepper, Spot)

[0810] software:

[0811] emotion_recognition module: A library for recognizing user emotions

[0812] meeting_summary module: A library for summarizing meeting content

[0813] robot_control module: A library for controlling robots

[0814] Program processing explanation

[0815] The server first receives instructions from the user. The instructions are analyzed using natural language processing, and an appropriate task is selected. For example, if the user instructs the robot to "attend the meeting tomorrow at 10:00 on my behalf and share the meeting summary," the server will launch the meeting proxy attendance module and have the robot join the meeting at the specified time.

[0816] After the meeting ends, the server summarizes the meeting contents and shares them with the user. The emotion engine recognizes the user's emotions and shares the summary with high priority if the user is feeling stressed, and with normal priority if the user is feeling happy.

[0817] Specific examples

[0818] For example, if a factory manager is feeling stressed, the robot can share a summary of the meeting with high priority, allowing the manager to quickly obtain important information and reduce their workload.

[0819] Prompt Sentence Examples

[0820] Examples of prompts for a generative AI model might include:

[0821] If the user is stressed, generate a robot assistant program that shares a meeting summary with high priority.

[0822] In this way, it becomes possible to carry out work efficiently while taking into account the user's emotions, and proxy attendance and summary sharing are also possible in meetings and discussions within the factory.

[0823] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0824] Step 1:

[0825] The server receives instructions from the user. The user instructs, "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." This instruction becomes the input data. The server passes this instruction to the natural language processing module for analysis. As a result of the analysis, it outputs the tasks of attending the meeting on my behalf and sharing the summary.

[0826] Step 2:

[0827] The server launches the meeting proxy attendance module based on the analysis results. It then sets up a schedule for the robot to participate in the meeting at the specified time (in this case, 10:00 tomorrow). The input data are the analysis results and the meeting schedule information, and the output data is the robot's schedule setting information.

[0828] Step 3:

[0829] At the specified time, the server controls the robot to join the meeting. The robot records the meeting audio and video in real time. The input data is the meeting schedule information and the meeting audio and video data, and the output data is the recorded meeting data.

[0830] Step 4:

[0831] After the meeting ends, the server passes the recorded meeting data to the meeting summarization module, which uses speech recognition and natural language processing to summarize the meeting content. The input data is the recorded meeting data, and the output data is the summarized meeting content.

[0832] Step 5:

[0833] The server passes the summarized meeting content to the emotion engine. The emotion engine recognizes the user's emotion and prioritizes summaries based on that emotion. The input data are the summarized meeting content and the user's emotion data, and the output data are the prioritized summary content.

[0834] Step 6:

[0835] The server shares prioritized summary content with the user. If the user is feeling stressed, it is shared with high priority, and if the user is feeling happy, it is shared with normal priority. The input data is the prioritized summary content, and the output data is the summary content sent to the user.

[0836] In this way, it becomes possible to carry out work efficiently while taking into account the user's emotions, and proxy attendance and summary sharing are also possible in meetings and discussions within the factory.

[0837] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0838] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0839] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0840] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0841] [Second embodiment]

[0842] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0843] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0844] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0845] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0846] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0847] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0848] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0849] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0850] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0851] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0852] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0853] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0854] "Example 1"

[0855] The system of the present invention has an interface for receiving instructions from a user. This interface supports formats that allow users to give instructions naturally, such as text input and voice input. For example, if a user instructs, "Please make travel arrangements for a conference in Tokyo next Monday," the system analyzes this instruction and selects the task of making travel arrangements.

[0856] "Example 2"

[0857] The system of the present invention includes modules for executing selected tasks. These modules have functions corresponding to various tasks. For example, if a user selects the task of arranging a business trip, the system will launch the business trip arrangement module and make the necessary arrangements for the business trip, such as reserving an airline ticket, booking a hotel, and arranging transportation.

[0858] "Example 3"

[0859] The system of the present invention can also attend meetings on behalf of users and share the summary after the meeting. For example, if a user requests, "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting," the system will launch the meeting proxy attendance module and join the meeting at the specified time. After the meeting ends, the system will summarize the contents of the meeting and share it with the user.

[0860] The processing flow of each embodiment will be described below.

[0861] "Example 1"

[0862] Step 1: Receive instructions from the user through the interface. For example, receive an instruction such as "Please make travel arrangements for a meeting in Tokyo next Monday."

[0863] Step 2: The received instruction is analyzed using natural language processing. Through this analysis, the task of "business trip arrangements" is selected.

[0864] Step 3: The module corresponding to the selected task is started. In this case, the business trip arrangement module is started.

[0865] Step 4: The travel arrangements module makes the necessary arrangements for the business trip, such as booking air tickets, hotel reservations, and transportation.

[0866] "Example 2"

[0867] Step 1: Receive instructions from the user through the interface. For example, receive an instruction such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting."

[0868] Step 2: The received instruction is analyzed using natural language processing. This analysis results in the selection of the task "attend a meeting on behalf of another person."

[0869] Step 3: Start the module corresponding to the selected task. In this case, the conference proxy attendance module is started.

[0870] Step 4: The meeting proxy attendance module joins the meeting at the specified time.

[0871] Step 5: After the meeting is over, the meeting proxy attendance module summarizes the contents of the meeting.

[0872] Step 6: Share the summarized meeting content with users.

[0873] Example 1

[0874] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0875] Conventional administrative services require users to manually input instructions for various tasks, which does not improve work efficiency. Furthermore, the difficulty of properly analyzing user instructions and responding quickly results in a lack of reduction in workload. Furthermore, the lack of an interface that can accept instructions in natural language poses a problem of low user convenience.

[0876] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0877] In this invention, the server includes means having an interface for receiving instructions from a user via text input or voice input, means for analyzing the user's instructions using natural language processing technology, means for selecting an appropriate task based on the analysis results, and means for executing the selected task. This makes it possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute an appropriate task.

[0878] "User" means any person or entity that uses the System to issue instructions.

[0879] "Instructions" refer to requests or commands given by a user to a system.

[0880] "Interface" refers to the means by which a user inputs instructions into a system, including text input and voice input.

[0881] "Natural language processing technology" refers to technology for analyzing user instructions and understanding their meaning.

[0882] "Parsing" refers to the process of breaking down a user's instructions and extracting meaning.

[0883] "Business" refers to the tasks or work that a system performs based on user instructions.

[0884] "Selection" refers to the process of determining the appropriate task based on the analysis results.

[0885] "Execution" refers to the process of specifically carrying out the selected task.

[0886] "Administrative agency" refers to tasks such as meeting coordination, travel arrangements, expense settlement, attendance management, and document preparation that the system performs on behalf of the user.

[0887] "Meeting coordination" refers to the task of arranging the date and location of a meeting.

[0888] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[0889] "Expense settlement" refers to the work of calculating and settling users' expenses.

[0890] "Time attendance management" refers to the task of managing users' attendance and departure records.

[0891] "Document creation" refers to the task of creating documents and presentation materials based on user instructions.

[0892] "Proxy attendance" refers to the task of attending a meeting on behalf of a user.

[0893] "Summary sharing" refers to the task of summarizing the contents of a meeting and sharing it with users.

[0894] MODE FOR CARRYING OUT THE INVENTION

[0895] The present invention is a system that receives instructions from a user, analyzes them, and selects and executes an appropriate task. A specific embodiment of this system will be described below.

[0896] 1. Receive instructions from the user

[0897] Users input instructions using the device's interface. This interface supports text input and voice input, allowing users to give instructions naturally. For example, a user opens a smartphone app and enters text, such as "Please arrange travel for a meeting in Tokyo next Monday." In the case of voice input, the user speaks the same instruction into the microphone.

[0898] 2. Parsing the instructions

[0899] The server receives instructions sent from the device. It analyzes the text of the instructions using natural language processing technologies such as Google Cloud Natural Language API and IBM Watson Natural Language Understanding. Through this analysis, it extracts the content of the instructions (e.g., "next Monday," "Tokyo," "meeting," "business trip arrangements," etc.).

[0900] 3. Selecting a task

[0901] The server selects the appropriate task based on the analysis results. For example, if the keyword "business trip arrangement" is included, the server selects the task of business trip arrangement. This selection can be done using a predefined rule-based system or a machine learning model (for example, a model using TensorFlow or PyTorch).

[0902] 4. Execute the task

[0903] The server executes the selected task. Specifically, it calls the travel agency's API (for example, Expedia API or Amadeus API) to make business trip arrangements. The server sends the necessary information (date, time, location, purpose, etc.) to the API to complete the reservation. Once the reservation is complete, the server notifies the user of the result. For example, the app may display a message saying, "Business trip arrangements completed."

[0904] Specific examples

[0905] User instructions

[0906] The user instructs the system, "Please arrange travel for a meeting in Tokyo next Monday."

[0907] Prompt Sentence Examples

[0908] Examples of prompts to input to a generative AI model might include:

[0909] The user has instructed us to "make travel arrangements for a conference in Tokyo next Monday." Analyze this instruction and select the task of making travel arrangements.

[0910] This prompt sentence can be used to have a generative AI model parse the instructions and select a task.

[0911] In this way, it becomes possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute the appropriate tasks. This system reduces the user's workload and supports efficient work execution.

[0912] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0913] Step 1:

[0914] The user inputs instructions using the device's interface. The interface supports text input and voice input. For example, a user opens a smartphone app and inputs text such as, "Please arrange a business trip for a meeting in Tokyo next Monday." In the case of voice input, the user speaks the same instruction into the microphone. The input instruction is sent from the device to the server.

[0915] Step 2:

[0916] The server receives instructions sent from the device and analyzes them using natural language processing technology. Specifically, it uses software such as Google Cloud Natural Language API and IBM Watson Natural Language Understanding. The input for analysis is the user's instruction text, and the output is keywords and phrases resulting from the analysis (such as "next Monday," "Tokyo," "meeting," and "business trip arrangements").

[0917] Step 3:

[0918] The server selects the appropriate task based on the analysis results. It selects the task using a predefined rule-based system or machine learning model (for example, a model using TensorFlow or PyTorch) based on keywords and phrases extracted from the analysis results. The input is the analysis results, and the output is the selected task (in this case, "business trip arrangements").

[0919] Step 4:

[0920] The server executes the selected task. Specifically, it calls the travel agency's API (for example, Expedia API or Amadeus API) to make business trip arrangements. The server sends the necessary information (date, time, location, purpose, etc.) to the API to complete the reservation. The input is the selected task and the necessary information, and the output is a notification that the reservation has been completed.

[0921] Step 5:

[0922] The server notifies the user that the reservation is complete. For example, the app displays "Business trip arrangements completed." The input is the reservation completion notification, and the output is a notification message to the user.

[0923] In this way, it becomes possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute the appropriate tasks. This system reduces the user's workload and supports efficient work execution.

[0924] (Application example 1)

[0925] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0926] Conventional administrative services systems often have limited interfaces for receiving user instructions, and in particular do not support voice input. Furthermore, they lack the functionality to analyze user instructions and generate optimal suggestions, making it impossible to fully reduce the workload of users. Furthermore, in certain operations, such as food delivery, it is necessary to efficiently process user instructions and provide optimal suggestions.

[0927] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0928] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, business trip arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for converting the user's voice input into text, means for analyzing the user's instructions using a generative AI model and generating optimal suggestions, and means for completing orders based on the generated suggestions. This makes it possible to efficiently process the user's voice instructions and provide optimal suggestions.

[0929] "User instructions" refer to requests or commands given by a user to a system.

[0930] "Meeting coordination" refers to the task of coordinating the date, location, and participants of a meeting.

[0931] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[0932] "Expense reimbursement" refers to the process of calculating and reimbursing expenses related to work.

[0933] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[0934] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[0935] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[0936] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and understanding the agenda.

[0937] "Post-meeting summary sharing" refers to the task of summarizing the minutes after the meeting and sharing them with relevant parties.

[0938] "Voice input" refers to a method in which a user inputs instructions by voice through a microphone.

[0939] "Convert to text" refers to the process of converting voice-input instructions into text information.

[0940] A "generative AI model" is an algorithm that uses artificial intelligence to analyze user instructions and generate appropriate suggestions.

[0941] An "optimal suggestion" is the most appropriate solution or option derived by the generative AI model based on the user's instructions.

[0942] "Completing an order" refers to the task of carrying out the necessary procedures based on the user's instructions and finalizing the order.

[0943] A system for implementing this invention has an interface for receiving instructions from a user and supports voice and text input. When a user inputs instructions by voice, the system converts the speech into text and analyzes it using a generative AI model. Based on the analysis results, an optimal proposal is generated and presented to the user. Once the user confirms and approves the proposal, the system completes the order.

[0944] Hardware and software used

[0945] Hardware: Smartphone, microphone

[0946] Software: Python, SpeechRecognition library, OpenAI API

[0947] Data processing and calculation

[0948] 1. Speech Recognition: Uses the smartphone's microphone to capture the user's voice input and converts this speech to text using the SpeechRecognition library.

[0949] 2. Generative AI model: Using the OpenAI API, the generative AI model analyzes the textual user instructions and generates optimal suggestions based on the user instructions.

[0950] 3. Present proposal and complete order: The generated proposal is presented to the user, and once the user approves it, the system completes the order.

[0951] Specific examples

[0952] If a user says, "I'd like to order pizza from a nearby Italian restaurant," the system converts the speech to text and analyzes it using a generative AI model. The analysis results suggest the best Italian restaurant and pizza menu. If the user accepts the suggestion, the system sends the order to the restaurant and completes the order.

[0953] Prompt Sentence Examples

[0954] A user says: 'I want to order pizza from an Italian restaurant near me.' Based on this instruction, suggest the best restaurant and menu and complete the order.

[0955] In this way, the user's voice instructions can be processed efficiently and optimal suggestions can be made.

[0956] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0957] Step 1:

[0958] The user uses the smartphone's microphone to input voice.

[0959] Input: User's spoken command (e.g., "I'd like to order pizza from a nearby Italian restaurant.")

[0960] Output: Audio data

[0961] Specific operation: The user activates the voice input function on their smartphone and inputs instructions by voice.

[0962] Step 2:

[0963] The device captures the voice data and converts it to text using the SpeechRecognition library.

[0964] Input: Audio data

[0965] Output: Text data (e.g., "I would like to order pizza from a nearby Italian restaurant")

[0966] Specific operation: The device passes the voice data to the SpeechRecognition library, performs voice recognition, and generates text data.

[0967] Step 3:

[0968] The device sends the text data to the generative AI model for analysis.

[0969] Input: Text data

[0970] Output: Analysis results (e.g., recommendations for the best restaurants and menus)

[0971] Specific operation: The device calls the OpenAI API, sends text data as a prompt, and has it analyzed by the generative AI model.

[0972] Step 4:

[0973] The server receives the analysis results of the generated AI model and presents optimal suggestions to the user.

[0974] Input: Analysis results

[0975] Output: Suggestions (e.g., best Italian restaurants and pizza menus)

[0976] Specific operation: The server sends the analysis results to the user's smartphone and displays suggestions to the user.

[0977] Step 5:

[0978] The user reviews and approves the proposal.

[0979] Input: Proposal

[0980] Output: Approval data

[0981] Specific operation: The user checks the proposal on the smartphone screen and presses the approval button.

[0982] Step 6:

[0983] The server receives the authorization data and completes the order.

[0984] Input: Approval data

[0985] Output: Order completion notification

[0986] What happens: The server uses the authorization data to send the order to the suggested restaurant and notifies the user that the order is complete.

[0987] Example 2

[0988] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0989] Conventional administrative services systems have the problem that it is time-consuming for users to select tasks and input the necessary information, making it difficult to carry out tasks efficiently. Furthermore, while it is required that the information entered by users be processed appropriately and results be provided quickly, there has been a lack of effective means to achieve this. Furthermore, a system that can reduce the workload on users and support efficient task execution is needed.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0991] In this invention, the server includes means for the user to select a task and input the necessary information, means for the terminal to receive the information input by the user and send it to the server, means for the server to start and execute an appropriate module based on the received information, and means for the server to send the arrangement results to the terminal and for the terminal to display the arrangement results to the user. This allows the user to easily select a task and input the necessary information, and the server to quickly start the appropriate module to process and provide the results to the user.

[0992] "User" means an individual or organization that uses the system to select a task and enter the required information.

[0993] "Instructions" refer to the operations that users perform on the system, such as selecting tasks and inputting information.

[0994] "Administrative agency" refers to the performance of tasks such as meeting coordination, travel arrangements, expense settlement, attendance management, and document preparation.

[0995] "Meeting coordination" refers to the task of arranging the date, time, and location of a meeting and notifying participants.

[0996] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[0997] "Expense settlement" refers to the task of organizing and settling expenses incurred for business trips and other work.

[0998] "Attendance management" refers to the task of managing employee attendance information such as attendance, departure, and vacation.

[0999] "Document creation" refers to the task of creating various documents necessary for business operations.

[1000] "Proxy attendance" refers to the task of attending a meeting on behalf of a user and understanding the content of the meeting.

[1001] "Summary sharing" refers to the task of summarizing the contents of a meeting and sharing it with users.

[1002] "Natural language processing" refers to technology for analyzing instructions from users and selecting appropriate tasks.

[1003] "Workload" refers to the effort and time required by a user to carry out their work.

[1004] "Efficient business execution" means performing business operations quickly and accurately.

[1005] "Terminal" refers to a device that allows a user to select a task and enter information.

[1006] "Server" refers to a computer system that receives information from a user, activates the appropriate module, and performs processing.

[1007] A "module" refers to a part of a program for performing a specific task.

[1008] "Arrangement result" refers to the result obtained after the server executes the task.

[1009] The present invention is a system in which a user selects a task and inputs the necessary information, and the server starts up an appropriate module to execute the task. A specific embodiment of this system will be described below.

[1010] Hardware and software used

[1011] server

[1012] The server is a high-performance computer system that manages and executes business modules. Specifically, it uses general server hardware (e.g., a high-performance server computer), and business modules are developed in Python and run on the server using the Django framework. MySQL is used as the database.

[1013] Terminal

[1014] The terminal is a device that allows the user to select a task and enter the necessary information. Specifically, the terminal is a PC or smartphone (e.g., Windows PC or smartphone) that the user uses.

[1015] System Operation

[1016] User Actions

[1017] The user uses the terminal interface to select the task they want to perform. For example, to select the task of arranging a business trip, the user clicks the "Business Trip Arrangements" button. The user then enters information such as the departure point, destination, departure date, and return date.

[1018] Terminal handling

[1019] The device receives the information entered by the user and sends it to the server using an HTTP request, which communicates the user's instructions to the server.

[1020] Server Processing

[1021] The server analyzes the received information and selects and launches the appropriate business module. For example, it launches a business trip arrangement module. This module makes the arrangements necessary for the business trip, such as booking an airline ticket, booking a hotel, and arranging transportation. To do this, it uses external APIs (e.g., airline APIs, hotel booking site APIs).

[1022] Displaying the results

[1023] Once the arrangements are complete, the server sends the results to the terminal. Specifically, it generates JSON data including detailed arrangement information (e.g., reservation confirmation number, hotel address, transportation details) and sends it as an HTTP response to the terminal. The terminal then displays the received arrangement results to the user. Specifically, it displays the arrangement results on the screen in an easy-to-read format. For example, it could display a list of the airline ticket reservation confirmation number, hotel address, and transportation details.

[1024] Specific examples

[1025] Specific examples of business trip arrangements

[1026] 1. The user clicks the "Business Trip Arrangements" button on the device.

[1027] 2. The user enters the departure location as "Tokyo," the destination as "New York," the departure date as "December 1, 2023," and the return date as "December 5, 2023."

[1028] 3. The device sends the input information to the server.

[1029] 4. The server launches the business trip arrangement module.

[1030] 5. The server will book your flight, hotel and transportation.

[1031] 6. The server sends the arrangement results to the terminal.

[1032] 7. The terminal displays the arrangement results to the user.

[1033] Example prompts for generative AI models

[1034] "Please select the business trip arrangement task, enter the departure point as Tokyo, the destination as New York, the departure date as December 1, 2023, and the return date as December 5, 2023."

[1035] In this way, users can easily make business trip arrangements.

[1036] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1037] Step 1:

[1038] The user selects a task.

[1039] The user uses the terminal interface to select the task they wish to perform. For example, to select the task of arranging a business trip, the user clicks the "Business Trip Arrangements" button. The input is an instruction to select the task, and the output is information about the selected task.

[1040] Step 2:

[1041] The terminal receives the user's input information.

[1042] The user inputs information such as the departure point, destination, departure date, and return date. The terminal receives this information and stores it as input information. The input includes various pieces of information entered by the user, and the output is stored in the terminal.

[1043] Step 3:

[1044] The terminal sends the input information to the server.

[1045] The device sends the received information to the server using an HTTP request. Specifically, it converts the input information into JSON format and sends it to the server's API endpoint. The input is the information entered by the user, and the output is the information sent to the server.

[1046] Step 4:

[1047] The server launches the appropriate module based on the information received.

[1048] The server analyzes the received information and selects and launches the appropriate business module. For example, it launches a business trip arrangement module. The input is the information sent from the terminal, and the output is the launched module.

[1049] Step 5:

[1050] The server executes the module and makes the necessary arrangements.

[1051] The server executes the launched module and makes the necessary arrangements. In the case of the business trip arrangement module, it makes flight reservations, hotel reservations, and transportation arrangements. For this, it uses external APIs (e.g., airline APIs, hotel reservation site APIs). The input is the user's business trip information, and the output is the arrangement results.

[1052] Step 6:

[1053] The server sends the arrangement result to the terminal.

[1054] Once the arrangements are complete, the server sends the results to the terminal. Specifically, it generates JSON data containing detailed arrangement information (e.g., reservation confirmation number, hotel address, transportation details) and sends it as an HTTP response to the terminal. The input is the arrangement results, and the output is the arrangement results sent to the terminal.

[1055] Step 7:

[1056] The terminal displays the results of the arrangement to the user.

[1057] The terminal displays the received arrangement results to the user. Specifically, the arrangement results are displayed on the screen in an easy-to-read format. For example, the airline ticket reservation confirmation number, hotel address, and transportation details are displayed in a list. The input is the arrangement results sent from the server, and the output is the arrangement results displayed to the user.

[1058] (Application example 2)

[1059] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1060] While conventional administrative services can efficiently handle tasks such as coordinating meetings, arranging business trips, and expense settlements, they do not address other needs in users' daily lives, such as arranging food delivery. This requires users to use a separate food delivery app, which creates a disconnect between managing work and daily life. Furthermore, there is no system that automatically selects the optimal restaurant based on the user's location and food preferences, arranges delivery methods, or adjusts delivery times, reducing user convenience.

[1061] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1062] In this invention, the server includes means for handling administrative tasks such as arranging meetings, arranging business trips, settling expenses, managing attendance, and preparing documents based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries afterward, and means for searching for the most suitable restaurant based on the user's location information and food preferences, ordering food, arranging delivery means, and adjusting delivery times. This allows users to efficiently manage both their work and daily lives using a single system.

[1063] "Instructions from the user" refers to the operations or inputs that the user makes to the system, and the system then performs a specific task based on these instructions.

[1064] "Meeting coordination" refers to the task of coordinating the date, time, location, and participants of a meeting, and finalizing the schedule.

[1065] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[1066] "Expense reimbursement" refers to the process of applying for, approving, and settling work-related expenses.

[1067] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and vacations.

[1068] "Document creation" refers to the task of creating documents and presentation materials needed for meetings, reports, etc.

[1069] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and preparing minutes and summaries.

[1070] "Post-event summary sharing" refers to the task of summarizing the contents of meetings and discussions and sharing them with relevant parties.

[1071] "Location information" refers to information about the user's current location or a specified location, and the system uses this information to perform its tasks.

[1072] "Food preferences" refers to the types of food and cuisine that a user prefers, and the system uses this information to select the most suitable restaurant.

[1073] "Restaurant search" refers to the task of finding suitable restaurants based on the user's location information and food preferences.

[1074] "Ordering food" refers to the task of ordering specific food from a selected restaurant.

[1075] "Arranging delivery means" refers to the task of arranging means to deliver the ordered food to the user's designated location.

[1076] "Delivery time adjustment" refers to the task of adjusting the delivery time so that the food arrives at the user's designated location at the appropriate time.

[1077] A system for implementing this invention includes means for carrying out administrative tasks such as coordinating meetings, arranging business trips, settling expenses, keeping attendance records, and preparing documents based on instructions from a user, means for attending meetings on behalf of a user and sharing summaries afterward, and means for searching for the most suitable restaurant based on the user's location information and food preferences, ordering food, arranging for delivery, and adjusting delivery times.

[1078] 1. System Program

[1079] The system is implemented using Python programs and REST APIs. The interface for receiving user instructions is provided as a smartphone application. When a user enters instructions through the application, the instructions are sent to the server, which then performs the appropriate tasks based on the instructions.

[1080] 2. Program Processing

[1081] The server processes and calculates data using the following hardware and software.

[1082] Hardware: Servers, smartphones

[1083] Software: Python, REST API, natural language processing libraries (e.g. spaCy, NLTK)

[1084] Processing flow

[1085] 1. Receiving user instructions: The user inputs instructions through a smartphone application. For example, "I want to eat sushi in Tokyo."

[1086] 2. Natural Language Processing: The server uses a natural language processing library to analyze the user's instructions and select the appropriate task.

[1087] 3. Execution of tasks: The server executes the following tasks based on the analysis results.

[1088] Restaurant search: Find the best restaurants based on your location and food preferences.

[1089] Food Ordering: Order specific food from selected restaurants.

[1090] Arranging delivery: Arranging delivery of the ordered food to the user's designated location.

[1091] Delivery time adjustment: Adjust delivery times to ensure food arrives at the user's desired location at the appropriate time.

[1092] 3. Specific Examples

[1093] As a concrete example, the prompt sentence when the user inputs "I live in Tokyo and I want to eat sushi" is as follows:

[1094] Prompt Sentence Examples

[1095] User location: Tokyo

[1096] Food preference: Sushi

[1097] Based on this prompt, the system operates as follows:

[1098] 1. Restaurant Search: Search for sushi restaurants in Tokyo and choose the best one.

[1099] 2. Order food: Order a "Sushi Set" at the selected restaurant.

[1100] 3. Arrange delivery: Once the order is completed, arrange delivery and schedule delivery based on the user's location.

[1101] In this way, users can easily use food delivery services.

[1102] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1103] Step 1:

[1104] A user inputs a command through a smartphone application, for example, "I want to eat sushi in Tokyo." This command is then sent from the application to the server.

[1105] Input: User instructions (e.g., "I want to eat sushi in Tokyo")

[1106] Output: User instructions sent to the server

[1107] Step 2:

[1108] The server uses a natural language processing library (e.g., spaCy, NLTK) to analyze the user's instructions, extracting their location (Tokyo) and food preference (sushi).

[1109] Input: User instructions

[1110] Output: Analysis results (location information and food preferences)

[1111] What it does: The server uses a natural language processing library to analyze the text and extract location information and food preferences.

[1112] Step 3:

[1113] Based on the analysis results, the server calls a restaurant search API and searches for restaurants that match the user's location information and food preferences.

[1114] Input: Analysis results (location information and food preferences)

[1115] Output: List of restaurants

[1116] Specific operation: The server passes location information and food preferences as parameters to the restaurant search API and obtains a list of the most suitable restaurants.

[1117] Step 4:

[1118] The server selects the most suitable restaurant from the acquired list of restaurants and places an order for food.

[1119] Input: A list of restaurants

[1120] Output: Order confirmation information

[1121] Specific operation: The server selects the most suitable restaurant from the list of restaurants based on criteria such as rating and distance, and calls the ordering API to order food.

[1122] Step 5:

[1123] After the order is completed, the server calls the delivery means arrangement API and arranges for the means to deliver the food to the location specified by the user.

[1124] Input: Order confirmation information

[1125] Output: Delivery arrangement confirmation information

[1126] Specific operation: The server calls the delivery means arrangement API based on the order confirmation information and arranges for delivery means.

[1127] Step 6:

[1128] After arranging the delivery means, the server adjusts the delivery time and notifies the user.

[1129] Input: Delivery means arrangement confirmation information

[1130] Output: Delivery time notification

[1131] Specific operation: The server adjusts the delivery time based on the delivery means arrangement confirmation information and notifies the user.

[1132] In this way, users can easily use food delivery services.

[1133] Example 3

[1134] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1135] Conventional administrative proxy systems did not efficiently handle proxy attendance at meetings or the subsequent sharing of summaries, and were unable to sufficiently reduce the user's workload. Furthermore, the process of recording, transcribing, and generating summaries of meeting content was not automated, requiring users to perform the tasks manually, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to quickly share the generated summaries, which sometimes resulted in delays in the transmission of information.

[1136] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1137] In this invention, the server includes means for performing administrative tasks such as meeting coordination, travel arrangements, expense settlement, time attendance, and document preparation based on instructions from the user, means for attending the meeting on behalf of the user and sharing a summary afterwards, means for recording the audio data of the meeting and transcribing it using speech recognition software, means for summarizing the transcript using a generative AI model, and means for sharing the summary with the user. This automates the entire process from attending the meeting on behalf of the user to generating and sharing the summary, significantly reducing the user's workload and enabling more efficient work performance.

[1138] "User" means any person or entity that uses the System to issue instructions.

[1139] An "instruction" is a request or command given by a user to a system.

[1140] "Meeting coordination" refers to the task of coordinating the date, location, and participants of a meeting.

[1141] "Business trip arrangements" refers to the work of arranging transportation and accommodation necessary for business trips.

[1142] "Expense settlement" refers to the process of calculating and settling expenses incurred for business trips and other work.

[1143] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[1144] "Document creation" refers to the task of creating documents and presentation materials required for meetings and business.

[1145] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[1146] "Meeting proxy attendance" refers to attending a meeting on behalf of a user.

[1147] "Summary sharing" refers to summarizing the contents of a meeting and communicating that summary to users.

[1148] "Audio data" refers to data that records the audio of a conference.

[1149] "Speech recognition software" means software for converting voice data into text.

[1150] "Transcription" is the process of converting audio data into text.

[1151] A "generative AI model" is a model that uses artificial intelligence to process data and perform summarization or other generative tasks.

[1152] A "summary" is a text that briefly summarizes the contents of a meeting.

[1153] "Sharing" refers to communicating the generated summary to a user.

[1154] MODE FOR CARRYING OUT THE INVENTION

[1155] This invention is a system that handles administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on user instructions. It also has the functionality to attend meetings on behalf of users and share summaries afterward. This system operates in cooperation with the server, the terminal, and the user.

[1156] Hardware and software used

[1157] server:

[1158] The server plays a central role in receiving instructions from users and carrying out various administrative tasks.

[1159] The server joins the meeting using video conferencing software (e.g., Zoom, Microsoft Teams).

[1160] The server transcribes the meeting audio data using speech recognition software (e.g., Google Cloud Speech-to-Text API).

[1161] The server summarizes the transcription data using a generative AI model (e.g., OpenAI's GPT-4).

[1162] Device:

[1163] A terminal is a device that allows a user to input instructions, such as a PC or smartphone.

[1164] The terminal provides an interface for transmitting user instructions to the server.

[1165] user:

[1166] A user is an individual or corporation that uses the system to instruct administrative services.

[1167] Users use terminals to input instructions into the system and receive results.

[1168] Program processing

[1169] Upon receiving instructions from the user, the server launches a meeting proxy attendance module, which joins the meeting using video conferencing software at the specified time. Audio data from the meeting is recorded in real time and transcribed using speech recognition software. The transcribed data is then summarized using a generative AI model. The generated summary is shared with the user via email or a chat app.

[1170] Specific examples

[1171] When a user instructs the server to "attend the meeting tomorrow at 10:00 on his behalf and share the summary of the meeting," the server performs the following process.

[1172] 1. The server starts the conference proxy attendance module and joins the designated conference using the video conferencing software.

[1173] 2. The server records the audio of the meeting and transcribes it using speech recognition software.

[1174] 3. The server inputs the transcription data into a generative AI model to generate a summary of the meeting.

[1175] 4. The server sends the generated summary to the user's email address.

[1176] Example prompt sentence:

[1177] "Please summarize the transcript of the following meeting. Please include key agenda items, decisions made, and upcoming action items."

[1178] In this way, the server attends the conference on behalf of the user based on the user's instructions, and generates and shares a summary of the conference. This significantly reduces the user's workload and enables efficient work execution. The flow of the specific processing in the third embodiment will be described with reference to FIG. 15.

[1179] Step 1:

[1180] The user inputs instructions into the system using a terminal. Specifically, the user inputs an instruction such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting," and presses the send button. This instruction is then sent to the server.

[1181] Input: User instructions (natural language)

[1182] Output: Instruction data sent to the server

[1183] Step 2:

[1184] The server receives instructions from the user, analyzes the instructions using natural language processing, and extracts information (date and time, conference URL, etc.) from the analyzed instructions to launch the conference proxy attendance module.

[1185] Input: User-specified data

[1186] Output: Parsed instruction information (date and time, meeting URL, etc.)

[1187] Step 3:

[1188] The server launches the meeting proxy attendance module at the specified time. Specifically, the server joins the meeting using video conferencing software (e.g., Zoom, Microsoft Teams). The server automatically connects to the meeting using the meeting URL.

[1189] Input: Parsed instruction information (date and time, meeting URL, etc.)

[1190] Output: Meeting participation status

[1191] Step 4:

[1192] The server records the audio data of the meeting in real time, and the recorded audio data is transcribed using speech recognition software (e.g., Google Cloud Speech-to-Text API).

[1193] Input: Meeting audio data

[1194] Output: Transcribed text data

[1195] Step 5:

[1196] The server uses a generative AI model (e.g., OpenAI's GPT-4) to summarize the transcription data. The server inputs the prompt "Please summarize the transcription data of the following meeting. Please include important agenda items, decisions, and next action items" into the generative AI model and generates a summary.

[1197] Input: Transcribed text data, prompt sentence

[1198] Output: Summarized text data

[1199] Step 6:

[1200] The server then shares the generated summary with the user. Specifically, it sends the summary to the user's email address or chat app (e.g., Slack, Microsoft Teams). The user can then check the summary and understand the content of the meeting.

[1201] Input: Summarized text data

[1202] Output: Summary data sent to the user

[1203] In this way, the server attends the meeting on behalf of the user and generates and shares a summary of the meeting based on the user's instructions, significantly reducing the user's workload and enabling them to carry out their work more efficiently.

[1204] (Application example 3)

[1205] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1206] In today's business environment, meetings are becoming more frequent, making it difficult for field workers and managers to attend every meeting. It is also important to accurately understand the content of meetings and efficiently summarize and share them, but this requires a great deal of time and effort. This increases the workload and hinders efficient business execution, creating a challenge.

[1207] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1208] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance management, and document preparation based on instructions from the user, means for attending the meeting on behalf of the user and sharing a summary afterwards, means for recording the audio of the meeting and converting it into text using speech recognition technology, means for summarizing the meeting content using a generative AI model, and means for sharing the summary results with the user. This enables the user to efficiently understand the meeting content without attending the meeting and reduces the workload.

[1209] "User" refers to any individual or legal entity that uses the System.

[1210] "Instructions" refer to commands or requests made by a user to a system.

[1211] "Meeting coordination" refers to the task of setting and coordinating the date, time, location, and participants of a meeting.

[1212] "Business trip arrangements" refers to the work of arranging transportation, accommodation, etc. necessary for business trips.

[1213] "Expense reimbursement" refers to the work of calculating and reimbursing expenses related to work.

[1214] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and vacations.

[1215] "Document creation" refers to the task of creating documents and presentation materials required for meetings and business.

[1216] "Administrative agency" refers to the business of performing administrative tasks on behalf of users.

[1217] "Meeting proxy attendance" refers to the task of attending a meeting on behalf of a user and recording the meeting contents.

[1218] "Post-meeting summary sharing" refers to the task of summarizing the meeting contents after the meeting and sharing them with users.

[1219] "Speech recognition technology" refers to technology that converts speech into text.

[1220] "Generative AI model" refers to a model that uses artificial intelligence to generate or summarize text.

[1221] "Summary result" refers to a text summarizing the contents of the meeting.

[1222] "Sharing" refers to providing the summary results to the user.

[1223] A system for implementing this invention includes a means for performing administrative tasks such as coordinating meetings, arranging travel, settling expenses, keeping attendance records, and preparing documents based on instructions from a user, a means for attending meetings on behalf of a user and sharing summaries afterward, a means for recording the audio of the meeting and converting it into text using speech recognition technology, a means for summarizing the meeting content using a generative AI model, and a means for sharing the summary results with the user.

[1224] When the server receives instructions from a user, it first analyzes them using natural language processing technology. Based on the analyzed instructions, it selects and executes the appropriate task. For example, if a user instructs the server to "attend the meeting tomorrow at 10:00 on my behalf and share the meeting summary," the server will launch the meeting proxy attendance module and join the meeting at the specified time.

[1225] When a meeting starts, a device (e.g., a microphone mounted on a factory robot) records the audio of the meeting. The recorded audio data is converted into text using speech recognition technology (e.g., Google Speech Recognition API). The converted text is summarized using a generative AI model (e.g., Hugging Face's transformers library).

[1226] The summary results are shared with users via the server, allowing users to efficiently understand the content of the meeting without attending the meeting.

[1227] As a concrete example, consider the case where a factory manager instructs, "Please attend the production meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." When this prompt sentence is entered into the system, the system will attend the meeting at the specified time, and after it is over, generate a summary and share it with the manager.

[1228] Example prompt sentence:

[1229] "I would like you to attend the production meeting tomorrow at 10:00 on my behalf and share a summary of the meeting."

[1230] In this way, it becomes possible to reduce the workload of the user and support efficient business execution.

[1231] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1232] Step 1:

[1233] The user enters instructions into the system.

[1234] Input: User instructions (e.g., "Please attend the production meeting tomorrow at 10:00 on my behalf and share the summary of the meeting.")

[1235] Output: Parsed instructions

[1236] Specific operation: The user inputs instructions in natural language, and the server receives the instructions.

[1237] Step 2:

[1238] The server analyzes the user's instructions using natural language processing technology.

[1239] Input: User instructions

[1240] Output: Parsed instructions (e.g., request for meeting date, time, content, and summary)

[1241] Specific operation: The server uses natural language processing technology (e.g., NLTK or spaCy) to parse the instructions and extract the necessary information.

[1242] Step 3:

[1243] The server starts the conference proxy attendance module and participates in the conference at the designated time.

[1244] Input: Parsed instructions (meeting date and time, content)

[1245] Output: Ready to join the meeting

[1246] Specific operation: The server starts the conference proxy attendance module and prepares to participate in the conference at the specified date and time.

[1247] Step 4:

[1248] The device records the audio of the meeting.

[1249] Input: Meeting audio

[1250] Output: Recorded audio data

[1251] Specific operation: A terminal (for example, a microphone mounted on a factory robot) records the audio of the meeting in real time.

[1252] Step 5:

[1253] The server uses voice recognition technology to convert the recorded voice data into text.

[1254] Input: Recorded audio data

[1255] Output: Text data

[1256] Specific operation: The server uses speech recognition technology (e.g., Google Speech Recognition API) to convert the voice data into text.

[1257] Step 6:

[1258] The server summarizes the text data using a generative AI model.

[1259] Input: Text data

[1260] Output: Summarized text

[1261] What it does: The server uses a generative AI model (e.g., Hugging Face's transformers library) to summarize the text data.

[1262] Step 7:

[1263] The server shares the summary results with the user.

[1264] Input: Summarized text

[1265] Output: Summary results shared with users

[1266] Specific operation: The server shares the summary results with the user by email, notification, etc.

[1267] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1268] "Example 1"

[1269] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system receives instructions from a user and performs tasks based on those instructions. The emotion engine recognizes the user's emotions and adjusts the priority and execution method of tasks according to those emotions. For example, if the emotion engine recognizes that the user is feeling stressed, it will prioritize tasks that have the potential to reduce stress. Also, if the emotion engine recognizes that the user is feeling joy, it will select tasks that will maintain that joy. In this way, the emotion engine performs tasks taking the user's emotions into consideration, thereby achieving more efficient task execution that provides greater user satisfaction.

[1270] "Example 2"

[1271] In another embodiment of the present invention, a system is provided in which an emotion engine adjusts a task execution method in accordance with a user's emotion. For example, if the emotion engine recognizes that the user is feeling angry, it selects a task to calm the anger and executes the task in a gentle tone. Alternatively, if the emotion engine recognizes that the user is feeling sad, it selects a task to ease the sadness and executes the task with encouraging words. In this way, the emotion engine adjusts the task execution method taking the user's emotion into consideration, thereby realizing task execution adapted to the user's emotional state.

[1272] "Example 3"

[1273] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system receives instructions from a user and performs tasks based on those instructions. The emotion engine recognizes the user's emotions and adjusts the priority and execution method of tasks according to those emotions. For example, if the emotion engine recognizes that the user is feeling stressed, it will prioritize tasks that have the potential to reduce stress. Also, if the emotion engine recognizes that the user is feeling joy, it will select tasks that will maintain that joy. In this way, the emotion engine performs tasks taking the user's emotions into consideration, thereby achieving more efficient task execution that provides greater user satisfaction.

[1274] The processing flow of each embodiment will be described below.

[1275] "Example 1"

[1276] Step 1: The system receives instructions from the user.

[1277] Step 2: The emotion engine recognizes the user's emotions.

[1278] Step 3: The emotion engine adjusts task priorities based on the emotions it recognizes.

[1279] Step 4: The system executes its work based on the adjusted priorities.

[1280] "Example 2"

[1281] Step 1: The system receives instructions from the user.

[1282] Step 2: The emotion engine recognizes the user's emotions.

[1283] Step 3: The emotion engine adjusts how work is performed depending on the emotions it recognizes.

[1284] Step 4: The system performs the task based on the adjusted execution method.

[1285] Example 1

[1286] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1287] Conventional administrative services systems are unable to consider the user's emotions or stress levels when carrying out tasks based on the user's instructions, making it difficult to carry out tasks efficiently and with high satisfaction. In addition, they are unable to properly prioritize tasks, making it difficult to reduce the user's workload.

[1288] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1289] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for analyzing the user's instructions using natural language processing and selecting and executing appropriate tasks, means for recognizing the user's emotions and adjusting the priority and execution method of tasks according to those emotions, and means for notifying the user when tasks are completed. This allows tasks to be performed efficiently and with a high level of satisfaction by taking the user's emotions into consideration.

[1290] "User" means an individual or corporation that uses the system to issue instructions and request work.

[1291] An "instruction" is a request or command given by a user to a system, and may take the form of text input or voice input.

[1292] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[1293] "Emotion recognition" is a technology that analyzes and recognizes a user's emotional state.

[1294] "Task priority" refers to the order in which a task is determined to be performed first among multiple tasks.

[1295] "Administrative agency" refers to performing tasks such as coordinating meetings, arranging business trips, settling expenses, managing attendance, and preparing documents on behalf of users.

[1296] "Attending a meeting on your behalf" means attending a meeting on your behalf and recording the contents of the meeting.

[1297] "Summary sharing" refers to providing users with a concise summary of the contents of a meeting or work.

[1298] "Notification of task completion" is the act of notifying the user that the system has completed a task.

[1299] An "emotion engine" is an engine that has the function of analyzing the user's emotions and adjusting the priorities and execution methods of work based on those emotions.

[1300] The system of the present invention has an interface for receiving instructions from a user. This interface supports formats that allow users to give instructions naturally, such as text input and voice input. For example, if a user instructs, "Please make travel arrangements for a conference in Tokyo next Monday," the system analyzes this instruction and selects the task of making travel arrangements.

[1301] Hardware and software used

[1302] Hardware: Servers, terminals (PCs, smartphones)

[1303] Software: Natural language processing engines (e.g., Google Cloud Natural Language API), emotion recognition engines (e.g., IBM Watson Tone Analyzer), generative AI models (e.g., OpenAI GPT-3)

[1304] Specific operation of the system

[1305] 1. User input instructions:

[1306] The user uses the device to input instructions using text or voice.

[1307] Specific operation: The user opens the app on their smartphone and gives a voice command such as, "Please make travel arrangements for a meeting in Tokyo next Monday."

[1308] 2. The device sends instructions to the server:

[1309] The terminal receives the user's input and sends it to the server.

[1310] Specific operation: The smartphone converts the voice data into text and sends the text data to the server.

[1311] 3. The server parses the instructions:

[1312] The server uses a natural language processing engine to parse the user's instructions.

[1313] Specific operation: The server analyzes the text "Please make travel arrangements for a meeting in Tokyo next Monday" and identifies the task of making travel arrangements.

[1314] 4. The server recognizes emotions:

[1315] The server uses an emotion recognition engine to analyze the user's emotions.

[1316] Specific behavior: The server recognizes that it is experiencing stress from the user's instructions.

[1317] 5. The server selects and prioritizes tasks:

[1318] The server uses an emotion engine to prioritize tasks based on the user's emotions.

[1319] Specific action: The server decides to prioritize business trip arrangements to reduce stress.

[1320] 6. The server performs the task:

[1321] The server collects the necessary data and arranges the trip.

[1322] What happens: The server accesses an airline booking website and reserves the best flight for the user, as well as a hotel reservation.

[1323] 7. The server sends the results to the device:

[1324] The server notifies the terminal of the completion of the task.

[1325] Specific operation: The server generates a message saying "Business trip arrangements have been completed. Details have been sent by email." and sends it to the terminal.

[1326] 8. The device notifies the user of the results:

[1327] The device receives the notification from the server and displays it to the user.

[1328] What happens: The smartphone displays a notification to let the user know that their travel arrangements have been completed.

[1329] Examples and prompts

[1330] Examples:

[1331] The user instructs, "Please arrange travel for a meeting in Tokyo next Monday."

[1332] The terminal sends this instruction to the server.

[1333] The server analyzes the instructions and identifies the business trip arrangement task.

[1334] The server uses an emotion recognition engine to determine that the user is feeling stressed.

[1335] The server decides to give top priority to business trip arrangements.

[1336] The server makes flight and hotel reservations.

[1337] The server notifies the terminal of the completion of the task.

[1338] The terminal notifies the user that the travel arrangements have been completed.

[1339] Example prompt sentence:

[1340] "A user has requested travel arrangements for a meeting in Tokyo next Monday. Analyze the user's emotions and make arrangements quickly if they are stressed."

[1341] In this way, the system of the present invention analyzes the user's instructions and performs tasks taking into account the user's emotions, thereby providing efficient and satisfying services.

[1342] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1343] Step 1:

[1344] The user inputs instructions.

[1345] Specific operation: The user uses a device (PC or smartphone) to input instructions via text or voice.

[1346] Input: User instructions (e.g., "Please make travel arrangements for a meeting in Tokyo next Monday.")

[1347] Output: User's instruction data (text or audio data)

[1348] Step 2:

[1349] The terminal sends an instruction to the server.

[1350] Specific operation: The device converts the voice data into text and sends the text data to the server.

[1351] Input: User's instruction data (text or voice data)

[1352] Output: Instruction data in text format (sent to the server)

[1353] Step 3:

[1354] The server parses the instructions.

[1355] What happens: The server uses a natural language processing engine to parse the user's instructions.

[1356] Input: Instruction data in text format

[1357] Output: Analysis results (identification of work, e.g., business trip arrangements)

[1358] Step 4:

[1359] The server recognizes emotions.

[1360] Specific operation: The server uses an emotion recognition engine to analyze the user's emotions.

[1361] Input: Instruction data in text format

[1362] Output: Sentiment analysis result (e.g., feeling stressed)

[1363] Step 5:

[1364] The server selects and prioritizes the work.

[1365] Specific operation: The server uses the emotion engine to prioritize tasks based on the user's emotions.

[1366] Input: Analysis results (job identification) and sentiment analysis results

[1367] Output: Priority-set tasks (e.g., business trip arrangements are given top priority)

[1368] Step 6:

[1369] The server performs the work.

[1370] Specific operation: The server collects the necessary data and arranges the trip.

[1371] Input: Prioritized tasks

[1372] Output: The result of the task (e.g., flight and hotel reservations completed)

[1373] Step 7:

[1374] The server sends the results to the terminal.

[1375] Specific operation: The server notifies the terminal that the task has been completed.

[1376] Input: Work execution results

[1377] Output: Completion notification message (e.g. "The business trip arrangements have been completed. Details have been sent by email.")

[1378] Step 8:

[1379] The terminal notifies the user of the result.

[1380] Specific behavior: The device receives the notification from the server and displays it to the user.

[1381] Input: Completion notification message

[1382] Output: Notification to the user (e.g., notification displayed on smartphone)

[1383] (Application example 1)

[1384] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1385] Conventional administrative outsourcing systems perform tasks without considering the user's emotions, which can result in reduced user satisfaction and operational efficiency. Food delivery services also face the problem of not being able to fully meet user needs because they do not suggest optimal menus based on the user's emotions. There is a need to solve these issues and realize optimal task execution and suggestions based on the user's emotions.

[1386] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1387] In this invention, the server includes means for performing administrative tasks such as arranging meetings, arranging business trips, settling expenses, managing attendance, and preparing documents based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries afterward, means for recognizing the user's emotions and adjusting the priority and execution method of tasks according to those emotions, means for making optimal suggestions based on the user's emotions, means for receiving the user's instructions through voice input or text input, and means for suggesting appropriate menus and options based on emotion analysis. This enables optimal task execution and suggestions according to the user's emotions.

[1388] "User instructions" refer to requests or commands given by a user to a system.

[1389] "Meeting coordination" refers to the task of coordinating the date, time, location, and participants of a meeting.

[1390] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[1391] "Expense settlement" refers to the work of calculating and settling expenses related to work.

[1392] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[1393] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[1394] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[1395] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and preparing minutes and summaries.

[1396] "Post-meeting summary sharing" refers to the task of sharing minutes and summaries with relevant parties after the meeting has ended.

[1397] "Emotion recognition" means analyzing emotions from a user's voice or text and understanding those emotions.

[1398] "Adjusting the priority and execution method of work" means changing the importance of work and execution procedures according to the user's feelings.

[1399] "Optimal suggestions" means presenting the most appropriate options based on the user's emotions and situation.

[1400] "Voice input and text input" refers to the way in which a user gives instructions to a system by voice or text.

[1401] "Emotion analysis" is a technology that analyzes emotions from a user's voice or text.

[1402] "Menu and option suggestions" refers to presenting optimal meal menus and service options based on the user's emotions.

[1403] The system for implementing this invention includes an interface for receiving instructions from a user, an emotion engine, a natural language processing engine, a task execution engine, and a proposal engine. The specific operation of each engine and the hardware and software used will be described below.

[1404] Interface

[1405] Users can communicate with the system through voice input or text input. Voice input uses a microphone and speech recognition software (e.g., Google Speech Recognition API), while text input uses a keyboard or touchscreen.

[1406] Emotion Engine

[1407] The emotion engine analyzes the emotion from the user's voice or text. It uses a natural language processing library (e.g., TextBlob) for emotion analysis. The emotion engine calculates the user's emotion score and classifies it as positive, negative, or neutral.

[1408] Natural Language Processing Engine

[1409] The natural language processing engine analyzes the user's instructions and selects the appropriate task. A generative AI model (e.g., GPT-3) is used for natural language processing. The generative AI model understands the user's instructions and generates the appropriate prompt.

[1410] Business execution engine

[1411] The business execution engine executes the selected business operations, such as meeting arrangements, business trip arrangements, expense settlement, attendance management, and document preparation, using various APIs and databases (e.g., Google Calendar API, Expedia API).

[1412] Suggestion Engine

[1413] The recommendation engine suggests the most suitable menus and options based on the user's emotions. It makes appropriate suggestions based on the results of emotion analysis. For example, if the user is feeling stressed, it will suggest menus that have a relaxing effect.

[1414] Specific examples

[1415] If a user says, "I'm tired today, so I want to have a relaxing meal," the emotion engine will recognize the negative emotion, and the suggestion engine will suggest a menu that has a relaxing effect.

[1416] Prompt Sentence Examples

[1417] Example prompts to input to a generative AI model:

[1418] If a user says, "I'm tired today, so I want a relaxing meal," design a system that judges the emotion score as negative and suggests a menu that has a relaxing effect.

[1419] In this way, it becomes possible to carry out tasks and make suggestions optimally according to the user's emotions.

[1420] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1421] Step 1:

[1422] Users give instructions to the system through voice input or text input. In the case of voice input, voice data is picked up using the device's microphone and converted into text data using speech recognition software (e.g., Google Speech Recognition API). In the case of text input, text data is entered using the device's keyboard or touch screen. The input data is sent to the server as user instructions.

[1423] Input: User voice or text data

[1424] Output: Text data (user instructions)

[1425] Step 2:

[1426] The server passes the received text data to the emotion engine, which analyzes the user's emotions. A natural language processing library (e.g., TextBlob) is used for emotion analysis. The emotion engine calculates an emotion score from the text data and classifies it as positive, negative, or neutral.

[1427] Input: Text data (user instructions)

[1428] Output: Sentiment score (positive, negative, neutral)

[1429] Step 3:

[1430] The server passes instructions to a natural language processing engine based on the emotion score and selects the appropriate task. A generative AI model (e.g., GPT-3) is used for natural language processing. The generative AI model understands the user's instructions and generates an appropriate prompt.

[1431] Input: Emotion score, text data (user instructions)

[1432] Output: prompt statement

[1433] Step 4:

[1434] The server passes the generated prompt to the business execution engine and executes the selected business. For example, it handles administrative tasks such as meeting arrangements, business trip arrangements, expense settlement, attendance management, and document preparation. This is done using various APIs and databases (e.g., Google Calendar API, Expedia API).

[1435] Input: prompt statement

[1436] Output: Business execution results

[1437] Step 5:

[1438] The server passes instructions to the recommendation engine based on the emotion score and the results of the task execution, and suggests the most appropriate menus and options based on the user's emotions. Appropriate suggestions are made based on the results of emotion analysis. For example, if the user is feeling stressed, a menu with a relaxing effect is suggested.

[1439] Input: Emotion score, business execution results

[1440] Output: Proposal content (menus and options)

[1441] Step 6:

[1442] The server notifies the user of the proposed content. The notification is made via the device's display or audio output. The user can then review the proposed content and make selections or changes as necessary.

[1443] Input: Proposal content (menu and options)

[1444] Output: User notification

[1445] Example 2

[1446] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1447] Conventional administrative services systems perform tasks without considering the user's emotional state, which makes it difficult to reduce the user's stress and frustration. Furthermore, there was a lack of efficient means to obtain the data necessary to perform tasks, which led to problems with reduced work efficiency. Furthermore, there was an insufficient means to properly analyze user instructions and select the most appropriate tasks, making it difficult to reduce the user's workload.

[1448] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1449] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance management, and document preparation based on user instructions; means for proxy attendance at meetings and sharing post-event summaries; means for analyzing the user's emotions and adjusting the way tasks are performed based on those emotions; means for using emotion analysis software to analyze the user's emotions; and means for acquiring data necessary for task execution from an external data provision service. This enables tasks to be performed in accordance with the user's emotional state, thereby reducing user stress and frustration. Furthermore, efficiently acquiring necessary data can improve task efficiency. Furthermore, by appropriately analyzing user instructions and selecting the optimal task, it is possible to reduce the user's workload.

[1450] "Instructions from the user" refers to the operations and inputs that the user makes to the system.

[1451] "Meeting coordination" refers to the task of setting the date, time, and location of a meeting and notifying participants.

[1452] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[1453] "Expense reimbursement" refers to the process of applying for, approving, and settling work-related expenses.

[1454] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and vacations.

[1455] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[1456] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and understanding the content of the meeting.

[1457] "Post-meeting summary sharing" refers to the task of summarizing the meeting contents after the meeting and sharing them with relevant parties.

[1458] "Emotion analysis software" refers to software that analyzes a user's facial expressions and voice to recognize their emotions.

[1459] "External data provision services" refers to external services and APIs that provide data necessary for business operations.

[1460] "Natural language processing" refers to technology for analyzing instructions from users and selecting appropriate tasks.

[1461] "Workload" refers to the burden and stress that users feel when performing their work.

[1462] "Efficient business execution" means performing business operations quickly and accurately.

[1463] This invention is a system that performs administrative tasks such as meeting arrangements, business trip arrangements, expense settlement, attendance management, and document preparation based on user instructions. The system has a function to analyze the user's emotions and adjust the way tasks are performed based on those emotions. It also includes a function to obtain data required for performing tasks from external data providing services.

[1464] Hardware and software used

[1465] Hardware:

[1466] Terminal: A device operated by a user (e.g., a computer, smartphone, tablet)

[1467] Server: A server that manages the entire system and executes various tasks.

[1468] software:

[1469] Emotion analysis software: Software for analyzing a user's facial expressions and voice (e.g., Microsoft Azure's Emotion API)

[1470] Natural language processing software: Software that analyzes user instructions and selects the appropriate task.

[1471] External data services: External services and APIs that provide data necessary for business operations (e.g., airline APIs, hotel booking site APIs)

[1472] Program processing

[1473] The server accepts the user's terminal and selects the task they wish to perform from the system interface. For example, if the user selects business trip arrangements, the server launches the business trip arrangement module and makes flight reservations, hotel reservations, and transportation arrangements. In this case, the server uses the airline's API and the hotel reservation site's API to obtain the necessary data and complete the reservation.

[1474] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice. It uses emotion analysis software, such as Microsoft Azure's Emotion API, to analyze the emotion. Based on the analysis results, the emotion engine adjusts how tasks are performed. For example, if the emotion engine recognizes that the user is angry, it will instruct the task to be performed in a calmer tone.

[1475] Specific examples

[1476] As a concrete example, consider the case where a user selects a business trip and the emotion engine recognizes the user's emotion as "anger." In this case, the server launches the business trip arrangement module and makes flight reservations, hotel reservations, and transportation arrangements. At the same time, the emotion engine performs the tasks in a calm tone to calm the user's anger.

[1477] Prompt Sentence Examples

[1478] If a user selects a business trip and the emotion engine identifies the user's emotion as "anger," how would the system perform its task?

[1479] If you feed this prompt to a generative AI model, you'll get the following response:

[1480] If a user selects a business trip and the emotion engine recognizes the user's emotion as "anger," the system will launch the business trip booking module to book flights, hotels, and transportation. At the same time, the emotion engine will perform the task in a gentle tone to calm the user's anger. For example, it will display a reservation confirmation message in gentle language to reduce the user's stress.

[1481] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1482] Step 1:

[1483] The user selects a task.

[1484] The user selects "Business Trip Arrangements" from the menu displayed on the terminal screen. As input, the user's selection is sent to the system. As output, the server receives the user's selection and proceeds to the next processing step.

[1485] Step 2:

[1486] The server starts a module corresponding to the selected task.

[1487] The server receives the click event of the "Business Trip Arrangements" button and starts the business trip arrangements module. The user's selection information is passed to the server as input. As output, the business trip arrangements module is initialized and proceeds to the next processing step.

[1488] Step 3:

[1489] The server retrieves the necessary data and executes the task.

[1490] The server sends a request to the airline's API based on the user's business trip itinerary to obtain the best flight information. It then sends a request to the hotel reservation site's API to reserve the accommodation. The user's business trip itinerary information is passed to the server as input. The obtained flight information and hotel reservation information are presented to the user as output.

[1491] Step 4:

[1492] The emotion engine analyzes the user's emotions.

[1493] The emotion engine captures the user's facial expressions with the device's camera and analyzes their emotions using emotion analysis software. As input, the user's facial image is sent to the emotion analysis software. As output, the analysis results are returned to the server.

[1494] Step 5:

[1495] The emotion engine adjusts how work is performed based on the analysis results.

[1496] Based on the analysis results, the emotion engine determines that the user is feeling angry and instructs the server to execute the task in a calmer tone. The emotion analysis results are passed to the emotion engine as input. As output, instructions for adjusting the task execution method are sent to the server.

[1497] Step 6:

[1498] The servers complete the work in a coordinated manner.

[1499] The server follows the instructions of the emotion engine to display a message confirming the reservation in a calm tone. As input, instructions for adjusting how the task is performed are passed to the server. As output, a message in a calm tone is displayed to the user.

[1500] (Application example 2)

[1501] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1502] Conventional administrative services systems perform tasks without considering the user's emotional state, making it impossible to reduce the user's psychological burden. Furthermore, they are unable to provide optimal suggestions based on the user's emotions, preventing an improved user experience. This leads to problems such as reduced work efficiency and increased stress for users.

[1503] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1504] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for recognizing the user's emotional state and adjusting the method of performing work accordingly, and means for making optimal suggestions based on the user's emotional state. This makes it possible to perform work according to the user's emotional state, reducing the user's psychological burden and improving work efficiency.

[1505] "User" means an individual or corporation that uses the system to direct business operations and receive services.

[1506] An "instruction" is a command or request that a user makes to a system to request the system to carry out a task.

[1507] "Meeting coordination" involves coordinating the date, location, and participants of a meeting, and finalizing the schedule.

[1508] "Business trip arrangements" refers to the task of booking and arranging the air tickets, hotels, transportation, etc. required for a business trip.

[1509] "Expense settlement" is the process of calculating and settling expenses related to work.

[1510] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[1511] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[1512] "Administrative agency" is a service that performs various administrative tasks on behalf of users.

[1513] "Proxy attendance" refers to the task of attending a meeting on behalf of a user and understanding the agenda.

[1514] "Summary sharing" is the task of summarizing the contents of a meeting or business and sharing it with the relevant parties.

[1515] "Emotional state" refers to the user's psychological state or mood, and is what the system recognizes.

[1516] "Adjusting how work is performed" means changing the way work is carried out or the way people respond depending on the user's emotional state.

[1517] "Optimal suggestions" means suggesting the most appropriate options or actions based on the user's emotional state and situation.

[1518] The system for implementing this invention performs various administrative tasks based on user instructions, and has the ability to recognize the user's emotional state and adjust the way tasks are performed accordingly. Specifically, the following hardware and software are used:

[1519] Hardware

[1520] Server: A central processing unit that receives and processes business instructions.

[1521] Smartphone: A device that allows users to input instructions and receive results.

[1522] Emotion Recognition Device: A device used to recognize a user's emotional state (e.g., a smartphone camera or microphone).

[1523] software

[1524] Natural language processing library: Software for analyzing user instructions and selecting appropriate tasks.

[1525] Emotion Recognizer: Software for recognizing a user's emotional state.

[1526] Food Recommender: Software for making optimal recommendations based on the user's emotional state.

[1527] Processing flow

[1528] 1. User instruction input: The user inputs instructions to the system using a smartphone. For example, the user inputs instructions that include emotions, such as "I'm very tired today."

[1529] 2. Emotion Recognition: The emotion recognition device recognizes the user's emotional state and analyzes it using an emotion recognition library.

[1530] 3. Job selection: A natural language processing library analyzes the user's instructions and selects the appropriate job.

[1531] 4. Adjusting the way work is performed: Adjusting the way work is performed based on the emotional state recognized by the emotion recognition library.

[1532] 5. Best Suggestion: The food recommendation library makes best suggestions based on the user's emotional state.

[1533] Specific examples

[1534] For example, if a user types, "I'm very tired today," the emotion recognition library will recognize "fatigue," and the food recommendation library will recommend "relaxing herbal tea and snacks."

[1535] Prompt Sentence Examples

[1536] "Tell me how you're feeling: I'm very tired today."

[1537] "Tell me how you feel: I'm very happy today."

[1538] In this way, it becomes possible to carry out work and make optimal suggestions according to the user's emotional state, thereby reducing the user's psychological burden and improving work efficiency.

[1539] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1540] Step 1:

[1541] A user inputs instructions into the system using a smartphone. The input instructions are sent to the server in text format. For example, an instruction including emotions such as "I'm very tired today" is input.

[1542] Step 2:

[1543] The server recognizes the user's emotional state using an emotion recognition device (smartphone camera or microphone). It uses an emotion recognition library (EmotionRecognizer) to analyze the input text data and identify the user's emotional state. The input is text data, and the output is an emotional state such as "fatigue" or "joy."

[1544] Step 3:

[1545] The server analyzes the user's instructions using a natural language processing library. Based on the analyzed instructions, it selects an appropriate task. For example, based on the emotional state of "fatigue," it selects a task with a relaxing effect. The input is the user's instruction text and emotional state, and the output is the selected task.

[1546] Step 4:

[1547] The server adjusts the way the task is performed based on the emotional state recognized by the emotion recognition library. For example, if the user feels tired, the server adjusts the tone of the task to be performed calmly. The input is the emotional state and the selected task, and the output is the adjusted way of performing the task.

[1548] Step 5:

[1549] The server uses a food recommendation library (FoodRecommender) to make optimal suggestions based on the user's emotional state. For example, based on the emotional state of "fatigue," it recommends "relaxing herbal tea and light meals." The input is the emotional state, and the output is the recommended menu.

[1550] Step 6:

[1551] The server sends the optimal proposal to the user's smartphone and displays it to the user. The user can check the proposed menu and task execution method, and make selections or changes as necessary. The input is the recommended menu, and the output is the display to the user.

[1552] Example 3

[1553] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1554] Conventional administrative proxy systems can perform tasks such as coordinating meetings and arranging business trips based on user instructions, but they have issues with not being able to attend meetings on behalf of others, sharing summaries of the meeting content, or adjusting tasks based on the user's emotions. Furthermore, there was a lack of a way to efficiently summarize the content of meetings and share it with users, making it difficult to reduce the user's workload.

[1555] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1556] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for recognizing the user's emotions and adjusting the priority and execution method of tasks according to the emotions, means for using a generative AI model to summarize the content of the meeting, and means for sharing the generated summary with the user. This makes it possible to attend meetings on behalf of the user based on instructions from the user, efficiently summarize and share the content, and also adjust tasks according to the user's emotions.

[1557] A "user" is an entity that uses the system and issues instructions.

[1558] An "instruction" is a request or command given by a user to a system.

[1559] "Meeting coordination" refers to the task of setting and coordinating the date, time, location, and participants of a meeting.

[1560] "Business trip arrangements" refers to the task of arranging transportation, accommodation, and other items necessary for a business trip.

[1561] "Expense settlement" refers to the process of calculating and settling expenses related to work.

[1562] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[1563] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[1564] "Administrative agency" refers to carrying out administrative tasks on behalf of the user.

[1565] "Meeting proxy attendance" refers to attending a meeting on behalf of a user.

[1566] "Summary sharing" refers to providing users with a concise summary of the meeting contents.

[1567] "Emotion recognition" refers to analyzing a user's emotions and understanding those emotions.

[1568] "Task priority adjustment" refers to changing the order in which tasks are performed depending on the user's emotions and situation.

[1569] A "generative AI model" is a model that uses artificial intelligence to analyze data and perform specific tasks.

[1570] "Summary generation" refers to analyzing the content of a meeting, extracting the important points, and summarizing them concisely.

[1571] "Natural language processing" is a technology that uses computers to analyze and understand human language.

[1572] The system of the present invention performs administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance management, and document preparation based on user instructions. It also has the ability to attend meetings on behalf of users and share summaries afterward. It also has the ability to recognize users' emotions and adjust the priorities and execution methods of tasks accordingly.

[1573] Hardware and software used

[1574] Hardware: Servers, terminals (PCs, smartphones)

[1575] Software: Meeting proxy attendance module, emotion engine, generative AI model

[1576] Specific operation of the system

[1577] Receiving user instructions

[1578] The user uses the device to instruct, "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." The device receives this instruction and sends it to the server. Specifically, the user opens the smartphone app and inputs instructions using voice input or text input. The device then sends this instruction to the server as text data.

[1579] Launching the meeting proxy attendance module

[1580] The server analyzes the received instructions and launches the meeting proxy attendance module. The server then prepares to join the meeting at the specified time. Specifically, the server checks the meeting schedule using the calendar API and automatically logs in to the online meeting tool (e.g., Zoom, Microsoft Teams) at the specified time.

[1581] Summary of the meeting

[1582] When the meeting ends, the server uses a generative AI model to summarize the meeting content. The server collects the meeting's audio and text data and inputs it into the generative AI model. Specifically, the server converts the meeting recording data into text using speech recognition software (e.g., Google Speech-to-Text), and then inputs the text data into a generative AI model (e.g., GPT-4) to generate a summary.

[1583] Share a summary

[1584] The server sends the generated summary to the user, who can then view it on their device. Specifically, the server sends the generated summary to the user via email or a chat app (e.g., Slack, Microsoft Teams). The user can then view the summary on their smartphone or PC.

[1585] Emotion Engine Operation

[1586] The user expresses their emotions to the system (e.g., feeling stressed, feeling happy). The server uses an emotion engine to recognize the user's emotions. The server then uses the emotion engine to adjust the priority and execution of tasks according to the user's emotions. Specifically, when a user enters "feeling stressed" in the app, the server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the emotion and prioritize reminders and tasks to reduce stress.

[1587] Specific examples

[1588] User instruction: "Please delegate to the meeting tomorrow at 10am and share the meeting summary."

[1589] Example prompt for a generative AI model: "Summarize the meeting. Extract the key points and summarize them concisely."

[1590] In this way, the system attends meetings on behalf of the user based on the user's instructions, generates and shares a summary of the meeting, and also adjusts tasks according to the user's emotions using an emotion engine. The flow of the specific processing in the third embodiment will be described with reference to FIG. 21.

[1591] Step 1:

[1592] The user uses the device to input instructions. The user opens an app on their smartphone or PC and inputs instructions by voice or text, such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." The device then sends this instruction to the server as text data.

[1593] Input: User's voice or text instructions

[1594] Output: Text data sent to the server

[1595] Step 2:

[1596] The server analyzes the received instructions and launches the meeting proxy attendance module. The server analyzes the instructions using natural language processing and extracts the meeting date, time, and content. The server checks the meeting schedule using a calendar API and prepares to automatically log in to the online meeting tool at the specified time.

[1597] Input: User's instruction text data

[1598] Output: Launch of meeting proxy attendance module

[1599] Step 3:

[1600] The server automatically logs in to the online conference tool at the specified time and participates in the conference. The server collects the conference audio and text data in real time.

[1601] Input: Meeting schedule information

[1602] Output: Meeting audio and text data

[1603] Step 4:

[1604] After the meeting ends, the server uses the generative AI model to summarize the meeting content. The server converts the meeting recording into text using speech recognition software, and then inputs the text data into the generative AI model to generate a summary.

[1605] Input: Meeting audio and text data

[1606] Output: Summary text

[1607] Step 5:

[1608] The server sends the generated summary to the user. The server sends the generated summary to the user via email or a chat app. The user can then check the summary on their smartphone or PC.

[1609] Input: Summary text

[1610] Output: Summary text sent to the user

[1611] Step 6:

[1612] Users express their emotions to the system. When a user enters "I'm feeling stressed" in the app, the server uses an emotion engine to analyze the emotion. The server then uses the emotion engine to adjust the priority and execution of tasks according to the user's emotions.

[1613] Input: User emote text

[1614] Output: Coordinated work priorities and execution methods

[1615] In this way, the system attends meetings on behalf of the user, summarizes the content efficiently, and shares it based on the user's instructions. It also adjusts work according to the user's emotions using an emotion engine.

[1616] (Application example 3)

[1617] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1618] Conventional administrative proxy systems perform tasks without considering the user's feelings, which can affect the user's stress and satisfaction. Furthermore, there was a lack of means for proxy attendance or sharing summaries during meetings and discussions within the factory, making it difficult to carry out tasks efficiently. This increased the user's workload and reduced work efficiency.

[1619] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1620] In this invention, the server includes means for performing administrative tasks such as arranging meetings, arranging business trips, settling expenses, keeping attendance records, and preparing documents based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries afterward, means including an emotion engine that recognizes the user's emotions and adjusts the priority and execution method of work in accordance with those emotions, and means including a robot assistant that attends meetings and conferences in a factory on behalf of the user and summarizes and shares the contents of the meetings. This enables efficient work execution that takes the user's emotions into consideration, and makes it possible to attend meetings and conferences in a factory on behalf of the user and share summaries.

[1621] "User" refers to an individual or organization that uses the system to issue instructions.

[1622] "Instructions" refer to specific requests or commands that a user gives to a system.

[1623] "Meeting coordination" refers to the task of coordinating the date, time, location, and participants of a meeting.

[1624] "Business trip arrangements" refers to the work of arranging transportation, accommodation, etc. necessary for business trips.

[1625] "Expense reimbursement" refers to the work of calculating and reimbursing expenses related to work.

[1626] "Time attendance" refers to the task of managing employee attendance information such as attendance, departure, and vacation.

[1627] "Document creation" refers to the task of creating materials necessary for meetings and business.

[1628] "Administrative agency" refers to carrying out administrative tasks on behalf of the user.

[1629] "Proxy attendance" refers to attending a meeting or conference on behalf of a user.

[1630] "Summary sharing" refers to summarizing the contents of a meeting or conference and sharing it with users.

[1631] An "emotion engine" refers to the part of the system that recognizes the user's emotions and adjusts the priorities and execution methods of work based on those emotions.

[1632] A "robot assistant" is a robot that attends meetings and discussions within a factory on behalf of employees and summarizes and shares the contents of the meetings.

[1633] A system for implementing this invention has functions to handle administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on user instructions. It also includes functions to attend meetings on behalf of users and share summaries afterward, and an emotion engine that recognizes users' emotions and adjusts task priorities and execution methods accordingly. It also includes a robot assistant that attends meetings and conferences in factories on behalf of users and summarizes and shares the contents of those meetings.

[1634] System Program Overview

[1635] The server implements the system using the following hardware and software.

[1636] Hardware: Robots that can move around the factory (e.g. Pepper, Spot)

[1637] software:

[1638] emotion_recognition module: A library for recognizing user emotions

[1639] meeting_summary module: A library for summarizing meeting content

[1640] robot_control module: A library for controlling robots

[1641] Program processing explanation

[1642] The server first receives instructions from the user. The instructions are analyzed using natural language processing, and an appropriate task is selected. For example, if the user instructs the robot to "attend the meeting tomorrow at 10:00 on my behalf and share the meeting summary," the server will launch the meeting proxy attendance module and have the robot join the meeting at the specified time.

[1643] After the meeting ends, the server summarizes the meeting contents and shares them with the user. The emotion engine recognizes the user's emotions and shares the summary with high priority if the user is feeling stressed, and with normal priority if the user is feeling happy.

[1644] Specific examples

[1645] For example, if a factory manager is feeling stressed, the robot can share a summary of the meeting with high priority, allowing the manager to quickly obtain important information and reduce their workload.

[1646] Prompt Sentence Examples

[1647] Examples of prompts for a generative AI model might include:

[1648] If the user is stressed, generate a robot assistant program that shares a meeting summary with high priority.

[1649] In this way, it becomes possible to carry out work efficiently while taking into account the user's emotions, and proxy attendance and summary sharing are also possible in meetings and discussions within the factory.

[1650] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1651] Step 1:

[1652] The server receives instructions from the user. The user instructs, "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting." This instruction becomes the input data. The server passes this instruction to the natural language processing module for analysis. As a result of the analysis, it outputs the tasks of attending the meeting on my behalf and sharing the summary.

[1653] Step 2:

[1654] The server launches the meeting proxy attendance module based on the analysis results. It then sets up a schedule for the robot to participate in the meeting at the specified time (in this case, 10:00 tomorrow). The input data are the analysis results and the meeting schedule information, and the output data is the robot's schedule setting information.

[1655] Step 3:

[1656] At the specified time, the server controls the robot to join the meeting. The robot records the meeting audio and video in real time. The input data is the meeting schedule information and the meeting audio and video data, and the output data is the recorded meeting data.

[1657] Step 4:

[1658] After the meeting ends, the server passes the recorded meeting data to the meeting summarization module, which uses speech recognition and natural language processing to summarize the meeting content. The input data is the recorded meeting data, and the output data is the summarized meeting content.

[1659] Step 5:

[1660] The server passes the summarized meeting content to the emotion engine. The emotion engine recognizes the user's emotion and prioritizes summaries based on that emotion. The input data are the summarized meeting content and the user's emotion data, and the output data are the prioritized summary content.

[1661] Step 6:

[1662] The server shares prioritized summary content with the user. If the user is feeling stressed, it is shared with high priority, and if the user is feeling happy, it is shared with normal priority. The input data is the prioritized summary content, and the output data is the summary content sent to the user.

[1663] In this way, it becomes possible to carry out work efficiently while taking into account the user's emotions, and proxy attendance and summary sharing are also possible in meetings and discussions within the factory.

[1664] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1665] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1666] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1667] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1668] [Third embodiment]

[1669] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1670] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1671] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1672] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1673] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1674] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1675] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1676] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1677] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1678] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1679] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1680] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1681] "Example 1"

[1682] The system of the present invention has an interface for receiving instructions from a user. This interface supports formats that allow users to give instructions naturally, such as text input and voice input. For example, if a user instructs, "Please make travel arrangements for a conference in Tokyo next Monday," the system analyzes this instruction and selects the task of making travel arrangements.

[1683] "Example 2"

[1684] The system of the present invention includes modules for executing selected tasks. These modules have functions corresponding to various tasks. For example, if a user selects the task of arranging a business trip, the system will launch the business trip arrangement module and make the necessary arrangements for the business trip, such as reserving an airline ticket, booking a hotel, and arranging transportation.

[1685] "Example 3"

[1686] The system of the present invention can also attend meetings on behalf of users and share the summary after the meeting. For example, if a user requests, "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting," the system will launch the meeting proxy attendance module and join the meeting at the specified time. After the meeting ends, the system will summarize the contents of the meeting and share it with the user.

[1687] The processing flow of each embodiment will be described below.

[1688] "Example 1"

[1689] Step 1: Receive instructions from the user through the interface. For example, receive an instruction such as "Please make travel arrangements for a meeting in Tokyo next Monday."

[1690] Step 2: The received instruction is analyzed using natural language processing. Through this analysis, the task of "business trip arrangements" is selected.

[1691] Step 3: The module corresponding to the selected task is started. In this case, the business trip arrangement module is started.

[1692] Step 4: The travel arrangements module makes the necessary arrangements for the business trip, such as booking air tickets, hotel reservations, and transportation.

[1693] "Example 2"

[1694] Step 1: Receive instructions from the user through the interface. For example, receive an instruction such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting."

[1695] Step 2: The received instruction is analyzed using natural language processing. Through this analysis, the task of "attending a meeting on behalf of another person" is selected.

[1696] Step 3: Start the module corresponding to the selected task. In this case, the conference proxy attendance module is started.

[1697] Step 4: The meeting proxy attendance module joins the meeting at the specified time.

[1698] Step 5: After the meeting is over, the meeting proxy attendance module summarizes the contents of the meeting.

[1699] Step 6: Share the summarized meeting content with users.

[1700] Example 1

[1701] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1702] Conventional administrative services require users to manually input instructions for various tasks, which does not improve work efficiency. Furthermore, the difficulty of properly analyzing user instructions and responding quickly results in a lack of reduction in workload. Furthermore, the lack of an interface that can accept instructions in natural language poses a problem of low user convenience.

[1703] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1704] In this invention, the server includes means having an interface for receiving instructions from a user via text input or voice input, means for analyzing the user's instructions using natural language processing technology, means for selecting an appropriate task based on the analysis results, and means for executing the selected task. This makes it possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute an appropriate task.

[1705] "User" means any person or entity that uses the System to issue instructions.

[1706] "Instructions" refer to requests or commands given by a user to a system.

[1707] "Interface" refers to the means by which a user inputs instructions into a system, including text input and voice input.

[1708] "Natural language processing technology" refers to technology for analyzing user instructions and understanding their meaning.

[1709] "Parsing" refers to the process of breaking down a user's instructions and extracting meaning.

[1710] "Business" refers to the tasks or work that a system performs based on user instructions.

[1711] "Selection" refers to the process of determining the appropriate task based on the analysis results.

[1712] "Execution" refers to the process of specifically carrying out the selected task.

[1713] "Administrative agency" refers to tasks such as meeting coordination, travel arrangements, expense settlement, attendance management, and document preparation that the system performs on behalf of the user.

[1714] "Meeting coordination" refers to the task of arranging the date and location of a meeting.

[1715] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[1716] "Expense settlement" refers to the work of calculating and settling users' expenses.

[1717] "Time attendance management" refers to the task of managing users' attendance and departure records.

[1718] "Document creation" refers to the task of creating documents and presentation materials based on user instructions.

[1719] "Proxy attendance" refers to the task of attending a meeting on behalf of a user.

[1720] "Summary sharing" refers to the task of summarizing the contents of a meeting and sharing it with users.

[1721] MODE FOR CARRYING OUT THE INVENTION

[1722] The present invention is a system that receives instructions from a user, analyzes them, and selects and executes an appropriate task. A specific embodiment of this system will be described below.

[1723] 1. Receive instructions from the user

[1724] Users input instructions using the device's interface. This interface supports text input and voice input, allowing users to give instructions naturally. For example, a user opens a smartphone app and enters text, such as "Please arrange travel for a meeting in Tokyo next Monday." In the case of voice input, the user speaks the same instruction into the microphone.

[1725] 2. Parsing the instructions

[1726] The server receives instructions sent from the device. It analyzes the text of the instructions using natural language processing technologies such as Google Cloud Natural Language API and IBM Watson Natural Language Understanding. Through this analysis, it extracts the content of the instructions (e.g., "next Monday," "Tokyo," "meeting," "business trip arrangements," etc.).

[1727] 3. Selecting a task

[1728] The server selects the appropriate task based on the analysis results. For example, if the keyword "business trip arrangement" is included, the server selects the task of business trip arrangement. This selection can be done using a predefined rule-based system or a machine learning model (for example, a model using TensorFlow or PyTorch).

[1729] 4. Execute the task

[1730] The server executes the selected task. Specifically, it calls the travel agency's API (for example, Expedia API or Amadeus API) to make business trip arrangements. The server sends the necessary information (date, time, location, purpose, etc.) to the API to complete the reservation. Once the reservation is complete, the server notifies the user of the result. For example, the app may display a message saying, "Business trip arrangements completed."

[1731] Specific examples

[1732] User instructions

[1733] The user instructs the system, "Please arrange travel for a meeting in Tokyo next Monday."

[1734] Prompt Sentence Examples

[1735] Examples of prompts to input to a generative AI model might include:

[1736] The user has instructed us to "make travel arrangements for a conference in Tokyo next Monday." Analyze this instruction and select the task of making travel arrangements.

[1737] This prompt sentence can be used to have a generative AI model parse the instructions and select a task.

[1738] In this way, it becomes possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute the appropriate tasks. This system reduces the user's workload and supports efficient work execution.

[1739] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1740] Step 1:

[1741] The user inputs instructions using the device's interface. The interface supports text input and voice input. For example, a user opens a smartphone app and inputs text such as, "Please arrange a business trip for a meeting in Tokyo next Monday." In the case of voice input, the user speaks the same instruction into the microphone. The input instruction is sent from the device to the server.

[1742] Step 2:

[1743] The server receives instructions sent from the device and analyzes them using natural language processing technology. Specifically, it uses software such as Google Cloud Natural Language API and IBM Watson Natural Language Understanding. The input for analysis is the user's instruction text, and the output is keywords and phrases resulting from the analysis (such as "next Monday," "Tokyo," "meeting," and "business trip arrangements").

[1744] Step 3:

[1745] The server selects the appropriate task based on the analysis results. It selects the task using a predefined rule-based system or machine learning model (for example, a model using TensorFlow or PyTorch) based on keywords and phrases extracted from the analysis results. The input is the analysis results, and the output is the selected task (in this case, "business trip arrangements").

[1746] Step 4:

[1747] The server executes the selected task. Specifically, it calls the travel agency's API (for example, Expedia API or Amadeus API) to make business trip arrangements. The server sends the necessary information (date, time, location, purpose, etc.) to the API to complete the reservation. The input is the selected task and the necessary information, and the output is a notification that the reservation has been completed.

[1748] Step 5:

[1749] The server notifies the user that the reservation is complete. For example, the app displays "Business trip arrangements completed." The input is the reservation completion notification, and the output is a notification message to the user.

[1750] In this way, it becomes possible to receive user instructions in natural language, analyze them quickly and accurately, and automatically select and execute the appropriate tasks. This system reduces the user's workload and supports efficient work execution.

[1751] (Application example 1)

[1752] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1753] Conventional administrative services systems often have limited interfaces for receiving user instructions, and in particular do not support voice input. Furthermore, they lack the functionality to analyze user instructions and generate optimal suggestions, making it impossible to fully reduce the workload of users. Furthermore, in certain operations, such as food delivery, it is necessary to efficiently process user instructions and provide optimal suggestions.

[1754] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1755] In this invention, the server includes means for performing administrative tasks such as meeting arrangements, business trip arrangements, expense settlement, attendance, and document preparation based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries after the meeting, means for converting the user's voice input into text, means for analyzing the user's instructions using a generative AI model and generating optimal suggestions, and means for completing orders based on the generated suggestions. This makes it possible to efficiently process the user's voice instructions and provide optimal suggestions.

[1756] "User instructions" refer to requests or commands given by a user to a system.

[1757] "Meeting coordination" refers to the task of coordinating the date, location, and participants of a meeting.

[1758] "Business trip arrangements" refers to the task of reserving transportation and accommodations necessary for business trips.

[1759] "Expense reimbursement" refers to the process of calculating and reimbursing expenses related to work.

[1760] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[1761] "Document creation" refers to the task of creating documents and presentation materials necessary for business.

[1762] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[1763] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and understanding the agenda.

[1764] "Post-meeting summary sharing" refers to the task of summarizing the minutes after the meeting and sharing them with relevant parties.

[1765] "Voice input" refers to a method in which a user inputs instructions by voice through a microphone.

[1766] "Convert to text" refers to the process of converting voice-input instructions into text information.

[1767] A "generative AI model" is an algorithm that uses artificial intelligence to analyze user instructions and generate appropriate suggestions.

[1768] An "optimal suggestion" is the most appropriate solution or option derived by the generative AI model based on the user's instructions.

[1769] "Completing an order" refers to the task of carrying out the necessary procedures based on the user's instructions and finalizing the order.

[1770] A system for implementing this invention has an interface for receiving instructions from a user and supports voice and text input. When a user inputs instructions by voice, the system converts the speech into text and analyzes it using a generative AI model. Based on the analysis results, an optimal proposal is generated and presented to the user. Once the user confirms and approves the proposal, the system completes the order.

[1771] Hardware and software used

[1772] Hardware: Smartphone, microphone

[1773] Software: Python, SpeechRecognition library, OpenAI API

[1774] Data processing and calculation

[1775] 1. Speech Recognition: Uses the smartphone's microphone to capture the user's voice input and converts this speech to text using the SpeechRecognition library.

[1776] 2. Generative AI model: Using the OpenAI API, the generative AI model analyzes the textual user instructions and generates optimal suggestions based on the user instructions.

[1777] 3. Present proposal and complete order: The generated proposal is presented to the user, and once the user approves it, the system completes the order.

[1778] Specific examples

[1779] If a user says, "I'd like to order pizza from a nearby Italian restaurant," the system converts the speech to text and analyzes it using a generative AI model. The analysis results suggest the best Italian restaurant and pizza menu. If the user accepts the suggestion, the system sends the order to the restaurant and completes the order.

[1780] Prompt Sentence Examples

[1781] A user says: 'I want to order pizza from an Italian restaurant near me.' Based on this instruction, suggest the best restaurant and menu and complete the order.

[1782] In this way, the user's voice instructions can be processed efficiently and optimal suggestions can be made.

[1783] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1784] Step 1:

[1785] The user uses the smartphone's microphone to input voice.

[1786] Input: User's spoken command (e.g., "I'd like to order pizza from a nearby Italian restaurant.")

[1787] Output: Audio data

[1788] Specific operation: The user activates the voice input function on their smartphone and inputs instructions by voice.

[1789] Step 2:

[1790] The device captures the voice data and converts it to text using the SpeechRecognition library.

[1791] Input: Audio data

[1792] Output: Text data (e.g., "I would like to order pizza from a nearby Italian restaurant")

[1793] Specific operation: The device passes the voice data to the SpeechRecognition library, performs voice recognition, and generates text data.

[1794] Step 3:

[1795] The device sends the text data to the generative AI model for analysis.

[1796] Input: Text data

[1797] Output: Analysis results (e.g., recommendations for the best restaurants and menus)

[1798] Specific operation: The device calls the OpenAI API, sends text data as a prompt, and has it analyzed by the generative AI model.

[1799] Step 4:

[1800] The server receives the analysis results of the generated AI model and presents optimal suggestions to the user.

[1801] Input: Analysis results

[1802] Output: Suggestions (e.g., best Italian restaurants and pizza menus)

[1803] Specific operation: The server sends the analysis results to the user's smartphone and displays suggestions to the user.

[1804] Step 5:

[1805] The user reviews and approves the proposal.

[1806] Input: Proposal

[1807] Output: Approval data

[1808] Specific operation: The user checks the proposal on the smartphone screen and presses the approval button.

[1809] Step 6:

[1810] The server receives the authorization data and completes the order.

[1811] Input: Approval data

[1812] Output: Order completion notification

[1813] What happens: The server uses the authorization data to send the order to the suggested restaurant and notifies the user that the order is complete.

[1814] Example 2

[1815] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1816] Conventional administrative services systems have the problem that it is time-consuming for users to select tasks and input the necessary information, making it difficult to carry out tasks efficiently. Furthermore, while it is required that the information entered by users be processed appropriately and results be provided quickly, there has been a lack of effective means to achieve this. Furthermore, a system that can reduce the workload on users and support efficient task execution is needed.

[1817] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1818] In this invention, the server includes means for the user to select a task and input the necessary information, means for the terminal to receive the information input by the user and send it to the server, means for the server to start and execute an appropriate module based on the received information, and means for the server to send the arrangement results to the terminal and for the terminal to display the arrangement results to the user. This allows the user to easily select a task and input the necessary information, and the server to quickly start the appropriate module to process and provide the results to the user.

[1819] "User" means an individual or organization that uses the system to select a task and enter the required information.

[1820] "Instructions" refer to the operations that users perform on the system, such as selecting tasks and inputting information.

[1821] "Administrative agency" refers to the performance of tasks such as meeting coordination, travel arrangements, expense settlement, attendance management, and document preparation.

[1822] "Meeting coordination" refers to the task of arranging the date, time, and location of a meeting and notifying participants.

[1823] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[1824] "Expense settlement" refers to the task of organizing and settling expenses incurred for business trips and other work.

[1825] "Attendance management" refers to the task of managing employee attendance information such as attendance, departure, and vacation.

[1826] "Document creation" refers to the task of creating various documents necessary for business operations.

[1827] "Proxy attendance" refers to the task of attending a meeting on behalf of a user and understanding the content of the meeting.

[1828] "Summary sharing" refers to the task of summarizing the contents of a meeting and sharing it with users.

[1829] "Natural language processing" refers to technology for analyzing instructions from users and selecting appropriate tasks.

[1830] "Workload" refers to the effort and time required by a user to carry out their work.

[1831] "Efficient business execution" means performing business operations quickly and accurately.

[1832] "Terminal" refers to a device that allows a user to select a task and enter information.

[1833] "Server" refers to a computer system that receives information from a user, activates the appropriate module, and performs processing.

[1834] A "module" refers to a part of a program for performing a specific task.

[1835] "Arrangement result" refers to the result obtained after the server executes the task.

[1836] The present invention is a system in which a user selects a task and inputs the necessary information, and the server starts up an appropriate module to execute the task. A specific embodiment of this system will be described below.

[1837] Hardware and software used

[1838] server

[1839] The server is a high-performance computer system that manages and executes business modules. Specifically, it uses general server hardware (e.g., a high-performance server computer), and business modules are developed in Python and run on the server using the Django framework. MySQL is used as the database.

[1840] Terminal

[1841] The terminal is a device that allows the user to select a task and enter the necessary information. Specifically, the terminal is a PC or smartphone (e.g., Windows PC or smartphone) that the user uses.

[1842] System Operation

[1843] User Actions

[1844] The user uses the terminal interface to select the task they want to perform. For example, to select the task of arranging a business trip, the user clicks the "Business Trip Arrangements" button. The user then enters information such as the departure point, destination, departure date, and return date.

[1845] Terminal handling

[1846] The device receives the information entered by the user and sends it to the server using an HTTP request, which communicates the user's instructions to the server.

[1847] Server Processing

[1848] The server analyzes the received information and selects and launches the appropriate business module. For example, it launches a business trip arrangement module. This module makes the arrangements necessary for the business trip, such as booking an airline ticket, booking a hotel, and arranging transportation. To do this, it uses external APIs (e.g., airline APIs, hotel booking site APIs).

[1849] Displaying the results

[1850] Once the arrangements are complete, the server sends the results to the terminal. Specifically, it generates JSON data including detailed arrangement information (e.g., reservation confirmation number, hotel address, transportation details) and sends it as an HTTP response to the terminal. The terminal then displays the received arrangement results to the user. Specifically, it displays the arrangement results on the screen in an easy-to-read format. For example, it could display a list of the airline ticket reservation confirmation number, hotel address, and transportation details.

[1851] Specific examples

[1852] Specific examples of business trip arrangements

[1853] 1. The user clicks the "Business Trip Arrangements" button on the device.

[1854] 2. The user enters the departure location as "Tokyo," the destination as "New York," the departure date as "December 1, 2023," and the return date as "December 5, 2023."

[1855] 3. The device sends the input information to the server.

[1856] 4. The server launches the business trip arrangement module.

[1857] 5. The server will book your flight, hotel and transportation.

[1858] 6. The server sends the arrangement results to the terminal.

[1859] 7. The terminal displays the arrangement results to the user.

[1860] Example prompts for generative AI models

[1861] "Please select the business trip arrangement task, enter the departure point as Tokyo, the destination as New York, the departure date as December 1, 2023, and the return date as December 5, 2023."

[1862] In this way, users can easily make business trip arrangements.

[1863] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1864] Step 1:

[1865] The user selects a task.

[1866] The user uses the terminal interface to select the task they wish to perform. For example, to select the task of arranging a business trip, the user clicks the "Business Trip Arrangements" button. The input is an instruction to select the task, and the output is information about the selected task.

[1867] Step 2:

[1868] The terminal receives the user's input information.

[1869] The user inputs information such as the departure point, destination, departure date, and return date. The terminal receives this information and stores it as input information. The input includes various pieces of information entered by the user, and the output is stored in the terminal.

[1870] Step 3:

[1871] The terminal sends the input information to the server.

[1872] The device sends the received information to the server using an HTTP request. Specifically, it converts the input information into JSON format and sends it to the server's API endpoint. The input is the information entered by the user, and the output is the information sent to the server.

[1873] Step 4:

[1874] The server launches the appropriate module based on the information received.

[1875] The server analyzes the received information and selects and launches the appropriate business module. For example, it launches a business trip arrangement module. The input is the information sent from the terminal, and the output is the launched module.

[1876] Step 5:

[1877] The server executes the module and makes the necessary arrangements.

[1878] The server executes the launched module and makes the necessary arrangements. In the case of the business trip arrangement module, it makes flight reservations, hotel reservations, and transportation arrangements. For this, it uses external APIs (e.g., airline APIs, hotel reservation site APIs). The input is the user's business trip information, and the output is the arrangement results.

[1879] Step 6:

[1880] The server sends the arrangement result to the terminal.

[1881] Once the arrangements are complete, the server sends the results to the terminal. Specifically, it generates JSON data containing detailed arrangement information (e.g., reservation confirmation number, hotel address, transportation details) and sends it as an HTTP response to the terminal. The input is the arrangement results, and the output is the arrangement results sent to the terminal.

[1882] Step 7:

[1883] The terminal displays the results of the arrangement to the user.

[1884] The terminal displays the received arrangement results to the user. Specifically, the arrangement results are displayed on the screen in an easy-to-read format. For example, the airline ticket reservation confirmation number, hotel address, and transportation details are displayed in a list. The input is the arrangement results sent from the server, and the output is the arrangement results displayed to the user.

[1885] (Application example 2)

[1886] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1887] While conventional administrative services can efficiently handle tasks such as coordinating meetings, arranging business trips, and expense settlements, they do not address other needs in users' daily lives, such as arranging food delivery. This requires users to use a separate food delivery app, which creates a disconnect between managing work and daily life. Furthermore, there is no system that automatically selects the optimal restaurant based on the user's location and food preferences, arranges delivery methods, or adjusts delivery times, reducing user convenience.

[1888] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1889] In this invention, the server includes means for handling administrative tasks such as arranging meetings, arranging business trips, settling expenses, managing attendance, and preparing documents based on instructions from the user, means for attending meetings on behalf of the user and sharing summaries afterward, and means for searching for the most suitable restaurant based on the user's location information and food preferences, ordering food, arranging delivery means, and adjusting delivery times. This allows users to efficiently manage both their work and daily lives using a single system.

[1890] "Instructions from the user" refers to the operations or inputs that the user makes to the system, and the system then performs a specific task based on these instructions.

[1891] "Meeting coordination" refers to the task of coordinating the date, time, location, and participants of a meeting, and finalizing the schedule.

[1892] "Business trip arrangements" refers to the work of booking airline tickets, hotel reservations, and arranging transportation necessary for business trips.

[1893] "Expense reimbursement" refers to the process of applying for, approving, and settling work-related expenses.

[1894] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and vacations.

[1895] "Document creation" refers to the task of creating documents and presentation materials needed for meetings, reports, etc.

[1896] "Attending a meeting on behalf of a user" refers to the task of attending a meeting on behalf of a user and preparing minutes and summaries.

[1897] "Post-event summary sharing" refers to the task of summarizing the contents of meetings and discussions and sharing them with relevant parties.

[1898] "Location information" refers to information about the user's current location or a specified location, and the system uses this information to perform its tasks.

[1899] "Food preferences" refers to the types of food and cuisine that a user prefers, and the system uses this information to select the most suitable restaurant.

[1900] "Restaurant search" refers to the task of finding suitable restaurants based on the user's location information and food preferences.

[1901] "Ordering food" refers to the task of ordering specific food from a selected restaurant.

[1902] "Arranging delivery means" refers to the task of arranging means to deliver the ordered food to the user's designated location.

[1903] "Delivery time adjustment" refers to the task of adjusting the delivery time so that the food arrives at the user's designated location at the appropriate time.

[1904] A system for implementing this invention includes means for carrying out administrative tasks such as coordinating meetings, arranging business trips, settling expenses, keeping attendance records, and preparing documents based on instructions from a user, means for attending meetings on behalf of a user and sharing summaries afterward, and means for searching for the most suitable restaurant based on the user's location information and food preferences, ordering food, arranging for delivery, and adjusting delivery times.

[1905] 1. System Program

[1906] The system is implemented using Python programs and REST APIs. The interface for receiving user instructions is provided as a smartphone application. When a user enters instructions through the application, the instructions are sent to the server, which then performs the appropriate tasks based on the instructions.

[1907] 2. Program Processing

[1908] The server processes and calculates data using the following hardware and software.

[1909] Hardware: Servers, smartphones

[1910] Software: Python, REST API, natural language processing libraries (e.g. spaCy, NLTK)

[1911] Processing flow

[1912] 1. Receiving user instructions: The user inputs instructions through a smartphone application. For example, "I want to eat sushi in Tokyo."

[1913] 2. Natural Language Processing: The server uses a natural language processing library to analyze the user's instructions and select the appropriate task.

[1914] 3. Execution of tasks: The server executes the following tasks based on the analysis results.

[1915] Restaurant search: Find the best restaurants based on your location and food preferences.

[1916] Food Ordering: Order specific food from selected restaurants.

[1917] Arranging delivery: Arranging delivery of the ordered food to the user's designated location.

[1918] Delivery time adjustment: Adjust delivery times to ensure food arrives at the user's desired location at the appropriate time.

[1919] 3. Specific Examples

[1920] As a concrete example, the prompt sentence when the user inputs "I live in Tokyo and I want to eat sushi" is as follows:

[1921] Prompt Sentence Examples

[1922] User location: Tokyo

[1923] Food preference: Sushi

[1924] Based on this prompt, the system operates as follows:

[1925] 1. Restaurant Search: Search for sushi restaurants in Tokyo and choose the best one.

[1926] 2. Order food: Order a "Sushi Set" at the selected restaurant.

[1927] 3. Arrange delivery: Once the order is completed, arrange delivery and schedule delivery based on the user's location.

[1928] In this way, users can easily use food delivery services.

[1929] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1930] Step 1:

[1931] A user inputs a command through a smartphone application, for example, "I want to eat sushi in Tokyo." This command is then sent from the application to the server.

[1932] Input: User instructions (e.g., "I want to eat sushi in Tokyo")

[1933] Output: User instructions sent to the server

[1934] Step 2:

[1935] The server uses a natural language processing library (e.g., spaCy, NLTK) to analyze the user's instructions, extracting their location (Tokyo) and food preference (sushi).

[1936] Input: User instructions

[1937] Output: Analysis results (location information and food preferences)

[1938] What it does: The server uses a natural language processing library to analyze the text and extract location information and food preferences.

[1939] Step 3:

[1940] Based on the analysis results, the server calls a restaurant search API and searches for restaurants that match the user's location information and food preferences.

[1941] Input: Analysis results (location information and food preferences)

[1942] Output: List of restaurants

[1943] Specific operation: The server passes location information and food preferences as parameters to the restaurant search API and obtains a list of the most suitable restaurants.

[1944] Step 4:

[1945] The server selects the most suitable restaurant from the acquired list of restaurants and places an order for food.

[1946] Input: A list of restaurants

[1947] Output: Order confirmation information

[1948] Specific operation: The server selects the most suitable restaurant from the list of restaurants based on criteria such as rating and distance, and calls the ordering API to order food.

[1949] Step 5:

[1950] After the order is completed, the server calls the delivery means arrangement API and arranges for the means to deliver the food to the location specified by the user.

[1951] Input: Order confirmation information

[1952] Output: Delivery arrangement confirmation information

[1953] Specific operation: The server calls the delivery means arrangement API based on the order confirmation information and arranges for delivery means.

[1954] Step 6:

[1955] After arranging the delivery means, the server adjusts the delivery time and notifies the user.

[1956] Input: Delivery means arrangement confirmation information

[1957] Output: Delivery time notification

[1958] Specific operation: The server adjusts the delivery time based on the delivery means arrangement confirmation information and notifies the user.

[1959] In this way, users can easily use food delivery services.

[1960] Example 3

[1961] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1962] Conventional administrative proxy systems did not efficiently handle proxy attendance at meetings or the subsequent sharing of summaries, and were unable to sufficiently reduce the user's workload. Furthermore, the process of recording, transcribing, and generating summaries of meeting content was not automated, requiring users to perform the tasks manually, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to quickly share the generated summaries, which sometimes resulted in delays in the transmission of information.

[1963] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1964] In this invention, the server includes means for performing administrative tasks such as meeting coordination, travel arrangements, expense settlement, time attendance, and document preparation based on instructions from the user, means for attending the meeting on behalf of the user and sharing a summary afterwards, means for recording the audio data of the meeting and transcribing it using speech recognition software, means for summarizing the transcript using a generative AI model, and means for sharing the summary with the user. This automates the entire process from attending the meeting on behalf of the user to generating and sharing the summary, significantly reducing the user's workload and enabling more efficient work performance.

[1965] "User" means any person or entity that uses the System to issue instructions.

[1966] An "instruction" is a request or command given by a user to a system.

[1967] "Meeting coordination" refers to the task of coordinating the date, location, and participants of a meeting.

[1968] "Business trip arrangements" refers to the work of arranging transportation and accommodation necessary for business trips.

[1969] "Expense settlement" refers to the process of calculating and settling expenses incurred for business trips and other business activities.

[1970] "Time attendance" refers to the task of managing employees' work status, such as their arrival and departure times and holidays.

[1971] "Document creation" refers to the task of creating documents and presentation materials required for meetings and business.

[1972] "Administrative agency" refers to a service that performs administrative tasks on behalf of users.

[1973] "Meeting proxy attendance" refers to attending a meeting on behalf of a user.

[1974] "Summary sharing" refers to summarizing the contents of a meeting and communicating that summary to users.

[1975] "Audio data" refers to data that records the audio of a conference.

[1976] "Speech recognition software" means software for converting voice data into text.

[1977] "Transcription" is the process of converting audio data into text.

[1978] A "generative AI model" is a model that uses artificial intelligence to process data and perform summarization or other generative tasks.

[1979] A "summary" is a text that briefly summarizes the contents of a meeting.

[1980] "Sharing" refers to communicating the generated summary to a user.

[1981] MODE FOR CARRYING OUT THE INVENTION

[1982] This invention is a system that handles administrative tasks such as meeting arrangements, travel arrangements, expense settlement, attendance, and document preparation based on user instructions. It also has the functionality to attend meetings on behalf of users and share summaries afterward. This system operates in cooperation with the server, the terminal, and the user.

[1983] Hardware and software used

[1984] server:

[1985] The server plays a central role in receiving instructions from users and carrying out various administrative tasks.

[1986] The server joins the meeting using video conferencing software (e.g., Zoom, Microsoft Teams).

[1987] The server transcribes the meeting audio data using speech recognition software (e.g., Google Cloud Speech-to-Text API).

[1988] The server summarizes the transcription data using a generative AI model (e.g., OpenAI's GPT-4).

[1989] Device:

[1990] A terminal is a device that allows a user to input instructions, such as a PC or smartphone.

[1991] The terminal provides an interface for transmitting user instructions to the server.

[1992] user:

[1993] A user is an individual or corporation that uses the system to instruct administrative services.

[1994] Users use terminals to input instructions into the system and receive results.

[1995] Program processing

[1996] Upon receiving instructions from the user, the server launches a meeting proxy attendance module, which joins the meeting using video conferencing software at the specified time. Audio data from the meeting is recorded in real time and transcribed using speech recognition software. The transcribed data is then summarized using a generative AI model. The generated summary is shared with the user via email or a chat app.

[1997] Specific examples

[1998] When a user instructs the server to "attend the meeting tomorrow at 10:00 on his behalf and share the summary of the meeting," the server performs the following process.

[1999] 1. The server starts the conference proxy attendance module and joins the designated conference using the video conferencing software.

[2000] 2. The server records the audio of the meeting and transcribes it using speech recognition software.

[2001] 3. The server inputs the transcription data into a generative AI model to generate a summary of the meeting.

[2002] 4. The server sends the generated summary to the user's email address.

[2003] Example prompt sentence:

[2004] "Please summarize the transcript of the following meeting. Please include key agenda items, decisions made, and upcoming action items."

[2005] In this way, the server attends the conference on behalf of the user based on the user's instructions, and generates and shares a summary of the conference. This significantly reduces the user's workload and enables efficient work execution. The flow of the specific processing in the third embodiment will be described with reference to FIG. 15.

[2006] Step 1:

[2007] The user inputs instructions into the system using a terminal. Specifically, the user inputs an instruction such as "Please attend the meeting tomorrow at 10:00 on my behalf and share the summary of the meeting," and presses the send button. This instruction is then sent to the server.

[2008] Input: User instructions (natural language)

[2009] Output: Instruction data sent to the server

[2010] Step 2:

[2011] The server receives instructions from the user, analyzes the instructions using natural language processing, and extracts information (date and time, conference URL, etc.) from the analyzed instructions to launch the confere...

Claims

1. A means for receiving instructions from a user; A means for analyzing an instruction from the user by natural language processing and determining whether the instruction from the user is for a task of meeting arrangement, meeting attendance, business trip arrangement, expense settlement, attendance, or document preparation; A means for executing the administrative action represented by the judgment result; A system including:

2. and if the determination result is that the person is to attend the meeting on behalf of the other person, a means for performing proxy attendance at the meeting and sharing a summary of the meeting after the meeting is included. The system of claim 1 .

3. means for recognizing an emotion of the user; When the user's instruction indicates a plurality of agency services, a means for adjusting the priority of the plurality of agency services according to the user's emotion; The system of claim 1 or claim 2, further comprising:

Images