System

Abstract

A system is provided.SOLUTION: A system comprising: means for obtaining an employee's schedule; means for suggesting an optimal meeting time using a generative AI model; means for approving or modifying the suggested meeting time; and means for adding the approved meeting time to the employee's calendar.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's corporate environment, many employees spend a lot of time on tedious administrative tasks, leaving them with little time to focus on creative work. This situation can lead to a decline in work efficiency and employee satisfaction, and can have a negative impact on the productivity of the entire company. The present invention aims to improve work efficiency by utilizing generative AI models to perform tedious tasks, allowing employees to focus on more important work. [Means for solving the problem]

[0005] The present invention provides a system including a means for acquiring employee schedules, a means for proposing optimal meeting times using a generative AI model, a means for approving or modifying the proposed meeting times, and a means for adding the approved meeting times to the employee's calendar (Claim 1). It also includes a means for creating an appropriate email using a generative AI model, a means for previewing the created email to the user, a means for approving or modifying the previewed email, and a means for sending the approved email (Claim 2). It also includes a means for monitoring employee calendars and task management databases, a means for generating reminders for document submission deadlines and meeting times, and a means for sending reminders to the user (Claim 3). It also provides a means for analyzing meeting recording data and text data, a means for automatically generating minutes based on the analyzed data, a means for converting the minutes into an internal wiki format, a means for previewing the generated minutes to the user, and a means for updating the minutes on the internal wiki (Claim 4). It also includes a means for collecting necessary data, a means for automatically generating documents based on the collected data, a means for displaying a preview of the generated document to the user, and a means for saving the approved document and completing the submission process (Claim 5).By using these means, we provide a system that streamlines employees' complicated work and allows them to concentrate on their original creative work.

[0006] "Employees" are employees who belong to a company and perform various tasks.

[0007] A "schedule" is a chronological plan of an individual's or group's plans and activities.

[0008] A "generative AI model" is a type of artificial intelligence that uses machine learning algorithms to generate text and data.

[0009] "Meeting time" refers to the specific date and time when the meeting will take place.

[0010] A "suggestion" is a recommendation or indication of a certain action or option.

[0011] "Approval" refers to the formal recognition of a proposal or plan, or the procedure for doing so.

[0012] "Correction" means the act or process of making a change to correct an error or inaccuracy.

[0013] A "calendar" is a tool or system for managing dates and appointments.

[0014] "Email" means a message sent using electronic means of communication.

[0015] A "reminder" is a message that notifies you of important events or deadlines.

[0016] "Meeting recording data" refers to data in which the contents of a meeting are recorded in audio format.

[0017] "Text data" is information stored in the form of characters or sentences.

[0018] "Minutes" are documents that record the contents of a meeting and the decisions made.

[0019] An "internal wiki" is an online document management system used to organize and share information within a company.

[0020] "Document" means a formally executed document containing specific information.

[0021] A "database" is a system for systematically managing data and making it available.

[0022] "Task management" is the process of tracking and managing the progress of a project or task. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks for employees, learning each employee's work process, internal tools, and databases to provide efficient support. Below, we will explain the system's program processing in natural language, with concrete examples.

[0045] Meeting scheduling and time coordination

[0046] server

[0047] Employee schedules are retrieved from the database and analyzed.

[0048] Calculate each employee's free time to suggest optimal meeting times.

[0049] Terminal

[0050] An interface is provided that displays the proposed meeting time to the user and asks for approval or modification.

[0051] User

[0052] Review the proposed time and approve or modify it. Once approved, the meeting will be automatically scheduled.

[0053] Specific examples

[0054] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[0055] Creating and replying to emails

[0056] server

[0057] Based on user requests, NLP (Natural Language Processing) models are used to create appropriate email content.

[0058] Refer to past email data to ensure consistency and appropriateness of content.

[0059] Terminal

[0060] A preview of the generated email is displayed to the user for approval or correction.

[0061] User

[0062] Review the preview and make any necessary changes. Once approved, you will receive an email.

[0063] Specific examples

[0064] When User B wants to send an email proposing a new product to a client, the server generates the appropriate content and displays a preview of the email on the device. Once User B checks the content and clicks the approve button, the email is automatically sent.

[0065] Reminder notifications

[0066] server

[0067] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[0068] Terminal

[0069] Display reminders to users and prompt them to take necessary action.

[0070] User

[0071] Check the notification and make the necessary preparations or take action.

[0072] Specific examples

[0073] When the deadline for User C's monthly report submission approaches, the server generates a reminder notification and displays it on the terminal. User C checks the notification and prepares to submit the report.

[0074] In-house Wiki creation support

[0075] server

[0076] Analyzes meeting recording data and text data to automatically generate minutes.

[0077] Generate documents suitable for internal Wiki formats.

[0078] Terminal

[0079] The generated minutes are previewed to the user and the user is asked to approve or correct the contents.

[0080] User

[0081] Check the minutes, make any necessary corrections, and click the Approve button.

[0082] Specific examples

[0083] After a meeting that User D participated in, the recording of the meeting is uploaded to the server. The server analyzes the recording, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company Wiki.

[0084] Document preparation support

[0085] server

[0086] The necessary data is collected from various internal databases and documents are generated based on specified templates.

[0087] Ensure documents are automatically formatted properly.

[0088] Terminal

[0089] A preview of the generated document is displayed to the user for approval or correction.

[0090] User

[0091] Review the document and make any necessary corrections. Once approved, the document will be saved and the submission process will be complete.

[0092] Specific examples

[0093] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information and automatically generates the documents. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[0094] This system frees employees from tedious tasks and provides an environment where they can focus on more creative work, which is expected to result in improved work efficiency and increased productivity across the company.

[0095] The processing flow will be explained below.

[0096] Meeting scheduling and time coordination

[0097] Step 1:

[0098] To request a conference, a user inputs information about the purpose of the conference and the necessary participants into the system.

[0099] Step 2:

[0100] The terminal transmits the input request to the server.

[0101] Step 3:

[0102] The server retrieves the schedules of all participants from the employee database.

[0103] Step 4:

[0104] The server analyzes the acquired schedule data and calculates the free time of each employee.

[0105] Step 5:

[0106] The server runs an algorithm to suggest the best meeting time and suggests a time.

[0107] Step 6:

[0108] The terminal displays the proposed meeting time to the user and provides an interface for approval or modification.

[0109] Step 7:

[0110] The user reviews the proposed time and takes action to approve or modify it.

[0111] Step 8:

[0112] The server adds the approved meeting time to the calendars of all participants, completing the meeting setup.

[0113] Creating and replying to emails

[0114] Step 1:

[0115] A user requests a new email or reply and enters a summary of the email and who it is for.

[0116] Step 2:

[0117] The device sends a request to the server.

[0118] Step 3:

[0119] The server uses the generative AI model to generate appropriate email content.

[0120] Step 4:

[0121] The server sends the generated email content to the terminal.

[0122] Step 5:

[0123] The terminal displays a preview of the generated email to the user.

[0124] Step 6:

[0125] The user checks the preview and makes any necessary corrections.

[0126] Step 7:

[0127] The user approves the email and sends it.

[0128] Step 8:

[0129] The server sends the approved email to the specified recipient.

[0130] Reminder notifications

[0131] Step 1:

[0132] The server regularly monitors employees' calendars and task management databases.

[0133] Step 2:

[0134] The server checks the deadline for submitting documents and the start time of the meeting.

[0135] Step 3:

[0136] Generate reminder notifications based on events and deadlines seen by the server.

[0137] Step 4:

[0138] The device displays a reminder notification to the user at the specified timing.

[0139] Step 5:

[0140] The user checks the notification and takes the necessary steps.

[0141] In-house Wiki creation support

[0142] Step 1:

[0143] A user uploads meeting recordings or text data to the system.

[0144] Step 2:

[0145] The device sends the recorded data or text data to the server.

[0146] Step 3:

[0147] The server analyzes the conference data and, if it is audio data, converts it into text using voice recognition technology.

[0148] Step 4:

[0149] The server automatically generates minutes based on the analyzed data.

[0150] Step 5:

[0151] The server converts the generated minutes into an internal Wiki format.

[0152] Step 6:

[0153] The terminal displays a preview of the generated minutes to the user.

[0154] Step 7:

[0155] The user checks the minutes and makes any necessary corrections.

[0156] Step 8:

[0157] The user approves the minutes and updates them on the company wiki.

[0158] Document preparation support

[0159] Step 1:

[0160] The user requests a document and enters the required information.

[0161] Step 2:

[0162] The device sends a request to the server.

[0163] Step 3:

[0164] The server collects the required data from each database.

[0165] Step 4:

[0166] The server automatically generates documents by inputting the collected data into a specified template.

[0167] Step 5:

[0168] The server sends the generated document to the terminal.

[0169] Step 6:

[0170] The terminal displays a preview of the generated document to the user.

[0171] Step 7:

[0172] The user reviews the document and makes any necessary corrections.

[0173] Step 8:

[0174] The user approves the document and the server stores the document, completing the submission process.

[0175] Through these steps, the system efficiently supports each task and provides an environment in which employees can focus on their creative work.

[0176] Example 1

[0177] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0178] In today's corporate environment, employees are often overwhelmed with a large amount of miscellaneous tasks, which results in insufficient time for creative work or important tasks that they should be focusing on. In addition, tasks such as coordinating meetings, writing appropriate emails, and managing reminder notifications are cumbersome and cause efficiency to decline. There is a need to improve this situation and increase employee productivity.

[0179] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0180] In this invention, the server includes a means for acquiring employee schedules, a means for calculating optimal meeting times using a generative AI model, and a means for proposing meeting times to a terminal. This enables efficient scheduling of meetings. It also includes a means for creating appropriate emails using a generative AI model, a means for previewing the created email on the terminal, and a means for approving or modifying the previewed email. This improves the efficiency of email creation tasks. It also includes a means for monitoring employee event schedules and a work management database, a means for generating reminders for important events and deadlines, and a means for displaying the reminders to the user. This makes it possible to prevent important tasks from being forgotten.

[0181] "Means for obtaining employee schedules" refers to hardware and software for obtaining employee schedule data, specifically including APIs and database access.

[0182] "Means for calculating optimal meeting times using a generative AI model" refers to algorithms and software that use a generative AI model to analyze the free time of all employees and calculate optimal meeting times.

[0183] "Means for proposing meeting times to a terminal" refers to the interface and software that transmits the calculated optimal meeting time from the server to the terminal and allows the user to confirm, approve, or modify it.

[0184] "Means for Approving or Modifying Proposed Meeting Times" refers to the interface and software that allows a user to review, modify, if necessary, and ultimately approve proposed meeting times.

[0185] "Means for adding approved meeting times to an employee's calendar" means software and APIs that automatically add user-approved meeting times to an employee's calendar.

[0186] "Means for creating appropriate emails using a generative AI model" refers to software that uses a generative AI model to automatically generate appropriate email content based on information provided by a user.

[0187] The "means for displaying a preview of the created e-mail on the terminal" refers to an interface and software for displaying a preview of the content of the created e-mail on the user's terminal.

[0188] "Means for approving or correcting previewed email" refers to the interface and software that allows a user to review the contents of a previewed email, make corrections as necessary, and ultimately approve it.

[0189] "Means for sending approved email" refers to software and APIs that automatically send user-approved email through an external mail server.

[0190] "Means for monitoring employee calendars and work management databases" refers to software and APIs that regularly monitor employee calendars and task management systems to detect important events and task deadlines.

[0191] "Means for generating reminders for important events and deadlines" refers to software that automatically generates appropriate reminders for important events and task deadlines detected through monitoring.

[0192] "Means for displaying a reminder to a user" refers to an interface and software that displays the generated reminder on the user's terminal and allows the user to take appropriate action.

[0193] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks on behalf of employees. In this system, the server mainly performs the processing, and the terminal provides the interface with the user, who confirms and approves specific tasks.

[0194] Meeting scheduling and time coordination

[0195] server

[0196] An API (for example, a calendar API) is used to obtain employee calendar data. The server analyzes the obtained data and runs an algorithm to calculate the free time of all employees. This algorithm analyzes the free time using a weighted average method or a heuristic algorithm to determine the optimal meeting time. The generated meeting time is sent from the server to the terminal.

[0197] Terminal

[0198] The system suggests optimal meeting times to users and displays them in an interface, allowing users to review the suggested meeting times and make adjustments as necessary.

[0199] User

[0200] Review the proposed time, make any necessary changes, and finally click the approve button, and the server will automatically add the meeting to your calendar.

[0201] Specific examples

[0202] When User A wants to schedule a new project meeting, the server uses the calendar API to retrieve the schedules of User A and related employees and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[0203] Prompt Sentence Examples

[0204] "Please suggest the best time for a new project meeting."

[0205] Creating and replying to emails

[0206] server

[0207] Based on the user's request, a natural language processing (NLP) model (e.g., a generative AI model) is used to generate appropriate email content. The generated email content is temporarily stored on the server and then sent to the device.

[0208] Terminal

[0209] A preview of the generated email is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[0210] User

[0211] Check the preview and make any necessary corrections. Finally, click the approve button and the server will automatically send the email.

[0212] Specific examples

[0213] If User B wants to send an email proposing a new product to a client, the server uses a generative AI model to generate appropriate email content, displays a preview of the email on the device, and once User B confirms the content and clicks the approve button, the email is automatically sent.

[0214] Prompt Sentence Examples

[0215] "Write an email to pitch a new product to a client."

[0216] Reminder notifications

[0217] server

[0218] Regularly monitor employee calendars and work management databases to detect important events and task deadlines, and use appropriate APIs and scripts to generate reminder notifications based on the detected data.

[0219] Terminal

[0220] The generated reminder notification is displayed to the user, and an interface is provided to prompt the user to take appropriate action.

[0221] User

[0222] Check the notification and take the necessary preparations or actions. Clicking on the notification will display more information and the next steps.

[0223] Specific examples

[0224] When the deadline for User C's monthly report submission approaches, the server uses the event schedule API to generate a reminder notification and displays it on the device. User C checks the notification and prepares to submit the report.

[0225] Prompt Sentence Examples

[0226] "Show me a reminder when my monthly report is due."

[0227] In-house Wiki creation support

[0228] server

[0229] Meeting recording data is acquired and converted into text using a speech recognition service (for example, a speech recognition API). Based on this text data, minutes are automatically generated using a generative AI model and formatted in a format suitable for the in-house wiki.

[0230] Terminal

[0231] A preview of the generated minutes is displayed to the user, and an interface is provided to request confirmation and correction of the contents.

[0232] User

[0233] Check the minutes, make any necessary corrections, and finally click the approve button.

[0234] Specific examples

[0235] The recording data of the meeting that User D participated in is uploaded to the server. The server analyzes the recording using a speech recognition API, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company wiki.

[0236] Prompt Sentence Examples

[0237] "Please create minutes based on the meeting recording data and post them on the company wiki."

[0238] Document preparation support

[0239] server

[0240] The necessary data is collected from the internal database, and documents are automatically generated based on document generation templates (e.g., document template APIs). The generated documents are formatted in the appropriate format.

[0241] Terminal

[0242] A preview of the generated document is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[0243] User

[0244] Review the documents, make any necessary corrections, and finally click the approve button.

[0245] Specific examples

[0246] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information from the company database and automatically generates the documents using the document template API. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[0247] Prompt Sentence Examples

[0248] Please fill out your year-end tax adjustment documents and enter the necessary information.

[0249] This system frees employees from tedious tasks and provides an environment where they can focus on more important tasks, which is expected to result in improved work efficiency and increased productivity across the company.

[0250] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0251] Meeting scheduling and time coordination

[0252] Step 1: Get employee calendar data

[0253] server

[0254] Input: User A sends a request to set up a conference.

[0255] Process: The server sends a request to the calendar API to retrieve employee schedule data.

[0256] Output: The retrieved schedule data.

[0257] Specific operation: The server sends a request to " / user / schedules / {user_id}" and receives schedule data in JSON format.

[0258] Step 2: Calculate the best meeting time

[0259] server

[0260] Input: Retrieved schedule data.

[0261] Processing: The server analyzes the data and calculates the free time of all employees.

[0262] Output: Optimal meeting time.

[0263] Specific behavior: Calls the "find_optimal_time(slots)" function to calculate the best non-overlapping time.

[0264] Step 3: Propose a meeting time

[0265] Terminal

[0266] Input: The best meeting time sent by the server.

[0267] Processing: The terminal displays the meeting time on the user interface.

[0268] Output: The suggested time displayed to the user.

[0269] Specific behavior: The terminal uses the "display_suggested_time(time)" method to display the meeting time in the interface.

[0270] Step 4: Review, revise, and approve the proposal

[0271] User

[0272] Input: Proposed meeting time.

[0273] Process: The user reviews the proposed time, makes any necessary corrections, and finally approves it.

[0274] Output: The revised or approved meeting time.

[0275] Specific action: The user clicks the "confirm_time()" or "modify_time()" button.

[0276] Step 5: Schedule a meeting

[0277] server

[0278] Input: The revised or approved meeting time.

[0279] Process: The server automatically sets up the meeting through the calendar API.

[0280] Output: The configured meeting.

[0281] Specific behavior: The server sends a POST request to " / calendar / events" to add the meeting to the calendar.

[0282] Creating and replying to emails

[0283] Step 1: Receive the user request

[0284] server

[0285] Input: User B sends an email composition request.

[0286] Processing: The server receives and analyzes the request.

[0287] Output: Email input prompt.

[0288] Specific behavior: A user fills out the " / create_email" form and submits the request.

[0289] Step 2: Generate email content

[0290] server

[0291] Input: Prompt to enter email.

[0292] Processing: The server generates the email content using the generative AI model.

[0293] Output: The generated email content.

[0294] Specific behavior: Calls the "generate_email_content(prompt)" function to generate text using a generative AI model.

[0295] Step 3: Preview your email

[0296] Terminal

[0297] Input: The generated email content.

[0298] Processing: The terminal displays the email contents on the user interface.

[0299] Output: The email preview shown to the user.

[0300] What happens: The device displays a preview of the email using "display_email_preview(email)".

[0301] Step 4: Check, edit, and approve the email content

[0302] User

[0303] Input: A preview of the generated email.

[0304] Processing: The user reviews the email, makes any necessary corrections, and approves it.

[0305] Output: The corrected or approved email content.

[0306] Specific behavior: The user clicks the "confirm_email()" or "modify_email()" button.

[0307] Step 5: Send an email

[0308] server

[0309] Input: The corrected or approved email content.

[0310] Process: The server sends the email through the mail server.

[0311] Output: The email sent.

[0312] Specific behavior: The server sends an email to the SMTP server using the "send_email(email)" function.

[0313] Reminder notifications

[0314] Step 1: Monitor your calendar and task management data

[0315] server

[0316] Input: Calendar and task management data.

[0317] Processing: The server periodically monitors the data to detect important events and task deadlines.

[0318] Output: Detected events and deadline data.

[0319] Specific behavior: The server periodically monitors " / calendar / events" and " / tasks" and collects data.

[0320] Step 2: Generate a reminder notification

[0321] server

[0322] Input: Detected event and deadline data.

[0323] Processing: The server generates a reminder notification.

[0324] Output: The generated reminder notification.

[0325] Specific behavior: Generates a reminder notification using the "generate_reminder(event)" function.

[0326] Step 3: Display the notification to the user

[0327] Terminal

[0328] Input: The generated reminder notification.

[0329] Processing: The device displays the reminder notification on the user interface.

[0330] Output: The reminder notification shown to the user.

[0331] Specific behavior: The device displays the notification using the "display_reminder(reminder)" method.

[0332] Step 4: Review and respond to notifications

[0333] User

[0334] Input: The reminder notification shown to the user.

[0335] Action: The user checks the notification and takes the necessary action.

[0336] Output: The action that was performed.

[0337] Specific action: The user clicks on the reminder notification and takes the necessary steps.

[0338] In-house Wiki creation support

[0339] Step 1: Get the meeting recording data

[0340] server

[0341] Input: Meeting recording data.

[0342] Processing: The server receives and stores the recording.

[0343] Output: Saved meeting recording data.

[0344] Specific behavior: The user uploads the recording to " / upload_meeting_audio".

[0345] Step 2: Analyze the recording data and generate transcripts

[0346] server

[0347] Input: Saved meeting recording data.

[0348] Processing: The server uses a speech recognition API to convert the speech to text, and then uses a generative AI model to generate the transcript.

[0349] Output: The generated transcript.

[0350] Specific behavior: The server converts the audio to text using the "transcribe_audio(audio_file)" function, and generates the transcript using the "generate_minutes(text)" function.

[0351] Step 3: Preview the transcript

[0352] Terminal

[0353] Input: The generated minutes.

[0354] Processing: The terminal displays the minutes on the user interface.

[0355] Output: A preview of the transcript as displayed to the user.

[0356] Specific behavior: The terminal displays the transcript using the "display_transcription(transcript)" method.

[0357] Step 4: Review, revise, and approve the minutes

[0358] User

[0359] Input: A preview of the generated transcript.

[0360] Process: The user reviews the minutes, makes any necessary corrections, and approves them.

[0361] Output: Amended or approved minutes.

[0362] Specific behavior: The user clicks the "confirm_transcript()" or "edit_transcript()" button.

[0363] Step 5: Post the minutes on the company wiki

[0364] server

[0365] Enter: Amended or approved minutes.

[0366] Process: The server posts the minutes to the company wiki.

[0367] Output: Published minutes.

[0368] Specific operation: The server sends a POST request to " / wiki / add_entry" to add the minutes to the internal Wiki.

[0369] Document preparation support

[0370] Step 1: Collect the necessary data

[0371] server

[0372] Input: Data collection request.

[0373] Processing: The server collects the necessary data from the company database.

[0374] Output: Collected data.

[0375] Specific operation: The server sends a request to " / database / get_data" to collect the data.

[0376] Step 2: Auto-generate documents

[0377] server

[0378] Input: Collected data.

[0379] Processing: The server automatically generates the document using the document template API.

[0380] Output: The generated document.

[0381] Specific operation: Use the "generate_document(template, data)" function to call the Document Template API and generate a document.

[0382] Step 3: Preview your document

[0383] Terminal

[0384] Input: The generated document.

[0385] Processing: The device displays a preview of the document in its user interface.

[0386] Output: A preview of the document as it appears to the user.

[0387] Specific behavior: The device displays a preview of the document using the "display_document_preview(document)" method.

[0388] Step 4: Review, correct, and approve the document

[0389] User

[0390] Input: A preview of the generated document.

[0391] Processing: The user reviews the document, makes any necessary corrections, and approves it.

[0392] Output: The corrected or approved document.

[0393] Specific behavior: The user clicks the "confirm_document()" or "edit_document()" button.

[0394] Step 5: Save and submit your documents

[0395] server

[0396] Enter: Amended or approved document.

[0397] Processing: The server saves the document in the specified location and processes it for submission.

[0398] Output: Submitted documents.

[0399] Specific operation: The server sends a POST request to " / documents / submit" to submit the document.

[0400] (Application example 1)

[0401] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0402] In factories, manual robot task management and anomaly detection requires a great deal of effort and time. It is also difficult to manually create an optimal schedule when multiple robots are operating simultaneously, which can lead to reduced efficiency. Furthermore, delayed response to anomalies can lead to a decline in quality and stalled production lines. To solve these problems, a system is needed that can automatically acquire robot status, generate optimal task schedules, detect anomalies, and propose countermeasures.

[0403] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0404] In this invention, the server includes means for acquiring the robot's status, means for proposing an optimal task schedule using a generative AI model, means for approving or modifying the proposed task schedule, means for controlling the robot's operation based on the approved task schedule, means for collecting and analyzing feedback data, and means for proposing the next action based on the analysis results. This improves the robot's operation efficiency and makes it possible to prevent production line stoppages by automating anomaly detection and countermeasures.

[0405] A "robot" is a mechanical device designed to automatically perform specific tasks in a factory or on a production line.

[0406] "Status" is data that indicates the current condition or status of the robot's operating state or function.

[0407] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal solutions and predictions based on large amounts of data.

[0408] A "task schedule" is a plan that efficiently arranges a series of tasks or duties to be performed by a robot and allocates them by time.

[0409] "Suggestion" refers to the generative AI model calculating optimal schedules and actions and providing them to the user.

[0410] "Approval" means that the user reviews and accepts the proposed schedule and actions.

[0411] "Control" refers to the proper operation of a robot's movements and functions based on programs and systems.

[0412] "Feedback data" refers to various sensor data and operation logs collected while the robot is performing its tasks.

[0413] "Analysis" is a method of analyzing collected feedback data to detect anomalies and evaluate performance.

[0414] "Action" refers to the specific actions or countermeasures to be taken next based on the analysis results.

[0415] This invention is a system that automates robot task management and anomaly detection in factories. The system acquires robot status data, generates an optimal task schedule using a generative AI model, analyzes feedback data to detect anomalies, and proposes next actions.

[0416] Hardware and software used

[0417] Hardware: Factory robots, various sensors, and servers.

[0418] Software: Python, NLP (Natural Language Processing) models, database management systems (e.g., PostgreSQL).

[0419] 1. Task schedule generation

[0420] The server first obtains the status data of the factory robots (e.g., current working status, operating hours, maintenance history), then uses a generative AI model to generate an optimal task schedule based on the obtained status data, and the generated task schedule is approved or modified to ensure the robots' efficient operation.

[0421] As a concrete example, if a robot is in charge of assembling part A, the generative AI model will generate the following schedule:

[0422] Assembly start time for part A: 09:00

[0423] Assembly of part A completed at 10:00

[0424] Next task start time: 10:05

[0425] 2. Feedback Analysis

[0426] The server collects feedback data from the robot in operation and analyzes it using an NLP model. This feedback data includes various sensor data such as temperature, vibration, and operation logs. Based on the analysis results, the next action is suggested. For example, if an abnormal temperature is detected, the next action suggested would be "Check for errors in part X and correct them."

[0427] 3. Implementing the proposed action

[0428] The actions proposed by the server can be approved or modified by the user. Once approved, the server again instructs the robot to execute the action. This automates the robot's response to abnormalities and maintenance, resulting in efficient operation.

[0429] Prompt Sentence Examples

[0430] Imagine a system that generates optimal task schedules for factory robots and prescribes next actions based on feedback data. Specifically, the system optimizes the robot schedule using NLP models, analyzes sensor feedback data (e.g., temperature, vibration), and automatically suggests countermeasures when anomalies are detected.

[0431] The system of the present invention improves the operational efficiency of robots and prevents production line stoppages by automating abnormality detection and countermeasures, thereby significantly improving the productivity of the entire factory and significantly reducing the need for manual management.

[0432] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0433] Step 1:

[0434] The server collects status data from factory robots, including their current working status, operating hours, and maintenance history. This data is used to aggregate information that forms the basis for the next processing step. The input is the robot's sensor data and operation log, and the output is the robot's status data.

[0435] Step 2:

[0436] The server uses a generative AI model to generate an optimal task schedule based on the acquired robot status data. The input is the status data, and the output is a proposed task schedule. The generative AI model analyzes the data and calculates the efficient work order for the robots.

[0437] Step 3:

[0438] The terminal displays the proposed task schedule to the user, who can then approve or modify it. The input is the task schedule generated by the server, and the output is the user's approved or modified schedule.

[0439] Step 4:

[0440] The user reviews the proposed task schedule and approves or modifies it. The user's actions are used to control the next step. The input is the proposed task schedule, and the output is the user's approval or modification.

[0441] Step 5:

[0442] The server controls the robot's actions based on the approved task schedule. The input is the schedule approved by the user, and the output is the control signal to the robot, which then performs the task as specified.

[0443] Step 6:

[0444] The server collects feedback data while the robot is operating, including temperature, vibration, and operation logs. The input is the robot's sensor data, and the output is the feedback data.

[0445] Step 7:

[0446] The server analyzes the collected feedback data and uses an NLP model to detect anomalies and maintenance needs. The input is the feedback data and the output is the analysis result.

[0447] Step 8:

[0448] The server proposes the next action based on the analysis results. For example, if an anomaly is detected, it suggests that a specific part needs to be repaired or maintained. The input is the analysis results, and the output is the proposed action.

[0449] Step 9:

[0450] The terminal displays the proposed action to the user, who can review it and accept or modify it as needed. The input is the server's proposed action, and the output is the user's accepted or modified action.

[0451] Step 10:

[0452] The user reviews the proposed action and approves or modifies it. The input is the proposed action and the output is the user's approval or modification.

[0453] Step 11:

[0454] The server executes the approved actions and sends control signals to the robot, which then performs the appropriate repairs or maintenance. The input is the action approved by the user, and the output is the command for the robot to execute.

[0455] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0456] This invention is a system that combines a generative AI model with an emotion engine to streamline employees' daily work and provide optimal support based on the user's emotions. The system learns employees' work processes, internal tools, and databases, recognizes the user's emotions using the emotion engine, and responds appropriately.

[0457] Meeting scheduling and time coordination

[0458] server

[0459] Employee schedules are retrieved from the database and analyzed.

[0460] It uses an emotion engine to recognize the user's current emotions and runs an algorithm that suggests optimal meeting times that take into consideration the emotions.

[0461] Terminal

[0462] The system provides an interface that displays proposed meeting times to users and asks for approval or modification based on sentiment.

[0463] User

[0464] Review the proposed time and accept or modify it. Emotion-based suggestions make meeting scheduling smoother.

[0465] Specific examples

[0466] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members. The emotion engine suggests the optimal time to reduce User A's stress level. Once User A confirms and approves the suggested time on their device, the meeting is added to their calendar.

[0467] Creating and replying to emails

[0468] server

[0469] Based on user requests, NLP models are used to generate appropriate email content.

[0470] The emotional engine recognizes the user's emotions and creates emails with tone and content based on those emotions.

[0471] Terminal

[0472] Show users a preview of the generated email and ask for sentiment-based approval or revision.

[0473] User

[0474] Check the preview and make any necessary corrections. By generating emails that are sensitive to customer sentiment, communication becomes smoother.

[0475] Specific examples

[0476] If User B wants to send an email proposing a new product to a client, the server will use the emotion engine to recognize that User B is nervous. Therefore, it will generate an email with a relaxed tone and display a preview on the device. If User B checks the content and approves it, the email will be sent.

[0477] Reminder notifications

[0478] server

[0479] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[0480] It uses an emotion engine to recognize the user's emotions and generate reminder notifications with optimal timing and content.

[0481] Terminal

[0482] Display emotional reminders to users and encourage them to take appropriate action.

[0483] User

[0484] Check the notification and make the necessary preparations or take action.

[0485] Specific examples

[0486] When the deadline for User C's monthly report is approaching, the server uses the emotion engine to recognize User C's current stress level. Therefore, a reminder notification is generated and displayed on the device at a time that minimizes stress. User C checks the notification and prepares the report.

[0487] In-house Wiki creation support

[0488] server

[0489] Analyzes meeting recording data and text data to automatically generate minutes.

[0490] Use an emotion engine to adjust the content and expression of meeting minutes based on user emotions.

[0491] Terminal

[0492] A preview of the generated meeting minutes is displayed to the user for approval or revision based on sentiment.

[0493] User

[0494] Check the minutes, make any necessary corrections, and click the approve button.

[0495] Specific examples

[0496] After a meeting that User D attends, the recording of the meeting is uploaded to the server. The emotion engine recognizes User D's fatigue and automatically generates concise, to-the-point minutes. Once User D checks and approves the content, the minutes are posted on the company's internal wiki.

[0497] Document preparation support

[0498] server

[0499] The necessary data is collected from various internal databases and documents are generated based on specified templates.

[0500] Use an emotion engine to recognize user emotions and adjust tone and format accordingly.

[0501] Terminal

[0502] A preview of the generated document is displayed to the user for approval or modification based on their sentiment.

[0503] User

[0504] Review the document and make any necessary corrections. Once approved, the document is saved and submitted.

[0505] Specific examples

[0506] When User E needs to create documents for year-end tax adjustment, the server uses an emotion engine to recognize User E's current emotional state. Therefore, the server automatically generates documents in a format that is easy for User E to use and displays a preview. Once User E confirms and approves the contents, the documents are saved and submitted.

[0507] This system provides support that takes users' emotions into consideration, improving work efficiency and the quality of communication. As a result, it is expected that an environment will be created where employees can focus on their core creative work, thereby increasing productivity across the company.

[0508] The processing flow will be explained below.

[0509] Meeting scheduling and time coordination

[0510] Step 1:

[0511] To request a conference, a user inputs information about the purpose of the conference and the necessary participants into the system.

[0512] Step 2:

[0513] The terminal transmits the user's input data to the server.

[0514] Step 3:

[0515] The server retrieves the schedules of all participants from the employee database.

[0516] Step 4:

[0517] The server analyzes the acquired schedule data and calculates the free time of each employee.

[0518] Step 5:

[0519] The server uses an emotion engine to recognize the user's current emotion.

[0520] Step 6:

[0521] The server runs an algorithm that suggests optimal meeting times based on sentiment.

[0522] Step 7:

[0523] The terminal displays the proposed meeting time to the user and provides an interface for sentiment-based approval or modification.

[0524] Step 8:

[0525] The user reviews the proposed time and takes action to approve or modify it.

[0526] Step 9:

[0527] The server adds the approved meeting time to the calendars of all participants, completing the meeting setup.

[0528] Creating and replying to emails

[0529] Step 1:

[0530] A user requests a new email or reply and enters a summary of the email and who it is for.

[0531] Step 2:

[0532] The device sends a request to the server.

[0533] Step 3:

[0534] The server uses the generative AI model to generate appropriate email content.

[0535] Step 4:

[0536] The server uses an emotion engine to recognize the user's emotions and generates emails with a tone and content based on those emotions.

[0537] Step 5:

[0538] The server sends the generated email content to the terminal.

[0539] Step 6:

[0540] The terminal displays a preview of the generated email to the user.

[0541] Step 7:

[0542] The user checks the preview and makes any necessary corrections.

[0543] Step 8:

[0544] The user approves the email and sends it.

[0545] Step 9:

[0546] The server sends the approved email to the specified recipient.

[0547] Reminder notifications

[0548] Step 1:

[0549] The server regularly monitors employees' calendars and task management databases.

[0550] Step 2:

[0551] The server checks the deadline for submitting documents and the start time of the meeting.

[0552] Step 3:

[0553] The server uses an emotion engine to recognize the user's emotions and generates a reminder notification with optimal timing and content.

[0554] Step 4:

[0555] The device displays emotion-based reminder notifications to the user.

[0556] Step 5:

[0557] The user checks the notification and takes the necessary steps.

[0558] In-house Wiki creation support

[0559] Step 1:

[0560] A user uploads meeting recordings or text data to the system.

[0561] Step 2:

[0562] The device sends the recorded data or text data to the server.

[0563] Step 3:

[0564] The server analyzes the conference data and, if it is audio data, converts it into text using voice recognition technology.

[0565] Step 4:

[0566] The server automatically generates minutes based on the analyzed data.

[0567] Step 5:

[0568] The server recognizes the user's emotions using an emotion engine and adjusts the content and expression of the minutes.

[0569] Step 6:

[0570] The server converts the generated minutes into an internal Wiki format.

[0571] Step 7:

[0572] The terminal displays a preview of the generated minutes to the user.

[0573] Step 8:

[0574] The user checks the minutes and makes any necessary corrections.

[0575] Step 9:

[0576] The user approves the minutes, and the server updates the minutes to the company Wiki.

[0577] Document preparation support

[0578] Step 1:

[0579] The user requests a document and enters the required information.

[0580] Step 2:

[0581] The device sends a request to the server.

[0582] Step 3:

[0583] The server collects the required data from each database.

[0584] Step 4:

[0585] The server automatically generates documents by inputting the collected data into a specified template.

[0586] Step 5:

[0587] The server uses an emotion engine to recognize the user's emotions and adjust the tone and format accordingly.

[0588] Step 6:

[0589] The server sends the generated document to the terminal.

[0590] Step 7:

[0591] The terminal displays a preview of the generated document to the user.

[0592] Step 8:

[0593] The user reviews the document and makes any necessary corrections.

[0594] Step 9:

[0595] The user approves the document and the server stores the document, completing the submission process.

[0596] Through these steps, the system will be able to efficiently support each task and respond in a way that takes into consideration the user's feelings. This will create an environment where employees can focus on their creative work, which is what we believe is essential, and is expected to improve productivity across the company.

[0597] Example 2

[0598] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0599] In conventional systems, managing employee schedules and communications is often complex and time-consuming, resulting in reduced work efficiency. Furthermore, because processes proceed mechanically without considering the user's emotions, stress and discord in communication can occur. Furthermore, reminder notifications and email creation do not take the user's emotional state into consideration, which can lead to problems such as insufficient stress reduction and work efficiency.

[0600] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0601] In this invention, the server includes a means for acquiring employee schedules, a means for recognizing user emotions using an emotion engine, and a means for proposing optimal meeting times using a generative AI model, thereby enabling the proposal of optimal meeting times that take user emotions into consideration.

[0602] The server includes means for recognizing a user's emotion and creating an email with an appropriate tone and expression based on the emotion, means for displaying a preview of the created email to the user, means for approving or correcting the previewed email based on the emotion, and means for sending the approved email. This allows an email with an appropriate tone according to the user's emotion to be created, facilitating smooth communication.

[0603] The server includes a means for monitoring employees' calendars and task management databases, and a means for generating reminder notifications of important events and deadlines at optimal timing based on the user's emotions, and a means for sending reminder notifications based on the emotions to the user, thereby enabling reminder notifications at optimal timing in line with the user's emotional state, thereby reducing stress and improving work efficiency.

[0604] An "employee schedule" is a record of the timetables of work hours, meetings, tasks, etc. scheduled by employees belonging to an organization.

[0605] "Emotion engine" is a general term for software or hardware that analyzes and recognizes a user's emotional state (e.g., stress level, fatigue, joy, etc.).

[0606] A "generative AI model" is an artificial intelligence model that generates output for natural language processing or specific tasks based on given input data, and primarily uses machine learning algorithms.

[0607] "Meeting time suggestion" refers to the act of the generative AI model presenting the optimal meeting time for the user based on the analysis results.

[0608] "Preview display" refers to a function that allows the user to view the generated content (for example, email or minutes) in advance and check or modify it.

[0609] A "reminder notification" is a notification sent to a user before an important event or deadline, to help the user remember.

[0610] The "optimal timing" refers to the time when the notification or action will be most effectively received, taking into account the user's current emotional state and work situation.

[0611] A "calendar" is a tool for visually managing employees' schedules and tasks, and is generally composed of dates and times.

[0612] A "task management database" is a database system for digitally managing and recording the tasks and business processes that employees must complete.

[0613] "Tone" refers to the emotional nuances and style of writing or communication, including friendly and formal tones.

[0614] This invention is a system that combines a generative AI model with an emotion engine to streamline employees' daily work and provide optimal support based on the user's emotions. This system operates using three elements: a server, a terminal, and the user.

[0615] Meeting scheduling and time coordination

[0616] server

[0617] The server retrieves employee schedules from a database and performs analysis. Specifically, the server executes a database query to retrieve each employee's schedule data. The server then uses an emotion engine to recognize user emotions and runs an algorithm to suggest optimal meeting times based on the analysis results.

[0618] Specific examples

[0619] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members from the schedule database. Then, the emotion engine suggests the best time to reduce User A's stress level.

[0620] Prompt Sentence Examples

[0621] "Check User A's schedule and suggest the best meeting time based on the sentiment engine data."

[0622] Creating and replying to emails

[0623] server

[0624] The server uses NLP models to generate appropriate email content based on user requests, and an emotion engine to recognize user emotions and create emails with appropriate tone and content.

[0625] Specific examples

[0626] If User B wants to send an email proposing a new product to a client, the server uses the emotion engine to recognize that User B is nervous, generates an email with a relaxed tone, and displays a preview on the device.

[0627] Prompt Sentence Examples

[0628] "User B is nervous, so please generate a new product proposal email with a relaxed tone."

[0629] Reminder notifications

[0630] server

[0631] The server monitors employees' calendars and task management databases to generate reminders for important events and deadlines. It uses an emotion engine to recognize users' emotions and generates reminders with optimal timing and content.

[0632] Specific examples

[0633] When User C's monthly report submission deadline approaches, the server uses the emotion engine to recognize User C's current stress level. A reminder notification is generated and displayed on the device at a time that minimizes stress.

[0634] Prompt Sentence Examples

[0635] "Please remind User C to submit the monthly report at a time when he is least stressed."

[0636] In-house Wiki creation support

[0637] server

[0638] The server analyzes meeting recording data and text data to automatically generate minutes, and uses an emotion engine to adjust the content and expression of the minutes based on the user's emotions.

[0639] Specific examples

[0640] After a meeting that User D participated in, the recording of the meeting is uploaded to the server. The emotion engine recognizes User D's sense of fatigue and automatically generates concise, to-the-point minutes.

[0641] Prompt Sentence Examples

[0642] "Please consider User D's fatigue and generate concise minutes."

[0643] Document preparation support

[0644] server

[0645] The server collects the necessary data from various company databases, generates documents based on specified templates, and uses an emotion engine to recognize the user's emotions and adjust the tone and format accordingly.

[0646] Specific examples

[0647] When User E needs to create documents for year-end tax adjustment, the server uses an emotion engine to recognize User E's current emotional state, automatically generates documents in a format that is easy to use, and displays a preview.

[0648] Prompt Sentence Examples

[0649] "Please generate the year-end tax adjustment documents in a format that is less burdensome for User E."

[0650] This system provides support that takes users' emotions into consideration, improving work efficiency and the quality of communication. As a result, it is expected that an environment will be created where employees can focus on their core creative work, thereby increasing productivity across the company.

[0651] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0652] Meeting scheduling and time coordination

[0653] Step 1: Get the schedule

[0654] The server connects to the database to retrieve employee schedule data. Specifically, the server runs a database query to pull schedule data from the "employee_schedules" table.

[0655] Input: Employee ID

[0656] Output: Employee schedule data

[0657] What it does: Retrieves schedule information from a database using an SQL query.

[0658] Step 2: Emotion recognition and analysis

[0659] The server uses the emotion engine to recognize the user's current emotional state and calls the emotion engine API to obtain the emotion data.

[0660] Input: User ID

[0661] Output: User emotion data

[0662] Specific operation: Calls the emotion engine API and collects emotion data.

[0663] Step 3: Suggest the best meeting time

[0664] The server runs an algorithm that suggests optimal meeting times based on the acquired schedule and emotion data, using a generative AI model to calculate the suggested times.

[0665] Input: Employee schedule data, user emotion data

[0666] Output: Best meeting time

[0667] What it does: Runs an algorithm using a generative AI model to calculate optimal meeting times.

[0668] Step 4: View and review the proposal

[0669] The terminal receives the proposed meeting time from the server and displays it to the user, who can then review and approve or modify the proposed time.

[0670] Input: Best Meeting Time

[0671] Output: Suggested time displayed to the user

[0672] Specific behavior: Show the suggested time to the user via notification or popup.

[0673] Step 5: Set up a meeting

[0674] The user reviews the proposed meeting time and can approve or modify it, and once approved, the update is sent to the server and the meeting is set in their calendar.

[0675] Input: User approval or correction information

[0676] Output: Meeting times set in the calendar

[0677] Specific operation: When the user clicks the approve button, the server calls the calendar API to add the meeting.

[0678] Creating and replying to emails

[0679] Step 1: Receiving a user request

[0680] The server receives a request to compose an email from the user: The user clicks the "Compose new email" button in the email sending interface.

[0681] Input: Email creation request

[0682] Output: Start of email creation process

[0683] Specific behavior: Receives an HTTP request and starts the email creation process.

[0684] Step 2: Emotion recognition and analysis

[0685] The server uses an emotion engine to recognize the user's emotions, and acquires and analyzes the emotion data.

[0686] Input: User ID

[0687] Output: User emotion data

[0688] Specific operation: Call the emotion engine API and obtain the user's emotional state.

[0689] Step 3: Generate email content

[0690] The server uses NLP models to generate appropriate email content based on the user's emotional data.

[0691] Input: User emotion data, basic email information

[0692] Output: Generated email content

[0693] Specific operation: A prompt sentence is input into the generative AI model to generate email content.

[0694] Step 4: Preview and check

[0695] The terminal displays a preview of the generated email to the user and provides an interface for the user to review and modify the content.

[0696] Input: Generated email content

[0697] Output: Email preview shown to the user

[0698] Specific behavior: Displays the email preview screen and allows the user to edit it.

[0699] Step 5: Sending an email

[0700] The user checks the contents of the email and clicks the send button. Once approval is complete, the sending information is sent to the server and the email is actually sent.

[0701] Input: User authorization information

[0702] Output: Email sent

[0703] Specific behavior: When you click the send button, the server sends the email using the SMTP server.

[0704] Reminder notifications

[0705] Step 1: Monitor your calendar and task data

[0706] The server periodically monitors employees' calendars and task management databases to detect important events and deadlines.

[0707] Input: Calendar and task data

[0708] Output: Detected important events and deadlines

[0709] Specific behavior: Runs a database query as a background job to obtain task information.

[0710] Step 2: Emotion recognition and analysis

[0711] The server uses an emotion engine to analyze the user's emotions, such as assessing stress levels due to an approaching deadline.

[0712] Input: User ID

[0713] Output: Emotion data

[0714] Specific operation: Utilize the emotion engine API to obtain emotion data.

[0715] Step 3: Generate a reminder notification

[0716] The server generates the optimal timing and content for the reminder notification based on the acquired data and emotional data.

[0717] Input: Calendar, task data, emotion data

[0718] Output: Reminder notification

[0719] Specific behavior: Generates notification content and manages notification schedules.

[0720] Step 4: View and review notifications

[0721] The device receives the reminder notification from the server and displays it to the user, allowing the user to check the notification and take appropriate action.

[0722] Input: Reminder notification

[0723] Output: The notification that is displayed to the user

[0724] Specific behavior: A popup notification will be displayed and an action will be triggered when the user presses the confirmation button.

[0725] (Application example 2)

[0726] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0727] Conventional employee work support systems are limited to functions such as setting up meetings, creating emails, and sending reminders, making it difficult to provide services such as emotion recognition in customer service or optimal product recommendations. Furthermore, there was a lack of a way to provide customer service staff with appropriate responses based on customer emotions in real time. This resulted in issues such as a decline in customer satisfaction and insufficient improvement in work efficiency.

[0728] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring employee schedules, means for proposing optimal meeting times using a generative AI model, means for approving or modifying the proposed meeting times, means for recognizing customer emotions, means for recommending optimal products based on the customer emotions, and means for proposing ways for customer service staff to respond based on the customer emotions. This makes it possible to significantly improve not only the daily work of employees but also the quality of customer service in physical stores, thereby increasing customer satisfaction.

[0729] "Means for obtaining employee schedules" refers to a function that obtains schedule information such as employee calendars and timetables from a database and analyzes it.

[0730] "Means for suggesting optimal meeting times using a generative AI model" is a function that uses a generative AI model to automatically suggest optimal meeting times for everyone based on the acquired employee schedule information.

[0731] A "means for approving or amending a proposed meeting time" is an interface or operating means for an employee to approve or amend a proposed meeting time.

[0732] "Means for adding approved meeting times to employees' calendars" refers to a function that automatically adds the final approved or amended meeting time to the employees' individual calendars.

[0733] "Means for recognizing customer emotions" refers to algorithms and sensor devices that analyze and recognize emotions in real time from customers' facial expressions, tone of voice, etc.

[0734] The "means for recommending optimal products based on customer emotions" is a function that automatically recommends products that are likely to be desired by a customer based on the recognized customer emotions.

[0735] "A means to suggest how to respond to customer service staff based on customer emotions" is a function that analyzes the emotional state of the customer and suggests in real time how the customer service staff should respond.

[0736] "Means for creating emails" refers to a function that uses a generative AI model to automatically create appropriate email content based on user requests.

[0737] The "means for displaying a preview of an email" is an interface for displaying a preview of the generated email to the user, allowing the user to check and modify the contents.

[0738] The "means for approving or correcting the previewed email" is an operation means for the user to approve or correct the previewed email.

[0739] "Means for monitoring employee calendars and task management databases" refers to a function that constantly monitors employee calendars and task management systems to track important events and deadlines.

[0740] The "means for generating reminders for document submission deadlines and meeting times" is a function that extracts important deadlines and meeting times from monitored data and generates reminders based on them.

[0741] The "means for sending a reminder notification to a user" is a system for sending the generated reminder notification to the user's terminal and prompting the user to take the necessary action.

[0742] This invention is a method for optimizing customer service in brick-and-mortar stores by using a system that combines a generative AI model and an emotion engine to streamline the daily work of employees. The hardware environment includes cameras, microphones, servers, and devices (smartphones and tablets) used by customer service staff. The software used includes OpenCV, DeepFace, and OpenAI API.

[0743] Customer Emotion Recognition

[0744] The server collects data in real time from cameras and microphones installed in the store. The video data from the cameras is captured using OpenCV, and DeepFace is used to analyze emotions from customers' facial expressions and tone of voice, allowing the system to recognize the customer's current emotional state.

[0745] Product recommendation

[0746] The server uses a generative AI model (OpenAI API) to recommend optimal products based on the recognized customer emotions. For example, if the customer is recognized as tired, the system generates a prompt such as, "Please recommend the best products for a tired customer. For example, aromatic candles with a relaxing effect or massage equipment," and receives the product recommendation through the OpenAI API.

[0747] Proposal for ways to deal with customer service staff

[0748] Furthermore, the server suggests appropriate responses to the customer service staff based on the customer's emotions. For example, if the customer is tired, the server generates a prompt such as, "Please suggest the best response to a tired customer. It is important to maintain a calm and relaxed tone of voice." The server then receives the appropriate response from the staff via the OpenAI API.

[0749] Meeting scheduling and time coordination

[0750] The server retrieves employee schedules from a database and uses a generative AI model to suggest optimal meeting times. The suggested times are displayed on the device, and the user can accept or modify them. Approved meeting times are automatically added to the employee's calendar.

[0751] Composing and replying to emails

[0752] The server uses a generative AI model to generate appropriate email content based on the user's request. The emotion engine creates emails with tone and content based on the user's emotions. The generated email is previewed on the device, and the user can approve or modify it before sending it.

[0753] Reminder notifications

[0754] The server monitors employees' calendars and task management databases, and generates reminders for deadlines and meeting times. These notifications are generated with optimal timing and content using an emotion engine and sent to users' devices.

[0755] Specific examples

[0756] For example, if a customer is tired, the server recognizes their emotion through the camera and microphone and uses the OpenAI API to send a prompt such as, "Please recommend the best products for a tired customer. For example, aroma candles or massage equipment that have a relaxing effect." This makes it possible to recommend products with a relaxing effect to the customer and to suggest a customer service method to staff based on the prompt, "Please suggest the best way to serve a tired customer. It is important to speak in a calm and relaxed tone."

[0757] This system can improve customer satisfaction and operational efficiency.

[0758] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0759] Step 1: Recognize customer emotions

[0760] The server captures video and audio data in real time from cameras and microphones installed in the store. The captured video data is processed using OpenCV, and DeepFace is used to analyze emotions from the customer's facial expressions and tone of voice. The input to this step is the video and audio data from the cameras and microphones, and the output is the customer's emotional information as an analysis result.

[0761] Step 2: Generate product recommendations

[0762] The server uses a generative AI model (OpenAI API) to create a prompt sentence that recommends the most suitable product based on the customer's emotional information recognized in step 1. For example, if the customer is recognized as "tired," it generates a prompt sentence such as "Please recommend the most suitable product for a tired customer" and inputs it into the generative AI model. The inputs in this step are the customer's emotional information and the prompt sentence, and the output is the generated product recommendation information.

[0763] Step 3: Propose ways to respond to staff

[0764] The server uses a generative AI model to create a prompt that generates a response method for the customer service staff based on the customer's emotional information recognized in step 1. For example, it generates a prompt such as "Please suggest the best response to a tired customer" and inputs it into the generative AI model. The input in this step is the customer's emotional information and the prompt, and the output is a generated response method suggestion.

[0765] Step 4: Schedule and schedule a meeting

[0766] The server retrieves employee schedules from the company database. Based on the retrieved schedule information, it uses a generative AI model to suggest optimal meeting times. The input for this step is employee schedule information, and the output is the suggested meeting time.

[0767] Step 5: Confirm meeting times

[0768] The terminal provides an interface that displays the proposed meeting time to the user and asks the user for approval or modification. The user reviews the proposed meeting time and modifies it if necessary. The input of this step is the proposed meeting time, and the output is the user's approval or modification of the meeting time.

[0769] Step 6: Add meeting times

[0770] The server automatically adds the meeting times approved or modified by the user to the employee's calendar. The input to this step is the approved or modified meeting times, and the output is the updated employee's calendar information.

[0771] Step 7: Compose and preview your email

[0772] The server generates appropriate email content based on the user's request using a generative AI model. The generated email is displayed as a preview on the device, and the user can review the content and make corrections as necessary. The inputs to this step are the user's request and the generated email content, and the output is the previewed email.

[0773] Step 8: Approve and send emails

[0774] The terminal provides an interface for the user to approve or modify the previewed email. The user approves the email, and the server sends the final email. The input of this step is the previewed email, and the output is the final email sent.

[0775] Step 9: Reminders

[0776] The server monitors employees' calendars and task management databases and generates reminders for important events and deadlines. It uses an emotion engine to create reminders with optimal timing and content and sends them to devices. The input to this step is the information in the calendar and task management databases, and the output is the generated reminder.

[0777] Through each step, it is expected that customer service and employee work efficiency will improve significantly, leading to increased productivity across the company.

[0778] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0779] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0780] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0781] [Second embodiment]

[0782] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0783] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0784] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0785] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0786] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0787] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0788] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0789] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0790] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0791] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0792] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0793] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0794] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks for employees, learning each employee's work process, internal tools, and databases to provide efficient support. Below, we will explain the system's program processing in natural language, with concrete examples.

[0795] Meeting scheduling and time coordination

[0796] server

[0797] Employee schedules are retrieved from the database and analyzed.

[0798] Calculate each employee's free time to suggest optimal meeting times.

[0799] Terminal

[0800] An interface is provided that displays the proposed meeting time to the user and asks for approval or modification.

[0801] User

[0802] Review the proposed time and approve or modify it. Once approved, the meeting will be automatically scheduled.

[0803] Specific examples

[0804] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[0805] Creating and replying to emails

[0806] server

[0807] Based on user requests, NLP (Natural Language Processing) models are used to create appropriate email content.

[0808] Refer to past email data to ensure consistency and appropriateness of content.

[0809] Terminal

[0810] A preview of the generated email is displayed to the user for approval or correction.

[0811] User

[0812] Review the preview and make any necessary changes. Once approved, you will receive an email.

[0813] Specific examples

[0814] When User B wants to send an email proposing a new product to a client, the server generates the appropriate content and displays a preview of the email on the device. Once User B checks the content and clicks the approve button, the email is automatically sent.

[0815] Reminder notifications

[0816] server

[0817] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[0818] Terminal

[0819] Display reminders to users and prompt them to take necessary action.

[0820] User

[0821] Check the notification and make the necessary preparations or take action.

[0822] Specific examples

[0823] When the deadline for User C's monthly report submission approaches, the server generates a reminder notification and displays it on the terminal. User C checks the notification and prepares to submit the report.

[0824] In-house Wiki creation support

[0825] server

[0826] Analyzes meeting recording data and text data to automatically generate minutes.

[0827] Generate documents suitable for internal Wiki formats.

[0828] Terminal

[0829] The generated minutes are previewed to the user and the user is asked to approve or correct the contents.

[0830] User

[0831] Check the minutes, make any necessary corrections, and click the Approve button.

[0832] Specific examples

[0833] After a meeting that User D participated in, the recording of the meeting is uploaded to the server. The server analyzes the recording, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company Wiki.

[0834] Document preparation support

[0835] server

[0836] The necessary data is collected from various internal databases and documents are generated based on specified templates.

[0837] Ensure documents are automatically formatted properly.

[0838] Terminal

[0839] A preview of the generated document is displayed to the user for approval or correction.

[0840] User

[0841] Review the document and make any necessary corrections. Once approved, the document will be saved and the submission process will be complete.

[0842] Specific examples

[0843] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information and automatically generates the documents. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[0844] This system frees employees from tedious tasks and provides an environment where they can focus on more creative work, which is expected to result in improved work efficiency and increased productivity across the company.

[0845] The processing flow will be explained below.

[0846] Meeting scheduling and time coordination

[0847] Step 1:

[0848] To request a conference, a user inputs information about the purpose of the conference and the necessary participants into the system.

[0849] Step 2:

[0850] The terminal transmits the input request to the server.

[0851] Step 3:

[0852] The server retrieves the schedules of all participants from the employee database.

[0853] Step 4:

[0854] The server analyzes the acquired schedule data and calculates the free time of each employee.

[0855] Step 5:

[0856] The server runs an algorithm to suggest the best meeting time and suggests a time.

[0857] Step 6:

[0858] The terminal displays the proposed meeting time to the user and provides an interface for approval or modification.

[0859] Step 7:

[0860] The user reviews the proposed time and takes action to approve or modify it.

[0861] Step 8:

[0862] The server adds the approved meeting time to the calendars of all participants, completing the meeting setup.

[0863] Creating and replying to emails

[0864] Step 1:

[0865] A user requests a new email or reply and enters a summary of the email and who it is for.

[0866] Step 2:

[0867] The device sends a request to the server.

[0868] Step 3:

[0869] The server uses the generative AI model to generate appropriate email content.

[0870] Step 4:

[0871] The server sends the generated email content to the terminal.

[0872] Step 5:

[0873] The terminal displays a preview of the generated email to the user.

[0874] Step 6:

[0875] The user checks the preview and makes any necessary corrections.

[0876] Step 7:

[0877] The user approves the email and sends it.

[0878] Step 8:

[0879] The server sends the approved email to the specified recipient.

[0880] Reminder notifications

[0881] Step 1:

[0882] The server regularly monitors employees' calendars and task management databases.

[0883] Step 2:

[0884] The server checks the deadline for submitting documents and the start time of the meeting.

[0885] Step 3:

[0886] Generate reminder notifications based on events and deadlines seen by the server.

[0887] Step 4:

[0888] The device displays a reminder notification to the user at the specified timing.

[0889] Step 5:

[0890] The user checks the notification and takes the necessary steps.

[0891] In-house Wiki creation support

[0892] Step 1:

[0893] A user uploads meeting recordings or text data to the system.

[0894] Step 2:

[0895] The device sends the recorded data or text data to the server.

[0896] Step 3:

[0897] The server analyzes the conference data and, if it is audio data, converts it into text using voice recognition technology.

[0898] Step 4:

[0899] The server automatically generates minutes based on the analyzed data.

[0900] Step 5:

[0901] The server converts the generated minutes into an internal Wiki format.

[0902] Step 6:

[0903] The terminal displays a preview of the generated minutes to the user.

[0904] Step 7:

[0905] The user checks the minutes and makes any necessary corrections.

[0906] Step 8:

[0907] The user approves the minutes and updates them on the company wiki.

[0908] Document preparation support

[0909] Step 1:

[0910] The user requests a document and enters the required information.

[0911] Step 2:

[0912] The device sends a request to the server.

[0913] Step 3:

[0914] The server collects the required data from each database.

[0915] Step 4:

[0916] The server automatically generates documents by inputting the collected data into a specified template.

[0917] Step 5:

[0918] The server sends the generated document to the terminal.

[0919] Step 6:

[0920] The terminal displays a preview of the generated document to the user.

[0921] Step 7:

[0922] The user reviews the document and makes any necessary corrections.

[0923] Step 8:

[0924] The user approves the document and the server stores the document, completing the submission process.

[0925] Through these steps, the system efficiently supports each task and provides an environment in which employees can focus on their creative work.

[0926] Example 1

[0927] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0928] In today's corporate environment, employees are often overwhelmed with a large amount of miscellaneous tasks, which results in insufficient time for creative work or important tasks that they should be focusing on. In addition, tasks such as coordinating meetings, writing appropriate emails, and managing reminder notifications are cumbersome and cause efficiency to decline. There is a need to improve this situation and increase employee productivity.

[0929] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0930] In this invention, the server includes a means for acquiring employee schedules, a means for calculating optimal meeting times using a generative AI model, and a means for proposing meeting times to a terminal. This enables efficient scheduling of meetings. It also includes a means for creating appropriate emails using a generative AI model, a means for previewing the created email on the terminal, and a means for approving or modifying the previewed email. This improves the efficiency of email creation tasks. It also includes a means for monitoring employee event schedules and a work management database, a means for generating reminders for important events and deadlines, and a means for displaying the reminders to the user. This makes it possible to prevent important tasks from being forgotten.

[0931] "Means for obtaining employee schedules" refers to hardware and software for obtaining employee schedule data, specifically including APIs and database access.

[0932] "Means for calculating optimal meeting times using a generative AI model" refers to algorithms and software that use a generative AI model to analyze the free time of all employees and calculate optimal meeting times.

[0933] "Means for proposing meeting times to a terminal" refers to the interface and software that transmits the calculated optimal meeting time from the server to the terminal and allows the user to confirm, approve, or modify it.

[0934] "Means for Approving or Modifying Proposed Meeting Times" refers to the interface and software that allows a user to review, modify, if necessary, and ultimately approve proposed meeting times.

[0935] "Means for adding approved meeting times to an employee's calendar" means software and APIs that automatically add user-approved meeting times to an employee's calendar.

[0936] "Means for creating appropriate emails using a generative AI model" refers to software that uses a generative AI model to automatically generate appropriate email content based on information provided by a user.

[0937] The "means for displaying a preview of the created e-mail on the terminal" refers to an interface and software for displaying a preview of the content of the created e-mail on the user's terminal.

[0938] "Means for approving or correcting previewed email" refers to the interface and software that allows a user to review the contents of a previewed email, make corrections as necessary, and ultimately approve it.

[0939] "Means for sending approved email" refers to software and APIs that automatically send user-approved email through an external mail server.

[0940] "Means for monitoring employee calendars and work management databases" refers to software and APIs that regularly monitor employee calendars and task management systems to detect important events and task deadlines.

[0941] "Means for generating reminders for important events and deadlines" refers to software that automatically generates appropriate reminders for important events and task deadlines detected through monitoring.

[0942] "Means for displaying a reminder to a user" refers to an interface and software that displays the generated reminder on the user's terminal and allows the user to take appropriate action.

[0943] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks on behalf of employees. In this system, the server mainly performs the processing, and the terminal provides the interface with the user, who confirms and approves specific tasks.

[0944] Meeting scheduling and time coordination

[0945] server

[0946] An API (for example, a calendar API) is used to obtain employee calendar data. The server analyzes the obtained data and runs an algorithm to calculate the free time of all employees. This algorithm analyzes the free time using a weighted average method or a heuristic algorithm to determine the optimal meeting time. The generated meeting time is sent from the server to the terminal.

[0947] Terminal

[0948] The system suggests optimal meeting times to users and displays them in an interface, allowing users to review the suggested meeting times and make adjustments as necessary.

[0949] User

[0950] Review the proposed time, make any necessary changes, and finally click the approve button, and the server will automatically add the meeting to your calendar.

[0951] Specific examples

[0952] When User A wants to schedule a new project meeting, the server uses the calendar API to retrieve the schedules of User A and related employees and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[0953] Prompt Sentence Examples

[0954] "Please suggest the best time for a new project meeting."

[0955] Creating and replying to emails

[0956] server

[0957] Based on the user's request, a natural language processing (NLP) model (e.g., a generative AI model) is used to generate appropriate email content. The generated email content is temporarily stored on the server and then sent to the device.

[0958] Terminal

[0959] A preview of the generated email is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[0960] User

[0961] Check the preview and make any necessary corrections. Finally, click the approve button and the server will automatically send the email.

[0962] Specific examples

[0963] If User B wants to send an email proposing a new product to a client, the server uses a generative AI model to generate appropriate email content, displays a preview of the email on the device, and once User B confirms the content and clicks the approve button, the email is automatically sent.

[0964] Prompt Sentence Examples

[0965] "Write an email to pitch a new product to a client."

[0966] Reminder notifications

[0967] server

[0968] Regularly monitor employee calendars and work management databases to detect important events and task deadlines, and use appropriate APIs and scripts to generate reminder notifications based on the detected data.

[0969] Terminal

[0970] The generated reminder notification is displayed to the user, and an interface is provided to prompt the user to take appropriate action.

[0971] User

[0972] Check the notification and take the necessary preparations or actions. Clicking on the notification will display more information and the next steps.

[0973] Specific examples

[0974] When the deadline for User C's monthly report submission approaches, the server uses the event schedule API to generate a reminder notification and displays it on the device. User C checks the notification and prepares to submit the report.

[0975] Prompt Sentence Examples

[0976] "Show me a reminder when my monthly report is due."

[0977] In-house Wiki creation support

[0978] server

[0979] Meeting recording data is acquired and converted into text using a speech recognition service (for example, a speech recognition API). Based on this text data, minutes are automatically generated using a generative AI model and formatted in a format suitable for the in-house wiki.

[0980] Terminal

[0981] A preview of the generated minutes is displayed to the user, and an interface is provided to request confirmation and correction of the contents.

[0982] User

[0983] Check the minutes, make any necessary corrections, and finally click the approve button.

[0984] Specific examples

[0985] The recording data of the meeting that User D participated in is uploaded to the server. The server analyzes the recording using a speech recognition API, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company wiki.

[0986] Prompt Sentence Examples

[0987] "Please create minutes based on the meeting recording data and post them on the company wiki."

[0988] Document preparation support

[0989] server

[0990] The necessary data is collected from the internal database, and documents are automatically generated based on document generation templates (e.g., document template APIs). The generated documents are formatted in the appropriate format.

[0991] Terminal

[0992] A preview of the generated document is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[0993] User

[0994] Review the documents, make any necessary corrections, and finally click the approve button.

[0995] Specific examples

[0996] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information from the company database and automatically generates the documents using the document template API. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[0997] Prompt Sentence Examples

[0998] Please fill out your year-end tax adjustment documents and enter the necessary information.

[0999] This system frees employees from tedious tasks and provides an environment where they can focus on more important tasks, which is expected to result in improved work efficiency and increased productivity across the company.

[1000] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1001] Meeting scheduling and time coordination

[1002] Step 1: Get employee calendar data

[1003] server

[1004] Input: User A sends a request to set up a conference.

[1005] Process: The server sends a request to the calendar API to retrieve employee schedule data.

[1006] Output: The retrieved schedule data.

[1007] Specific operation: The server sends a request to " / user / schedules / {user_id}" and receives schedule data in JSON format.

[1008] Step 2: Calculate the best meeting time

[1009] server

[1010] Input: Retrieved schedule data.

[1011] Processing: The server analyzes the data and calculates the free time of all employees.

[1012] Output: Optimal meeting time.

[1013] Specific behavior: Calls the "find_optimal_time(slots)" function to calculate the best non-overlapping time.

[1014] Step 3: Propose a meeting time

[1015] Terminal

[1016] Input: The best meeting time sent by the server.

[1017] Processing: The terminal displays the meeting time on the user interface.

[1018] Output: The suggested time displayed to the user.

[1019] Specific behavior: The terminal uses the "display_suggested_time(time)" method to display the meeting time in the interface.

[1020] Step 4: Review, revise, and approve the proposal

[1021] User

[1022] Input: Proposed meeting time.

[1023] Process: The user reviews the proposed time, makes any necessary corrections, and finally approves it.

[1024] Output: The revised or approved meeting time.

[1025] Specific action: The user clicks the "confirm_time()" or "modify_time()" button.

[1026] Step 5: Schedule a meeting

[1027] server

[1028] Input: The revised or approved meeting time.

[1029] Process: The server automatically sets up the meeting through the calendar API.

[1030] Output: The configured meeting.

[1031] Specific behavior: The server sends a POST request to " / calendar / events" to add the meeting to the calendar.

[1032] Creating and replying to emails

[1033] Step 1: Receive the user request

[1034] server

[1035] Input: User B sends an email composition request.

[1036] Processing: The server receives and analyzes the request.

[1037] Output: Email input prompt.

[1038] Specific behavior: A user fills out the " / create_email" form and submits the request.

[1039] Step 2: Generate email content

[1040] server

[1041] Input: Prompt to enter email.

[1042] Processing: The server generates the email content using the generative AI model.

[1043] Output: The generated email content.

[1044] Specific behavior: Calls the "generate_email_content(prompt)" function to generate text using a generative AI model.

[1045] Step 3: Preview your email

[1046] Terminal

[1047] Input: The generated email content.

[1048] Processing: The terminal displays the email contents on the user interface.

[1049] Output: The email preview shown to the user.

[1050] What happens: The device displays a preview of the email using "display_email_preview(email)".

[1051] Step 4: Check, edit, and approve the email content

[1052] User

[1053] Input: A preview of the generated email.

[1054] Processing: The user reviews the email, makes any necessary corrections, and approves it.

[1055] Output: The corrected or approved email content.

[1056] Specific behavior: The user clicks the "confirm_email()" or "modify_email()" button.

[1057] Step 5: Send an email

[1058] server

[1059] Input: The corrected or approved email content.

[1060] Process: The server sends the email through the mail server.

[1061] Output: The email sent.

[1062] Specific behavior: The server sends an email to the SMTP server using the "send_email(email)" function.

[1063] Reminder notifications

[1064] Step 1: Monitor your calendar and task management data

[1065] server

[1066] Input: Calendar and task management data.

[1067] Processing: The server periodically monitors the data to detect important events and task deadlines.

[1068] Output: Detected events and deadline data.

[1069] Specific behavior: The server periodically monitors " / calendar / events" and " / tasks" and collects data.

[1070] Step 2: Generate a reminder notification

[1071] server

[1072] Input: Detected event and deadline data.

[1073] Processing: The server generates a reminder notification.

[1074] Output: The generated reminder notification.

[1075] Specific behavior: Generates a reminder notification using the "generate_reminder(event)" function.

[1076] Step 3: Display the notification to the user

[1077] Terminal

[1078] Input: The generated reminder notification.

[1079] Processing: The device displays the reminder notification on the user interface.

[1080] Output: The reminder notification shown to the user.

[1081] Specific behavior: The device displays the notification using the "display_reminder(reminder)" method.

[1082] Step 4: Review and respond to notifications

[1083] User

[1084] Input: The reminder notification shown to the user.

[1085] Action: The user checks the notification and takes the necessary action.

[1086] Output: The action that was performed.

[1087] Specific action: The user clicks on the reminder notification and takes the necessary steps.

[1088] In-house Wiki creation support

[1089] Step 1: Get the meeting recording data

[1090] server

[1091] Input: Meeting recording data.

[1092] Processing: The server receives and stores the recording.

[1093] Output: Saved meeting recording data.

[1094] Specific behavior: The user uploads the recording to " / upload_meeting_audio".

[1095] Step 2: Analyze the recording data and generate transcripts

[1096] server

[1097] Input: Saved meeting recording data.

[1098] Processing: The server uses a speech recognition API to convert the speech to text, and then uses a generative AI model to generate the transcript.

[1099] Output: The generated transcript.

[1100] Specific behavior: The server converts the audio to text using the "transcribe_audio(audio_file)" function, and generates the transcript using the "generate_minutes(text)" function.

[1101] Step 3: Preview the transcript

[1102] Terminal

[1103] Input: The generated minutes.

[1104] Processing: The terminal displays the minutes on the user interface.

[1105] Output: A preview of the transcript as displayed to the user.

[1106] Specific behavior: The terminal displays the transcript using the "display_transcription(transcript)" method.

[1107] Step 4: Review, revise, and approve the minutes

[1108] User

[1109] Input: A preview of the generated transcript.

[1110] Process: The user reviews the minutes, makes any necessary corrections, and approves them.

[1111] Output: Amended or approved minutes.

[1112] Specific behavior: The user clicks the "confirm_transcript()" or "edit_transcript()" button.

[1113] Step 5: Post the minutes on the company wiki

[1114] server

[1115] Enter: Amended or approved minutes.

[1116] Process: The server posts the minutes to the company wiki.

[1117] Output: Published minutes.

[1118] Specific operation: The server sends a POST request to " / wiki / add_entry" to add the minutes to the internal Wiki.

[1119] Document preparation support

[1120] Step 1: Collect the necessary data

[1121] server

[1122] Input: Data collection request.

[1123] Processing: The server collects the necessary data from the company database.

[1124] Output: Collected data.

[1125] Specific operation: The server sends a request to " / database / get_data" to collect the data.

[1126] Step 2: Auto-generate documents

[1127] server

[1128] Input: Collected data.

[1129] Processing: The server automatically generates the document using the document template API.

[1130] Output: The generated document.

[1131] Specific operation: Use the "generate_document(template, data)" function to call the Document Template API and generate a document.

[1132] Step 3: Preview your document

[1133] Terminal

[1134] Input: The generated document.

[1135] Processing: The device displays a preview of the document in its user interface.

[1136] Output: A preview of the document as it appears to the user.

[1137] Specific behavior: The device displays a preview of the document using the "display_document_preview(document)" method.

[1138] Step 4: Review, correct, and approve the document

[1139] User

[1140] Input: A preview of the generated document.

[1141] Processing: The user reviews the document, makes any necessary corrections, and approves it.

[1142] Output: The corrected or approved document.

[1143] Specific behavior: The user clicks the "confirm_document()" or "edit_document()" button.

[1144] Step 5: Save and submit your documents

[1145] server

[1146] Enter: Amended or approved document.

[1147] Processing: The server saves the document in the specified location and processes it for submission.

[1148] Output: Submitted documents.

[1149] Specific operation: The server sends a POST request to " / documents / submit" to submit the document.

[1150] (Application example 1)

[1151] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1152] In factories, manual robot task management and anomaly detection requires a great deal of effort and time. It is also difficult to manually create an optimal schedule when multiple robots are operating simultaneously, which can lead to reduced efficiency. Furthermore, delayed response to anomalies can lead to a decline in quality and stalled production lines. To solve these problems, a system is needed that can automatically acquire robot status, generate optimal task schedules, detect anomalies, and propose countermeasures.

[1153] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1154] In this invention, the server includes means for acquiring the robot's status, means for proposing an optimal task schedule using a generative AI model, means for approving or modifying the proposed task schedule, means for controlling the robot's operation based on the approved task schedule, means for collecting and analyzing feedback data, and means for proposing the next action based on the analysis results. This improves the robot's operation efficiency and makes it possible to prevent production line stoppages by automating anomaly detection and countermeasures.

[1155] A "robot" is a mechanical device designed to automatically perform specific tasks in a factory or on a production line.

[1156] "Status" is data that indicates the current condition or status of the robot's operating state or function.

[1157] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal solutions and predictions based on large amounts of data.

[1158] A "task schedule" is a plan that efficiently arranges a series of tasks or duties to be performed by a robot and allocates them by time.

[1159] "Suggestion" refers to the generative AI model calculating optimal schedules and actions and providing them to the user.

[1160] "Approval" means that the user reviews and accepts the proposed schedule and actions.

[1161] "Control" refers to the proper operation of a robot's movements and functions based on programs and systems.

[1162] "Feedback data" refers to various sensor data and operation logs collected while the robot is performing its tasks.

[1163] "Analysis" is a method of analyzing collected feedback data to detect anomalies and evaluate performance.

[1164] "Action" refers to the specific actions or countermeasures to be taken next based on the analysis results.

[1165] This invention is a system that automates robot task management and anomaly detection in factories. The system acquires robot status data, generates an optimal task schedule using a generative AI model, analyzes feedback data to detect anomalies, and proposes next actions.

[1166] Hardware and software used

[1167] Hardware: Factory robots, various sensors, and servers.

[1168] Software: Python, NLP (Natural Language Processing) models, database management systems (e.g., PostgreSQL).

[1169] 1. Task schedule generation

[1170] The server first obtains the status data of the factory robots (e.g., current working status, operating hours, maintenance history), then uses a generative AI model to generate an optimal task schedule based on the obtained status data, and the generated task schedule is approved or modified to ensure the robots' efficient operation.

[1171] As a concrete example, if a robot is in charge of assembling part A, the generative AI model will generate the following schedule:

[1172] Assembly start time for part A: 09:00

[1173] Assembly of part A completed at 10:00

[1174] Next task start time: 10:05

[1175] 2. Feedback Analysis

[1176] The server collects feedback data from the robot in operation and analyzes it using an NLP model. This feedback data includes various sensor data such as temperature, vibration, and operation logs. Based on the analysis results, the next action is suggested. For example, if an abnormal temperature is detected, the next action suggested would be "Check for errors in part X and correct them."

[1177] 3. Implementing the proposed action

[1178] The actions proposed by the server can be approved or modified by the user. Once approved, the server again instructs the robot to execute the action. This automates the robot's response to abnormalities and maintenance, resulting in efficient operation.

[1179] Prompt Sentence Examples

[1180] Imagine a system that generates optimal task schedules for factory robots and prescribes next actions based on feedback data. Specifically, the system optimizes the robot schedule using NLP models, analyzes sensor feedback data (e.g., temperature, vibration), and automatically suggests countermeasures when anomalies are detected.

[1181] The system of the present invention improves the operational efficiency of robots and prevents production line stoppages by automating abnormality detection and countermeasures, thereby significantly improving the productivity of the entire factory and significantly reducing the need for manual management.

[1182] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1183] Step 1:

[1184] The server collects status data from factory robots, including their current working status, operating hours, and maintenance history. This data is used to aggregate information that forms the basis for the next processing step. The input is the robot's sensor data and operation log, and the output is the robot's status data.

[1185] Step 2:

[1186] The server uses a generative AI model to generate an optimal task schedule based on the acquired robot status data. The input is the status data, and the output is a proposed task schedule. The generative AI model analyzes the data and calculates the efficient work order for the robots.

[1187] Step 3:

[1188] The terminal displays the proposed task schedule to the user, who can then approve or modify it. The input is the task schedule generated by the server, and the output is the user's approved or modified schedule.

[1189] Step 4:

[1190] The user reviews the proposed task schedule and approves or modifies it. The user's actions are used to control the next step. The input is the proposed task schedule, and the output is the user's approval or modification.

[1191] Step 5:

[1192] The server controls the robot's actions based on the approved task schedule. The input is the schedule approved by the user, and the output is the control signal to the robot, which then performs the task as specified.

[1193] Step 6:

[1194] The server collects feedback data while the robot is operating, including temperature, vibration, and operation logs. The input is the robot's sensor data, and the output is the feedback data.

[1195] Step 7:

[1196] The server analyzes the collected feedback data and uses an NLP model to detect anomalies and maintenance needs. The input is the feedback data and the output is the analysis result.

[1197] Step 8:

[1198] The server proposes the next action based on the analysis results. For example, if an anomaly is detected, it suggests that a specific part needs to be repaired or maintained. The input is the analysis results, and the output is the proposed action.

[1199] Step 9:

[1200] The terminal displays the proposed action to the user, who can review it and accept or modify it as needed. The input is the server's proposed action, and the output is the user's accepted or modified action.

[1201] Step 10:

[1202] The user reviews the proposed action and approves or modifies it. The input is the proposed action and the output is the user's approval or modification.

[1203] Step 11:

[1204] The server executes the approved actions and sends control signals to the robot, which then performs the appropriate repairs or maintenance. The input is the action approved by the user, and the output is the command for the robot to execute.

[1205] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1206] This invention is a system that combines a generative AI model with an emotion engine to streamline employees' daily work and provide optimal support based on the user's emotions. The system learns employees' work processes, internal tools, and databases, recognizes the user's emotions using the emotion engine, and responds appropriately.

[1207] Meeting scheduling and time coordination

[1208] server

[1209] Employee schedules are retrieved from the database and analyzed.

[1210] It uses an emotion engine to recognize the user's current emotions and runs an algorithm that suggests optimal meeting times that take into consideration the emotions.

[1211] Terminal

[1212] The system provides an interface that displays proposed meeting times to users and asks for approval or modification based on sentiment.

[1213] User

[1214] Review the proposed time and accept or modify it. Emotion-based suggestions make meeting scheduling smoother.

[1215] Specific examples

[1216] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members. The emotion engine suggests the optimal time to reduce User A's stress level. Once User A confirms and approves the suggested time on their device, the meeting is added to their calendar.

[1217] Creating and replying to emails

[1218] server

[1219] Based on user requests, NLP models are used to generate appropriate email content.

[1220] The emotional engine recognizes the user's emotions and creates emails with tone and content based on those emotions.

[1221] Terminal

[1222] Show users a preview of the generated email and ask for sentiment-based approval or revision.

[1223] User

[1224] Check the preview and make any necessary corrections. By generating emails that are sensitive to customer sentiment, communication becomes smoother.

[1225] Specific examples

[1226] If User B wants to send an email proposing a new product to a client, the server will use the emotion engine to recognize that User B is nervous. Therefore, it will generate an email with a relaxed tone and display a preview on the device. If User B checks the content and approves it, the email will be sent.

[1227] Reminder notifications

[1228] server

[1229] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[1230] It uses an emotion engine to recognize the user's emotions and generate reminder notifications with optimal timing and content.

[1231] Terminal

[1232] Display emotional reminders to users and encourage them to take appropriate action.

[1233] User

[1234] Check the notification and make the necessary preparations or take action.

[1235] Specific examples

[1236] When the deadline for User C's monthly report is approaching, the server uses the emotion engine to recognize User C's current stress level. Therefore, a reminder notification is generated and displayed on the device at a time that minimizes stress. User C checks the notification and prepares the report.

[1237] In-house Wiki creation support

[1238] server

[1239] Analyzes meeting recording data and text data to automatically generate minutes.

[1240] Use an emotion engine to adjust the content and expression of meeting minutes based on user emotions.

[1241] Terminal

[1242] A preview of the generated meeting minutes is displayed to the user for approval or revision based on sentiment.

[1243] User

[1244] Check the minutes, make any necessary corrections, and click the approve button.

[1245] Specific examples

[1246] After a meeting that User D attends, the recording of the meeting is uploaded to the server. The emotion engine recognizes User D's fatigue and automatically generates concise, to-the-point minutes. Once User D checks and approves the content, the minutes are posted on the company's internal wiki.

[1247] Document preparation support

[1248] server

[1249] The necessary data is collected from various internal databases and documents are generated based on specified templates.

[1250] Use an emotion engine to recognize user emotions and adjust tone and format accordingly.

[1251] Terminal

[1252] A preview of the generated document is displayed to the user for approval or modification based on their sentiment.

[1253] User

[1254] Review the document and make any necessary corrections. Once approved, the document is saved and submitted.

[1255] Specific examples

[1256] When User E needs to create documents for year-end tax adjustment, the server uses an emotion engine to recognize User E's current emotional state. Therefore, the server automatically generates documents in a format that is easy for User E to use and displays a preview. Once User E confirms and approves the contents, the documents are saved and submitted.

[1257] This system provides support that takes users' emotions into consideration, improving work efficiency and the quality of communication. As a result, it is expected that an environment will be created where employees can focus on their core creative work, thereby increasing productivity across the company.

[1258] The processing flow will be explained below.

[1259] Meeting scheduling and time coordination

[1260] Step 1:

[1261] To request a conference, a user inputs information about the purpose of the conference and the necessary participants into the system.

[1262] Step 2:

[1263] The terminal transmits the user's input data to the server.

[1264] Step 3:

[1265] The server retrieves the schedules of all participants from the employee database.

[1266] Step 4:

[1267] The server analyzes the acquired schedule data and calculates the free time of each employee.

[1268] Step 5:

[1269] The server uses an emotion engine to recognize the user's current emotion.

[1270] Step 6:

[1271] The server runs an algorithm that suggests optimal meeting times based on sentiment.

[1272] Step 7:

[1273] The terminal displays the proposed meeting time to the user and provides an interface for sentiment-based approval or modification.

[1274] Step 8:

[1275] The user reviews the proposed time and takes action to approve or modify it.

[1276] Step 9:

[1277] The server adds the approved meeting time to the calendars of all participants, completing the meeting setup.

[1278] Creating and replying to emails

[1279] Step 1:

[1280] A user requests a new email or reply and enters a summary of the email and who it is for.

[1281] Step 2:

[1282] The device sends a request to the server.

[1283] Step 3:

[1284] The server uses the generative AI model to generate appropriate email content.

[1285] Step 4:

[1286] The server uses an emotion engine to recognize the user's emotions and generates emails with a tone and content based on those emotions.

[1287] Step 5:

[1288] The server sends the generated email content to the terminal.

[1289] Step 6:

[1290] The terminal displays a preview of the generated email to the user.

[1291] Step 7:

[1292] The user checks the preview and makes any necessary corrections.

[1293] Step 8:

[1294] The user approves the email and sends it.

[1295] Step 9:

[1296] The server sends the approved email to the specified recipient.

[1297] Reminder notifications

[1298] Step 1:

[1299] The server regularly monitors employees' calendars and task management databases.

[1300] Step 2:

[1301] The server checks the deadline for submitting documents and the start time of the meeting.

[1302] Step 3:

[1303] The server uses an emotion engine to recognize the user's emotions and generates a reminder notification with optimal timing and content.

[1304] Step 4:

[1305] The device displays emotion-based reminder notifications to the user.

[1306] Step 5:

[1307] The user checks the notification and takes the necessary steps.

[1308] In-house Wiki creation support

[1309] Step 1:

[1310] A user uploads meeting recordings or text data to the system.

[1311] Step 2:

[1312] The device sends the recorded data or text data to the server.

[1313] Step 3:

[1314] The server analyzes the conference data and, if it is audio data, converts it into text using voice recognition technology.

[1315] Step 4:

[1316] The server automatically generates minutes based on the analyzed data.

[1317] Step 5:

[1318] The server recognizes the user's emotions using an emotion engine and adjusts the content and expression of the minutes.

[1319] Step 6:

[1320] The server converts the generated minutes into an internal Wiki format.

[1321] Step 7:

[1322] The terminal displays a preview of the generated minutes to the user.

[1323] Step 8:

[1324] The user checks the minutes and makes any necessary corrections.

[1325] Step 9:

[1326] The user approves the minutes, and the server updates the minutes to the company Wiki.

[1327] Document preparation support

[1328] Step 1:

[1329] The user requests a document and enters the required information.

[1330] Step 2:

[1331] The device sends a request to the server.

[1332] Step 3:

[1333] The server collects the required data from each database.

[1334] Step 4:

[1335] The server automatically generates documents by inputting the collected data into a specified template.

[1336] Step 5:

[1337] The server uses an emotion engine to recognize the user's emotions and adjust the tone and format accordingly.

[1338] Step 6:

[1339] The server sends the generated document to the terminal.

[1340] Step 7:

[1341] The terminal displays a preview of the generated document to the user.

[1342] Step 8:

[1343] The user reviews the document and makes any necessary corrections.

[1344] Step 9:

[1345] The user approves the document and the server stores the document, completing the submission process.

[1346] Through these steps, the system will be able to efficiently support each task and respond in a way that takes into consideration the user's feelings. This will create an environment where employees can focus on their creative work, which is what we believe is essential, and is expected to improve productivity across the company.

[1347] Example 2

[1348] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1349] In conventional systems, managing employee schedules and communications is often complex and time-consuming, resulting in reduced work efficiency. Furthermore, because processes proceed mechanically without considering the user's emotions, stress and discord in communication can occur. Furthermore, reminder notifications and email creation do not take the user's emotional state into consideration, which can lead to problems such as insufficient stress reduction and work efficiency.

[1350] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1351] In this invention, the server includes a means for acquiring employee schedules, a means for recognizing user emotions using an emotion engine, and a means for proposing optimal meeting times using a generative AI model, thereby enabling the proposal of optimal meeting times that take user emotions into consideration.

[1352] The server includes means for recognizing a user's emotion and creating an email with an appropriate tone and expression based on the emotion, means for displaying a preview of the created email to the user, means for approving or correcting the previewed email based on the emotion, and means for sending the approved email. This allows an email with an appropriate tone according to the user's emotion to be created, facilitating smooth communication.

[1353] The server includes a means for monitoring employees' calendars and task management databases, and a means for generating reminder notifications of important events and deadlines at optimal timing based on the user's emotions, and a means for sending reminder notifications based on the emotions to the user, thereby enabling reminder notifications at optimal timing in line with the user's emotional state, thereby reducing stress and improving work efficiency.

[1354] An "employee schedule" is a record of the timetables of work hours, meetings, tasks, etc. scheduled by employees belonging to an organization.

[1355] "Emotion engine" is a general term for software or hardware that analyzes and recognizes a user's emotional state (e.g., stress level, fatigue, joy, etc.).

[1356] A "generative AI model" is an artificial intelligence model that generates output for natural language processing or specific tasks based on given input data, and primarily uses machine learning algorithms.

[1357] "Meeting time suggestion" refers to the act of the generative AI model presenting the optimal meeting time for the user based on the analysis results.

[1358] "Preview display" refers to a function that allows the user to view the generated content (for example, email or minutes) in advance and check or modify it.

[1359] A "reminder notification" is a notification sent to a user before an important event or deadline, to help the user remember.

[1360] The "optimal timing" refers to the time when the notification or action will be most effectively received, taking into account the user's current emotional state and work situation.

[1361] A "calendar" is a tool for visually managing employees' schedules and tasks, and is generally composed of dates and times.

[1362] A "task management database" is a database system for digitally managing and recording the tasks and business processes that employees must complete.

[1363] "Tone" refers to the emotional nuances and style of writing or communication, including friendly and formal tones.

[1364] This invention is a system that combines a generative AI model with an emotion engine to streamline employees' daily work and provide optimal support based on the user's emotions. This system operates using three elements: a server, a terminal, and the user.

[1365] Meeting scheduling and time coordination

[1366] server

[1367] The server retrieves employee schedules from a database and performs analysis. Specifically, the server executes a database query to retrieve each employee's schedule data. The server then uses an emotion engine to recognize user emotions and runs an algorithm to suggest optimal meeting times based on the analysis results.

[1368] Specific examples

[1369] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members from the schedule database. Then, the emotion engine suggests the best time to reduce User A's stress level.

[1370] Prompt Sentence Examples

[1371] "Check User A's schedule and suggest the best meeting time based on the sentiment engine data."

[1372] Creating and replying to emails

[1373] server

[1374] The server uses NLP models to generate appropriate email content based on user requests, and an emotion engine to recognize user emotions and create emails with appropriate tone and content.

[1375] Specific examples

[1376] If User B wants to send an email proposing a new product to a client, the server uses the emotion engine to recognize that User B is nervous, generates an email with a relaxed tone, and displays a preview on the device.

[1377] Prompt Sentence Examples

[1378] "User B is nervous, so please generate a new product proposal email with a relaxed tone."

[1379] Reminder notifications

[1380] server

[1381] The server monitors employees' calendars and task management databases to generate reminders for important events and deadlines. It uses an emotion engine to recognize users' emotions and generates reminders with optimal timing and content.

[1382] Specific examples

[1383] When User C's monthly report submission deadline approaches, the server uses the emotion engine to recognize User C's current stress level. A reminder notification is generated and displayed on the device at a time that minimizes stress.

[1384] Prompt Sentence Examples

[1385] "Please remind User C to submit the monthly report at a time when he is least stressed."

[1386] In-house Wiki creation support

[1387] server

[1388] The server analyzes meeting recording data and text data to automatically generate minutes, and uses an emotion engine to adjust the content and expression of the minutes based on the user's emotions.

[1389] Specific examples

[1390] After a meeting that User D participated in, the recording of the meeting is uploaded to the server. The emotion engine recognizes User D's sense of fatigue and automatically generates concise, to-the-point minutes.

[1391] Prompt Sentence Examples

[1392] "Please consider User D's fatigue and generate concise minutes."

[1393] Document preparation support

[1394] server

[1395] The server collects the necessary data from various company databases, generates documents based on specified templates, and uses an emotion engine to recognize the user's emotions and adjust the tone and format accordingly.

[1396] Specific examples

[1397] When User E needs to create documents for year-end tax adjustment, the server uses an emotion engine to recognize User E's current emotional state, automatically generates documents in a format that is easy to use, and displays a preview.

[1398] Prompt Sentence Examples

[1399] "Please generate the year-end tax adjustment documents in a format that is less burdensome for User E."

[1400] This system provides support that takes users' emotions into consideration, improving work efficiency and the quality of communication. As a result, it is expected that an environment will be created where employees can focus on their core creative work, thereby increasing productivity across the company.

[1401] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1402] Meeting scheduling and time coordination

[1403] Step 1: Get the schedule

[1404] The server connects to the database to retrieve employee schedule data. Specifically, the server runs a database query to pull schedule data from the "employee_schedules" table.

[1405] Input: Employee ID

[1406] Output: Employee schedule data

[1407] What it does: Retrieves schedule information from a database using an SQL query.

[1408] Step 2: Emotion recognition and analysis

[1409] The server uses the emotion engine to recognize the user's current emotional state and calls the emotion engine API to obtain the emotion data.

[1410] Input: User ID

[1411] Output: User emotion data

[1412] Specific operation: Calls the emotion engine API and collects emotion data.

[1413] Step 3: Suggest the best meeting time

[1414] The server runs an algorithm that suggests optimal meeting times based on the acquired schedule and emotion data, using a generative AI model to calculate the suggested times.

[1415] Input: Employee schedule data, user emotion data

[1416] Output: Best meeting time

[1417] What it does: Runs an algorithm using a generative AI model to calculate optimal meeting times.

[1418] Step 4: View and review the proposal

[1419] The terminal receives the proposed meeting time from the server and displays it to the user, who can then review and approve or modify the proposed time.

[1420] Input: Best Meeting Time

[1421] Output: Suggested time displayed to the user

[1422] Specific behavior: Show the suggested time to the user via notification or popup.

[1423] Step 5: Set up a meeting

[1424] The user reviews the proposed meeting time and can approve or modify it, and once approved, the update is sent to the server and the meeting is set in their calendar.

[1425] Input: User approval or correction information

[1426] Output: Meeting times set in the calendar

[1427] Specific operation: When the user clicks the approve button, the server calls the calendar API to add the meeting.

[1428] Creating and replying to emails

[1429] Step 1: Receiving a user request

[1430] The server receives a request to compose an email from the user: The user clicks the "Compose new email" button in the email sending interface.

[1431] Input: Email creation request

[1432] Output: Start of email creation process

[1433] Specific behavior: Receives an HTTP request and starts the email creation process.

[1434] Step 2: Emotion recognition and analysis

[1435] The server uses an emotion engine to recognize the user's emotions, and acquires and analyzes the emotion data.

[1436] Input: User ID

[1437] Output: User emotion data

[1438] Specific operation: Call the emotion engine API and obtain the user's emotional state.

[1439] Step 3: Generate email content

[1440] The server uses NLP models to generate appropriate email content based on the user's emotional data.

[1441] Input: User emotion data, basic email information

[1442] Output: Generated email content

[1443] Specific operation: A prompt sentence is input into the generative AI model to generate email content.

[1444] Step 4: Preview and check

[1445] The terminal displays a preview of the generated email to the user and provides an interface for the user to review and modify the content.

[1446] Input: Generated email content

[1447] Output: Email preview shown to the user

[1448] Specific behavior: Displays the email preview screen and allows the user to edit it.

[1449] Step 5: Sending an email

[1450] The user checks the contents of the email and clicks the send button. Once approval is complete, the sending information is sent to the server and the email is actually sent.

[1451] Input: User authorization information

[1452] Output: Email sent

[1453] Specific behavior: When you click the send button, the server sends the email using the SMTP server.

[1454] Reminder notifications

[1455] Step 1: Monitor your calendar and task data

[1456] The server periodically monitors employees' calendars and task management databases to detect important events and deadlines.

[1457] Input: Calendar and task data

[1458] Output: Detected important events and deadlines

[1459] Specific behavior: Runs a database query as a background job to obtain task information.

[1460] Step 2: Emotion recognition and analysis

[1461] The server uses an emotion engine to analyze the user's emotions, such as assessing stress levels due to an approaching deadline.

[1462] Input: User ID

[1463] Output: Emotion data

[1464] Specific operation: Utilize the emotion engine API to obtain emotion data.

[1465] Step 3: Generate a reminder notification

[1466] The server generates the optimal timing and content for the reminder notification based on the acquired data and emotional data.

[1467] Input: Calendar, task data, emotion data

[1468] Output: Reminder notification

[1469] Specific behavior: Generates notification content and manages notification schedules.

[1470] Step 4: View and review notifications

[1471] The device receives the reminder notification from the server and displays it to the user, allowing the user to check the notification and take appropriate action.

[1472] Input: Reminder notification

[1473] Output: The notification that is displayed to the user

[1474] Specific behavior: A popup notification will be displayed and an action will be triggered when the user presses the confirmation button.

[1475] (Application example 2)

[1476] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1477] Conventional employee work support systems are limited to functions such as setting up meetings, creating emails, and sending reminders, making it difficult to provide services such as emotion recognition in customer service or optimal product recommendations. Furthermore, there was a lack of a way to provide customer service staff with appropriate responses based on customer emotions in real time. This resulted in issues such as a decline in customer satisfaction and insufficient improvement in work efficiency.

[1478] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring employee schedules, means for proposing optimal meeting times using a generative AI model, means for approving or modifying the proposed meeting times, means for recognizing customer emotions, means for recommending optimal products based on the customer emotions, and means for proposing ways for customer service staff to respond based on the customer emotions. This makes it possible to significantly improve not only the daily work of employees but also the quality of customer service in physical stores, thereby increasing customer satisfaction.

[1479] "Means for obtaining employee schedules" refers to a function that obtains schedule information such as employee calendars and timetables from a database and analyzes it.

[1480] "Means for suggesting optimal meeting times using a generative AI model" is a function that uses a generative AI model to automatically suggest optimal meeting times for everyone based on the acquired employee schedule information.

[1481] A "means for approving or amending a proposed meeting time" is an interface or operating means for an employee to approve or amend a proposed meeting time.

[1482] "Means for adding approved meeting times to employees' calendars" refers to a function that automatically adds the final approved or amended meeting time to the employees' individual calendars.

[1483] "Means for recognizing customer emotions" refers to algorithms and sensor devices that analyze and recognize emotions in real time from customers' facial expressions, tone of voice, etc.

[1484] The "means for recommending optimal products based on customer emotions" is a function that automatically recommends products that are likely to be desired by a customer based on the recognized customer emotions.

[1485] "A means to suggest how to respond to customer service staff based on customer emotions" is a function that analyzes the emotional state of the customer and suggests in real time how the customer service staff should respond.

[1486] "Means for creating emails" refers to a function that uses a generative AI model to automatically create appropriate email content based on user requests.

[1487] The "means for displaying a preview of an email" is an interface for displaying a preview of the generated email to the user, allowing the user to check and modify the contents.

[1488] The "means for approving or correcting the previewed email" is an operation means for the user to approve or correct the previewed email.

[1489] "Means for monitoring employee calendars and task management databases" refers to a function that constantly monitors employee calendars and task management systems to track important events and deadlines.

[1490] The "means for generating reminders for document submission deadlines and meeting times" is a function that extracts important deadlines and meeting times from monitored data and generates reminders based on them.

[1491] The "means for sending a reminder notification to a user" is a system for sending the generated reminder notification to the user's terminal and prompting the user to take the necessary action.

[1492] This invention is a method for optimizing customer service in brick-and-mortar stores by using a system that combines a generative AI model and an emotion engine to streamline the daily work of employees. The hardware environment includes cameras, microphones, servers, and devices (smartphones and tablets) used by customer service staff. The software used includes OpenCV, DeepFace, and OpenAI API.

[1493] Customer Emotion Recognition

[1494] The server collects data in real time from cameras and microphones installed in the store. The video data from the cameras is captured using OpenCV, and DeepFace is used to analyze emotions from customers' facial expressions and tone of voice, allowing the system to recognize the customer's current emotional state.

[1495] Product recommendation

[1496] The server uses a generative AI model (OpenAI API) to recommend optimal products based on the recognized customer emotions. For example, if the customer is recognized as tired, the system generates a prompt such as, "Please recommend the best products for a tired customer. For example, aromatic candles with a relaxing effect or massage equipment," and receives the product recommendation through the OpenAI API.

[1497] Proposal for ways to deal with customer service staff

[1498] Furthermore, the server suggests appropriate responses to the customer service staff based on the customer's emotions. For example, if the customer is tired, the server generates a prompt such as, "Please suggest the best response to a tired customer. It is important to maintain a calm and relaxed tone of voice." The server then receives the appropriate response from the staff via the OpenAI API.

[1499] Meeting scheduling and time coordination

[1500] The server retrieves employee schedules from a database and uses a generative AI model to suggest optimal meeting times. The suggested times are displayed on the device, and the user can accept or modify them. Approved meeting times are automatically added to the employee's calendar.

[1501] Composing and replying to emails

[1502] The server uses a generative AI model to generate appropriate email content based on the user's request. The emotion engine creates emails with tone and content based on the user's emotions. The generated email is previewed on the device, and the user can approve or modify it before sending it.

[1503] Reminder notifications

[1504] The server monitors employees' calendars and task management databases, and generates reminders for deadlines and meeting times. These notifications are generated with optimal timing and content using an emotion engine and sent to users' devices.

[1505] Specific examples

[1506] For example, if a customer is tired, the server recognizes their emotion through the camera and microphone and uses the OpenAI API to send a prompt such as, "Please recommend the best products for a tired customer. For example, aroma candles or massage equipment that have a relaxing effect." This makes it possible to recommend products with a relaxing effect to the customer and to suggest a customer service method to staff based on the prompt, "Please suggest the best way to serve a tired customer. It is important to speak in a calm and relaxed tone."

[1507] This system can improve customer satisfaction and operational efficiency.

[1508] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1509] Step 1: Recognize customer emotions

[1510] The server captures video and audio data in real time from cameras and microphones installed in the store. The captured video data is processed using OpenCV, and DeepFace is used to analyze emotions from the customer's facial expressions and tone of voice. The input to this step is the video and audio data from the cameras and microphones, and the output is the customer's emotional information as an analysis result.

[1511] Step 2: Generate product recommendations

[1512] The server uses a generative AI model (OpenAI API) to create a prompt sentence that recommends the most suitable product based on the customer's emotional information recognized in step 1. For example, if the customer is recognized as "tired," it generates a prompt sentence such as "Please recommend the most suitable product for a tired customer" and inputs it into the generative AI model. The inputs in this step are the customer's emotional information and the prompt sentence, and the output is the generated product recommendation information.

[1513] Step 3: Propose ways to respond to staff

[1514] The server uses a generative AI model to create a prompt that generates a response method for the customer service staff based on the customer's emotional information recognized in step 1. For example, it generates a prompt such as "Please suggest the best response to a tired customer" and inputs it into the generative AI model. The input in this step is the customer's emotional information and the prompt, and the output is a generated response method suggestion.

[1515] Step 4: Schedule and schedule a meeting

[1516] The server retrieves employee schedules from the company database. Based on the retrieved schedule information, it uses a generative AI model to suggest optimal meeting times. The input for this step is employee schedule information, and the output is the suggested meeting time.

[1517] Step 5: Confirm meeting times

[1518] The terminal provides an interface that displays the proposed meeting time to the user and asks the user for approval or modification. The user reviews the proposed meeting time and modifies it if necessary. The input of this step is the proposed meeting time, and the output is the user's approval or modification of the meeting time.

[1519] Step 6: Add meeting times

[1520] The server automatically adds the meeting times approved or modified by the user to the employee's calendar. The input to this step is the approved or modified meeting times, and the output is the updated employee's calendar information.

[1521] Step 7: Compose and preview your email

[1522] The server generates appropriate email content based on the user's request using a generative AI model. The generated email is displayed as a preview on the device, and the user can review the content and make corrections as necessary. The inputs to this step are the user's request and the generated email content, and the output is the previewed email.

[1523] Step 8: Approve and send emails

[1524] The terminal provides an interface for the user to approve or modify the previewed email. The user approves the email, and the server sends the final email. The input of this step is the previewed email, and the output is the final email sent.

[1525] Step 9: Reminders

[1526] The server monitors employees' calendars and task management databases and generates reminders for important events and deadlines. It uses an emotion engine to create reminders with optimal timing and content and sends them to devices. The input to this step is the information in the calendar and task management databases, and the output is the generated reminder.

[1527] Through each step, it is expected that customer service and employee work efficiency will improve significantly, leading to increased productivity across the company.

[1528] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1529] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1530] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1531] [Third embodiment]

[1532] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1533] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1534] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1535] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1536] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1537] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1538] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1539] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1540] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1541] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1542] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1543] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1544] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks for employees, learning each employee's work process, internal tools, and databases to provide efficient support. Below, we will explain the system's program processing in natural language, with concrete examples.

[1545] Meeting scheduling and time coordination

[1546] server

[1547] Employee schedules are retrieved from the database and analyzed.

[1548] Calculate each employee's free time to suggest optimal meeting times.

[1549] Terminal

[1550] An interface is provided that displays the proposed meeting time to the user and asks for approval or modification.

[1551] User

[1552] Review the proposed time and approve or modify it. Once approved, the meeting will be automatically scheduled.

[1553] Specific examples

[1554] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[1555] Creating and replying to emails

[1556] server

[1557] Based on user requests, NLP (Natural Language Processing) models are used to create appropriate email content.

[1558] Refer to past email data to ensure consistency and appropriateness of content.

[1559] Terminal

[1560] A preview of the generated email is displayed to the user for approval or correction.

[1561] User

[1562] Review the preview and make any necessary changes. Once approved, you will receive an email.

[1563] Specific examples

[1564] When User B wants to send an email proposing a new product to a client, the server generates the appropriate content and displays a preview of the email on the device. Once User B checks the content and clicks the approve button, the email is automatically sent.

[1565] Reminder notifications

[1566] server

[1567] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[1568] Terminal

[1569] Display reminders to users and prompt them to take necessary action.

[1570] User

[1571] Check the notification and make the necessary preparations or take action.

[1572] Specific examples

[1573] When the deadline for User C's monthly report submission approaches, the server generates a reminder notification and displays it on the terminal. User C checks the notification and prepares to submit the report.

[1574] In-house Wiki creation support

[1575] server

[1576] Analyzes meeting recording data and text data to automatically generate minutes.

[1577] Generate documents suitable for internal Wiki formats.

[1578] Terminal

[1579] The generated minutes are previewed to the user and the user is asked to approve or correct the contents.

[1580] User

[1581] Check the minutes, make any necessary corrections, and click the Approve button.

[1582] Specific examples

[1583] After a meeting that User D participated in, the recording of the meeting is uploaded to the server. The server analyzes the recording, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company Wiki.

[1584] Document preparation support

[1585] server

[1586] The necessary data is collected from various internal databases and documents are generated based on specified templates.

[1587] Ensure documents are automatically formatted properly.

[1588] Terminal

[1589] A preview of the generated document is displayed to the user for approval or correction.

[1590] User

[1591] Review the document and make any necessary corrections. Once approved, the document will be saved and the submission process will be complete.

[1592] Specific examples

[1593] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information and automatically generates the documents. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[1594] This system frees employees from tedious tasks and provides an environment where they can focus on more creative work, which is expected to result in improved work efficiency and increased productivity across the company.

[1595] The processing flow will be explained below.

[1596] Meeting scheduling and time coordination

[1597] Step 1:

[1598] To request a conference, a user inputs information about the purpose of the conference and the necessary participants into the system.

[1599] Step 2:

[1600] The terminal transmits the input request to the server.

[1601] Step 3:

[1602] The server retrieves the schedules of all participants from the employee database.

[1603] Step 4:

[1604] The server analyzes the acquired schedule data and calculates the free time of each employee.

[1605] Step 5:

[1606] The server runs an algorithm to suggest the best meeting time and suggests a time.

[1607] Step 6:

[1608] The terminal displays the proposed meeting time to the user and provides an interface for approval or modification.

[1609] Step 7:

[1610] The user reviews the proposed time and takes action to approve or modify it.

[1611] Step 8:

[1612] The server adds the approved meeting time to the calendars of all participants, completing the meeting setup.

[1613] Creating and replying to emails

[1614] Step 1:

[1615] A user requests a new email or reply and enters a summary of the email and who it is for.

[1616] Step 2:

[1617] The device sends a request to the server.

[1618] Step 3:

[1619] The server uses the generative AI model to generate appropriate email content.

[1620] Step 4:

[1621] The server sends the generated email content to the terminal.

[1622] Step 5:

[1623] The terminal displays a preview of the generated email to the user.

[1624] Step 6:

[1625] The user checks the preview and makes any necessary corrections.

[1626] Step 7:

[1627] The user approves the email and sends it.

[1628] Step 8:

[1629] The server sends the approved email to the specified recipient.

[1630] Reminder notifications

[1631] Step 1:

[1632] The server regularly monitors employees' calendars and task management databases.

[1633] Step 2:

[1634] The server checks the deadline for submitting documents and the start time of the meeting.

[1635] Step 3:

[1636] Generate reminder notifications based on events and deadlines seen by the server.

[1637] Step 4:

[1638] The device displays a reminder notification to the user at the specified timing.

[1639] Step 5:

[1640] The user checks the notification and takes the necessary steps.

[1641] In-house Wiki creation support

[1642] Step 1:

[1643] A user uploads meeting recordings or text data to the system.

[1644] Step 2:

[1645] The device sends the recorded data or text data to the server.

[1646] Step 3:

[1647] The server analyzes the conference data and, if it is audio data, converts it into text using voice recognition technology.

[1648] Step 4:

[1649] The server automatically generates minutes based on the analyzed data.

[1650] Step 5:

[1651] The server converts the generated minutes into an internal Wiki format.

[1652] Step 6:

[1653] The terminal displays a preview of the generated minutes to the user.

[1654] Step 7:

[1655] The user checks the minutes and makes any necessary corrections.

[1656] Step 8:

[1657] The user approves the minutes and updates them on the company wiki.

[1658] Document preparation support

[1659] Step 1:

[1660] The user requests a document and enters the required information.

[1661] Step 2:

[1662] The device sends a request to the server.

[1663] Step 3:

[1664] The server collects the required data from each database.

[1665] Step 4:

[1666] The server automatically generates documents by inputting the collected data into a specified template.

[1667] Step 5:

[1668] The server sends the generated document to the terminal.

[1669] Step 6:

[1670] The terminal displays a preview of the generated document to the user.

[1671] Step 7:

[1672] The user reviews the document and makes any necessary corrections.

[1673] Step 8:

[1674] The user approves the document and the server stores the document, completing the submission process.

[1675] Through these steps, the system efficiently supports each task and provides an environment in which employees can focus on their creative work.

[1676] Example 1

[1677] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1678] In today's corporate environment, employees are often overwhelmed with a large amount of miscellaneous tasks, which results in insufficient time for creative work or important tasks that they should be focusing on. In addition, tasks such as coordinating meetings, writing appropriate emails, and managing reminder notifications are cumbersome and cause efficiency to decline. There is a need to improve this situation and increase employee productivity.

[1679] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1680] In this invention, the server includes a means for acquiring employee schedules, a means for calculating optimal meeting times using a generative AI model, and a means for proposing meeting times to a terminal. This enables efficient scheduling of meetings. It also includes a means for creating appropriate emails using a generative AI model, a means for previewing the created email on the terminal, and a means for approving or modifying the previewed email. This improves the efficiency of email creation tasks. It also includes a means for monitoring employee event schedules and a work management database, a means for generating reminders for important events and deadlines, and a means for displaying the reminders to the user. This makes it possible to prevent important tasks from being forgotten.

[1681] "Means for obtaining employee schedules" refers to hardware and software for obtaining employee schedule data, specifically including APIs and database access.

[1682] "Means for calculating optimal meeting times using a generative AI model" refers to algorithms and software that use a generative AI model to analyze the free time of all employees and calculate optimal meeting times.

[1683] "Means for proposing meeting times to a terminal" refers to the interface and software that transmits the calculated optimal meeting time from the server to the terminal and allows the user to confirm, approve, or modify it.

[1684] "Means for Approving or Modifying Proposed Meeting Times" refers to the interface and software that allows a user to review, modify, if necessary, and ultimately approve proposed meeting times.

[1685] "Means for adding approved meeting times to an employee's calendar" means software and APIs that automatically add user-approved meeting times to an employee's calendar.

[1686] "Means for creating appropriate emails using a generative AI model" refers to software that uses a generative AI model to automatically generate appropriate email content based on information provided by a user.

[1687] The "means for displaying a preview of the created e-mail on the terminal" refers to an interface and software for displaying a preview of the content of the created e-mail on the user's terminal.

[1688] "Means for approving or correcting previewed email" refers to the interface and software that allows a user to review the contents of a previewed email, make corrections as necessary, and ultimately approve it.

[1689] "Means for sending approved email" refers to software and APIs that automatically send user-approved email through an external mail server.

[1690] "Means for monitoring employee calendars and work management databases" refers to software and APIs that regularly monitor employee calendars and task management systems to detect important events and task deadlines.

[1691] "Means for generating reminders for important events and deadlines" refers to software that automatically generates appropriate reminders for important events and task deadlines detected through monitoring.

[1692] "Means for displaying a reminder to a user" refers to an interface and software that displays the generated reminder on the user's terminal and allows the user to take appropriate action.

[1693] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks on behalf of employees. In this system, the server mainly performs the processing, and the terminal provides the interface with the user, who confirms and approves specific tasks.

[1694] Meeting scheduling and time coordination

[1695] server

[1696] An API (for example, a calendar API) is used to obtain employee calendar data. The server analyzes the obtained data and runs an algorithm to calculate the free time of all employees. This algorithm analyzes the free time using a weighted average method or a heuristic algorithm to determine the optimal meeting time. The generated meeting time is sent from the server to the terminal.

[1697] Terminal

[1698] The system suggests optimal meeting times to users and displays them in an interface, allowing users to review the suggested meeting times and make adjustments as necessary.

[1699] User

[1700] Review the proposed time, make any necessary changes, and finally click the approve button, and the server will automatically add the meeting to your calendar.

[1701] Specific examples

[1702] When User A wants to schedule a new project meeting, the server uses the calendar API to retrieve the schedules of User A and related employees and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[1703] Prompt Sentence Examples

[1704] "Please suggest the best time for a new project meeting."

[1705] Creating and replying to emails

[1706] server

[1707] Based on the user's request, a natural language processing (NLP) model (e.g., a generative AI model) is used to generate appropriate email content. The generated email content is temporarily stored on the server and then sent to the device.

[1708] Terminal

[1709] A preview of the generated email is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[1710] User

[1711] Check the preview and make any necessary corrections. Finally, click the approve button and the server will automatically send the email.

[1712] Specific examples

[1713] If User B wants to send an email proposing a new product to a client, the server uses a generative AI model to generate appropriate email content, displays a preview of the email on the device, and once User B confirms the content and clicks the approve button, the email is automatically sent.

[1714] Prompt Sentence Examples

[1715] "Write an email to pitch a new product to a client."

[1716] Reminder notifications

[1717] server

[1718] Regularly monitor employee calendars and work management databases to detect important events and task deadlines, and use appropriate APIs and scripts to generate reminder notifications based on the detected data.

[1719] Terminal

[1720] The generated reminder notification is displayed to the user, and an interface is provided to prompt the user to take appropriate action.

[1721] User

[1722] Check the notification and take the necessary preparations or actions. Clicking on the notification will display more information and the next steps.

[1723] Specific examples

[1724] When the deadline for User C's monthly report submission approaches, the server uses the event schedule API to generate a reminder notification and displays it on the device. User C checks the notification and prepares to submit the report.

[1725] Prompt Sentence Examples

[1726] "Show me a reminder when my monthly report is due."

[1727] In-house Wiki creation support

[1728] server

[1729] Meeting recording data is acquired and converted into text using a speech recognition service (for example, a speech recognition API). Based on this text data, minutes are automatically generated using a generative AI model and formatted in a format suitable for the in-house wiki.

[1730] Terminal

[1731] A preview of the generated minutes is displayed to the user, and an interface is provided to request confirmation and correction of the contents.

[1732] User

[1733] Check the minutes, make any necessary corrections, and finally click the approve button.

[1734] Specific examples

[1735] The recording data of the meeting that User D participated in is uploaded to the server. The server analyzes the recording using a speech recognition API, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company wiki.

[1736] Prompt Sentence Examples

[1737] "Please create minutes based on the meeting recording data and post them on the company wiki."

[1738] Document preparation support

[1739] server

[1740] The necessary data is collected from the internal database, and documents are automatically generated based on document generation templates (e.g., document template APIs). The generated documents are formatted in the appropriate format.

[1741] Terminal

[1742] A preview of the generated document is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[1743] User

[1744] Review the documents, make any necessary corrections, and finally click the approve button.

[1745] Specific examples

[1746] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information from the company database and automatically generates the documents using the document template API. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[1747] Prompt Sentence Examples

[1748] Please fill out your year-end tax adjustment documents and enter the necessary information.

[1749] This system frees employees from tedious tasks and provides an environment where they can focus on more important tasks, which is expected to result in improved work efficiency and increased productivity across the company.

[1750] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1751] Meeting scheduling and time coordination

[1752] Step 1: Get employee calendar data

[1753] server

[1754] Input: User A sends a request to set up a conference.

[1755] Process: The server sends a request to the calendar API to retrieve employee schedule data.

[1756] Output: The retrieved schedule data.

[1757] Specific operation: The server sends a request to " / user / schedules / {user_id}" and receives schedule data in JSON format.

[1758] Step 2: Calculate the best meeting time

[1759] server

[1760] Input: Retrieved schedule data.

[1761] Processing: The server analyzes the data and calculates the free time of all employees.

[1762] Output: Optimal meeting time.

[1763] Specific behavior: Calls the "find_optimal_time(slots)" function to calculate the best non-overlapping time.

[1764] Step 3: Propose a meeting time

[1765] Terminal

[1766] Input: The best meeting time sent by the server.

[1767] Processing: The terminal displays the meeting time on the user interface.

[1768] Output: The suggested time displayed to the user.

[1769] Specific behavior: The terminal uses the "display_suggested_time(time)" method to display the meeting time in the interface.

[1770] Step 4: Review, revise, and approve the proposal

[1771] User

[1772] Input: Proposed meeting time.

[1773] Process: The user reviews the proposed time, makes any necessary corrections, and finally approves it.

[1774] Output: The revised or approved meeting time.

[1775] Specific action: The user clicks the "confirm_time()" or "modify_time()" button.

[1776] Step 5: Schedule a meeting

[1777] server

[1778] Input: The revised or approved meeting time.

[1779] Process: The server automatically sets up the meeting through the calendar API.

[1780] Output: The configured meeting.

[1781] Specific behavior: The server sends a POST request to " / calendar / events" to add the meeting to the calendar.

[1782] Creating and replying to emails

[1783] Step 1: Receive the user request

[1784] server

[1785] Input: User B sends an email composition request.

[1786] Processing: The server receives and analyzes the request.

[1787] Output: Email input prompt.

[1788] Specific behavior: A user fills out the " / create_email" form and submits the request.

[1789] Step 2: Generate email content

[1790] server

[1791] Input: Prompt to enter email.

[1792] Processing: The server generates the email content using the generative AI model.

[1793] Output: The generated email content.

[1794] Specific behavior: Calls the "generate_email_content(prompt)" function to generate text using a generative AI model.

[1795] Step 3: Preview your email

[1796] Terminal

[1797] Input: The generated email content.

[1798] Processing: The terminal displays the email contents on the user interface.

[1799] Output: The email preview shown to the user.

[1800] What happens: The device displays a preview of the email using "display_email_preview(email)".

[1801] Step 4: Check, edit, and approve the email content

[1802] User

[1803] Input: A preview of the generated email.

[1804] Processing: The user reviews the email, makes any necessary corrections, and approves it.

[1805] Output: The corrected or approved email content.

[1806] Specific behavior: The user clicks the "confirm_email()" or "modify_email()" button.

[1807] Step 5: Send an email

[1808] server

[1809] Input: The corrected or approved email content.

[1810] Process: The server sends the email through the mail server.

[1811] Output: The email sent.

[1812] Specific behavior: The server sends an email to the SMTP server using the "send_email(email)" function.

[1813] Reminder notifications

[1814] Step 1: Monitor your calendar and task management data

[1815] server

[1816] Input: Calendar and task management data.

[1817] Processing: The server periodically monitors the data to detect important events and task deadlines.

[1818] Output: Detected events and deadline data.

[1819] Specific behavior: The server periodically monitors " / calendar / events" and " / tasks" and collects data.

[1820] Step 2: Generate a reminder notification

[1821] server

[1822] Input: Detected event and deadline data.

[1823] Processing: The server generates a reminder notification.

[1824] Output: The generated reminder notification.

[1825] Specific behavior: Generates a reminder notification using the "generate_reminder(event)" function.

[1826] Step 3: Display the notification to the user

[1827] Terminal

[1828] Input: The generated reminder notification.

[1829] Processing: The device displays the reminder notification on the user interface.

[1830] Output: The reminder notification shown to the user.

[1831] Specific behavior: The device displays the notification using the "display_reminder(reminder)" method.

[1832] Step 4: Review and respond to notifications

[1833] User

[1834] Input: The reminder notification shown to the user.

[1835] Action: The user checks the notification and takes the necessary action.

[1836] Output: The action that was performed.

[1837] Specific action: The user clicks on the reminder notification and takes the necessary steps.

[1838] In-house Wiki creation support

[1839] Step 1: Get the meeting recording data

[1840] server

[1841] Input: Meeting recording data.

[1842] Processing: The server receives and stores the recording.

[1843] Output: Saved meeting recording data.

[1844] Specific behavior: The user uploads the recording to " / upload_meeting_audio".

[1845] Step 2: Analyze the recording data and generate transcripts

[1846] server

[1847] Input: Saved meeting recording data.

[1848] Processing: The server uses a speech recognition API to convert the speech to text, and then uses a generative AI model to generate the transcript.

[1849] Output: The generated transcript.

[1850] Specific behavior: The server converts the audio to text using the "transcribe_audio(audio_file)" function, and generates the transcript using the "generate_minutes(text)" function.

[1851] Step 3: Preview the transcript

[1852] Terminal

[1853] Input: The generated minutes.

[1854] Processing: The terminal displays the minutes on the user interface.

[1855] Output: A preview of the transcript as displayed to the user.

[1856] Specific behavior: The terminal displays the transcript using the "display_transcription(transcript)" method.

[1857] Step 4: Review, revise, and approve the minutes

[1858] User

[1859] Input: A preview of the generated transcript.

[1860] Process: The user reviews the minutes, makes any necessary corrections, and approves them.

[1861] Output: Amended or approved minutes.

[1862] Specific behavior: The user clicks the "confirm_transcript()" or "edit_transcript()" button.

[1863] Step 5: Post the minutes on the company wiki

[1864] server

[1865] Enter: Amended or approved minutes.

[1866] Process: The server posts the minutes to the company wiki.

[1867] Output: Published minutes.

[1868] Specific operation: The server sends a POST request to " / wiki / add_entry" to add the minutes to the internal Wiki.

[1869] Document preparation support

[1870] Step 1: Collect the necessary data

[1871] server

[1872] Input: Data collection request.

[1873] Processing: The server collects the necessary data from the company database.

[1874] Output: Collected data.

[1875] Specific operation: The server sends a request to " / database / get_data" to collect the data.

[1876] Step 2: Auto-generate documents

[1877] server

[1878] Input: Collected data.

[1879] Processing: The server automatically generates the document using the document template API.

[1880] Output: The generated document.

[1881] Specific operation: Use the "generate_document(template, data)" function to call the Document Template API and generate a document.

[1882] Step 3: Preview your document

[1883] Terminal

[1884] Input: The generated document.

[1885] Processing: The device displays a preview of the document in its user interface.

[1886] Output: A preview of the document as it appears to the user.

[1887] Specific behavior: The device displays a preview of the document using the "display_document_preview(document)" method.

[1888] Step 4: Review, correct, and approve the document

[1889] User

[1890] Input: A preview of the generated document.

[1891] Processing: The user reviews the document, makes any necessary corrections, and approves it.

[1892] Output: The corrected or approved document.

[1893] Specific behavior: The user clicks the "confirm_document()" or "edit_document()" button.

[1894] Step 5: Save and submit your documents

[1895] server

[1896] Enter: Amended or approved document.

[1897] Processing: The server saves the document in the specified location and processes it for submission.

[1898] Output: Submitted documents.

[1899] Specific operation: The server sends a POST request to " / documents / submit" to submit the document.

[1900] (Application example 1)

[1901] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1902] In factories, manual robot task management and anomaly detection requires a great deal of effort and time. It is also difficult to manually create an optimal schedule when multiple robots are operating simultaneously, which can lead to reduced efficiency. Furthermore, delayed response to anomalies can lead to a decline in quality and stalled production lines. To solve these problems, a system is needed that can automatically acquire robot status, generate optimal task schedules, detect anomalies, and propose countermeasures.

[1903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1904] In this invention, the server includes means for acquiring the robot's status, means for proposing an optimal task schedule using a generative AI model, means for approving or modifying the proposed task schedule, means for controlling the robot's operation based on the approved task schedule, means for collecting and analyzing feedback data, and means for proposing the next action based on the analysis results. This improves the robot's operation efficiency and makes it possible to prevent production line stoppages by automating anomaly detection and countermeasures.

[1905] A "robot" is a mechanical device designed to automatically perform specific tasks in a factory or on a production line.

[1906] "Status" is data that indicates the current condition or status of the robot's operating state or function.

[1907] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal solutions and predictions based on large amounts of data.

[1908] A "task schedule" is a plan that efficiently arranges a series of tasks or duties to be performed by a robot and allocates them by time.

[1909] "Suggestion" refers to the generative AI model calculating optimal schedules and actions and providing them to the user.

[1910] "Approval" means that the user reviews and accepts the proposed schedule and actions.

[1911] "Control" refers to the proper operation of a robot's movements and functions based on programs and systems.

[1912] "Feedback data" refers to various sensor data and operation logs collected while the robot is performing its tasks.

[1913] "Analysis" is a method of analyzing collected feedback data to detect anomalies and evaluate performance.

[1914] "Action" refers to the specific actions or countermeasures to be taken next based on the analysis results.

[1915] This invention is a system that automates robot task management and anomaly detection in factories. The system acquires robot status data, generates an optimal task schedule using a generative AI model, analyzes feedback data to detect anomalies, and proposes next actions.

[1916] Hardware and software used

[1917] Hardware: Factory robots, various sensors, and servers.

[1918] Software: Python, NLP (Natural Language Processing) models, database management systems (e.g., PostgreSQL).

[1919] 1. Task schedule generation

[1920] The server first obtains the status data of the factory robots (e.g., current working status, operating hours, maintenance history), then uses a generative AI model to generate an optimal task schedule based on the obtained status data, and the generated task schedule is approved or modified to ensure the robots' efficient operation.

[1921] As a concrete example, if a robot is in charge of assembling part A, the generative AI model will generate the following schedule:

[1922] Assembly start time for part A: 09:00

[1923] Assembly of part A completed at 10:00

[1924] Next task start time: 10:05

[1925] 2. Feedback Analysis

[1926] The server collects feedback data from the robot in operation and analyzes it using an NLP model. This feedback data includes various sensor data such as temperature, vibration, and operation logs. Based on the analysis results, the next action is suggested. For example, if an abnormal temperature is detected, the next action suggested would be "Check for errors in part X and correct them."

[1927] 3. Implementing the proposed action

[1928] The actions proposed by the server can be approved or modified by the user. Once approved, the server again instructs the robot to execute the action. This automates the robot's response to abnormalities and maintenance, resulting in efficient operation.

[1929] Prompt Sentence Examples

[1930] Imagine a system that generates optimal task schedules for factory robots and prescribes next actions based on feedback data. Specifically, the system optimizes the robot schedule using NLP models, analyzes sensor feedback data (e.g., temperature, vibration), and automatically suggests countermeasures when anomalies are detected.

[1931] The system of the present invention improves the operational efficiency of robots and prevents production line stoppages by automating abnormality detection and countermeasures, thereby significantly improving the productivity of the entire factory and significantly reducing the need for manual management.

[1932] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1933] Step 1:

[1934] The server collects status data from factory robots, including their current working status, operating hours, and maintenance history. This data is used to aggregate information that forms the basis for the next processing step. The input is the robot's sensor data and operation log, and the output is the robot's status data.

[1935] Step 2:

[1936] The server uses a generative AI model to generate an optimal task schedule based on the acquired robot status data. The input is the status data, and the output is a proposed task schedule. The generative AI model analyzes the data and calculates the efficient work order for the robots.

[1937] Step 3:

[1938] The terminal displays the proposed task schedule to the user, who can then approve or modify it. The input is the task schedule generated by the server, and the output is the user's approved or modified schedule.

[1939] Step 4:

[1940] The user reviews the proposed task schedule and approves or modifies it. The user's actions are used to control the next step. The input is the proposed task schedule, and the output is the user's approval or modification.

[1941] Step 5:

[1942] The server controls the robot's actions based on the approved task schedule. The input is the schedule approved by the user, and the output is the control signal to the robot, which then performs the task as specified.

[1943] Step 6:

[1944] The server collects feedback data while the robot is operating, including temperature, vibration, and operation logs. The input is the robot's sensor data, and the output is the feedback data.

[1945] Step 7:

[1946] The server analyzes the collected feedback data and uses an NLP model to detect anomalies and maintenance needs. The input is the feedback data and the output is the analysis result.

[1947] Step 8:

[1948] The server proposes the next action based on the analysis results. For example, if an anomaly is detected, it suggests that a specific part needs to be repaired or maintained. The input is the analysis results, and the output is the proposed action.

[1949] Step 9:

[1950] The terminal displays the proposed action to the user, who can review it and accept or modify it as needed. The input is the server's proposed action, and the output is the user's accepted or modified action.

[1951] Step 10:

[1952] The user reviews the proposed action and approves or modifies it. The input is the proposed action and the output is the user's approval or modification.

[1953] Step 11:

[1954] The server executes the approved actions and sends control signals to the robot, which then performs the appropriate repairs or maintenance. The input is the action approved by the user, and the output is the command for the robot to execute.

[1955] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1956] This invention is a system that combines a generative AI model with an emotion engine to streamline employees' daily work and provide optimal support based on the user's emotions. The system learns employees' work processes, internal tools, and databases, recognizes the user's emotions using the emotion engine, and responds appropriately.

[1957] Meeting scheduling and time coordination

[1958] server

[1959] Employee schedules are retrieved from the database and analyzed.

[1960] It uses an emotion engine to recognize the user's current emotions and runs an algorithm that suggests optimal meeting times that take into consideration the emotions.

[1961] Terminal

[1962] The system provides an interface that displays proposed meeting times to users and asks for approval or modification based on sentiment.

[1963] User

[1964] Review the proposed time and accept or modify it. Emotion-based suggestions make meeting scheduling smoother.

[1965] Specific examples

[1966] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members. The emotion engine suggests the optimal time to reduce User A's stress level. Once User A confirms and approves the suggested time on their device, the meeting is added to their calendar.

[1967] Creating and replying to emails

[1968] server

[1969] Based on user requests, NLP models are used to generate appropriate email content.

[1970] The emotional engine recognizes the user's emotions and creates emails with tone and content based on those emotions.

[1971] Terminal

[1972] Show users a preview of the generated email and ask for sentiment-based approval or revision.

[1973] User

[1974] Check the preview and make any necessary corrections. By generating emails that are sensitive to customer sentiment, communication becomes smoother.

[1975] Specific examples

[1976] If User B wants to send an email proposing a new product to a client, the server will use the emotion engine to recognize that User B is nervous. Therefore, it will generate an email with a relaxed tone and display a preview on the device. If User B checks the content and approves it, the email will be sent.

[1977] Reminder notifications

[1978] server

[1979] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[1980] It uses an emotion engine to recognize the user's emotions and generate reminder notifications with optimal timing and content.

[1981] Terminal

[1982] Display emotional reminders to users and encourage them to take appropriate action.

[1983] User

[1984] Check the notification and make the necessary preparations or take action.

[1985] Specific examples

[1986] When the deadline for User C's monthly report is approaching, the server uses the emotion engine to recognize User C's current stress level. Therefore, a reminder notification is generated and displayed on the device at a time that minimizes stress. User C checks the notification and prepares the report.

[1987] In-house Wiki creation support

[1988] server

[1989] Analyzes meeting recording data and text data to automatically generate minutes.

[1990] Use an emotion engine to adjust the content and expression of meeting minutes based on user emotions.

[1991] Terminal

[1992] A preview of the generated meeting minutes is displayed to the user for approval or revision based on sentiment.

[1993] User

[1994] Check the minutes, make any necessary corrections, and click the approve button.

[1995] Specific examples

[1996] After a meeting that User D attends, the recording of the meeting is uploaded to the server. The emotion engine recognizes User D's fatigue and automatically generates concise, to-the-point minutes. Once User D checks and approves the content, the minutes are posted on the company's internal wiki.

[1997] Document preparation support

[1998] server

[1999] The necessary data is collected from various internal databases and documents are generated based on specified templates.

[2000] Use an emotion engine to recognize user emotions and adjust tone and format accordingly.

[2001] Terminal

[2002] A preview of the generated document is displayed to the user for approval or modification based on their sentiment.

[2003] User

[2004] Review the document and make any necessary corrections. Once approved, the document is saved and submitted.

[2005] Specific examples

[2006] When User E needs to create documents for year-end tax adjustment, the server uses an emotion engine to recognize User E's current emotional state. Therefore, the server automatically generates documents in a format that is easy for User E to use and displays a preview. Once User E confirms and approves the contents, the documents are saved and submitted.

[2007] This system provides support that takes users' emotions into consideration, improving work efficiency and the quality of communication. As a result, it is expected that an environment will be created where employees can focus on their core creative work, thereby increasing productivity across the company.

[2008] The processing flow will be explained below.

[2009] Meeting scheduling and time coordination

[2010] Step 1:

[2011] To request a conference, a user inputs information about the purpose of the conference and the necessary participants into the system.

[2012] Step 2:

[2013] The terminal transmits the user's input data to the server.

[2014] Step 3:

[2015] The server retrieves the schedules of all participants from the employee database.

[2016] Step 4:

[2017] The server analyzes the acquired schedule data and calculates the free time of each employee.

[2018] Step 5:

[2019] The server uses an emotion engine to recognize the user's current emotion.

[2020] Step 6:

[2021] The server runs an algorithm that suggests optimal meeting times based on sentiment.

[2022] Step 7:

[2023] The terminal displays the proposed meeting time to the user and provides an interface for sentiment-based approval or modification.

[2024] Step 8:

[2025] The user reviews the proposed time and takes action to approve or modify it.

[2026] Step 9:

[2027] The server adds the approved meeting time to the calendars of all participants, completing the meeting setup.

[2028] Creating and replying to emails

[2029] Step 1:

[2030] A user requests a new email or reply and enters a summary of the email and who it is for.

[2031] Step 2:

[2032] The device sends a request to the server.

[2033] Step 3:

[2034] The server uses the generative AI model to generate appropriate email content.

[2035] Step 4:

[2036] The server uses an emotion engine to recognize the user's emotions and generates emails with a tone and content based on those emotions.

[2037] Step 5:

[2038] The server sends the generated email content to the terminal.

[2039] Step 6:

[2040] The terminal displays a preview of the generated email to the user.

[2041] Step 7:

[2042] The user checks the preview and makes any necessary corrections.

[2043] Step 8:

[2044] The user approves the email and sends it.

[2045] Step 9:

[2046] The server sends the approved email to the specified recipient.

[2047] Reminder notifications

[2048] Step 1:

[2049] The server regularly monitors employees' calendars and task management databases.

[2050] Step 2:

[2051] The server checks the deadline for submitting documents and the start time of the meeting.

[2052] Step 3:

[2053] The server uses an emotion engine to recognize the user's emotions and generates a reminder notification with optimal timing and content.

[2054] Step 4:

[2055] The device displays emotion-based reminder notifications to the user.

[2056] Step 5:

[2057] The user checks the notification and takes the necessary steps.

[2058] In-house Wiki creation support

[2059] Step 1:

[2060] A user uploads meeting recordings or text data to the system.

[2061] Step 2:

[2062] The device sends the recorded data or text data to the server.

[2063] Step 3:

[2064] The server analyzes the conference data and, if it is audio data, converts it into text using voice recognition technology.

[2065] Step 4:

[2066] The server automatically generates minutes based on the analyzed data.

[2067] Step 5:

[2068] The server recognizes the user's emotions using an emotion engine and adjusts the content and expression of the minutes.

[2069] Step 6:

[2070] The server converts the generated minutes into an internal Wiki format.

[2071] Step 7:

[2072] The terminal displays a preview of the generated minutes to the user.

[2073] Step 8:

[2074] The user checks the minutes and makes any necessary corrections.

[2075] Step 9:

[2076] The user approves the minutes, and the server updates the minutes to the company Wiki.

[2077] Document preparation support

[2078] Step 1:

[2079] The user requests a document and enters the required information.

[2080] Step 2:

[2081] The device sends a request to the server.

[2082] Step 3:

[2083] The server collects the required data from each database.

[2084] Step 4:

[2085] The server automatically generates documents by inputting the collected data into a specified template.

[2086] Step 5:

[2087] The server uses an emotion engine to recognize the user's emotions and adjust the tone and format accordingly.

[2088] Step 6:

[2089] The server sends the generated document to the terminal.

[2090] Step 7:

[2091] The terminal displays a preview of the generated document to the user.

[2092] Step 8:

[2093] The user reviews the document and makes any necessary corrections.

[2094] Step 9:

[2095] The user approves the document and the server stores the document, completing the submission process.

[2096] Through these steps, the system will be able to efficiently support each task and respond in a way that takes into consideration the user's feelings. This will create an environment where employees can focus on their creative work, which is what we believe is essential, and is expected to improve productivity across the company.

[2097] Example 2

[2098] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2099] In conventional systems, managing employee schedules and communications is often complex and time-consuming, resulting in reduced work efficiency. Furthermore, because processes proceed mechanically without considering the user's emotions, stress and discord in communication can occur. Furthermore, reminder notifications and email creation do not take the user's emotional state into consideration, which can lead to problems such as insufficient stress reduction and work efficiency.

[2100] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2101] In this invention, the server includes a means for acquiring employee schedules, a means for recognizing user emotions using an emotion engine, and a means for proposing optimal meeting times using a generative AI model, thereby enabling the proposal of optimal meeting times that take user emotions into consideration.

[2102] The server includes means for recognizing a user's emotion and creating an email with an appropriate tone and expression based on the emotion, means for displaying a preview of the created email to the user, means for approving or correcting the previewed email based on the emotion, and means for sending the approved email. This allows an email with an appropriate tone according to the user's emotion to be created, facilitating smooth communication.

[2103] The server includes a means for monitoring employees' calendars and task management databases, and a means for generating reminder notifications of important events and deadlines at optimal timing based on the user's emotions, and a means for sending reminder notifications based on the emotions to the user, thereby enabling reminder notifications at optimal timing in line with the user's emotional state, thereby reducing stress and improving work efficiency.

[2104] An "employee schedule" is a record of the timetables of work hours, meetings, tasks, etc. scheduled by employees belonging to an organization.

[2105] "Emotion engine" is a general term for software or hardware that analyzes and recognizes a user's emotional state (e.g., stress level, fatigue, joy, etc.).

[2106] A "generative AI model" is an artificial intelligence model that generates output for natural language processing or specific tasks based on given input data, and primarily uses machine learning algorithms.

[2107] "Meeting time suggestion" refers to the act of the generative AI model presenting the optimal meeting time for the user based on the analysis results.

[2108] "Preview display" refers to a function that allows the user to view the generated content (for example, email or minutes) in advance and check or modify it.

[2109] A "reminder notification" is a notification sent to a user before an important event or deadline, to help the user remember.

[2110] The "optimal timing" refers to the time when the notification or action will be most effectively received, taking into account the user's current emotional state and work situation.

[2111] A "calendar" is a tool for visually managing employees' schedules and tasks, and is generally composed of dates and times.

[2112] A "task management database" is a database system for digitally managing and recording the tasks and business processes that employees must complete.

[2113] "Tone" refers to the emotional nuances and style of writing or communication, including friendly and formal tones.

[2114] This invention is a system that combines a generative AI model with an emotion engine to streamline employees' daily work and provide optimal support based on the user's emotions. This system operates using three elements: a server, a terminal, and the user.

[2115] Meeting scheduling and time coordination

[2116] server

[2117] The server retrieves employee schedules from a database and performs analysis. Specifically, the server executes a database query to retrieve each employee's schedule data. The server then uses an emotion engine to recognize user emotions and runs an algorithm to suggest optimal meeting times based on the analysis results.

[2118] Specific examples

[2119] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members from the schedule database. Then, the emotion engine suggests the best time to reduce User A's stress level.

[2120] Prompt Sentence Examples

[2121] "Check User A's schedule and suggest the best meeting time based on the sentiment engine data."

[2122] Creating and replying to emails

[2123] server

[2124] The server uses NLP models to generate appropriate email content based on user requests, and an emotion engine to recognize user emotions and create emails with appropriate tone and content.

[2125] Specific examples

[2126] If User B wants to send an email proposing a new product to a client, the server uses the emotion engine to recognize that User B is nervous, generates an email with a relaxed tone, and displays a preview on the device.

[2127] Prompt Sentence Examples

[2128] "User B is nervous, so please generate a new product proposal email with a relaxed tone."

[2129] Reminder notifications

[2130] server

[2131] The server monitors employees' calendars and task management databases to generate reminders for important events and deadlines. It uses an emotion engine to recognize users' emotions and generates reminders with optimal timing and content.

[2132] Specific examples

[2133] When User C's monthly report submission deadline approaches, the server uses the emotion engine to recognize User C's current stress level. A reminder notification is generated and displayed on the device at a time that minimizes stress.

[2134] Prompt Sentence Examples

[2135] "Please remind User C to submit the monthly report at a time when he is least stressed."

[2136] In-house Wiki creation support

[2137] server

[2138] The server analyzes meeting recording data and text data to automatically generate minutes, and uses an emotion engine to adjust the content and expression of the minutes based on the user's emotions.

[2139] Specific examples

[2140] After a meeting that User D participated in, the recording of the meeting is uploaded to the server. The emotion engine recognizes User D's sense of fatigue and automatically generates concise, to-the-point minutes.

[2141] Prompt Sentence Examples

[2142] "Please consider User D's fatigue and generate concise minutes."

[2143] Document preparation support

[2144] server

[2145] The server collects the necessary data from various company databases, generates documents based on specified templates, and uses an emotion engine to recognize the user's emotions and adjust the tone and format accordingly.

[2146] Specific examples

[2147] When User E needs to create documents for year-end tax adjustment, the server uses an emotion engine to recognize User E's current emotional state, automatically generates documents in a format that is easy to use, and displays a preview.

[2148] Prompt Sentence Examples

[2149] "Please generate the year-end tax adjustment documents in a format that is less burdensome for User E."

[2150] This system provides support that takes users' emotions into consideration, improving work efficiency and the quality of communication. As a result, it is expected that an environment will be created where employees can focus on their core creative work, thereby increasing productivity across the company.

[2151] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2152] Meeting scheduling and time coordination

[2153] Step 1: Get the schedule

[2154] The server connects to the database to retrieve employee schedule data. Specifically, the server runs a database query to pull schedule data from the "employee_schedules" table.

[2155] Input: Employee ID

[2156] Output: Employee schedule data

[2157] What it does: Retrieves schedule information from a database using an SQL query.

[2158] Step 2: Emotion recognition and analysis

[2159] The server uses the emotion engine to recognize the user's current emotional state and calls the emotion engine API to obtain the emotion data.

[2160] Input: User ID

[2161] Output: User emotion data

[2162] Specific operation: Calls the emotion engine API and collects emotion data.

[2163] Step 3: Suggest the best meeting time

[2164] The server runs an algorithm that suggests optimal meeting times based on the acquired schedule and emotion data, using a generative AI model to calculate the suggested times.

[2165] Input: Employee schedule data, user emotion data

[2166] Output: Best meeting time

[2167] What it does: Runs an algorithm using a generative AI model to calculate optimal meeting times.

[2168] Step 4: View and review the proposal

[2169] The terminal receives the proposed meeting time from the server and displays it to the user, who can then review and approve or modify the proposed time.

[2170] Input: Best Meeting Time

[2171] Output: Suggested time displayed to the user

[2172] Specific behavior: Show the suggested time to the user via notification or popup.

[2173] Step 5: Set up a meeting

[2174] The user reviews the proposed meeting time and can approve or modify it, and once approved, the update is sent to the server and the meeting is set in their calendar.

[2175] Input: User approval or correction information

[2176] Output: Meeting times set in the calendar

[2177] Specific operation: When the user clicks the approve button, the server calls the calendar API to add the meeting.

[2178] Creating and replying to emails

[2179] Step 1: Receiving a user request

[2180] The server receives a request to compose an email from the user: The user clicks the "Compose new email" button in the email sending interface.

[2181] Input: Email creation request

[2182] Output: Start of email creation process

[2183] Specific behavior: Receives an HTTP request and starts the email creation process.

[2184] Step 2: Emotion recognition and analysis

[2185] The server uses an emotion engine to recognize the user's emotions, and acquires and analyzes the emotion data.

[2186] Input: User ID

[2187] Output: User emotion data

[2188] Specific operation: Call the emotion engine API and obtain the user's emotional state.

[2189] Step 3: Generate email content

[2190] The server uses NLP models to generate appropriate email content based on the user's emotional data.

[2191] Input: User emotion data, basic email information

[2192] Output: Generated email content

[2193] Specific operation: A prompt sentence is input into the generative AI model to generate email content.

[2194] Step 4: Preview and check

[2195] The terminal displays a preview of the generated email to the user and provides an interface for the user to review and modify the content.

[2196] Input: Generated email content

[2197] Output: Email preview shown to the user

[2198] Specific behavior: Displays the email preview screen and allows the user to edit it.

[2199] Step 5: Sending an email

[2200] The user checks the contents of the email and clicks the send button. Once approval is complete, the sending information is sent to the server and the email is actually sent.

[2201] Input: User authorization information

[2202] Output: Email sent

[2203] Specific behavior: When you click the send button, the server sends the email using the SMTP server.

[2204] Reminder notifications

[2205] Step 1: Monitor your calendar and task data

[2206] The server periodically monitors employees' calendars and task management databases to detect important events and deadlines.

[2207] Input: Calendar and task data

[2208] Output: Detected important events and deadlines

[2209] Specific behavior: Runs a database query as a background job to obtain task information.

[2210] Step 2: Emotion recognition and analysis

[2211] The server uses an emotion engine to analyze the user's emotions, such as assessing stress levels due to an approaching deadline.

[2212] Input: User ID

[2213] Output: Emotion data

[2214] Specific operation: Utilize the emotion engine API to obtain emotion data.

[2215] Step 3: Generate a reminder notification

[2216] The server generates the optimal timing and content for the reminder notification based on the acquired data and emotional data.

[2217] Input: Calendar, task data, emotion data

[2218] Output: Reminder notification

[2219] Specific behavior: Generates notification content and manages notification schedules.

[2220] Step 4: View and review notifications

[2221] The device receives the reminder notification from the server and displays it to the user, allowing the user to check the notification and take appropriate action.

[2222] Input: Reminder notification

[2223] Output: The notification that is displayed to the user

[2224] Specific behavior: A popup notification will be displayed and an action will be triggered when the user presses the confirmation button.

[2225] (Application example 2)

[2226] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2227] Conventional employee work support systems are limited to functions such as setting up meetings, creating emails, and sending reminders, making it difficult to provide services such as emotion recognition in customer service or optimal product recommendations. Furthermore, there was a lack of a way to provide customer service staff with appropriate responses based on customer emotions in real time. This resulted in issues such as a decline in customer satisfaction and insufficient improvement in work efficiency.

[2228] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring employee schedules, means for proposing optimal meeting times using a generative AI model, means for approving or modifying the proposed meeting times, means for recognizing customer emotions, means for recommending optimal products based on the customer emotions, and means for proposing ways for customer service staff to respond based on the customer emotions. This makes it possible to significantly improve not only the daily work of employees but also the quality of customer service in physical stores, thereby increasing customer satisfaction.

[2229] "Means for obtaining employee schedules" refers to a function that obtains schedule information such as employee calendars and timetables from a database and analyzes it.

[2230] "Means for suggesting optimal meeting times using a generative AI model" is a function that uses a generative AI model to automatically suggest optimal meeting times for everyone based on the acquired employee schedule information.

[2231] A "means for approving or amending a proposed meeting time" is an interface or operating means for an employee to approve or amend a proposed meeting time.

[2232] "Means for adding approved meeting times to employees' calendars" refers to a function that automatically adds the final approved or amended meeting time to the employees' individual calendars.

[2233] "Means for recognizing customer emotions" refers to algorithms and sensor devices that analyze and recognize emotions in real time from customers' facial expressions, tone of voice, etc.

[2234] The "means for recommending optimal products based on customer emotions" is a function that automatically recommends products that are likely to be desired by a customer based on the recognized customer emotions.

[2235] "A means to suggest how to respond to customer service staff based on customer emotions" is a function that analyzes the emotional state of the customer and suggests in real time how the customer service staff should respond.

[2236] "Means for creating emails" refers to a function that uses a generative AI model to automatically create appropriate email content based on user requests.

[2237] The "means for displaying a preview of an email" is an interface for displaying a preview of the generated email to the user, allowing the user to check and modify the contents.

[2238] The "means for approving or correcting the previewed email" is an operation means for the user to approve or correct the previewed email.

[2239] "Means for monitoring employee calendars and task management databases" refers to a function that constantly monitors employee calendars and task management systems to track important events and deadlines.

[2240] The "means for generating reminders for document submission deadlines and meeting times" is a function that extracts important deadlines and meeting times from monitored data and generates reminders based on them.

[2241] The "means for sending a reminder notification to a user" is a system for sending the generated reminder notification to the user's terminal and prompting the user to take the necessary action.

[2242] This invention is a method for optimizing customer service in brick-and-mortar stores by using a system that combines a generative AI model and an emotion engine to streamline the daily work of employees. The hardware environment includes cameras, microphones, servers, and devices (smartphones and tablets) used by customer service staff. The software used includes OpenCV, DeepFace, and OpenAI API.

[2243] Customer Emotion Recognition

[2244] The server collects data in real time from cameras and microphones installed in the store. The video data from the cameras is captured using OpenCV, and DeepFace is used to analyze emotions from customers' facial expressions and tone of voice, allowing the system to recognize the customer's current emotional state.

[2245] Product recommendation

[2246] The server uses a generative AI model (OpenAI API) to recommend optimal products based on the recognized customer emotions. For example, if the customer is recognized as tired, the system generates a prompt such as, "Please recommend the best products for a tired customer. For example, aromatic candles with a relaxing effect or massage equipment," and receives the product recommendation through the OpenAI API.

[2247] Proposal for ways to deal with customer service staff

[2248] Furthermore, the server suggests appropriate responses to the customer service staff based on the customer's emotions. For example, if the customer is tired, the server generates a prompt such as, "Please suggest the best response to a tired customer. It is important to maintain a calm and relaxed tone of voice." The server then receives the appropriate response from the staff via the OpenAI API.

[2249] Meeting scheduling and time coordination

[2250] The server retrieves employee schedules from a database and uses a generative AI model to suggest optimal meeting times. The suggested times are displayed on the device, and the user can accept or modify them. Approved meeting times are automatically added to the employee's calendar.

[2251] Composing and replying to emails

[2252] The server uses a generative AI model to generate appropriate email content based on the user's request. The emotion engine creates emails with tone and content based on the user's emotions. The generated email is previewed on the device, and the user can approve or modify it before sending it.

[2253] Reminder notifications

[2254] The server monitors employees' calendars and task management databases, and generates reminders for deadlines and meeting times. These notifications are generated with optimal timing and content using an emotion engine and sent to users' devices.

[2255] Specific examples

[2256] For example, if a customer is tired, the server recognizes their emotion through the camera and microphone and uses the OpenAI API to send a prompt such as, "Please recommend the best products for a tired customer. For example, aroma candles or massage equipment that have a relaxing effect." This makes it possible to recommend products with a relaxing effect to the customer and to suggest a customer service method to staff based on the prompt, "Please suggest the best way to serve a tired customer. It is important to speak in a calm and relaxed tone."

[2257] This system can improve customer satisfaction and operational efficiency.

[2258] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2259] Step 1: Recognize customer emotions

[2260] The server captures video and audio data in real time from cameras and microphones installed in the store. The captured video data is processed using OpenCV, and DeepFace is used to analyze emotions from the customer's facial expressions and tone of voice. The input to this step is the video and audio data from the cameras and microphones, and the output is the customer's emotional information as an analysis result.

[2261] Step 2: Generate product recommendations

[2262] The server uses a generative AI model (OpenAI API) to create a prompt sentence that recommends the most suitable product based on the customer's emotional information recognized in step 1. For example, if the customer is recognized as "tired," it generates a prompt sentence such as "Please recommend the most suitable product for a tired customer" and inputs it into the generative AI model. The inputs in this step are the customer's emotional information and the prompt sentence, and the output is the generated product recommendation information.

[2263] Step 3: Propose ways to respond to staff

[2264] The server uses a generative AI model to create a prompt that generates a response method for the customer service staff based on the customer's emotional information recognized in step 1. For example, it generates a prompt such as "Please suggest the best response to a tired customer" and inputs it into the generative AI model. The input in this step is the customer's emotional information and the prompt, and the output is a generated response method suggestion.

[2265] Step 4: Schedule and schedule a meeting

[2266] The server retrieves employee schedules from the company database. Based on the retrieved schedule information, it uses a generative AI model to suggest optimal meeting times. The input for this step is employee schedule information, and the output is the suggested meeting time.

[2267] Step 5: Confirm meeting times

[2268] The terminal provides an interface that displays the proposed meeting time to the user and asks the user for approval or modification. The user reviews the proposed meeting time and modifies it if necessary. The input of this step is the proposed meeting time, and the output is the user's approval or modification of the meeting time.

[2269] Step 6: Add meeting times

[2270] The server automatically adds the meeting times approved or modified by the user to the employee's calendar. The input to this step is the approved or modified meeting times, and the output is the updated employee's calendar information.

[2271] Step 7: Compose and preview your email

[2272] The server generates appropriate email content based on the user's request using a generative AI model. The generated email is displayed as a preview on the device, and the user can review the content and make corrections as necessary. The inputs to this step are the user's request and the generated email content, and the output is the previewed email.

[2273] Step 8: Approve and send emails

[2274] The terminal provides an interface for the user to approve or modify the previewed email. The user approves the email, and the server sends the final email. The input of this step is the previewed email, and the output is the final email sent.

[2275] Step 9: Reminders

[2276] The server monitors employees' calendars and task management databases and generates reminders for important events and deadlines. It uses an emotion engine to create reminders with optimal timing and content and sends them to devices. The input to this step is the information in the calendar and task management databases, and the output is the generated reminder.

[2277] Through each step, it is expected that customer service and employee work efficiency will improve significantly, leading to increased productivity across the company.

[2278] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2279] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2280] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2281] [Fourth embodiment]

[2282] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2283] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2284] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2285] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2286] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2287] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2288] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2289] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2290] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2291] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2292] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2293] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2294] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2295] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks for employees, learning each employee's work process, internal tools, and databases to provide efficient support. Below, we will explain the system's program processing in natural language, with concrete examples.

[2296] Meeting scheduling and time coordination

[2297] server

[2298] Employee schedules are retrieved from the database and analyzed.

[2299] Calculate each employee's free time to suggest optimal meeting times.

[2300] Terminal

[2301] An interface is provided that displays the proposed meeting time to the user and asks for approval or modification.

[2302] User

[2303] Review the proposed time and approve or modify it. Once approved, the meeting will be automatically scheduled.

[2304] Specific examples

[2305] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[2306] Creating and replying to emails

[2307] server

[2308] Based on user requests, NLP (Natural Language Processing) models are used to create appropriate email content.

[2309] Refer to past email data to ensure consistency and appropriateness of content.

[2310] Terminal

[2311] A preview of the generated email is displayed to the user for approval or correction.

[2312] User

[2313] Review the preview and make any necessary changes. Once approved, you will receive an email.

[2314] Specific examples

[2315] When User B wants to send an email proposing a new product to a client, the server generates the appropriate content and displays a preview of the email on the device. Once User B checks the content and clicks the approve button, the email is automatically sent.

[2316] Reminder notifications

[2317] server

[2318] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[2319] Terminal

[2320] Display reminders to users and prompt them to take necessary action.

[2321] User

[2322] Check the notification and make the necessary preparations or take action.

[2323] Specific examples

[2324] When the deadline for User C's monthly report submission approaches, the server generates a reminder notification and displays it on the terminal. User C checks the notification and prepares to submit the report.

[2325] In-house Wiki creation support

[2326] server

[2327] Analyzes meeting recording data and text data to automatically generate minutes.

[2328] Generate documents suitable for internal Wiki formats.

[2329] Terminal

[2330] The generated minutes are previewed to the user and the user is asked to approve or correct the contents.

[2331] User

[2332] Check the minutes, make any necessary corrections, and click the Approve button.

[2333] Specific examples

[2334] After a meeting that User D participated in, the recording of the meeting is uploaded to the server. The server analyzes the recording, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company Wiki.

[2335] Document preparation support

[2336] server

[2337] The necessary data is collected from various internal databases and documents are generated based on specified templates.

[2338] Ensure documents are automatically formatted properly.

[2339] Terminal

[2340] A preview of the generated document is displayed to the user for approval or correction.

[2341] User

[2342] Review the document and make any necessary corrections. Once approved, the document will be saved and the submission process will be complete.

[2343] Specific examples

[2344] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information and automatically generates the documents. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[2345] This system frees employees from tedious tasks and provides an environment where they can focus on more creative work, which is expected to result in improved work efficiency and increased productivity across the company.

[2346] The processing flow will be explained below.

[2347] Meeting scheduling and time coordination

[2348] Step 1:

[2349] To request a conference, a user inputs information about the purpose of the conference and the necessary participants into the system.

[2350] Step 2:

[2351] The terminal transmits the input request to the server.

[2352] Step 3:

[2353] The server retrieves the schedules of all participants from the employee database.

[2354] Step 4:

[2355] The server analyzes the acquired schedule data and calculates the free time of each employee.

[2356] Step 5:

[2357] The server runs an algorithm to suggest the best meeting time and suggests a time.

[2358] Step 6:

[2359] The terminal displays the proposed meeting time to the user and provides an interface for approval or modification.

[2360] Step 7:

[2361] The user reviews the proposed time and takes action to approve or modify it.

[2362] Step 8:

[2363] The server adds the approved meeting time to the calendars of all participants, completing the meeting setup.

[2364] Creating and replying to emails

[2365] Step 1:

[2366] A user requests a new email or reply and enters a summary of the email and who it is for.

[2367] Step 2:

[2368] The device sends a request to the server.

[2369] Step 3:

[2370] The server uses the generative AI model to generate appropriate email content.

[2371] Step 4:

[2372] The server sends the generated email content to the terminal.

[2373] Step 5:

[2374] The terminal displays a preview of the generated email to the user.

[2375] Step 6:

[2376] The user checks the preview and makes any necessary corrections.

[2377] Step 7:

[2378] The user approves the email and sends it.

[2379] Step 8:

[2380] The server sends the approved email to the specified recipient.

[2381] Reminder notifications

[2382] Step 1:

[2383] The server regularly monitors employees' calendars and task management databases.

[2384] Step 2:

[2385] The server checks the deadline for submitting documents and the start time of the meeting.

[2386] Step 3:

[2387] Generate reminder notifications based on events and deadlines seen by the server.

[2388] Step 4:

[2389] The device displays a reminder notification to the user at the specified timing.

[2390] Step 5:

[2391] The user checks the notification and takes the necessary steps.

[2392] In-house Wiki creation support

[2393] Step 1:

[2394] A user uploads meeting recordings or text data to the system.

[2395] Step 2:

[2396] The device sends the recorded data or text data to the server.

[2397] Step 3:

[2398] The server analyzes the conference data and, if it is audio data, converts it into text using voice recognition technology.

[2399] Step 4:

[2400] The server automatically generates minutes based on the analyzed data.

[2401] Step 5:

[2402] The server converts the generated minutes into an internal Wiki format.

[2403] Step 6:

[2404] The terminal displays a preview of the generated minutes to the user.

[2405] Step 7:

[2406] The user checks the minutes and makes any necessary corrections.

[2407] Step 8:

[2408] The user approves the minutes and updates them on the company wiki.

[2409] Document preparation support

[2410] Step 1:

[2411] The user requests a document and enters the required information.

[2412] Step 2:

[2413] The device sends a request to the server.

[2414] Step 3:

[2415] The server collects the required data from each database.

[2416] Step 4:

[2417] The server automatically generates documents by inputting the collected data into a specified template.

[2418] Step 5:

[2419] The server sends the generated document to the terminal.

[2420] Step 6:

[2421] The terminal displays a preview of the generated document to the user.

[2422] Step 7:

[2423] The user reviews the document and makes any necessary corrections.

[2424] Step 8:

[2425] The user approves the document and the server stores the document, completing the submission process.

[2426] Through these steps, the system efficiently supports each task and provides an environment in which employees can focus on their creative work.

[2427] Example 1

[2428] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2429] In today's corporate environment, employees are often overwhelmed with a large amount of miscellaneous tasks, which results in insufficient time for creative work or important tasks that they should be focusing on. In addition, tasks such as coordinating meetings, writing appropriate emails, and managing reminder notifications are cumbersome and cause efficiency to decline. There is a need to improve this situation and increase employee productivity.

[2430] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2431] In this invention, the server includes a means for acquiring employee schedules, a means for calculating optimal meeting times using a generative AI model, and a means for proposing meeting times to a terminal. This enables efficient scheduling of meetings. It also includes a means for creating appropriate emails using a generative AI model, a means for previewing the created email on the terminal, and a means for approving or modifying the previewed email. This improves the efficiency of email creation tasks. It also includes a means for monitoring employee event schedules and a work management database, a means for generating reminders for important events and deadlines, and a means for displaying the reminders to the user. This makes it possible to prevent important tasks from being forgotten.

[2432] "Means for obtaining employee schedules" refers to hardware and software for obtaining employee schedule data, specifically including APIs and database access.

[2433] "Means for calculating optimal meeting times using a generative AI model" refers to algorithms and software that use a generative AI model to analyze the free time of all employees and calculate optimal meeting times.

[2434] "Means for proposing meeting times to a terminal" refers to the interface and software that transmits the calculated optimal meeting time from the server to the terminal and allows the user to confirm, approve, or modify it.

[2435] "Means for Approving or Modifying Proposed Meeting Times" refers to the interface and software that allows a user to review, modify, if necessary, and ultimately approve proposed meeting times.

[2436] "Means for adding approved meeting times to an employee's calendar" means software and APIs that automatically add user-approved meeting times to an employee's calendar.

[2437] "Means for creating appropriate emails using a generative AI model" refers to software that uses a generative AI model to automatically generate appropriate email content based on information provided by a user.

[2438] The "means for displaying a preview of the created e-mail on the terminal" refers to an interface and software for displaying a preview of the content of the created e-mail on the user's terminal.

[2439] "Means for approving or correcting previewed email" refers to the interface and software that allows a user to review the contents of a previewed email, make corrections as necessary, and ultimately approve it.

[2440] "Means for sending approved email" refers to software and APIs that automatically send user-approved email through an external mail server.

[2441] "Means for monitoring employee calendars and work management databases" refers to software and APIs that regularly monitor employee calendars and task management systems to detect important events and task deadlines.

[2442] "Means for generating reminders for important events and deadlines" refers to software that automatically generates appropriate reminders for important events and task deadlines detected through monitoring.

[2443] "Means for displaying a reminder to a user" refers to an interface and software that displays the generated reminder on the user's terminal and allows the user to take appropriate action.

[2444] This invention is a system that uses a generative AI model to perform a series of miscellaneous tasks on behalf of employees. In this system, the server mainly performs the processing, and the terminal provides the interface with the user, who confirms and approves specific tasks.

[2445] Meeting scheduling and time coordination

[2446] server

[2447] An API (for example, a calendar API) is used to obtain employee calendar data. The server analyzes the obtained data and runs an algorithm to calculate the free time of all employees. This algorithm analyzes the free time using a weighted average method or a heuristic algorithm to determine the optimal meeting time. The generated meeting time is sent from the server to the terminal.

[2448] Terminal

[2449] The system suggests optimal meeting times to users and displays them in an interface, allowing users to review the suggested meeting times and make adjustments as necessary.

[2450] User

[2451] Review the proposed time, make any necessary changes, and finally click the approve button, and the server will automatically add the meeting to your calendar.

[2452] Specific examples

[2453] When User A wants to schedule a new project meeting, the server uses the calendar API to retrieve the schedules of User A and related employees and proposes the best meeting time. User A checks the proposed time on their device and clicks the approve button, and the meeting is added to their calendar.

[2454] Prompt Sentence Examples

[2455] "Please suggest the best time for a new project meeting."

[2456] Creating and replying to emails

[2457] server

[2458] Based on the user's request, a natural language processing (NLP) model (e.g., a generative AI model) is used to generate appropriate email content. The generated email content is temporarily stored on the server and then sent to the device.

[2459] Terminal

[2460] A preview of the generated email is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[2461] User

[2462] Check the preview and make any necessary corrections. Finally, click the approve button and the server will automatically send the email.

[2463] Specific examples

[2464] If User B wants to send an email proposing a new product to a client, the server uses a generative AI model to generate appropriate email content, displays a preview of the email on the device, and once User B confirms the content and clicks the approve button, the email is automatically sent.

[2465] Prompt Sentence Examples

[2466] "Write an email to pitch a new product to a client."

[2467] Reminder notifications

[2468] server

[2469] Regularly monitor employee calendars and work management databases to detect important events and task deadlines, and use appropriate APIs and scripts to generate reminder notifications based on the detected data.

[2470] Terminal

[2471] The generated reminder notification is displayed to the user, and an interface is provided to prompt the user to take appropriate action.

[2472] User

[2473] Check the notification and take the necessary preparations or actions. Clicking on the notification will display more information and the next steps.

[2474] Specific examples

[2475] When the deadline for User C's monthly report submission approaches, the server uses the event schedule API to generate a reminder notification and displays it on the device. User C checks the notification and prepares to submit the report.

[2476] Prompt Sentence Examples

[2477] "Show me a reminder when my monthly report is due."

[2478] In-house Wiki creation support

[2479] server

[2480] Meeting recording data is acquired and converted into text using a speech recognition service (for example, a speech recognition API). Based on this text data, minutes are automatically generated using a generative AI model and formatted in a format suitable for the in-house wiki.

[2481] Terminal

[2482] A preview of the generated minutes is displayed to the user, and an interface is provided to request confirmation and correction of the contents.

[2483] User

[2484] Check the minutes, make any necessary corrections, and finally click the approve button.

[2485] Specific examples

[2486] The recording data of the meeting that User D participated in is uploaded to the server. The server analyzes the recording using a speech recognition API, automatically generates minutes, and displays a preview on the device. Once User D approves the content, the minutes are posted to the company wiki.

[2487] Prompt Sentence Examples

[2488] "Please create minutes based on the meeting recording data and post them on the company wiki."

[2489] Document preparation support

[2490] server

[2491] The necessary data is collected from the internal database, and documents are automatically generated based on document generation templates (e.g., document template APIs). The generated documents are formatted in the appropriate format.

[2492] Terminal

[2493] A preview of the generated document is displayed to the user, and an interface is provided for requesting confirmation and correction of the contents.

[2494] User

[2495] Review the documents, make any necessary corrections, and finally click the approve button.

[2496] Specific examples

[2497] When User E needs to create documents for year-end tax adjustment, the server collects the necessary information from the company database and automatically generates the documents using the document template API. A preview is displayed on the terminal, and once User E approves, the documents are saved and submitted.

[2498] Prompt Sentence Examples

[2499] Please fill out your year-end tax adjustment documents and enter the necessary information.

[2500] This system frees employees from tedious tasks and provides an environment where they can focus on more important tasks, which is expected to result in improved work efficiency and increased productivity across the company.

[2501] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2502] Meeting scheduling and time coordination

[2503] Step 1: Get employee calendar data

[2504] server

[2505] Input: User A sends a request to set up a conference.

[2506] Process: The server sends a request to the calendar API to retrieve employee schedule data.

[2507] Output: The retrieved schedule data.

[2508] Specific operation: The server sends a request to " / user / schedules / {user_id}" and receives schedule data in JSON format.

[2509] Step 2: Calculate the best meeting time

[2510] server

[2511] Input: Retrieved schedule data.

[2512] Processing: The server analyzes the data and calculates the free time of all employees.

[2513] Output: Optimal meeting time.

[2514] Specific behavior: Calls the "find_optimal_time(slots)" function to calculate the best non-overlapping time.

[2515] Step 3: Propose a meeting time

[2516] Terminal

[2517] Input: The best meeting time sent by the server.

[2518] Processing: The terminal displays the meeting time on the user interface.

[2519] Output: The suggested time displayed to the user.

[2520] Specific behavior: The terminal uses the "display_suggested_time(time)" method to display the meeting time in the interface.

[2521] Step 4: Review, revise, and approve the proposal

[2522] User

[2523] Input: Proposed meeting time.

[2524] Process: The user reviews the proposed time, makes any necessary corrections, and finally approves it.

[2525] Output: The revised or approved meeting time.

[2526] Specific action: The user clicks the "confirm_time()" or "modify_time()" button.

[2527] Step 5: Schedule a meeting

[2528] server

[2529] Input: The revised or approved meeting time.

[2530] Process: The server automatically sets up the meeting through the calendar API.

[2531] Output: The configured meeting.

[2532] Specific behavior: The server sends a POST request to " / calendar / events" to add the meeting to the calendar.

[2533] Creating and replying to emails

[2534] Step 1: Receive the user request

[2535] server

[2536] Input: User B sends an email composition request.

[2537] Processing: The server receives and analyzes the request.

[2538] Output: Email input prompt.

[2539] Specific behavior: A user fills out the " / create_email" form and submits the request.

[2540] Step 2: Generate email content

[2541] server

[2542] Input: Prompt to enter email.

[2543] Processing: The server generates the email content using the generative AI model.

[2544] Output: The generated email content.

[2545] Specific behavior: Calls the "generate_email_content(prompt)" function to generate text using a generative AI model.

[2546] Step 3: Preview your email

[2547] Terminal

[2548] Input: The generated email content.

[2549] Processing: The terminal displays the email contents on the user interface.

[2550] Output: The email preview shown to the user.

[2551] What happens: The device displays a preview of the email using "display_email_preview(email)".

[2552] Step 4: Check, edit, and approve the email content

[2553] User

[2554] Input: A preview of the generated email.

[2555] Processing: The user reviews the email, makes any necessary corrections, and approves it.

[2556] Output: The corrected or approved email content.

[2557] Specific behavior: The user clicks the "confirm_email()" or "modify_email()" button.

[2558] Step 5: Send an email

[2559] server

[2560] Input: The corrected or approved email content.

[2561] Process: The server sends the email through the mail server.

[2562] Output: The email sent.

[2563] Specific behavior: The server sends an email to the SMTP server using the "send_email(email)" function.

[2564] Reminder notifications

[2565] Step 1: Monitor your calendar and task management data

[2566] server

[2567] Input: Calendar and task management data.

[2568] Processing: The server periodically monitors the data to detect important events and task deadlines.

[2569] Output: Detected events and deadline data.

[2570] Specific behavior: The server periodically monitors " / calendar / events" and " / tasks" and collects data.

[2571] Step 2: Generate a reminder notification

[2572] server

[2573] Input: Detected event and deadline data.

[2574] Processing: The server generates a reminder notification.

[2575] Output: The generated reminder notification.

[2576] Specific behavior: Generates a reminder notification using the "generate_reminder(event)" function.

[2577] Step 3: Display the notification to the user

[2578] Terminal

[2579] Input: The generated reminder notification.

[2580] Processing: The device displays the reminder notification on the user interface.

[2581] Output: The reminder notification shown to the user.

[2582] Specific behavior: The device displays the notification using the "display_reminder(reminder)" method.

[2583] Step 4: Review and respond to notifications

[2584] User

[2585] Input: The reminder notification shown to the user.

[2586] Action: The user checks the notification and takes the necessary action.

[2587] Output: The action that was performed.

[2588] Specific action: The user clicks on the reminder notification and takes the necessary steps.

[2589] In-house Wiki creation support

[2590] Step 1: Get the meeting recording data

[2591] server

[2592] Input: Meeting recording data.

[2593] Processing: The server receives and stores the recording.

[2594] Output: Saved meeting recording data.

[2595] Specific behavior: The user uploads the recording to " / upload_meeting_audio".

[2596] Step 2: Analyze the recording data and generate transcripts

[2597] server

[2598] Input: Saved meeting recording data.

[2599] Processing: The server uses a speech recognition API to convert the speech to text, and then uses a generative AI model to generate the transcript.

[2600] Output: The generated transcript.

[2601] Specific behavior: The server converts the audio to text using the "transcribe_audio(audio_file)" function, and generates the transcript using the "generate_minutes(text)" function.

[2602] Step 3: Preview the transcript

[2603] Terminal

[2604] Input: The generated minutes.

[2605] Processing: The terminal displays the minutes on the user interface.

[2606] Output: A preview of the transcript as displayed to the user.

[2607] Specific behavior: The terminal displays the transcript using the "display_transcription(transcript)" method.

[2608] Step 4: Review, revise, and approve the minutes

[2609] User

[2610] Input: A preview of the generated transcript.

[2611] Process: The user reviews the minutes, makes any necessary corrections, and approves them.

[2612] Output: Amended or approved minutes.

[2613] Specific behavior: The user clicks the "confirm_transcript()" or "edit_transcript()" button.

[2614] Step 5: Post the minutes on the company wiki

[2615] server

[2616] Enter: Amended or approved minutes.

[2617] Process: The server posts the minutes to the company wiki.

[2618] Output: Published minutes.

[2619] Specific operation: The server sends a POST request to " / wiki / add_entry" to add the minutes to the internal Wiki.

[2620] Document preparation support

[2621] Step 1: Collect the necessary data

[2622] server

[2623] Input: Data collection request.

[2624] Processing: The server collects the necessary data from the company database.

[2625] Output: Collected data.

[2626] Specific operation: The server sends a request to " / database / get_data" to collect the data.

[2627] Step 2: Auto-generate documents

[2628] server

[2629] Input: Collected data.

[2630] Processing: The server automatically generates the document using the document template API.

[2631] Output: The generated document.

[2632] Specific operation: Use the "generate_document(template, data)" function to call the Document Template API and generate a document.

[2633] Step 3: Preview your document

[2634] Terminal

[2635] Input: The generated document.

[2636] Processing: The device displays a preview of the document in its user interface.

[2637] Output: A preview of the document as it appears to the user.

[2638] Specific behavior: The device displays a preview of the document using the "display_document_preview(document)" method.

[2639] Step 4: Review, correct, and approve the document

[2640] User

[2641] Input: A preview of the generated document.

[2642] Processing: The user reviews the document, makes any necessary corrections, and approves it.

[2643] Output: The corrected or approved document.

[2644] Specific behavior: The user clicks the "confirm_document()" or "edit_document()" button.

[2645] Step 5: Save and submit your documents

[2646] server

[2647] Enter: Amended or approved document.

[2648] Processing: The server saves the document in the specified location and processes it for submission.

[2649] Output: Submitted documents.

[2650] Specific operation: The server sends a POST request to " / documents / submit" to submit the document.

[2651] (Application example 1)

[2652] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2653] In factories, manual robot task management and anomaly detection requires a great deal of effort and time. It is also difficult to manually create an optimal schedule when multiple robots are operating simultaneously, which can lead to reduced efficiency. Furthermore, delayed response to anomalies can lead to a decline in quality and stalled production lines. To solve these problems, a system is needed that can automatically acquire robot status, generate optimal task schedules, detect anomalies, and propose countermeasures.

[2654] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2655] In this invention, the server includes means for acquiring the robot's status, means for proposing an optimal task schedule using a generative AI model, means for approving or modifying the proposed task schedule, means for controlling the robot's operation based on the approved task schedule, means for collecting and analyzing feedback data, and means for proposing the next action based on the analysis results. This improves the robot's operation efficiency and makes it possible to prevent production line stoppages by automating anomaly detection and countermeasures.

[2656] A "robot" is a mechanical device designed to automatically perform specific tasks in a factory or on a production line.

[2657] "Status" is data that indicates the current condition or status of the robot's operating state or function.

[2658] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal solutions and predictions based on large amounts of data.

[2659] A "task schedule" is a plan that efficiently arranges a series of tasks or duties to be performed by a robot and allocates them by time.

[2660] "Suggestion" refers to the generative AI model calculating optimal schedules and actions and providing them to the user.

[2661] "Approval" means that the user reviews and accepts the proposed schedule and actions.

[2662] "Control" refers to the proper operation of a robot's movements and functions based on programs and systems.

[2663] "Feedback data" refers to various sensor data and operation logs collected while the robot is performing its tasks.

[2664] "Analysis" is a method of analyzing collected feedback data to detect anomalies and evaluate performance.

[2665] "Action" refers to the specific actions or countermeasures to be taken next based on the analysis results.

[2666] This invention is a system that automates robot task management and anomaly detection in factories. The system acquires robot status data, generates an optimal task schedule using a generative AI model, analyzes feedback data to detect anomalies, and proposes next actions.

[2667] Hardware and software used

[2668] Hardware: Factory robots, various sensors, and servers.

[2669] Software: Python, NLP (Natural Language Processing) models, database management systems (e.g., PostgreSQL).

[2670] 1. Task schedule generation

[2671] The server first obtains the status data of the factory robots (e.g., current working status, operating hours, maintenance history), then uses a generative AI model to generate an optimal task schedule based on the obtained status data, and the generated task schedule is approved or modified to ensure the robots' efficient operation.

[2672] As a concrete example, if a robot is in charge of assembling part A, the generative AI model will generate the following schedule:

[2673] Assembly start time for part A: 09:00

[2674] Assembly of part A completed at 10:00

[2675] Next task start time: 10:05

[2676] 2. Feedback Analysis

[2677] The server collects feedback data from the robot in operation and analyzes it using an NLP model. This feedback data includes various sensor data such as temperature, vibration, and operation logs. Based on the analysis results, the next action is suggested. For example, if an abnormal temperature is detected, the next action suggested would be "Check for errors in part X and correct them."

[2678] 3. Implementing the proposed action

[2679] The actions proposed by the server can be approved or modified by the user. Once approved, the server again instructs the robot to execute the action. This automates the robot's response to abnormalities and maintenance, resulting in efficient operation.

[2680] Prompt Sentence Examples

[2681] Imagine a system that generates optimal task schedules for factory robots and prescribes next actions based on feedback data. Specifically, the system optimizes the robot schedule using NLP models, analyzes sensor feedback data (e.g., temperature, vibration), and automatically suggests countermeasures when anomalies are detected.

[2682] The system of the present invention improves the operational efficiency of robots and prevents production line stoppages by automating abnormality detection and countermeasures, thereby significantly improving the productivity of the entire factory and significantly reducing the need for manual management.

[2683] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2684] Step 1:

[2685] The server collects status data from factory robots, including their current working status, operating hours, and maintenance history. This data is used to aggregate information that forms the basis for the next processing step. The input is the robot's sensor data and operation log, and the output is the robot's status data.

[2686] Step 2:

[2687] The server uses a generative AI model to generate an optimal task schedule based on the acquired robot status data. The input is the status data, and the output is a proposed task schedule. The generative AI model analyzes the data and calculates the efficient work order for the robots.

[2688] Step 3:

[2689] The terminal displays the proposed task schedule to the user, who can then approve or modify it. The input is the task schedule generated by the server, and the output is the user's approved or modified schedule.

[2690] Step 4:

[2691] The user reviews the proposed task schedule and approves or modifies it. The user's actions are used to control the next step. The input is the proposed task schedule, and the output is the user's approval or modification.

[2692] Step 5:

[2693] The server controls the robot's actions based on the approved task schedule. The input is the schedule approved by the user, and the output is the control signal to the robot, which then performs the task as specified.

[2694] Step 6:

[2695] The server collects feedback data while the robot is operating, including temperature, vibration, and operation logs. The input is the robot's sensor data, and the output is the feedback data.

[2696] Step 7:

[2697] The server analyzes the collected feedback data and uses an NLP model to detect anomalies and maintenance needs. The input is the feedback data and the output is the analysis result.

[2698] Step 8:

[2699] The server proposes the next action based on the analysis results. For example, if an anomaly is detected, it suggests that a specific part needs to be repaired or maintained. The input is the analysis results, and the output is the proposed action.

[2700] Step 9:

[2701] The terminal displays the proposed action to the user, who can review it and accept or modify it as needed. The input is the server's proposed action, and the output is the user's accepted or modified action.

[2702] Step 10:

[2703] The user reviews the proposed action and approves or modifies it. The input is the proposed action and the output is the user's approval or modification.

[2704] Step 11:

[2705] The server executes the approved actions and sends control signals to the robot, which then performs the appropriate repairs or maintenance. The input is the action approved by the user, and the output is the command for the robot to execute.

[2706] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2707] This invention is a system that combines a generative AI model with an emotion engine to streamline employees' daily work and provide optimal support based on the user's emotions. The system learns employees' work processes, internal tools, and databases, recognizes the user's emotions using the emotion engine, and responds appropriately.

[2708] Meeting scheduling and time coordination

[2709] server

[2710] Employee schedules are retrieved from the database and analyzed.

[2711] It uses an emotion engine to recognize the user's current emotions and runs an algorithm that suggests optimal meeting times that take into consideration the emotions.

[2712] Terminal

[2713] The system provides an interface that displays proposed meeting times to users and asks for approval or modification based on sentiment.

[2714] User

[2715] Review the proposed time and accept or modify it. Emotion-based suggestions make meeting scheduling smoother.

[2716] Specific examples

[2717] When User A wants to schedule a new project meeting, the server retrieves the schedules of User A and related members. The emotion engine suggests the optimal time to reduce User A's stress level. Once User A confirms and approves the suggested time on their device, the meeting is added to their calendar.

[2718] Creating and replying to emails

[2719] server

[2720] Based on user requests, NLP models are used to generate appropriate email content.

[2721] The emotional engine recognizes the user's emotions and creates emails with tone and content based on those emotions.

[2722] Terminal

[2723] Show users a preview of the generated email and ask for sentiment-based approval or revision.

[2724] User

[2725] Check the preview and make any necessary corrections. By generating emails that are sensitive to customer sentiment, communication becomes smoother.

[2726] Specific examples

[2727] If User B wants to send an email proposing a new product to a client, the server will use the emotion engine to recognize that User B is nervous. Therefore, it will generate an email with a relaxed tone and display a preview on the device. If User B checks the content and approves it, the email will be sent.

[2728] Reminder notifications

[2729] server

[2730] Monitor employee calendars and task management databases to generate reminders of important events and deadlines.

[2731] It uses an emotion engine to recognize the user's emotions and generate reminder notifications with optimal timing and content.

[2732] Terminal

[2733] Display emotional reminders to users and encourage them to take appropriate action.

[2734] User

[2735] Check the notification and make the necessary preparations or take action.

[2736] Specific examples

[2737] When the deadline for User C's monthly report is approaching, the server uses the emotion engine to recognize User C's current stress level. Therefore, a reminder notification is generated and displayed on the device at a time that minimizes stress. User C checks the notification and prepares the report.

[2738] In-house Wiki creation support

[2739] server

[2740] Analyzes meeting recording data and text data to automatically generate minute...

Claims

1. A means of obtaining employee schedules; A means to suggest optimal meeting times using generative AI models; a means to approve or amend the proposed meeting time; A way to add approved meeting times to employees' calendars; A system including:

2. A way to create appropriate emails using generative AI models; A means for displaying a preview of the created email to the user; A means to approve or modify the previewed email; a means for sending approved emails; The system of claim 1 , comprising:

3. A means of monitoring employee calendars and task management databases; A means to generate reminders for deadlines and meeting times; means for sending reminder notifications to the user; The system of claim 1 , comprising:

4. A means of analyzing recording data and text data from meetings, A means to automatically generate minutes based on the analyzed data, A way to convert minutes into an internal Wiki format, a means for displaying a preview of the generated minutes to a user; A way to reflect minutes on the company Wiki, The system of claim 1 , comprising:

5. means of collecting the necessary data; A means of automatically generating documents based on the collected data; means for displaying a preview of the generated document to a user; A means to store approved documents and complete the submission process; The system of claim 1 , comprising:

Images