System

Abstract

To provide a system.SOLUTION: A system includes: means for acquiring messages posted on a communication tool that is used for information sharing among a plurality of users and that is for each user to send and receive messages; means for inputting a prompt sentence including contents of the acquired message into a generative AI model, thereby causing the generative AI model to share the contents of the message with other members and generate advice for the message; and means for presenting the information generated by the generative AI model to each user via the communication tool.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Lack of communication in remote work can lead to delays in information sharing and misunderstandings between team members, which is an issue that can lead to reduced work efficiency. [Means for solving the problem]

[0005] The present invention uses posts from communication tools as input to a generation AI, and visualizes the work that team members are currently doing and the advice they will need in the future. This allows you to see what work other members are doing and what advice you should give them. In particular, the generation AI automatically generates the work status of team members and the advice they need, solving the problem of lack of communication in remote work. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The embodiment of the present invention uses a chat tool that is commonly used as a communication tool. The content posted on the chat tool is input to the generation AI, which analyzes it. Based on the analysis results, the work that team members are currently doing and the advice they will need in the future are visualized. Specifically, if a member posts on the chat tool, "I will be writing a report today," the generation AI analyzes this information and visualizes to other members the information that the member is writing a report. In addition, the generation AI takes into account the content of past posts and the members' skill sets, etc., and generates advice on writing reports, which it presents to team members.

[0029] "Example 2"

[0030] In another embodiment of the present invention, the generation AI automatically generates the work status of team members and necessary advice. Specifically, the generation AI grasps the work status of team members in real time and automatically generates necessary advice. For example, when a member is performing programming work, the generation AI analyzes the member's code and generates advice pointing out improvements to the code and possible bugs. This enables team members to proceed with their work more efficiently.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: A team member posts their work on a chat tool. For example, they post, "Today, we will be creating a report."

[0034] Step 2: The generation AI analyzes the content posted in the chat tool. Through this analysis, it learns that a member is writing a report.

[0035] Step 3: Based on the analysis results, the generative AI visualizes the work status of team members. Specifically, it displays information such as "Member A is writing a report" to other members on the chat tool.

[0036] Step 4: The generation AI considers past posts and the skill sets of team members, etc., and generates advice on report creation. The generated advice is presented to team members. "Form Example 2"

[0037] Step 1: Team members start working. In this example, we assume that a team member starts programming work.

[0038] Step 2: The generative AI grasps the member's work status in real time. Specifically, it analyzes the code that the member is writing.

[0039] Step 3: Based on the results of the code analysis, the generative AI generates advice pointing out areas for improvement and potential bugs.

[0040] Step 4: The generative AI presents the generated advice to team members, allowing them to work more efficiently.

[0041] Example 1

[0042] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0043] In remote work and distributed teams, it is difficult to know in real time what team members are currently doing and what advice they need. In addition, there is a lack of ways for each member to share with other members what work they are doing, making efficient communication and cooperation difficult. There is a need to improve this situation and improve the productivity of the entire team.

[0044] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0045] In this invention, the server includes a means for a user to post to the communication tool, a means for the server to acquire the posted content, a means for the server to transmit the posted content to the generative AI model, a means for the generative AI model to analyze the posted content, a means for the generative AI model to return the analysis result to the server, a means for the server to display the analysis result on the communication tool, and a means for the user to check the analysis result. This makes it possible to visualize the work currently being done by team members and any advice they need in real time, enabling efficient communication and cooperation.

[0046] "User" means an individual or organization that uses the system to post to a communication tool.

[0047] A "communication tool" is a software or platform used by multiple users to share information and for each user to send and receive messages.

[0048] The "server" is a computer system that acquires user posted content, transmits it to the generative AI model, and displays the analysis results in the communication tool.

[0049] A "generative AI model" is an artificial intelligence model that analyzes the content posted by users and generates information about work status and necessary advice.

[0050] "Posted content" refers to messages or information that a user inputs into a communication tool and sends.

[0051] "Analysis results" refer to the information and advice obtained by the generative AI model when it analyzes the posted content.

[0052] "Visualization" refers to displaying the analysis results in a form that is easy for other team members to understand.

[0053] "Remote work" is a way of working in which work is carried out from a remote location without going to a physical office.

[0054] A "prompt sentence" is an instruction sentence input to a generative AI model to perform analysis.

[0055] The present invention provides a system for improving the efficiency of communication in remote work and distributed teams, and visualizing the work status of team members and necessary advice in real time. A specific embodiment of the system will be described below.

[0056] Hardware and Software Used

[0057] Hardware: Servers, terminals (PCs, smartphones, etc.)

[0058] Software: communication tools (e.g., Slack, MICROSOFT® TEAMS®), generative AI models (e.g., GPT-4®)

[0059] System Configuration

[0060] 1. A user posts to a communication tool

[0061] A user logs into a communication tool, enters a message about work, and sends it.

[0062] Example: A user posts, "Today I will be working on a report."

[0063] 2. The server retrieves the post

[0064] The server periodically checks for and retrieves new posted content using the communication tool's API.

[0065] Example: A server calls the Slack API to get new messages.

[0066] 3. The server sends the post to the generative AI model

[0067] The server sends the acquired post content to the generative AI model as a prompt sentence.

[0068] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[0069] 4. The generative AI model analyzes the post

[0070] The generative AI model analyzes the received prompt and identifies the user's current task.

[0071] Example: A generative AI model extracts the keyword "report writing" and determines that the user is writing a report.

[0072] 5. The generative AI model returns the analysis results to the server

[0073] The generative AI model generates analysis results and sends them back to the server.

[0074] Example: A generative AI model generates information that "the user is creating a report" and "advice on creating the report," and returns this to the server.

[0075] 6. The server displays the analysis results in the communication tool.

[0076] The server displays the analysis results received from the generative AI model on the communication tool.

[0077] Example: The server posts a message to a communication tool saying, "A user is creating a report. When creating the report, it may be a good idea to refer to past project reports."

[0078] 7. The user checks the analysis results

[0079] The user checks the analysis results and advice displayed on the communication tool.

[0080] Example: A user reviews "report writing advice" and modifies their actions as necessary.

[0081] The system provides real-time visibility into what team members are currently doing and what advice they need, enabling efficient communication and collaboration.

[0082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0083] Step 1:

[0084] A user posts to a communication tool.

[0085] Specific behavior: A user logs into a communications tool, types and sends a work-related message.

[0086] Input: A message typed by the user (e.g. "Today I will be writing a report").

[0087] Output: The message posted to the conversations tool.

[0088] Step 2:

[0089] The server retrieves the post.

[0090] Specific operation: The server uses the communication tool's API to periodically check for and retrieve new post content.

[0091] Input: The message posted in the conversations tool.

[0092] Output: The post retrieved by the server.

[0093] Step 3:

[0094] The server sends the post to the generative AI model.

[0095] Specific operation: The server sends the post content obtained as a prompt to the generative AI model.

[0096] Input: The post content retrieved by the server.

[0097] Output: The prompt sent to the generative AI model.

[0098] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[0099] Step 4:

[0100] A generative AI model analyzes the content of the post.

[0101] Specific behavior: The generative AI model analyzes the received prompt and identifies the user's current task.

[0102] Input: The prompt sent to the generative AI model.

[0103] Output: Analysis results (user actions and advice).

[0104] Step 5:

[0105] The generative AI model returns the analysis results to the server.

[0106] Specific operation: The generative AI model generates analysis results and sends them back to the server.

[0107] Input: Analysis results from a generative AI model.

[0108] Output: Analysis results sent back to the server.

[0109] Step 6:

[0110] The server displays the analysis results on the communication tool.

[0111] Specific operation: The server displays the analysis results received from the generative AI model on the communication tool.

[0112] Input: The analysis result sent back to the server.

[0113] Output: Analysis results displayed in a communication tool.

[0114] Example: Post a message in a communication tool saying, "A user is creating a report. When creating the report, it would be a good idea to refer to past project reports."

[0115] Step 7:

[0116] The user checks the analysis results.

[0117] Specific action: The user checks the analysis results and advice displayed on the communication tool.

[0118] Input: Analysis results displayed in communication tool.

[0119] Output: Analysis results and advice reviewed by the user.

[0120] (Application example 1)

[0121] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] With conventional work management systems in factories, it was difficult to grasp the working status of each machine in real time, and it was not possible to provide efficient work methods or points of caution appropriately. In addition, sharing of work status and provision of advice between team members was often done manually, making efficient work management difficult.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for inputting a post on a communication tool to the generating AI, a means for visualizing the work currently being performed by team members and advice that will be required in the future, a means for making it possible to know what work other members are performing and what advice should be given, a means for visualizing the work status of machines operating in a factory in real time, and a means for providing efficient work methods and points of caution using the generating AI. This makes it possible to grasp the work status in a factory in real time and provide efficient work methods and points of caution appropriately.

[0124] A "communication tool" is software or an application that enables the exchange of information between users.

[0125] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate appropriate output.

[0126] "Visualization" is the process of displaying data or information visually so that it can be easily understood by users.

[0127] "Machinery operating in a factory" refers to automated equipment and robots used in a factory to carry out manufacturing operations.

[0128] "Real-time" refers to data and information being processed instantly and provided without delay.

[0129] An "efficient work method" is the optimal procedure or method for completing a task with the least amount of time and effort.

[0130] "Points to note" are matters or points that require special attention when performing work.

[0131] As a form for implementing this invention, a system will be constructed that visualizes the working status of machines operating in a factory in real time and uses generative AI to provide efficient work methods and points to note.

[0132] System Configuration

[0133] 1. Hardware:

[0134] Machines operating in factories (robots, etc.)

[0135] Administrator's PC or tablet

[0136] 2. Software:

[0137] Communication tools (e.g. Slack)

[0138] Generative AI (e.g. OpenAI(R) GPT-3(R))

[0139] Data processing and calculation

[0140] Data Acquisition

[0141] The server receives real-time data on the operation status of machines operating in the factory, collected from the machine's sensors and log files.

[0142] 4. Data Analysis

[0143] The server posts the acquired work status data to the communication tool. The generation AI analyzes the posted content and understands the work status.

[0144] Visualization and Advice

[0145] The server visualizes the work status in real time based on the analysis results of the Generative AI. The Generative AI also automatically generates efficient work methods and points to note, taking into account past data and the skillset of the machine, and provides these to managers and other machines.

[0146] Examples

[0147] For example, if a post is made on a communication tool within a factory saying, "Robot A has started assembling parts," the generative AI would analyze this information and generate advice such as, "Robot A is assembling parts. According to past data, efficiency can be improved by paying attention to the order in which the parts are installed."

[0148] Examples of prompt statements

[0149] Please analyze the following tasks and provide your advice.

[0150] Task: Robot A starts assembling parts.

[0151] In this way, it is possible to grasp the work status within the factory in real time and provide efficient work methods and points of caution appropriately.

[0152] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0153] Step 1:

[0154] The server obtains data on the operation status from the machines operating in the factory in real time. The input is data from the machine's sensors and log files, and the output is the obtained operation status data. Specifically, the server collects data such as temperature, pressure, and operating status from the machine's sensors.

[0155] Step 2:

[0156] The server posts the acquired work status data to the communication tool. The input is the work status data acquired in step 1, and the output is the content to be posted to the communication tool. Specifically, the server converts the data into text format and posts it using the API of the communication tool.

[0157] Step 3:

[0158] The generation AI analyzes the content posted to the communication tool. The input is the content posted to the communication tool, and the output is the analysis result. In concrete terms, the generation AI analyzes the posted content using natural language processing technology and grasps the work status.

[0159] Step 4:

[0160] The server visualizes the work status in real time based on the analysis results of the generative AI. The input is the analysis results of the generative AI, and the output is the visualized work status. Specifically, the server displays the analysis results on a dashboard or monitor.

[0161] Step 5:

[0162] Generative AI automatically generates efficient work methods and points to note by taking into account past data and the skillset of the machine. The input is past data and the skillset of the machine, and the output is efficient work methods and points to note. In concrete terms, generative AI retrieves past work history from the database and generates the optimal work method.

[0163] Step 6:

[0164] The server provides the efficient work methods and cautions generated by the generative AI to managers and other machines. The input is the advice generated by the generative AI, and the output is a notification to the manager or other machines. Specifically, the server converts the advice into text format and notifies the user through a communication tool or dashboard.

[0165] Example 2

[0166] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] In remote work and distributed teams, it is difficult to grasp the work status of team members in real time and provide appropriate advice. In particular, it is difficult for other members to understand what work each member is doing and what kind of advice is needed, which can hinder efficient work progress and quality improvement.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0169] In this invention, the server includes means for collecting task data from the terminals of the team members, means for analyzing the collected data, means for generating advice based on the analysis results, means for notifying the terminals of each member of the generated advice, and means for letting each member know what tasks the other members are performing and what advice to give them. This makes it possible to grasp the task status of team members in real time and provide appropriate advice.

[0170] "Team members" refers to two or more individuals who come together to work on a project or task.

[0171] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[0172] "Work Data" means information related to work being performed by a Team Member, including code, logs, notes, etc.

[0173] "Means for collecting" refers to a method or device for acquiring work data from a terminal and transmitting it to a server.

[0174] "Means for analyzing" refers to methods and devices for analyzing and evaluating collected work data.

[0175] "Means for generating advice" refers to a method or device for suggesting improvements or modifications based on the results of the analysis.

[0176] The "means for notifying" refers to a method or device for transmitting the generated advice to the terminals of the team members.

[0177] "Visualization means" refers to a method or device for visually displaying what work other members are doing and what advice is needed.

[0178] "Generative AI" refers to artificial intelligence that uses machine learning models to analyze data and automatically generate advice.

[0179] "Communication tools" refers to software or platforms that allow team members to share information and stay in touch.

[0180] "Remote work" refers to a way of working in which work is carried out via the internet from a physically distant location.

[0181] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0182] The present invention is a system for grasping the working status of team members in real time and automatically generating necessary advice. A specific embodiment of this system will be described below.

[0183] 1. Generate the system program

[0184] The server uses the generative AI model to understand the work status of team members in real time and generate a program that automatically generates the necessary advice. This program is implemented using Python and utilizes machine learning libraries such as TENSORFLOW (registered trademark) and PyTorch.

[0185] 2. Explain the program's processing in natural language

[0186] The server collects data from each team member's device to monitor the work status of team members in real time. This data includes information from the IDE (integrated development environment) and editor (e.g. Visual Studio Code, PyCharm) used by the team members. The device periodically sends the work data to the server.

[0187] The server analyzes the collected data and inputs it into the generative AI model. The generative AI model evaluates the work status of each member based on the collected data and generates necessary advice. For example, if a member is working on programming, the generative AI model analyzes the member's code and generates advice pointing out areas to improve the code and possible bugs. This advice is sent to the member's device in real time.

[0188] 3. Add concrete examples

[0189] As a concrete example, let's say a member is developing a web application in Python. As the member writes code using Visual Studio Code, the server collects the code in real time and inputs it into the generative AI model. The generative AI model evaluates the quality of the code and generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the scope of the variable."

[0190] Example prompt:

[0191] "Analyze this code, written in Python, and generate suggestions for improvements or potential bugs."

[0192] In this way, generative AI models help team members work more efficiently and improve quality.

[0193] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0194] Step 1: Data collection

[0195] The server collects work data from each team member's device. The device obtains the code and work log in real time from the IDE or editor (e.g., Visual Studio Code, PyCharm) used by the team member and sends it to the server. The input is the code and work log sent from the device, and the output is the work data stored on the server.

[0196] Step 2: Data analysis

[0197] The server analyzes the collected work data. Specifically, it runs Python scripts and uses machine learning libraries such as TensorFlow and PyTorch to evaluate the quality of the code and possible bugs. The input is the work data stored on the server, and the output is the analysis results, for example checking function redundancy and variable scope.

[0198] Step 3: Advice Generation

[0199] The server uses a generative AI model to generate advice based on the analysis results. The generative AI model takes the analysis results as input and outputs advice including specific improvements and correction methods. For example, it generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the variable scope."

[0200] Step 4: Advice Notification

[0201] The server notifies each member's terminal of the generated advice. The terminal receives the notification and displays it as a pop-up message to the user. The input is the generated advice, and the output is the advice displayed to the user. The user checks the displayed advice and modifies the code.

[0202] In this way, generative AI models help team members work more efficiently and improve quality.

[0203] (Application example 2)

[0204] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0205] Conventional communication tools and generative AI systems can visualize the work status of team members and necessary advice, but they have the problem that it is difficult to provide specific advice in real time to improve work efficiency in the factory. In addition, there is a lack of systems that can monitor the work status in the factory in real time and suggest optimal operation patterns and improvements. This has led to problems such as a decrease in work efficiency in the factory and a lack of improvement in productivity.

[0206] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0207] In this invention, the server includes a means for inputting posts from the communication tool to the generation AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for making it possible to know what work other members are doing and what advice should be given, a means for monitoring the work situation in the factory in real time and automatically generating advice for improving work efficiency, and a means for displaying the generated advice. This makes it possible to provide specific advice for improving work efficiency in the factory in real time.

[0208] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[0209] "Generative AI" is a system that uses artificial intelligence technology to analyze data and automatically generate necessary information and advice.

[0210] "Visualization" is the process of displaying data or information visually and making it easier to understand.

[0211] "Real-time" refers to processing ongoing events and situations immediately and providing results.

[0212] "Work efficiency" is an index that indicates how much work can be effectively performed within a certain amount of time.

[0213] "Advice" is guidance or a suggestion given to address a particular situation or problem.

[0214] "Surveillance" is the act of continuously observing a specific object or situation and collecting necessary information.

[0215] A "behavior pattern" refers to a series of actions or steps in a specific task or process.

[0216] "Areas for improvement" are specific proposals or methods for identifying current problems or inefficiencies and making them better.

[0217] "Display" refers to showing information or data visually, often using a device such as a screen or display.

[0218] A system for implementing the present invention provides specific advice in real time to improve work efficiency in a factory. The system includes a camera, a generative AI model, and a display device.

[0219] First, a camera (e.g., Logitech C920) captures the work situation in the factory in real time. The captured video is sent to a server. The server analyzes the video using a generative AI model (e.g., a model built with TensorFlow or PyTorch). The generative AI model analyzes the work situation in the video and automatically generates optimal operation patterns and areas for improvement.

[0220] The generated advice is displayed on a display device (e.g., a monitor or smart glasses), allowing workers in the factory to receive the advice in real time and improve their work efficiency.

[0221] As a concrete example, consider a case where a robot is tightening screws in a factory. The generative AI model analyzes how and at what speed the screws are tightened, and suggests the optimal tightening method and speed. For example, by inputting a prompt such as "Analyze the current work situation and suggest the optimal tightening method and speed for the screws" into the generative AI model, specific advice can be obtained.

[0222] This system makes it possible to provide specific advice in real time to improve work efficiency within the factory.

[0223] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0224] Step 1:

[0225] Cameras capture the work situation inside the factory in real time.

[0226] Input: Work status in the factory

[0227] Output: Captured video data

[0228] Specific operation: A camera (e.g., Logitech C920) captures work being done in the factory and sends the video data to a server.

[0229] Step 2:

[0230] The server receives the captured video data and inputs it into the generative AI model.

[0231] Input: Captured video data

[0232] Output: Data input to the generative AI model

[0233] How it works: The server receives the video data and preprocesses it for input into a generative AI model (e.g., a model built with TensorFlow or PyTorch).

[0234] Step 3:

[0235] A generative AI model analyzes the video data and evaluates the work situation.

[0236] Input: Preprocessed video data

[0237] Output: Evaluation result of the work situation

[0238] How it works: A generative AI model analyzes video data to evaluate the progress and efficiency of a task, for example analysing how and how fast a screw is tightened.

[0239] Step 4:

[0240] The generative AI model automatically generates optimal movement patterns and areas for improvement.

[0241] Input: Evaluation result of work status

[0242] Output: Optimal operation patterns and advice on areas for improvement

[0243] Specific behavior: The generative AI model generates optimal behavior patterns and improvements based on the evaluation results. For example, it uses a prompt sentence such as, "Analyze the current work situation and suggest the optimal way and speed to tighten the screws."

[0244] Step 5:

[0245] The server displays the generated advice on a display device.

[0246] Input: Optimal behavior patterns and suggestions for improvement

[0247] Output: Advice displayed on your display device

[0248] Specific operation: The server receives the generated advice and displays it on a display device (e.g., a monitor or smart glasses). The worker can check the advice in real time to improve work efficiency.

[0249] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0250] "Example 1"

[0251] One embodiment of the present invention is a system that combines an emotion engine. This system recognizes the user's emotions and adjusts advice based on the user's emotional state. Specifically, if the user feels stressed, the emotion engine communicates this information to the generation AI. The generation AI then adjusts the advice to the user based on this information. For example, the system provides advice to a user who is stressed that is kinder or includes encouraging words.

[0252] "Example 2"

[0253] In addition, the emotion engine visualizes the user's emotional state to other team members. This enables team members to understand the emotional state of other members and communicate appropriately. Specifically, the emotion engine visualizes the user's emotional state on a chat tool. For example, if a user feels happy, this information is displayed on the chat tool as "Member A is happy." This allows other members to know that Member A is happy and to communicate based on that information.

[0254] The process flow of each embodiment will be described below.

[0255] "Example 1"

[0256] Step 1: Users communicate using a chat tool.

[0257] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[0258] Step 3: The emotion engine communicates the recognized emotional state to the generative AI.

[0259] Step 4: The generative AI tailors the advice to the user based on their emotional state.

[0260] "Example 2"

[0261] Step 1: Users communicate using a chat tool.

[0262] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[0263] Step 3: The emotional state recognized by the emotion engine is visualized on the chat tool.

[0264] Step 4: Other team members see the visualized emotional state and communicate based on it.

[0265] Example 1

[0266] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0267] In remote work and distributed teams, it is difficult to grasp the work status of team members and the advice they need in real time. It is also difficult to provide appropriate advice that takes into account the emotional state of members. This can lead to reduced communication efficiency and affect the productivity of the entire team.

[0268] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for inputting a post of a communication tool to a generative AI model, a means for the generative AI model to analyze the posted content, a means for visualizing the work currently being performed by team members and advice that will be required in the future based on the analysis result, and a means for recognizing the user's emotions and adjusting advice based on the emotional state. This makes it possible to grasp the work status and necessary advice of team members in real time and provide appropriate advice according to the emotional state.

[0269] A "communication tool" is software or a platform that enables users to send and receive messages.

[0270] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing to analyze text data and generate appropriate responses or advice.

[0271] "Posted content" refers to messages and text data sent by users on communication tools.

[0272] "Analysis" is the process by which the generative AI model understands the content of a post and extracts meaning from it.

[0273] "Visualization" is the process of visually displaying information based on analysis results so that users can easily understand it.

[0274] An “emotion engine” is software or algorithms for recognizing a user’s emotional state and providing that information to a generative AI model.

[0275] "Advice" is advice or suggestions that the generative AI model provides to the user based on the analysis results and emotional state.

[0276] "Remote work" refers to working in which work is carried out via the Internet from a physically distant location.

[0277] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0278] This invention is a system for streamlining communication in remote work and distributed teams, and understanding the work status of team members and necessary advice in real time. This system is realized by combining communication tools, generative AI models, and emotion engines.

[0279] Hardware and Software Used

[0280] Hardware: Servers, terminals (PCs, smartphones)

[0281] Software: communication tools (e.g. Slack, Microsoft Teams), generative AI models (e.g. GPT-4), emotion engines

[0282] System Configuration

[0283] 1. User Submissions

[0284] A user uses a terminal to post a message on a communication tool. For example, the user posts, "I will be writing a report today."

[0285] 2. Acquiring posted content

[0286] The server uses the communication tool's API to obtain user posts in real time.

[0287] 3. Analysis of posted content

[0288] The server sends the acquired post content to the generative AI model for analysis. The generative AI model understands the post content and recognizes that the user is creating a report.

[0289] 4. Information Visualization

[0290] Based on the analysis results from the generative AI model, the server makes the information that the relevant member is writing a report visible to other team members. Specifically, it displays "Mr. / Ms. XX is writing a report" on the chat tool.

[0291] 5. Generating Advice

[0292] The generative AI model takes into account past posts and the skill sets of team members to generate advice on report creation. The server presents the advice to team members.

[0293] 6. Use of Emotion Engines

[0294] The emotion engine recognizes the user's emotions and conveys that information to the generative AI model. For example, if the emotion engine determines that the user is feeling stressed, it will send that information to the generative AI model.

[0295] The generative AI model uses information from the emotion engine to tailor its advice to users, for example providing gentler or more encouraging advice to a user who is feeling stressed.

[0296] Examples

[0297] User submission examples

[0298] I will be writing a report today.

[0299] Example prompts for generative AI models

[0300] "A user posts, 'Today I'm going to create a report.' Use this information to make it visible to other members that the user is creating a report, and generate advice on creating the report."

[0301] Advice examples

[0302] "When preparing a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports."

[0303] Example of using the emotion engine

[0304] The emotion engine detects the user's stress and passes that information to the generative AI model.

[0305] The generative AI model generates gentle advice such as, "When writing a report, it is most efficient to first consider the overall structure. Take your time and take breaks as you go," and returns this advice to the server.

[0306] In this way, the system can analyze the content posted by users, visualize the work status and necessary advice of team members in real time, and provide appropriate advice according to their emotional state.

[0307] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0308] Step 1:

[0309] A user uses a terminal to post a message on a communication tool. The input is the user's message, and the output is the message sent to the communication tool. A specific example of this is when a user accesses a chat tool from a PC or smartphone, types "I will be writing a report today," and sends it.

[0310] Step 2:

[0311] The server uses the API of the communication tool to obtain user posts in real time. The input is the message data obtained from the communication tool, and the output is the message data stored on the server. Specifically, the server calls the Slack API and obtains new messages from a specific channel.

[0312] Step 3:

[0313] The server sends the acquired post content to the generative AI model for analysis. The input is the message data stored on the server, and the output is the analysis result returned from the generative AI model. In concrete terms, the server sends the message "Today I will create a report" as a prompt to the generative AI model, and receives the analysis result.

[0314] Step 4:

[0315] Based on the analysis results from the generative AI model, the server visualizes to other team members the information that the relevant member is writing a report. The input is the analysis results from the generative AI model, and the output is the information displayed in the chat tool. In concrete terms, the server automatically posts to the chat tool, "Mr. / Ms. XX is writing a report."

[0316] Step 5:

[0317] The generative AI model takes into account the content of past posts and the skill sets of members, and generates advice on creating reports. The input is the content of past posts and the skill sets of members, and the output is the generated advice. In concrete terms, the generative AI model generates advice such as "When creating a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports," and returns this advice to the server.

[0318] Step 6:

[0319] The emotion engine recognizes the user's emotions and transmits that information to the generative AI model. The input is the user's emotional state, and the output is the emotional information sent to the generative AI model. Specifically, the emotion engine determines that the user is feeling stressed based on the content of the user's posts and facial expression recognition data, and sends that information to the generative AI model.

[0320] Step 7:

[0321] The generative AI model adjusts advice to the user based on information from the emotion engine. The input is emotion information from the emotion engine, and the output is adjusted advice. In concrete terms, the generative AI model generates gentle advice such as, "When writing a report, it is efficient to first consider the overall structure. Don't push yourself, and take breaks as you go along," and returns this advice to the server.

[0322] (Application example 1)

[0323] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0324] In order to improve work efficiency in factories, it is important to facilitate communication between workers and robots and to provide appropriate information about the work situation and necessary advice. However, with conventional systems, it was difficult to provide advice that took into account the emotional state of the worker, which limited the improvement of work efficiency. In addition, it was difficult for workers to give instructions to the robot and the robot's work status was not sufficiently visualized, making it difficult to manage the progress of work.

[0325] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0326] In this invention, the server includes a means for inputting a post made via a communication tool to the generating AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for letting the user know what work other members are doing and what advice to give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot via a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, thereby improving work efficiency and facilitating progress management.

[0327] A "communication tool" is software or applications that allow users to exchange information through text or voice.

[0328] "Generative AI" is an artificial intelligence model that analyzes input data and generates appropriate output.

[0329] "Visualization" refers to displaying data or information visually so that users can easily understand it.

[0330] An "emotion engine" is a system that recognizes the user's emotional state and provides appropriate feedback based on that information.

[0331] A "robot" is a mechanical device that performs tasks automatically according to programmed instructions.

[0332] A "worker" is a human worker who performs a specific task in a factory.

[0333] An "instruction" is an order to perform a particular action or task.

[0334] "Analysis" refers to examining the input data in detail and understanding its meaning and structure.

[0335] "Advice" is advice or a suggestion intended to encourage appropriate action or decision in a particular situation.

[0336] "Progress management" is a management activity that monitors the progress of work or projects and keeps them moving forward as planned.

[0337] The system for implementing this invention includes a communication tool, a generation AI, an emotion engine, a robot, and a worker's terminal. The specific operation of each element and its cooperation will be described below.

[0338] System Configuration

[0339] 1. Communication tools:

[0340] Software or applications that allow workers to exchange information through text or voice.

[0341] Workers use chat tools to give instructions to the robot.

[0342] 2. Generation AI:

[0343] It uses generative AI models such as the OpenAI API to analyze chat content as it is typed.

[0344] Based on the analysis results, appropriate work instructions are issued to the robot.

[0345] Automatically generate work status and necessary advice.

[0346] 3. Emotion Engine:

[0347] Use emotion recognition libraries such as EmotionEngine to recognize the emotional state of workers.

[0348] Tailor the advice provided by the generative AI based on the perceived emotional state.

[0349] 4. Robots:

[0350] It is a mechanical device that performs tasks automatically according to programmed instructions.

[0351] It receives instructions from the generative AI and carries out the appropriate tasks.

[0352] 5. Worker terminal:

[0353] These are devices used by workers, such as smartphones and tablets.

[0354] Instructions are given to the robot through a communication tool and advice is received from the generating AI.

[0355] System Operation

[0356] The server includes a means for inputting posts from the communication tool to the generating AI, a means for visualizing the work that team members are currently doing and the advice they will need in the future, a means for letting the user know what work other members are doing and what advice they should give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot using a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, improving work efficiency and facilitating progress management.

[0357] Examples

[0358] When a worker types "I'm going to write a report today" into the chat tool, the generative AI analyzes the content and instructs the robot to support the report writing. At the same time, the emotion engine recognizes the worker's emotional state and provides advice including encouraging words such as "Do your best! You can do it!" if the worker is feeling stressed.

[0359] Examples of prompt statements

[0360] Please analyze the following chat and provide your suggestions:

[0361] I'm going to create a report today.

[0362] In this way, the work efficiency in the factory can be improved.

[0363] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0364] Step 1:

[0365] A user inputs text into a chat tool using a terminal. For example, the user inputs "Today I will be writing a report." This text becomes input data into the system.

[0366] Step 2:

[0367] The server receives the text entered from the chat tool. It generates a prompt sentence to send the received text to the generative AI model. This prompt sentence becomes the input data for the generative AI model to analyze.

[0368] Step 3:

[0369] The server sends a prompt to the generative AI model, which analyzes the text. The generative AI model analyzes the input text and generates work content and advice. The results of this analysis become the system's output data.

[0370] Step 4:

[0371] The server receives the analysis results from the generative AI model, which include action content (creation of a report) and advice. The server visualizes this information and shares it with other team members.

[0372] Step 5:

[0373] The server uses an emotion engine to recognize the user's emotional state. Based on chat content and past data, it determines whether the user is feeling stressed. This emotional state becomes the input data for the system.

[0374] Step 6:

[0375] The server receives the emotional state from the emotion engine and adjusts the advice from the generative AI model. For example, if the user is feeling stressed, it adds words of encouragement to the advice. This adjusted advice becomes the output data of the system.

[0376] Step 7:

[0377] The server sends the adjusted advice to the user's device, and the user receives the advice from the generative AI model and the adjustments made by the emotion engine through the device.

[0378] Step 8:

[0379] The user issues instructions to the robot using a terminal. The server receives the instructions and sends them to the generative AI model. The generative AI model analyzes the instructions and generates instructions for the robot to perform the appropriate tasks.

[0380] Step 9:

[0381] The server receives instructions from the generative AI model and sends them to the robot, which then follows the instructions to perform a task, such as writing a report. The progress of this task becomes the output data of the system.

[0382] Step 10:

[0383] The server receives the progress of the robot's work and makes it visible to the user through a chat tool. The user can then check the robot's progress through their device.

[0384] Example 2

[0385] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0386] In remote work and distributed teams, it is difficult to grasp the work situation and emotional state of team members in real time and provide appropriate advice and communication. In particular, communication between members tends to be lacking, which can lead to a decrease in work efficiency and team unity.

[0387] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0388] In this invention, the server includes a means for a user to start a task on a terminal, a means for the terminal to transmit task data to the server, a means for the server to analyze the data received, a means for the server to generate advice using a generative AI model, a means for the server to analyze and visualize an emotional state using an emotion engine, a means for the server to transmit the advice and emotional information generated by the server to the terminal, and a means for displaying the information received by the terminal to the user. This makes it possible to grasp the task status and emotional state of team members in real time and provide appropriate advice and communication.

[0389] "User" refers to an individual or group that performs work using the system.

[0390] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[0391] "Server" refers to a central processing unit that receives, analyzes, generates, and transmits data.

[0392] "Task data" refers to information related to the task a user is performing (e.g., programming code or emotion data).

[0393] A "generative AI model" refers to an artificial intelligence model that analyzes a user's work data and generates appropriate advice.

[0394] An "emotion engine" refers to software or hardware for analyzing a user's emotional state and visualizing the results.

[0395] "Advice" refers to suggestions for improvement or comments that the generative AI model generates based on the user's work data.

[0396] "Emotion information" refers to information about the user's emotional state analyzed by the emotion engine.

[0397] "Visualization" refers to displaying analyzed data in a form that is easy for users to understand.

[0398] "Communication tools" refers to software or platforms for sharing information among team members.

[0399] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0400] This invention is a system for grasping the work situation and emotional state of team members in real time in remote work and distributed teams, and providing appropriate advice and communication. Below, we will show how to specifically implement this system.

[0401] 1. Generate the system program

[0402] This system combines a generative AI model and an emotion engine to grasp the work situation and emotional state of team members in real time and provide necessary advice and emotional information. For example, OpenAI's GPT-4 is used as the generative AI model, and the emotion engine uses Microsoft's Azure (registered trademark) Cognitive Services Emotion API.

[0403] 2. Explain the program's processing in natural language

[0404] The server hosts a generative AI model and an emotion engine. The generative AI model analyzes the user's work data (e.g., programming code) and generates appropriate advice. The emotion engine analyzes the user's emotional state (e.g., facial expressions and voice) and visualizes the results.

[0405] The terminals are PCs or smartphones used by users, and send data to the server. Specifically, the terminals send the user's task data and emotion data to the server.

[0406] The server analyzes the data it receives and uses a generative AI model to generate advice. For example, if a member is working on a program, the server analyzes the member's code and generates advice on how to improve the code or identify potential bugs.

[0407] The emotion engine also analyzes the user's emotional state and visualizes the results on the chat tool. For example, if the emotion engine determines that the user is happy, the server will display "Member A is happy" on the chat tool.

[0408] 3. Add examples to your description

[0409] As a concrete example, consider the following scenario.

[0410] Scenario 1: Programming task advice

[0411] User A is programming on a device. User A's device sends the code he is working on to the server. The server analyzes the code using a generative AI model (GPT-4) and generates advice such as the following:

[0412] "This part of the code is inefficient and can be improved as follows."

[0413] "There may be a bug in this line. Details below."

[0414] Example prompt:

[0415] Analyze User A's code and generate advice pointing out areas for improvement and potential bugs.

[0416] Code: [User A's code]

[0417] Scenario 2: Visualizing Emotional States

[0418] User B is working on a device. The emotion engine analyzes User B's facial expression and determines that User B is happy. The server displays this information on the chat tool.

[0419] "Member B is happy."

[0420] Example prompt:

[0421] Analyze User B's emotional state and visualize it on the chat tool.

[0422] Emotional state: Joy

[0423] This system enables users to efficiently carry out their own tasks and to understand the emotional states of other members and communicate appropriately.

[0424] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0425] Step 1:

[0426] The user begins working at the terminal.

[0427] Input: The device used by the user (PC or smartphone)

[0428] Output: Working data (programming code and emotion data)

[0429] Specific operation: A user starts programming or other work on his / her PC or smartphone. For example, User A opens Visual Studio Code and starts programming.

[0430] Step 2:

[0431] The terminal transmits the work data to the server.

[0432] Input: User-generated task data (programming code and emotion data)

[0433] Output: The work data sent to the server.

[0434] Specific operation: User A's device transmits the programming code written in real time and facial expression data captured by the webcam to the server.

[0435] Step 3:

[0436] The server parses the received data.

[0437] Input: Working data sent from the terminal (programming code and emotion data)

[0438] Output: Analysis results (code structure and content, emotional state)

[0439] Specific operation: The server analyzes the received programming code and understands the structure and content of the code. It also analyzes User A’s facial expressions from the webcam footage to understand his / her emotional state.

[0440] Step 4:

[0441] The server generates advice using a generative AI model.

[0442] Input: Parsed programming code

[0443] Output: Generated advice

[0444] Specific operation: The server uses a generative AI model (GPT-4) to generate advice for User A's programming code. For example, it generates specific advice such as "This loop is inefficient, so it would be better to use a list comprehension."

[0445] Step 5:

[0446] The server uses an emotion engine to analyze and visualize the emotional state.

[0447] Input: Parsed emotion data

[0448] Output: Visualized emotion information

[0449] Specific operation: The server uses the emotion engine (Emotion API) to determine that User A is happy. The server visualizes this information on the chat tool as "Member A is happy."

[0450] Step 6:

[0451] The server transmits the generated advice and emotion information to the terminal.

[0452] Input: Generated advice and visualized emotion information

[0453] Output: Advice and sentiment sent to the device

[0454] Specific operation: The server sends the generated advice and emotion information to the terminal of user A. For example, the server sends advice such as "This loop is inefficient, so it would be better to use a list comprehension" and emotion information such as "Member A is happy."

[0455] Step 7:

[0456] The terminal displays the received information to the user.

[0457] Input: Advice and sentiment information sent from the server

[0458] Output: Advice and emotion information displayed to the user

[0459] Specific operation: User A's device displays advice on Visual Studio Code and emotional information on the chat tool. For example, "This loop is inefficient, so it would be better to use a list comprehension" is displayed in the Visual Studio Code editor, and "Member A is happy" is displayed in the chat tool.

[0460] (Application example 2)

[0461] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0462] In conventional teamwork, it is difficult to grasp the working status of team members and necessary advice in real time, which often hinders efficient communication. In addition, there is no way to grasp the emotional state of team members, making it difficult to communicate appropriately. This can lead to a decrease in work efficiency and friction within the team.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0464] In this invention, the server includes a means for inputting posts from a communication tool to a generation AI, a means for visualizing the work that team members are currently doing and advice that they will need in the future, a means for letting users know what work other members are doing and what advice they should give, a means for recognizing and visualizing the emotional state of users in real time, and a means for generating and displaying necessary advice based on the work situation. This makes it possible to grasp the work situation and emotional state of team members in real time and provide appropriate advice.

[0465] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[0466] "Generative AI" is an artificial intelligence system that automatically generates advice and information based on input data.

[0467] "Visualization" is a means of visually displaying data or information to make it easier to understand.

[0468] "User's emotional state" refers to the emotions and psychological state that the user is currently feeling.

[0469] "Real-time" refers to processing and display occurring immediately, without delay.

[0470] "Work status" refers to the progress and content of the work that team members are currently doing.

[0471] "Advice" means advice or a suggestion given to improve a task or solve a problem.

[0472] A "display" is a means of visually presenting information to a user.

[0473] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The server uses a generative AI model to automatically generate the work status and necessary advice of team members, and has the function of recognizing and visualizing the emotional state of the user in real time. The terminal is a device such as smart glasses or a smartphone, and serves to provide this information to the user. The user receives advice and feedback of the emotional state in real time through these devices.

[0474] The server uses the following software and hardware:

[0475] OpenCV: A library for image processing

[0476] EmotionRecognizer: Software for Recognizing a User's Emotional State

[0477] CodeAnalyzer: software to analyze the work of team members

[0478] AIAdvisor: An artificial intelligence system for generating needed advice

[0479] The terminals are devices such as smart glasses or smartphones, which are equipped with cameras and displays. The cameras capture images of the user and send them to a server. The server analyzes the images and recognizes the user's emotional state and work situation. The recognized information is converted into appropriate advice using a generative AI model and displayed on the terminal's display.

[0480] As a concrete example, consider a scenario in which an employee working in a factory wears smart glasses and receives advice in real time based on the work situation and emotional state. While the employee is working, the camera in the smart glasses captures video and sends it to a server. The server analyzes the video and recognizes the employee's emotional state. At the same time, it analyzes the progress of the work the employee is doing and generates the necessary advice. This information is displayed in real time on the display of the smart glasses.

[0481] Example of a prompt to be input to a generative AI model:

[0482] "Employee A puts on smart glasses, which analyze the code he is working on and gives him advice. His emotional state is also recognized in real time and shared with his team members."

[0483] In this way, it is possible to improve the work efficiency and communication of team members.

[0484] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0485] Step 1:

[0486] The camera on the device (smart glasses) captures an image of the user.

[0487] Input: Real-time video of the user

[0488] Output: Captured video data

[0489] Specific operation: The camera in the smart glasses photographs the user's face and work environment and captures the video data.

[0490] Step 2:

[0491] The video data acquired by the terminal is transmitted to the server.

[0492] Input: Acquired video data

[0493] Output: Video data sent to the server

[0494] Specific operation: The smart glasses upload video data to the server using communication methods such as Wi-Fi or Bluetooth.

[0495] Step 3:

[0496] The server receives the video data and analyzes the user's emotional state using an emotion recognition engine (EmotionRecognizer).

[0497] Input: Video data sent to the server

[0498] Output: Parsed emotional state data

[0499] Specific operation: The server inputs the video data into EmotionRecognizer, which estimates the user's emotional state from their facial expressions and movements.

[0500] Step 4:

[0501] The server uses a code analysis engine (CodeAnalyzer) to analyze the user's work.

[0502] Input: Data about what the user is working on (e.g. code files)

[0503] Output: Analyzed work status data

[0504] What it does: The server inputs the user's work (e.g., program code) into CodeAnalyzer and evaluates the code's quality and progress.

[0505] Step 5:

[0506] The server uses a generative AI model (AIAdvisor) to generate the necessary advice based on the analyzed emotional state data and work situation data.

[0507] Input: Emotional state data, work situation data

[0508] Output: Generated advice

[0509] Specific operation: The server inputs emotional state data and task situation data into the AIAdvisor and generates appropriate advice.

[0510] Step 6:

[0511] The server transmits the generated advice and emotional state to the terminal.

[0512] Input: Generated advice, emotional state data

[0513] Output: Advice and emotional state data sent to the device

[0514] Specific operation: The server uses a communication means to transmit the generated advice and emotional state data to the terminal.

[0515] Step 7:

[0516] The terminal displays the received advice and emotional state to the user.

[0517] Input: Received advice, emotional state data

[0518] Output: Advice and emotional state displayed to the user

[0519] Specific operation: The smart glasses display visually displays the received advice and emotional state to the user.

[0520] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0521] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0522] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0523] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0524] [Second embodiment]

[0525] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0526] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0527] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0528] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0529] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0530] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0531] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0532] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0533] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0534] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0535] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0536] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0537] "Example 1"

[0538] The embodiment of the present invention uses a chat tool that is commonly used as a communication tool. The content posted on the chat tool is input to the generation AI, which analyzes it. Based on the analysis results, the work that team members are currently doing and the advice they will need in the future are visualized. Specifically, if a member posts on the chat tool, "I will be writing a report today," the generation AI analyzes this information and visualizes to other members the information that the member is writing a report. In addition, the generation AI takes into account the content of past posts and the members' skill sets, etc., and generates advice on writing reports, which it presents to team members.

[0539] "Example 2"

[0540] In another embodiment of the present invention, the generation AI automatically generates the work status of team members and necessary advice. Specifically, the generation AI grasps the work status of team members in real time and automatically generates necessary advice. For example, when a member is performing programming work, the generation AI analyzes the member's code and generates advice pointing out improvements to the code and possible bugs. This enables team members to proceed with their work more efficiently.

[0541] The process flow of each embodiment will be described below.

[0542] "Example 1"

[0543] Step 1: A team member posts their work on a chat tool. For example, they post, "Today, we will be creating a report."

[0544] Step 2: The generation AI analyzes the content posted in the chat tool. Through this analysis, it learns that a member is writing a report.

[0545] Step 3: Based on the analysis results, the generative AI visualizes the work status of team members. Specifically, it displays information such as "Member A is writing a report" to other members on the chat tool.

[0546] Step 4: The generation AI considers past posts and the skill sets of team members, etc., and generates advice on report creation. The generated advice is presented to team members. "Form Example 2"

[0547] Step 1: Team members start working. In this example, we assume that a team member starts programming work.

[0548] Step 2: The generative AI grasps the member's work status in real time. Specifically, it analyzes the code that the member is writing.

[0549] Step 3: Based on the results of the code analysis, the generative AI generates advice pointing out areas for improvement and potential bugs.

[0550] Step 4: The generative AI presents the generated advice to team members, enabling them to work more efficiently.

[0551] Example 1

[0552] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0553] In remote work and distributed teams, it is difficult to know in real time what team members are currently doing and what advice they need. In addition, there is a lack of ways for each member to share with other members what work they are doing, making efficient communication and cooperation difficult. There is a need to improve this situation and improve the productivity of the entire team.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0555] In this invention, the server includes a means for a user to post to the communication tool, a means for the server to acquire the posted content, a means for the server to transmit the posted content to the generative AI model, a means for the generative AI model to analyze the posted content, a means for the generative AI model to return the analysis result to the server, a means for the server to display the analysis result on the communication tool, and a means for the user to check the analysis result. This makes it possible to visualize the work currently being done by team members and any advice they need in real time, enabling efficient communication and cooperation.

[0556] "User" means an individual or organization that uses the system to post to a communication tool.

[0557] A "communication tool" is any software or platform that a user uses to send or receive messages.

[0558] The "server" is a computer system that acquires user posted content, transmits it to the generative AI model, and displays the analysis results in the communication tool.

[0559] A "generative AI model" is an artificial intelligence model that analyzes the content posted by users and generates information about work status and necessary advice.

[0560] "Posted content" refers to messages or information that a user inputs into a communication tool and sends.

[0561] "Analysis results" refer to the information and advice obtained by the generative AI model when it analyzes the posted content.

[0562] "Visualization" refers to displaying the analysis results in a form that is easy for other team members to understand.

[0563] "Remote work" is a way of working in which work is carried out from a remote location without going to a physical office.

[0564] A "prompt sentence" is an instruction sentence input to a generative AI model to perform analysis.

[0565] The present invention provides a system for improving the efficiency of communication in remote work and distributed teams, and visualizing the work status of team members and necessary advice in real time. A specific embodiment of the system will be described below.

[0566] Hardware and Software Used

[0567] Hardware: Servers, terminals (PCs, smartphones, etc.)

[0568] Software: communication tools (e.g. Slack, Microsoft Teams), generative AI models (e.g. GPT-4)

[0569] System Configuration

[0570] 1. A user posts to a communication tool

[0571] A user logs into a communication tool, enters a message about work, and sends it.

[0572] Example: A user posts, "Today I will be working on a report."

[0573] 2. The server retrieves the post

[0574] The server periodically checks for and retrieves new posted content using the communication tool's API.

[0575] Example: A server calls the Slack API to get new messages.

[0576] 3. The server sends the post to the generative AI model

[0577] The server sends the acquired post content to the generative AI model as a prompt sentence.

[0578] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[0579] 4. The generative AI model analyzes the post

[0580] The generative AI model analyzes the received prompt and identifies the user's current task.

[0581] Example: A generative AI model extracts the keyword "report writing" and determines that the user is writing a report.

[0582] 5. The generative AI model returns the analysis results to the server

[0583] The generative AI model generates analysis results and sends them back to the server.

[0584] Example: A generative AI model generates information that "the user is creating a report" and "advice on creating the report," and returns this to the server.

[0585] 6. The server displays the analysis results in the communication tool.

[0586] The server displays the analysis results received from the generative AI model on the communication tool.

[0587] Example: The server posts a message to a communication tool saying, "A user is creating a report. When creating the report, it may be a good idea to refer to past project reports."

[0588] 7. The user checks the analysis results

[0589] The user checks the analysis results and advice displayed on the communication tool.

[0590] Example: A user reviews "report writing advice" and modifies their actions as necessary.

[0591] The system provides real-time visibility into what team members are currently doing and what advice they need, enabling efficient communication and collaboration.

[0592] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0593] Step 1:

[0594] A user posts to a communication tool.

[0595] Specific behavior: A user logs into a communications tool, types and sends a work-related message.

[0596] Input: A message typed by the user (e.g. "Today I will be writing a report").

[0597] Output: The message posted to the conversations tool.

[0598] Step 2:

[0599] The server retrieves the post.

[0600] Specific operation: The server uses the communication tool's API to periodically check for and retrieve new post content.

[0601] Input: The message posted in the conversations tool.

[0602] Output: The post retrieved by the server.

[0603] Step 3:

[0604] The server sends the post to the generative AI model.

[0605] Specific operation: The server sends the post content obtained as a prompt to the generative AI model.

[0606] Input: The post content retrieved by the server.

[0607] Output: The prompt sent to the generative AI model.

[0608] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[0609] Step 4:

[0610] A generative AI model analyzes the content of the post.

[0611] Specific behavior: The generative AI model analyzes the received prompt and identifies the user's current task.

[0612] Input: The prompt sent to the generative AI model.

[0613] Output: Analysis results (user actions and advice).

[0614] Step 5:

[0615] The generative AI model returns the analysis results to the server.

[0616] Specific operation: The generative AI model generates analysis results and sends them back to the server.

[0617] Input: Analysis results from a generative AI model.

[0618] Output: Analysis results sent back to the server.

[0619] Step 6:

[0620] The server displays the analysis results on the communication tool.

[0621] Specific operation: The server displays the analysis results received from the generative AI model on the communication tool.

[0622] Input: The analysis result sent back to the server.

[0623] Output: Analysis results displayed in a communication tool.

[0624] Example: Post a message in a communication tool saying, "A user is creating a report. When creating the report, it would be a good idea to refer to past project reports."

[0625] Step 7:

[0626] The user checks the analysis results.

[0627] Specific action: The user checks the analysis results and advice displayed on the communication tool.

[0628] Input: Analysis results displayed in communication tool.

[0629] Output: Analysis results and advice reviewed by the user.

[0630] (Application example 1)

[0631] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0632] With conventional work management systems in factories, it was difficult to grasp the working status of each machine in real time, and it was not possible to provide efficient work methods or points of caution appropriately. In addition, sharing of work status and provision of advice between team members was often done manually, making efficient work management difficult.

[0633] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for inputting a post on a communication tool to the generating AI, a means for visualizing the work currently being performed by team members and advice that will be required in the future, a means for making it possible to know what work other members are performing and what advice should be given, a means for visualizing the work status of machines operating in a factory in real time, and a means for providing efficient work methods and points of caution using the generating AI. This makes it possible to grasp the work status in a factory in real time and provide efficient work methods and points of caution appropriately.

[0634] A "communication tool" is software or an application that enables the exchange of information between users.

[0635] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate appropriate output.

[0636] "Visualization" is the process of displaying data or information visually so that it can be easily understood by users.

[0637] "Machinery operating in a factory" refers to automated equipment and robots used in a factory to carry out manufacturing operations.

[0638] "Real-time" refers to data and information being processed instantly and provided without delay.

[0639] An "efficient work method" is the optimal procedure or method for completing a task with the least amount of time and effort.

[0640] "Points to note" are matters or points that require special attention when performing work.

[0641] As a form for implementing this invention, a system will be constructed that visualizes the working status of machines operating in a factory in real time and uses generative AI to provide efficient work methods and points to note.

[0642] System Configuration

[0643] 1. Hardware:

[0644] Machines operating in factories (robots, etc.)

[0645] Administrator's PC or tablet

[0646] 2. Software:

[0647] Communication tools (e.g. Slack)

[0648] Generative AI (e.g. OpenAI GPT-3)

[0649] Data processing and calculation

[0650] Data Acquisition

[0651] The server receives real-time data on the operation status of machines operating in the factory, collected from the machine's sensors and log files.

[0652] 4. Data Analysis

[0653] The server posts the acquired work status data to the communication tool. The generation AI analyzes the posted content and understands the work status.

[0654] Visualization and Advice

[0655] The server visualizes the work status in real time based on the analysis results of the Generative AI. The Generative AI also automatically generates efficient work methods and points to note, taking into account past data and the skillset of the machine, and provides these to managers and other machines.

[0656] Examples

[0657] For example, if a post is made on a communication tool within a factory saying, "Robot A has started assembling parts," the generative AI would analyze this information and generate advice such as, "Robot A is assembling parts. According to past data, efficiency can be improved by paying attention to the order in which the parts are installed."

[0658] Examples of prompt statements

[0659] Please analyze the following tasks and provide your advice.

[0660] Task: Robot A starts assembling parts.

[0661] In this way, it is possible to grasp the work status within the factory in real time and provide efficient work methods and points of caution appropriately.

[0662] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0663] Step 1:

[0664] The server obtains data on the operation status from the machines operating in the factory in real time. The input is data from the machine's sensors and log files, and the output is the obtained operation status data. Specifically, the server collects data such as temperature, pressure, and operating status from the machine's sensors.

[0665] Step 2:

[0666] The server posts the acquired work status data to the communication tool. The input is the work status data acquired in step 1, and the output is the content to be posted to the communication tool. Specifically, the server converts the data into text format and posts it using the API of the communication tool.

[0667] Step 3:

[0668] The generation AI analyzes the content posted to the communication tool. The input is the content posted to the communication tool, and the output is the analysis result. In concrete terms, the generation AI analyzes the posted content using natural language processing technology and grasps the work status.

[0669] Step 4:

[0670] The server visualizes the work status in real time based on the analysis results of the generative AI. The input is the analysis results of the generative AI, and the output is the visualized work status. Specifically, the server displays the analysis results on a dashboard or monitor.

[0671] Step 5:

[0672] Generative AI automatically generates efficient work methods and points to note by taking into account past data and the skillset of the machine. The input is past data and the skillset of the machine, and the output is efficient work methods and points to note. In concrete terms, generative AI retrieves past work history from the database and generates the optimal work method.

[0673] Step 6:

[0674] The server provides the efficient work methods and cautions generated by the generative AI to managers and other machines. The input is the advice generated by the generative AI, and the output is a notification to the manager or other machines. Specifically, the server converts the advice into text format and notifies the user through a communication tool or dashboard.

[0675] Example 2

[0676] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0677] In remote work and distributed teams, it is difficult to grasp the work status of team members in real time and provide appropriate advice. In particular, it is difficult for other members to understand what work each member is doing and what kind of advice is needed, which can hinder efficient work progress and quality improvement.

[0678] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0679] In this invention, the server includes means for collecting task data from the terminals of the team members, means for analyzing the collected data, means for generating advice based on the analysis results, means for notifying the terminals of each member of the generated advice, and means for letting each member know what tasks the other members are performing and what advice to give them. This makes it possible to grasp the task status of team members in real time and provide appropriate advice.

[0680] "Team members" refers to two or more individuals who come together to work on a project or task.

[0681] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[0682] "Work Data" means information related to work being performed by a Team Member, including code, logs, notes, etc.

[0683] "Means for collecting" refers to a method or device for acquiring work data from a terminal and transmitting it to a server.

[0684] "Means for analyzing" refers to methods and devices for analyzing and evaluating collected work data.

[0685] "Means for generating advice" refers to a method or device for suggesting improvements or modifications based on the results of the analysis.

[0686] The "means for notifying" refers to a method or device for transmitting the generated advice to the terminals of the team members.

[0687] "Visualization means" refers to a method or device for visually displaying what work other members are doing and what advice is needed.

[0688] "Generative AI" refers to artificial intelligence that uses machine learning models to analyze data and automatically generate advice.

[0689] "Communication tools" refers to software or platforms that allow team members to share information and stay in touch.

[0690] "Remote work" refers to a way of working in which work is carried out via the internet from a physically distant location.

[0691] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0692] The present invention is a system for grasping the working status of team members in real time and automatically generating necessary advice. A specific embodiment of this system will be described below.

[0693] 1. Generate the system program

[0694] The server uses a generative AI model to understand the work status of team members in real time and generate a program that automatically generates the necessary advice. This program is implemented using Python and utilizes machine learning libraries such as TensorFlow and PyTorch.

[0695] 2. Explain the program's processing in natural language

[0696] The server collects data from each team member's device to monitor the work status of team members in real time. This data includes information from the IDE (integrated development environment) and editor (e.g. Visual Studio Code, PyCharm) used by the team members. The device periodically sends the work data to the server.

[0697] The server analyzes the collected data and inputs it into the generative AI model. The generative AI model evaluates the work status of each member based on the collected data and generates necessary advice. For example, if a member is working on programming, the generative AI model analyzes the member's code and generates advice pointing out areas to improve the code and possible bugs. This advice is sent to the member's device in real time.

[0698] 3. Add concrete examples

[0699] As a concrete example, let's say a member is developing a web application in Python. As the member writes code using Visual Studio Code, the server collects the code in real time and inputs it into the generative AI model. The generative AI model evaluates the quality of the code and generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the scope of the variable."

[0700] Example prompt:

[0701] "Analyze this code, written in Python, and generate suggestions for improvements or potential bugs."

[0702] In this way, generative AI models help team members work more efficiently and improve quality.

[0703] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0704] Step 1: Data collection

[0705] The server collects work data from each team member's device. The device obtains the code and work log in real time from the IDE or editor (e.g., Visual Studio Code, PyCharm) used by the team member and sends it to the server. The input is the code and work log sent from the device, and the output is the work data stored on the server.

[0706] Step 2: Data analysis

[0707] The server analyzes the collected work data. Specifically, it runs Python scripts and uses machine learning libraries such as TensorFlow and PyTorch to evaluate the quality of the code and possible bugs. The input is the work data stored on the server, and the output is the analysis results, for example checking function redundancy and variable scope.

[0708] Step 3: Advice Generation

[0709] The server uses a generative AI model to generate advice based on the analysis results. The generative AI model takes the analysis results as input and outputs advice including specific improvements and correction methods. For example, it generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the variable scope."

[0710] Step 4: Advice Notification

[0711] The server notifies each member's terminal of the generated advice. The terminal receives the notification and displays it as a pop-up message to the user. The input is the generated advice, and the output is the advice displayed to the user. The user checks the displayed advice and modifies the code.

[0712] In this way, generative AI models help team members work more efficiently and improve quality.

[0713] (Application example 2)

[0714] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0715] Conventional communication tools and generative AI systems can visualize the work status of team members and necessary advice, but they have the problem that it is difficult to provide specific advice in real time to improve work efficiency in the factory. In addition, there is a lack of systems that can monitor the work status in the factory in real time and suggest optimal operation patterns and improvements. This has led to problems such as a decrease in work efficiency in the factory and a lack of improvement in productivity.

[0716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0717] In this invention, the server includes a means for inputting posts from the communication tool to the generation AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for making it possible to know what work other members are doing and what advice should be given, a means for monitoring the work situation in the factory in real time and automatically generating advice for improving work efficiency, and a means for displaying the generated advice. This makes it possible to provide specific advice for improving work efficiency in the factory in real time.

[0718] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[0719] "Generative AI" is a system that uses artificial intelligence technology to analyze data and automatically generate necessary information and advice.

[0720] "Visualization" is the process of displaying data or information visually and making it easier to understand.

[0721] "Real-time" refers to processing ongoing events and situations immediately and providing results.

[0722] "Work efficiency" is an index that indicates how much work can be effectively performed within a certain amount of time.

[0723] "Advice" is guidance or a suggestion given to address a particular situation or problem.

[0724] "Surveillance" is the act of continuously observing a specific object or situation and collecting necessary information.

[0725] A "behavior pattern" refers to a series of actions or steps in a specific task or process.

[0726] "Areas for improvement" are specific proposals or methods for identifying current problems or inefficiencies and making them better.

[0727] "Display" refers to showing information or data visually, often using a device such as a screen or display.

[0728] A system for implementing the present invention provides specific advice in real time to improve work efficiency in a factory. The system includes a camera, a generative AI model, and a display device.

[0729] First, a camera (e.g., Logitech C920) captures the work situation in the factory in real time. The captured video is sent to a server. The server analyzes the video using a generative AI model (e.g., a model built with TensorFlow or PyTorch). The generative AI model analyzes the work situation in the video and automatically generates optimal operation patterns and areas for improvement.

[0730] The generated advice is displayed on a display device (e.g., a monitor or smart glasses), allowing workers in the factory to receive the advice in real time and improve their work efficiency.

[0731] As a concrete example, consider a case where a robot is tightening screws in a factory. The generative AI model analyzes how and at what speed the screws are tightened, and suggests the optimal tightening method and speed. For example, by inputting a prompt such as "Analyze the current work situation and suggest the optimal tightening method and speed for the screws" into the generative AI model, specific advice can be obtained.

[0732] This system makes it possible to provide specific advice in real time to improve work efficiency within the factory.

[0733] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0734] Step 1:

[0735] Cameras capture the work situation inside the factory in real time.

[0736] Input: Work status in the factory

[0737] Output: Captured video data

[0738] Specific operation: A camera (e.g., Logitech C920) captures work being done in the factory and sends the video data to a server.

[0739] Step 2:

[0740] The server receives the captured video data and inputs it into the generative AI model.

[0741] Input: Captured video data

[0742] Output: Data input to the generative AI model

[0743] How it works: The server receives the video data and preprocesses it for input into a generative AI model (e.g., a model built with TensorFlow or PyTorch).

[0744] Step 3:

[0745] A generative AI model analyzes the video data and evaluates the work situation.

[0746] Input: Preprocessed video data

[0747] Output: Evaluation result of the work situation

[0748] How it works: A generative AI model analyzes video data to evaluate the progress and efficiency of a task, for example analysing how and how fast a screw is tightened.

[0749] Step 4:

[0750] The generative AI model automatically generates optimal movement patterns and areas for improvement.

[0751] Input: Evaluation result of work status

[0752] Output: Optimal operation patterns and advice on areas for improvement

[0753] Specific behavior: The generative AI model generates optimal behavior patterns and improvements based on the evaluation results. For example, it uses a prompt sentence such as, "Analyze the current work situation and suggest the optimal way and speed to tighten the screws."

[0754] Step 5:

[0755] The server displays the generated advice on a display device.

[0756] Input: Optimal behavior patterns and suggestions for improvement

[0757] Output: Advice displayed on your display device

[0758] Specific operation: The server receives the generated advice and displays it on a display device (e.g., a monitor or smart glasses). The worker can check the advice in real time to improve work efficiency.

[0759] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0760] "Example 1"

[0761] One embodiment of the present invention is a system that combines an emotion engine. This system recognizes the user's emotions and adjusts advice based on the user's emotional state. Specifically, if the user feels stressed, the emotion engine communicates this information to the generation AI. The generation AI then adjusts the advice to the user based on this information. For example, the system provides advice to a user who is stressed that is kinder or includes encouraging words.

[0762] "Example 2"

[0763] In addition, the emotion engine visualizes the user's emotional state to other team members. This enables team members to understand the emotional state of other members and communicate appropriately. Specifically, the emotion engine visualizes the user's emotional state on a chat tool. For example, if a user feels happy, this information is displayed on the chat tool as "Member A is happy." This allows other members to know that Member A is happy and to communicate based on that information.

[0764] The process flow of each embodiment will be described below.

[0765] "Example 1"

[0766] Step 1: Users communicate using a chat tool.

[0767] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[0768] Step 3: The emotion engine communicates the recognized emotional state to the generative AI.

[0769] Step 4: The generative AI tailors the advice to the user based on their emotional state.

[0770] "Example 2"

[0771] Step 1: Users communicate using a chat tool.

[0772] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[0773] Step 3: The emotional state recognized by the emotion engine is visualized on the chat tool.

[0774] Step 4: Other team members see the visualized emotional state and communicate based on it.

[0775] Example 1

[0776] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0777] In remote work and distributed teams, it is difficult to grasp the work status of team members and the advice they need in real time. It is also difficult to provide appropriate advice that takes into account the emotional state of members. This can lead to reduced communication efficiency and affect the productivity of the entire team.

[0778] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for inputting a post of a communication tool to a generative AI model, a means for the generative AI model to analyze the posted content, a means for visualizing the work currently being performed by team members and advice that will be required in the future based on the analysis result, and a means for recognizing the user's emotions and adjusting advice based on the emotional state. This makes it possible to grasp the work status and necessary advice of team members in real time and provide appropriate advice according to the emotional state.

[0779] A "communication tool" is software or a platform that enables users to send and receive messages.

[0780] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing to analyze text data and generate appropriate responses or advice.

[0781] "Posted content" refers to messages and text data sent by users on communication tools.

[0782] "Analysis" is the process by which the generative AI model understands the content of a post and extracts meaning from it.

[0783] "Visualization" is the process of visually displaying information based on analysis results so that users can easily understand it.

[0784] An “emotion engine” is software or algorithms for recognizing a user’s emotional state and providing that information to a generative AI model.

[0785] "Advice" is advice or suggestions that the generative AI model provides to the user based on the analysis results and emotional state.

[0786] "Remote work" refers to working in which work is carried out via the Internet from a physically distant location.

[0787] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0788] This invention is a system for streamlining communication in remote work and distributed teams, and understanding the work status of team members and necessary advice in real time. This system is realized by combining communication tools, generative AI models, and emotion engines.

[0789] Hardware and Software Used

[0790] Hardware: Servers, terminals (PCs, smartphones)

[0791] Software: communication tools (e.g. Slack, Microsoft Teams), generative AI models (e.g. GPT-4), emotion engines

[0792] System Configuration

[0793] 1. User Submissions

[0794] A user uses a terminal to post a message on a communication tool. For example, the user posts, "I will be writing a report today."

[0795] 2. Acquiring posted content

[0796] The server uses the communication tool's API to obtain user posts in real time.

[0797] 3. Analysis of posted content

[0798] The server sends the acquired post content to the generative AI model for analysis. The generative AI model understands the post content and recognizes that the user is creating a report.

[0799] 4. Information Visualization

[0800] Based on the analysis results from the generative AI model, the server makes the information that the relevant member is writing a report visible to other team members. Specifically, it displays "Mr. / Ms. XX is writing a report" on the chat tool.

[0801] 5. Generating Advice

[0802] The generative AI model takes into account past posts and the skill sets of team members to generate advice on report creation. The server presents the advice to team members.

[0803] 6. Use of Emotion Engines

[0804] The emotion engine recognizes the user's emotions and conveys that information to the generative AI model. For example, if the emotion engine determines that the user is feeling stressed, it will send that information to the generative AI model.

[0805] The generative AI model uses information from the emotion engine to tailor its advice to users, for example providing gentler or more encouraging advice to a user who is feeling stressed.

[0806] Examples

[0807] User submission examples

[0808] I will be writing a report today.

[0809] Example prompts for generative AI models

[0810] "A user posts, 'Today I'm going to create a report.' Use this information to make it visible to other members that the user is creating a report, and generate advice on creating the report."

[0811] Advice examples

[0812] "When preparing a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports."

[0813] Example of using the emotion engine

[0814] The emotion engine detects the user's stress and passes that information to the generative AI model.

[0815] The generative AI model generates gentle advice such as, "When writing a report, it is most efficient to first consider the overall structure. Take your time and take breaks as you go," and returns this advice to the server.

[0816] In this way, the system can analyze the content posted by users, visualize the work status and necessary advice of team members in real time, and provide appropriate advice according to their emotional state.

[0817] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0818] Step 1:

[0819] A user uses a terminal to post a message on a communication tool. The input is the user's message, and the output is the message sent to the communication tool. A specific example of this is when a user accesses a chat tool from a PC or smartphone, types "I will be writing a report today," and sends it.

[0820] Step 2:

[0821] The server uses the API of the communication tool to obtain user posts in real time. The input is the message data obtained from the communication tool, and the output is the message data stored on the server. Specifically, the server calls the Slack API and obtains new messages from a specific channel.

[0822] Step 3:

[0823] The server sends the acquired post content to the generative AI model for analysis. The input is the message data stored on the server, and the output is the analysis result returned from the generative AI model. In concrete terms, the server sends the message "Today I will create a report" as a prompt to the generative AI model, and receives the analysis result.

[0824] Step 4:

[0825] Based on the analysis results from the generative AI model, the server visualizes to other team members the information that the relevant member is writing a report. The input is the analysis results from the generative AI model, and the output is the information displayed in the chat tool. In concrete terms, the server automatically posts to the chat tool, "Mr. / Ms. XX is writing a report."

[0826] Step 5:

[0827] The generative AI model takes into account the content of past posts and the skill sets of members, and generates advice on creating reports. The input is the content of past posts and the skill sets of members, and the output is the generated advice. In concrete terms, the generative AI model generates advice such as "When creating a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports," and returns this advice to the server.

[0828] Step 6:

[0829] The emotion engine recognizes the user's emotions and transmits that information to the generative AI model. The input is the user's emotional state, and the output is the emotional information sent to the generative AI model. Specifically, the emotion engine determines that the user is feeling stressed based on the content of the user's posts and facial expression recognition data, and sends that information to the generative AI model.

[0830] Step 7:

[0831] The generative AI model adjusts advice to the user based on information from the emotion engine. The input is emotion information from the emotion engine, and the output is adjusted advice. In concrete terms, the generative AI model generates gentle advice such as, "When writing a report, it is efficient to first consider the overall structure. Don't push yourself, and take breaks as you go along," and returns this advice to the server.

[0832] (Application example 1)

[0833] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0834] In order to improve work efficiency in factories, it is important to facilitate communication between workers and robots and to provide appropriate information about the work situation and necessary advice. However, with conventional systems, it was difficult to provide advice that took into account the emotional state of the worker, which limited the improvement of work efficiency. In addition, it was difficult for workers to give instructions to the robot and the robot's work status was not sufficiently visualized, making it difficult to manage the progress of work.

[0835] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0836] In this invention, the server includes a means for inputting a post made via a communication tool to the generating AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for letting the user know what work other members are doing and what advice to give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot via a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, thereby improving work efficiency and facilitating progress management.

[0837] A "communication tool" is software or applications that allow users to exchange information through text or voice.

[0838] "Generative AI" is an artificial intelligence model that analyzes input data and generates appropriate output.

[0839] "Visualization" refers to displaying data or information visually so that users can easily understand it.

[0840] An "emotion engine" is a system that recognizes the user's emotional state and provides appropriate feedback based on that information.

[0841] A "robot" is a mechanical device that performs tasks automatically according to programmed instructions.

[0842] A "worker" is a human worker who performs a specific task in a factory.

[0843] An "instruction" is an order to perform a particular action or task.

[0844] "Analysis" refers to examining the input data in detail and understanding its meaning and structure.

[0845] "Advice" is advice or a suggestion intended to encourage appropriate action or decision in a particular situation.

[0846] "Progress management" is a management activity that monitors the progress of work or projects and keeps them moving forward as planned.

[0847] The system for implementing this invention includes a communication tool, a generation AI, an emotion engine, a robot, and a worker's terminal. The specific operation of each element and its cooperation will be described below.

[0848] System Configuration

[0849] 1. Communication tools:

[0850] Software or applications that allow workers to exchange information through text or voice.

[0851] Workers use chat tools to give instructions to the robot.

[0852] 2. Generation AI:

[0853] It uses generative AI models such as the OpenAI API to analyze chat content as it is typed.

[0854] Based on the analysis results, appropriate work instructions are issued to the robot.

[0855] Automatically generate work status and necessary advice.

[0856] 3. Emotion Engine:

[0857] Use emotion recognition libraries such as EmotionEngine to recognize the emotional state of workers.

[0858] Tailor the advice provided by the generative AI based on the perceived emotional state.

[0859] 4. Robots:

[0860] It is a mechanical device that performs tasks automatically according to programmed instructions.

[0861] It receives instructions from the generative AI and carries out the appropriate tasks.

[0862] 5. Worker terminal:

[0863] These are devices used by workers, such as smartphones and tablets.

[0864] Instructions are given to the robot through a communication tool and advice is received from the generating AI.

[0865] System Operation

[0866] The server includes a means for inputting posts from the communication tool to the generating AI, a means for visualizing the work that team members are currently doing and the advice they will need in the future, a means for letting the user know what work other members are doing and what advice they should give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot using a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, improving work efficiency and facilitating progress management.

[0867] Examples

[0868] When a worker types "I'm going to write a report today" into the chat tool, the generative AI analyzes the content and instructs the robot to support the report writing. At the same time, the emotion engine recognizes the worker's emotional state and provides advice including encouraging words such as "Do your best! You can do it!" if the worker is feeling stressed.

[0869] Examples of prompt statements

[0870] Please analyze the following chat and provide your suggestions:

[0871] I'm going to create a report today.

[0872] In this way, the work efficiency in the factory can be improved.

[0873] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0874] Step 1:

[0875] A user inputs text into a chat tool using a terminal. For example, the user inputs "Today I will be writing a report." This text becomes input data into the system.

[0876] Step 2:

[0877] The server receives text entered from the chat tool. It generates a prompt sentence to send the received text to the generative AI model. This prompt sentence becomes the input data for the generative AI model to analyze.

[0878] Step 3:

[0879] The server sends a prompt to the generative AI model, which analyzes the text. The generative AI model analyzes the input text and generates work content and advice. The analysis results become the system's output data.

[0880] Step 4:

[0881] The server receives the analysis results from the generative AI model, which include action content (creation of a report) and advice. The server visualizes this information and shares it with other team members.

[0882] Step 5:

[0883] The server uses an emotion engine to recognize the user's emotional state. Based on chat content and past data, it determines whether the user is feeling stressed. This emotional state becomes the input data for the system.

[0884] Step 6:

[0885] The server receives the emotional state from the emotion engine and adjusts the advice from the generative AI model. For example, if the user is feeling stressed, it adds words of encouragement to the advice. This adjusted advice becomes the output data of the system.

[0886] Step 7:

[0887] The server sends the adjusted advice to the user's device, and the user receives the advice from the generative AI model and the adjustments made by the emotion engine through the device.

[0888] Step 8:

[0889] The user issues instructions to the robot using a terminal. The server receives the instructions and sends them to the generative AI model. The generative AI model analyzes the instructions and generates instructions for the robot to perform the appropriate tasks.

[0890] Step 9:

[0891] The server receives instructions from the generative AI model and sends them to the robot, which then follows the instructions to perform a task, such as writing a report. The progress of this task becomes the output data of the system.

[0892] Step 10:

[0893] The server receives the progress of the robot's work and makes it visible to the user through a chat tool. The user can then check the robot's progress through their device.

[0894] Example 2

[0895] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0896] In remote work and distributed teams, it is difficult to grasp the work situation and emotional state of team members in real time and provide appropriate advice and communication. In particular, communication between members tends to be lacking, which can lead to a decrease in work efficiency and team unity.

[0897] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0898] In this invention, the server includes a means for a user to start a task on a terminal, a means for the terminal to transmit task data to the server, a means for the server to analyze the data received, a means for the server to generate advice using a generative AI model, a means for the server to analyze and visualize an emotional state using an emotion engine, a means for the server to transmit the advice and emotional information generated by the server to the terminal, and a means for displaying the information received by the terminal to the user. This makes it possible to grasp the task status and emotional state of team members in real time and provide appropriate advice and communication.

[0899] "User" refers to an individual or group that performs work using the system.

[0900] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[0901] "Server" refers to a central processing unit that receives, analyzes, generates, and transmits data.

[0902] "Task data" refers to information related to the task a user is performing (e.g., programming code or emotion data).

[0903] A "generative AI model" refers to an artificial intelligence model that analyzes a user's work data and generates appropriate advice.

[0904] An "emotion engine" refers to software or hardware for analyzing a user's emotional state and visualizing the results.

[0905] "Advice" refers to suggestions for improvement or comments that the generative AI model generates based on the user's work data.

[0906] "Emotion information" refers to information about the user's emotional state analyzed by the emotion engine.

[0907] "Visualization" refers to displaying analyzed data in a form that is easy for users to understand.

[0908] "Communication tools" refers to software or platforms for sharing information among team members.

[0909] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0910] This invention is a system for grasping the work situation and emotional state of team members in real time in remote work and distributed teams, and providing appropriate advice and communication. Below, we will show how to specifically implement this system.

[0911] 1. Generate the system program

[0912] This system combines a generative AI model and an emotion engine to grasp the work situation and emotional state of team members in real time and provide necessary advice and emotional information. For example, OpenAI's GPT-4 is used as the generative AI model, and the emotion engine uses Microsoft's Azure Cognitive Services Emotion API.

[0913] 2. Explain the program's processing in natural language

[0914] The server hosts a generative AI model and an emotion engine. The generative AI model analyzes the user's work data (e.g., programming code) and generates appropriate advice. The emotion engine analyzes the user's emotional state (e.g., facial expressions and voice) and visualizes the results.

[0915] The terminals are PCs or smartphones used by users, and send data to the server. Specifically, the terminals send the user's task data and emotion data to the server.

[0916] The server analyzes the data it receives and uses a generative AI model to generate advice. For example, if a member is working on a program, the server analyzes the member's code and generates advice on how to improve the code or identify potential bugs.

[0917] The emotion engine also analyzes the user's emotional state and visualizes the results on the chat tool. For example, if the emotion engine determines that the user is happy, the server will display "Member A is happy" on the chat tool.

[0918] 3. Add examples to your description

[0919] As a concrete example, consider the following scenario.

[0920] Scenario 1: Programming task advice

[0921] User A is programming on a device. User A's device sends the code he is working on to the server. The server analyzes the code using a generative AI model (GPT-4) and generates advice such as the following:

[0922] "This part of the code is inefficient and can be improved as follows."

[0923] "There may be a bug in this line. Details below."

[0924] Example prompt:

[0925] Analyze User A's code and generate advice pointing out areas for improvement and potential bugs.

[0926] Code: [User A's code]

[0927] Scenario 2: Visualizing Emotional States

[0928] User B is working on a device. The emotion engine analyzes User B's facial expression and determines that User B is happy. The server displays this information on the chat tool.

[0929] "Member B is happy."

[0930] Example prompt:

[0931] Analyze User B's emotional state and visualize it on the chat tool.

[0932] Emotional state: Joy

[0933] This system enables users to efficiently carry out their own tasks and to understand the emotional states of other members and communicate appropriately.

[0934] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0935] Step 1:

[0936] The user begins working at the terminal.

[0937] Input: The device used by the user (PC or smartphone)

[0938] Output: Working data (programming code and emotion data)

[0939] Specific operation: A user starts programming or other work on his / her PC or smartphone. For example, User A opens Visual Studio Code and starts programming.

[0940] Step 2:

[0941] The terminal transmits the work data to the server.

[0942] Input: User-generated task data (programming code and emotion data)

[0943] Output: The work data sent to the server.

[0944] Specific operation: User A's device transmits the programming code written in real time and facial expression data captured by the webcam to the server.

[0945] Step 3:

[0946] The server parses the received data.

[0947] Input: Working data sent from the terminal (programming code and emotion data)

[0948] Output: Analysis results (code structure and content, emotional state)

[0949] Specific operation: The server analyzes the received programming code and understands the structure and content of the code. It also analyzes User A’s facial expressions from the webcam footage to understand his / her emotional state.

[0950] Step 4:

[0951] The server generates advice using a generative AI model.

[0952] Input: Parsed programming code

[0953] Output: Generated advice

[0954] Specific operation: The server uses a generative AI model (GPT-4) to generate advice for User A's programming code. For example, it generates specific advice such as "This loop is inefficient, so it would be better to use a list comprehension."

[0955] Step 5:

[0956] The server uses an emotion engine to analyze and visualize the emotional state.

[0957] Input: Parsed emotion data

[0958] Output: Visualized emotion information

[0959] Specific operation: The server uses the emotion engine (Emotion API) to determine that User A is happy. The server visualizes this information on the chat tool as "Member A is happy."

[0960] Step 6:

[0961] The server transmits the generated advice and emotion information to the terminal.

[0962] Input: Generated advice and visualized emotion information

[0963] Output: Advice and sentiment sent to the device

[0964] Specific operation: The server sends the generated advice and emotion information to the terminal of user A. For example, the server sends advice such as "This loop is inefficient, so it would be better to use a list comprehension" and emotion information such as "Member A is happy."

[0965] Step 7:

[0966] The terminal displays the received information to the user.

[0967] Input: Advice and sentiment information sent from the server

[0968] Output: Advice and emotion information displayed to the user

[0969] Specific operation: User A's device displays advice on Visual Studio Code and emotional information on the chat tool. For example, "This loop is inefficient, so it would be better to use a list comprehension" is displayed in the Visual Studio Code editor, and "Member A is happy" is displayed in the chat tool.

[0970] (Application example 2)

[0971] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0972] In conventional teamwork, it is difficult to grasp the working status of team members and necessary advice in real time, which often hinders efficient communication. In addition, there is no way to grasp the emotional state of team members, making it difficult to communicate appropriately. This can lead to a decrease in work efficiency and friction within the team.

[0973] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0974] In this invention, the server includes a means for inputting posts from a communication tool to a generation AI, a means for visualizing the work that team members are currently doing and advice that they will need in the future, a means for letting users know what work other members are doing and what advice they should give, a means for recognizing and visualizing the emotional state of users in real time, and a means for generating and displaying necessary advice based on the work situation. This makes it possible to grasp the work situation and emotional state of team members in real time and provide appropriate advice.

[0975] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[0976] "Generative AI" is an artificial intelligence system that automatically generates advice and information based on input data.

[0977] "Visualization" is a means of visually displaying data or information to make it easier to understand.

[0978] "User's emotional state" refers to the emotions and psychological state that the user is currently feeling.

[0979] "Real-time" refers to processing and display occurring immediately, without delay.

[0980] "Work status" refers to the progress and content of the work that team members are currently doing.

[0981] "Advice" means advice or a suggestion given to improve a task or solve a problem.

[0982] A "display" is a means of visually presenting information to a user.

[0983] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The server uses a generative AI model to automatically generate the work status and necessary advice of team members, and has the function of recognizing and visualizing the emotional state of the user in real time. The terminal is a device such as smart glasses or a smartphone, and serves to provide this information to the user. The user receives advice and feedback of the emotional state in real time through these devices.

[0984] The server uses the following software and hardware:

[0985] OpenCV: A library for image processing

[0986] EmotionRecognizer: Software for Recognizing a User's Emotional State

[0987] CodeAnalyzer: software to analyze the work of team members

[0988] AIAdvisor: An artificial intelligence system for generating needed advice

[0989] The terminals are devices such as smart glasses or smartphones, which are equipped with cameras and displays. The cameras capture images of the user and send them to a server. The server analyzes the images and recognizes the user's emotional state and work situation. The recognized information is converted into appropriate advice using a generative AI model and displayed on the terminal's display.

[0990] As a concrete example, consider a scenario in which an employee working in a factory wears smart glasses and receives advice in real time based on the work situation and emotional state. While the employee is working, the camera in the smart glasses captures video and sends it to a server. The server analyzes the video and recognizes the employee's emotional state. At the same time, it analyzes the progress of the work the employee is doing and generates the necessary advice. This information is displayed in real time on the display of the smart glasses.

[0991] Example of a prompt to be input to a generative AI model:

[0992] "Employee A puts on smart glasses, which analyze the code he is working on and gives him advice. His emotional state is also recognized in real time and shared with his team members."

[0993] In this way, it is possible to improve the work efficiency and communication of team members.

[0994] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0995] Step 1:

[0996] The camera on the device (smart glasses) captures an image of the user.

[0997] Input: Real-time video of the user

[0998] Output: Captured video data

[0999] Specific operation: The camera in the smart glasses photographs the user's face and work environment and captures the video data.

[1000] Step 2:

[1001] The video data acquired by the terminal is transmitted to the server.

[1002] Input: Acquired video data

[1003] Output: Video data sent to the server

[1004] Specific operation: The smart glasses upload video data to the server using communication methods such as Wi-Fi or Bluetooth.

[1005] Step 3:

[1006] The server receives the video data and analyzes the user's emotional state using an emotion recognition engine (EmotionRecognizer).

[1007] Input: Video data sent to the server

[1008] Output: Parsed emotional state data

[1009] Specific operation: The server inputs the video data into EmotionRecognizer, which estimates the user's emotional state from their facial expressions and movements.

[1010] Step 4:

[1011] The server uses a code analysis engine (CodeAnalyzer) to analyze the user's work.

[1012] Input: Data about what the user is working on (e.g. code files)

[1013] Output: Analyzed work status data

[1014] What it does: The server inputs the user's work (e.g., program code) into CodeAnalyzer and evaluates the code's quality and progress.

[1015] Step 5:

[1016] The server uses a generative AI model (AIAdvisor) to generate the necessary advice based on the analyzed emotional state data and work situation data.

[1017] Input: Emotional state data, work situation data

[1018] Output: Generated advice

[1019] Specific operation: The server inputs emotional state data and task situation data into the AIAdvisor and generates appropriate advice.

[1020] Step 6:

[1021] The server transmits the generated advice and emotional state to the terminal.

[1022] Input: Generated advice, emotional state data

[1023] Output: Advice and emotional state data sent to the device

[1024] Specific operation: The server uses a communication means to transmit the generated advice and emotional state data to the terminal.

[1025] Step 7:

[1026] The terminal displays the received advice and emotional state to the user.

[1027] Input: Received advice, emotional state data

[1028] Output: Advice and emotional state displayed to the user

[1029] Specific operation: The smart glasses display visually displays the received advice and emotional state to the user.

[1030] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1031] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1032] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1033] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1034] [Third embodiment]

[1035] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1036] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1037] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1038] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1039] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1040] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1041] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1042] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1043] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1044] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1045] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1046] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1047] "Example 1"

[1048] The embodiment of the present invention uses a chat tool that is commonly used as a communication tool. The content posted on the chat tool is input to the generation AI, which analyzes it. Based on the analysis results, the work that team members are currently doing and the advice they will need in the future are visualized. Specifically, if a member posts on the chat tool, "I will be writing a report today," the generation AI analyzes this information and visualizes to other members the information that the member is writing a report. In addition, the generation AI takes into account the content of past posts and the members' skill sets, etc., and generates advice on writing reports, which it presents to team members.

[1049] "Example 2"

[1050] In another embodiment of the present invention, the generation AI automatically generates the work status of team members and necessary advice. Specifically, the generation AI grasps the work status of team members in real time and automatically generates necessary advice. For example, when a member is performing programming work, the generation AI analyzes the member's code and generates advice pointing out improvements to the code and possible bugs. This enables team members to proceed with their work more efficiently.

[1051] The process flow of each embodiment will be described below.

[1052] "Example 1"

[1053] Step 1: A team member posts their work on a chat tool. For example, they post, "Today, we will be creating a report."

[1054] Step 2: The generation AI analyzes the content posted in the chat tool. Through this analysis, it learns that a member is writing a report.

[1055] Step 3: Based on the analysis results, the generative AI visualizes the work status of team members. Specifically, it displays information such as "Member A is writing a report" to other members on the chat tool.

[1056] Step 4: The generation AI considers past posts and the skill sets of team members, etc., and generates advice on report creation. The generated advice is presented to team members. "Form Example 2"

[1057] Step 1: Team members start working. In this example, we assume that a team member starts programming work.

[1058] Step 2: The generative AI grasps the member's work status in real time. Specifically, it analyzes the code that the member is writing.

[1059] Step 3: Based on the results of the code analysis, the generative AI generates advice pointing out areas for improvement and potential bugs.

[1060] Step 4: The generative AI presents the generated advice to team members, enabling them to work more efficiently.

[1061] Example 1

[1062] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1063] In remote work and distributed teams, it is difficult to know in real time what team members are currently doing and what advice they need. In addition, there is a lack of ways for each member to share with other members what work they are doing, making efficient communication and cooperation difficult. There is a need to improve this situation and improve the productivity of the entire team.

[1064] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1065] In this invention, the server includes a means for a user to post to the communication tool, a means for the server to acquire the posted content, a means for the server to transmit the posted content to the generative AI model, a means for the generative AI model to analyze the posted content, a means for the generative AI model to return the analysis result to the server, a means for the server to display the analysis result on the communication tool, and a means for the user to check the analysis result. This makes it possible to visualize the work currently being done by team members and any advice they need in real time, enabling efficient communication and cooperation.

[1066] "User" means an individual or organization that uses the system to post to a communication tool.

[1067] A "communication tool" is any software or platform that a user uses to send or receive messages.

[1068] The "server" is a computer system that acquires user posted content, transmits it to the generative AI model, and displays the analysis results in the communication tool.

[1069] A "generative AI model" is an artificial intelligence model that analyzes the content posted by users and generates information about work status and necessary advice.

[1070] "Posted content" refers to messages or information that a user inputs into a communication tool and sends.

[1071] "Analysis results" refer to the information and advice obtained by the generative AI model when it analyzes the posted content.

[1072] "Visualization" refers to displaying the analysis results in a form that is easy for other team members to understand.

[1073] "Remote work" is a way of working in which work is carried out from a remote location without going to a physical office.

[1074] A "prompt sentence" is an instruction sentence input to a generative AI model to perform analysis.

[1075] The present invention provides a system for improving the efficiency of communication in remote work and distributed teams, and visualizing the work status of team members and necessary advice in real time. A specific embodiment of the system will be described below.

[1076] Hardware and Software Used

[1077] Hardware: Servers, terminals (PCs, smartphones, etc.)

[1078] Software: communication tools (e.g. Slack, Microsoft Teams), generative AI models (e.g. GPT-4)

[1079] System Configuration

[1080] 1. A user posts to a communication tool

[1081] A user logs into a communication tool, enters a message about work, and sends it.

[1082] Example: A user posts, "Today I will be working on a report."

[1083] 2. The server retrieves the post

[1084] The server periodically checks for and retrieves new posted content using the communication tool's API.

[1085] Example: A server calls the Slack API to get new messages.

[1086] 3. The server sends the post to the generative AI model

[1087] The server sends the acquired post content to the generative AI model as a prompt sentence.

[1088] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[1089] 4. The generative AI model analyzes the post

[1090] The generative AI model analyzes the received prompt and identifies the user's current task.

[1091] Example: A generative AI model extracts the keyword "report writing" and determines that the user is writing a report.

[1092] 5. The generative AI model returns the analysis results to the server

[1093] The generative AI model generates analysis results and sends them back to the server.

[1094] Example: A generative AI model generates information that "the user is creating a report" and "advice on creating the report," and returns this to the server.

[1095] 6. The server displays the analysis results in the communication tool.

[1096] The server displays the analysis results received from the generative AI model on the communication tool.

[1097] Example: The server posts a message to a communication tool saying, "A user is creating a report. When creating the report, it may be a good idea to refer to past project reports."

[1098] 7. The user checks the analysis results

[1099] The user checks the analysis results and advice displayed on the communication tool.

[1100] Example: A user reviews "report writing advice" and modifies their actions as necessary.

[1101] The system provides real-time visibility into what team members are currently doing and what advice they need, enabling efficient communication and collaboration.

[1102] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1103] Step 1:

[1104] A user posts to a communication tool.

[1105] Specific behavior: A user logs into a communications tool, types and sends a work-related message.

[1106] Input: A message typed by the user (e.g. "Today I will be writing a report").

[1107] Output: The message posted to the conversations tool.

[1108] Step 2:

[1109] The server retrieves the post.

[1110] Specific operation: The server uses the communication tool's API to periodically check for and retrieve new post content.

[1111] Input: The message posted in the conversations tool.

[1112] Output: The post retrieved by the server.

[1113] Step 3:

[1114] The server sends the post to the generative AI model.

[1115] Specific operation: The server sends the post content obtained as a prompt to the generative AI model.

[1116] Input: The post content retrieved by the server.

[1117] Output: The prompt sent to the generative AI model.

[1118] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[1119] Step 4:

[1120] A generative AI model analyzes the content of the post.

[1121] Specific behavior: The generative AI model analyzes the received prompt and identifies the user's current task.

[1122] Input: The prompt sent to the generative AI model.

[1123] Output: Analysis results (user actions and advice).

[1124] Step 5:

[1125] The generative AI model returns the analysis results to the server.

[1126] Specific operation: The generative AI model generates analysis results and sends them back to the server.

[1127] Input: Analysis results from a generative AI model.

[1128] Output: Analysis results sent back to the server.

[1129] Step 6:

[1130] The server displays the analysis results on the communication tool.

[1131] Specific operation: The server displays the analysis results received from the generative AI model on the communication tool.

[1132] Input: The analysis result sent back to the server.

[1133] Output: Analysis results displayed in a communication tool.

[1134] Example: Post a message in a communication tool saying, "A user is creating a report. When creating the report, it would be a good idea to refer to past project reports."

[1135] Step 7:

[1136] The user checks the analysis results.

[1137] Specific action: The user checks the analysis results and advice displayed on the communication tool.

[1138] Input: Analysis results displayed in communication tool.

[1139] Output: Analysis results and advice reviewed by the user.

[1140] (Application example 1)

[1141] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1142] With conventional work management systems in factories, it was difficult to grasp the working status of each machine in real time, and it was not possible to provide efficient work methods or points of caution appropriately. In addition, sharing of work status and provision of advice between team members was often done manually, making efficient work management difficult.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for inputting a post on a communication tool to the generating AI, a means for visualizing the work currently being performed by team members and advice that will be required in the future, a means for making it possible to know what work other members are performing and what advice should be given, a means for visualizing the work status of machines operating in a factory in real time, and a means for providing efficient work methods and points of caution using the generating AI. This makes it possible to grasp the work status in a factory in real time and provide efficient work methods and points of caution appropriately.

[1144] A "communication tool" is software or an application that enables the exchange of information between users.

[1145] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate appropriate output.

[1146] "Visualization" is the process of displaying data or information visually so that it can be easily understood by users.

[1147] "Machinery operating in a factory" refers to automated equipment and robots used in a factory to carry out manufacturing operations.

[1148] "Real-time" refers to a state in which data and information are processed immediately and provided without delay.

[1149] An "efficient work method" is the optimal procedure or method for completing a task with the least amount of time and effort.

[1150] "Points to note" are matters or points that require special attention when performing work.

[1151] As a form for implementing this invention, a system will be constructed that visualizes the working status of machines operating in a factory in real time and uses generative AI to provide efficient work methods and points to note.

[1152] System Configuration

[1153] 1. Hardware:

[1154] Machines operating in factories (robots, etc.)

[1155] Administrator's PC or tablet

[1156] 2. Software:

[1157] Communication tools (e.g. Slack)

[1158] Generative AI (e.g. OpenAI GPT-3)

[1159] Data processing and calculation

[1160] Data Acquisition

[1161] The server receives real-time data on the operation status of machines operating in the factory, collected from the machine's sensors and log files.

[1162] 4. Data Analysis

[1163] The server posts the acquired work status data to the communication tool. The generation AI analyzes the posted content and understands the work status.

[1164] Visualization and Advice

[1165] The server visualizes the work status in real time based on the analysis results of the Generative AI. The Generative AI also automatically generates efficient work methods and points to note, taking into account past data and the skillset of the machine, and provides these to managers and other machines.

[1166] Examples

[1167] For example, if a post is made on a communication tool within a factory saying, "Robot A has started assembling parts," the generative AI would analyze this information and generate advice such as, "Robot A is assembling parts. According to past data, efficiency can be improved by paying attention to the order in which the parts are installed."

[1168] Examples of prompt statements

[1169] Please analyze the following tasks and provide your advice.

[1170] Task: Robot A starts assembling parts.

[1171] In this way, it is possible to grasp the work status within the factory in real time and provide efficient work methods and points of caution appropriately.

[1172] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1173] Step 1:

[1174] The server obtains data on the operation status from the machines operating in the factory in real time. The input is data from the machine's sensors and log files, and the output is the obtained operation status data. Specifically, the server collects data such as temperature, pressure, and operating status from the machine's sensors.

[1175] Step 2:

[1176] The server posts the acquired work status data to the communication tool. The input is the work status data acquired in step 1, and the output is the content to be posted to the communication tool. Specifically, the server converts the data into text format and posts it using the API of the communication tool.

[1177] Step 3:

[1178] The generation AI analyzes the content posted on the communication tool. The input is the content posted on the communication tool, and the output is the analysis result. In concrete terms, the generation AI analyzes the posted content using natural language processing technology and grasps the work status.

[1179] Step 4:

[1180] The server visualizes the work status in real time based on the analysis results of the generative AI. The input is the analysis results of the generative AI, and the output is the visualized work status. Specifically, the server displays the analysis results on a dashboard or monitor.

[1181] Step 5:

[1182] Generative AI automatically generates efficient work methods and points to note by taking into account past data and the skillset of the machine. The input is past data and the skillset of the machine, and the output is efficient work methods and points to note. In concrete terms, generative AI retrieves past work history from the database and generates the optimal work method.

[1183] Step 6:

[1184] The server provides the efficient work methods and cautions generated by the generative AI to managers and other machines. The input is the advice generated by the generative AI, and the output is a notification to the manager or other machines. Specifically, the server converts the advice into text format and notifies the user through a communication tool or dashboard.

[1185] Example 2

[1186] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1187] In remote work and distributed teams, it is difficult to grasp the work status of team members in real time and provide appropriate advice. In particular, it is difficult for other members to understand what work each member is doing and what kind of advice is needed, which can hinder efficient work progress and quality improvement.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1189] In this invention, the server includes means for collecting task data from the terminals of the team members, means for analyzing the collected data, means for generating advice based on the analysis results, means for notifying the terminals of each member of the generated advice, and means for letting each member know what tasks the other members are performing and what advice to give them. This makes it possible to grasp the task status of team members in real time and provide appropriate advice.

[1190] "Team members" refers to two or more individuals who come together to work on a project or task.

[1191] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[1192] "Work Data" means information related to work being performed by a Team Member, including code, logs, notes, etc.

[1193] "Means for collecting" refers to a method or device for acquiring work data from a terminal and transmitting it to a server.

[1194] "Means for analyzing" refers to methods and devices for analyzing and evaluating collected work data.

[1195] "Means for generating advice" refers to a method or device for suggesting improvements or modifications based on the results of the analysis.

[1196] The "means for notifying" refers to a method or device for transmitting the generated advice to the terminals of the team members.

[1197] "Visualization means" refers to a method or device for visually displaying what work other members are doing and what advice is needed.

[1198] "Generative AI" refers to artificial intelligence that uses machine learning models to analyze data and automatically generate advice.

[1199] "Communication tools" refers to software or platforms that allow team members to share information and stay in touch.

[1200] "Remote work" refers to a way of working in which work is carried out via the internet from a physically distant location.

[1201] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1202] The present invention is a system for grasping the working status of team members in real time and automatically generating necessary advice. A specific embodiment of this system will be described below.

[1203] 1. Generate the system program

[1204] The server uses a generative AI model to understand the work status of team members in real time and generate a program that automatically generates the necessary advice. This program is implemented using Python and utilizes machine learning libraries such as TensorFlow and PyTorch.

[1205] 2. Explain the program's processing in natural language

[1206] The server collects data from each team member's device to monitor the work status of team members in real time. This data includes information from the IDE (integrated development environment) and editor (e.g. Visual Studio Code, PyCharm) used by the team members. The device periodically sends the work data to the server.

[1207] The server analyzes the collected data and inputs it into the generative AI model. The generative AI model evaluates the work status of each member based on the collected data and generates necessary advice. For example, if a member is working on programming, the generative AI model analyzes the member's code and generates advice pointing out areas to improve the code and possible bugs. This advice is sent to the member's device in real time.

[1208] 3. Add concrete examples

[1209] As a concrete example, let's say a member is developing a web application in Python. As the member writes code using Visual Studio Code, the server collects the code in real time and inputs it into the generative AI model. The generative AI model evaluates the quality of the code and generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the scope of the variable."

[1210] Example prompt:

[1211] "Analyze this code, written in Python, and generate suggestions for improvements or potential bugs."

[1212] In this way, generative AI models help team members work more efficiently and improve quality.

[1213] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1214] Step 1: Data collection

[1215] The server collects work data from each team member's device. The device obtains the code and work log in real time from the IDE or editor (e.g., Visual Studio Code, PyCharm) used by the team member and sends it to the server. The input is the code and work log sent from the device, and the output is the work data stored on the server.

[1216] Step 2: Data analysis

[1217] The server analyzes the collected work data. Specifically, it runs Python scripts and uses machine learning libraries such as TensorFlow and PyTorch to evaluate the quality of the code and possible bugs. The input is the work data stored on the server, and the output is the analysis results, for example checking function redundancy and variable scope.

[1218] Step 3: Advice Generation

[1219] The server uses a generative AI model to generate advice based on the analysis results. The generative AI model takes the analysis results as input and outputs advice including specific improvements and correction methods. For example, it generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the variable scope."

[1220] Step 4: Advice Notification

[1221] The server notifies each member's terminal of the generated advice. The terminal receives the notification and displays it as a pop-up message to the user. The input is the generated advice, and the output is the advice displayed to the user. The user checks the displayed advice and modifies the code.

[1222] In this way, generative AI models help team members work more efficiently and improve quality.

[1223] (Application example 2)

[1224] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1225] Conventional communication tools and generative AI systems can visualize the work status of team members and necessary advice, but they have the problem that it is difficult to provide specific advice in real time to improve work efficiency in the factory. In addition, there is a lack of systems that can monitor the work status in the factory in real time and suggest optimal operation patterns and improvements. This has led to problems such as a decrease in work efficiency in the factory and a lack of improvement in productivity.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1227] In this invention, the server includes a means for inputting posts from the communication tool to the generation AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for making it possible to know what work other members are doing and what advice should be given, a means for monitoring the work situation in the factory in real time and automatically generating advice for improving work efficiency, and a means for displaying the generated advice. This makes it possible to provide specific advice for improving work efficiency in the factory in real time.

[1228] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[1229] "Generative AI" is a system that uses artificial intelligence technology to analyze data and automatically generate necessary information and advice.

[1230] "Visualization" is the process of displaying data or information visually and making it easier to understand.

[1231] "Real-time" refers to processing ongoing events and situations immediately and providing results.

[1232] "Work efficiency" is an index that indicates how much work can be effectively performed within a certain amount of time.

[1233] "Advice" is guidance or a suggestion given to address a particular situation or problem.

[1234] "Surveillance" is the act of continuously observing a specific object or situation and collecting necessary information.

[1235] A "behavior pattern" refers to a series of actions or steps in a specific task or process.

[1236] "Areas for improvement" are specific proposals or methods for identifying current problems or inefficiencies and making them better.

[1237] "Display" refers to showing information or data visually, often using a device such as a screen or display.

[1238] A system for implementing the present invention provides specific advice in real time to improve work efficiency in a factory. The system includes a camera, a generative AI model, and a display device.

[1239] First, a camera (e.g., Logitech C920) captures the work situation in the factory in real time. The captured video is sent to a server. The server analyzes the video using a generative AI model (e.g., a model built with TensorFlow or PyTorch). The generative AI model analyzes the work situation in the video and automatically generates optimal operation patterns and areas for improvement.

[1240] The generated advice is displayed on a display device (e.g., a monitor or smart glasses), allowing workers in the factory to receive the advice in real time and improve their work efficiency.

[1241] As a concrete example, consider a case where a robot is tightening screws in a factory. The generative AI model analyzes how and at what speed the screws are tightened, and suggests the optimal tightening method and speed. For example, by inputting a prompt such as "Analyze the current work situation and suggest the optimal tightening method and speed for the screws" into the generative AI model, specific advice can be obtained.

[1242] This system makes it possible to provide specific advice in real time to improve work efficiency within the factory.

[1243] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1244] Step 1:

[1245] Cameras capture the work situation inside the factory in real time.

[1246] Input: Work status in the factory

[1247] Output: Captured video data

[1248] Specific operation: A camera (e.g., Logitech C920) captures work being done in the factory and sends the video data to a server.

[1249] Step 2:

[1250] The server receives the captured video data and inputs it into the generative AI model.

[1251] Input: Captured video data

[1252] Output: Data input to the generative AI model

[1253] How it works: The server receives the video data and preprocesses it for input into a generative AI model (e.g., a model built with TensorFlow or PyTorch).

[1254] Step 3:

[1255] A generative AI model analyzes the video data and evaluates the work situation.

[1256] Input: Preprocessed video data

[1257] Output: Evaluation result of the work situation

[1258] How it works: A generative AI model analyzes video data to evaluate the progress and efficiency of a task, for example analysing how and how fast a screw is tightened.

[1259] Step 4:

[1260] The generative AI model automatically generates optimal movement patterns and areas for improvement.

[1261] Input: Evaluation result of work status

[1262] Output: Optimal operation patterns and advice on areas for improvement

[1263] Specific behavior: The generative AI model generates optimal behavior patterns and improvements based on the evaluation results. For example, it uses a prompt sentence such as, "Analyze the current work situation and suggest the optimal way and speed to tighten the screws."

[1264] Step 5:

[1265] The server displays the generated advice on a display device.

[1266] Input: Optimal behavior patterns and suggestions for improvement

[1267] Output: Advice displayed on your display device

[1268] Specific operation: The server receives the generated advice and displays it on a display device (e.g., a monitor or smart glasses). The worker can check the advice in real time to improve work efficiency.

[1269] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1270] "Example 1"

[1271] One embodiment of the present invention is a system that combines an emotion engine. This system recognizes the user's emotions and adjusts advice based on the user's emotional state. Specifically, if the user feels stressed, the emotion engine communicates this information to the generation AI. The generation AI then adjusts the advice to the user based on this information. For example, the system provides advice to a user who is stressed that is kinder or includes encouraging words.

[1272] "Example 2"

[1273] In addition, the emotion engine visualizes the user's emotional state to other team members. This enables team members to understand the emotional state of other members and communicate appropriately. Specifically, the emotion engine visualizes the user's emotional state on a chat tool. For example, if a user feels happy, this information is displayed on the chat tool as "Member A is happy." This allows other members to know that Member A is happy and to communicate based on that information.

[1274] The process flow of each embodiment will be described below.

[1275] "Example 1"

[1276] Step 1: Users communicate using a chat tool.

[1277] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[1278] Step 3: The emotion engine communicates the recognized emotional state to the generative AI.

[1279] Step 4: The generative AI tailors the advice to the user based on their emotional state.

[1280] "Example 2"

[1281] Step 1: Users communicate using a chat tool.

[1282] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[1283] Step 3: The emotional state recognized by the emotion engine is visualized on the chat tool.

[1284] Step 4: Other team members see the visualized emotional state and communicate based on it.

[1285] Example 1

[1286] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1287] In remote work and distributed teams, it is difficult to grasp the work status of team members and the advice they need in real time. It is also difficult to provide appropriate advice that takes into account the emotional state of members. This can lead to reduced communication efficiency and affect the productivity of the entire team.

[1288] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for inputting a post of a communication tool to a generative AI model, a means for the generative AI model to analyze the posted content, a means for visualizing the work currently being performed by team members and advice that will be required in the future based on the analysis result, and a means for recognizing the user's emotions and adjusting advice based on the emotional state. This makes it possible to grasp the work status and necessary advice of team members in real time and provide appropriate advice according to the emotional state.

[1289] A "communication tool" is software or a platform that enables users to send and receive messages.

[1290] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing to analyze text data and generate appropriate responses or advice.

[1291] "Posted content" refers to messages and text data sent by users on communication tools.

[1292] "Analysis" is the process by which the generative AI model understands the content of a post and extracts meaning from it.

[1293] "Visualization" is the process of visually displaying information based on analysis results so that users can easily understand it.

[1294] An “emotion engine” is software or algorithms for recognizing a user’s emotional state and providing that information to a generative AI model.

[1295] "Advice" is advice or suggestions that the generative AI model provides to the user based on the analysis results and emotional state.

[1296] "Remote work" refers to working in which work is carried out via the Internet from a physically distant location.

[1297] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1298] This invention is a system for streamlining communication in remote work and distributed teams, and understanding the work status of team members and necessary advice in real time. This system is realized by combining communication tools, generative AI models, and emotion engines.

[1299] Hardware and Software Used

[1300] Hardware: Servers, terminals (PCs, smartphones)

[1301] Software: communication tools (e.g. Slack, Microsoft Teams), generative AI models (e.g. GPT-4), emotion engines

[1302] System Configuration

[1303] 1. User Submissions

[1304] A user uses a terminal to post a message on a communication tool. For example, the user posts, "I will be writing a report today."

[1305] 2. Acquiring posted content

[1306] The server uses the communication tool's API to obtain user posts in real time.

[1307] 3. Analysis of posted content

[1308] The server sends the acquired post content to the generative AI model for analysis. The generative AI model understands the post content and recognizes that the user is creating a report.

[1309] 4. Information Visualization

[1310] Based on the analysis results from the generative AI model, the server makes the information that the relevant member is writing a report visible to other team members. Specifically, it displays "Mr. / Ms. XX is writing a report" on the chat tool.

[1311] 5. Generating Advice

[1312] The generative AI model takes into account past posts and the skill sets of team members to generate advice on report creation. The server presents the advice to team members.

[1313] 6. Use of Emotion Engines

[1314] The emotion engine recognizes the user's emotions and conveys that information to the generative AI model. For example, if the emotion engine determines that the user is feeling stressed, it will send that information to the generative AI model.

[1315] The generative AI model uses information from the emotion engine to tailor its advice to users, for example providing gentler or more encouraging advice to a user who is feeling stressed.

[1316] Examples

[1317] User submission examples

[1318] I will be writing a report today.

[1319] Example prompts for generative AI models

[1320] "A user posts, 'Today I'm going to create a report.' Use this information to make it visible to other members that the user is creating a report, and generate advice on creating the report."

[1321] Advice examples

[1322] "When preparing a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports."

[1323] Example of using the emotion engine

[1324] The emotion engine detects the user's stress and passes that information to the generative AI model.

[1325] The generative AI model generates gentle advice such as, "When writing a report, it is most efficient to first consider the overall structure. Take your time and take breaks as you go," and returns this advice to the server.

[1326] In this way, the system can analyze the content posted by users, visualize the work status and necessary advice of team members in real time, and provide appropriate advice according to their emotional state.

[1327] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1328] Step 1:

[1329] A user uses a terminal to post a message on a communication tool. The input is the user's message, and the output is the message sent to the communication tool. A specific example of this is when a user accesses a chat tool from a PC or smartphone, types "I will be writing a report today," and sends it.

[1330] Step 2:

[1331] The server uses the API of the communication tool to obtain user posts in real time. The input is the message data obtained from the communication tool, and the output is the message data stored on the server. Specifically, the server calls the Slack API and obtains new messages from a specific channel.

[1332] Step 3:

[1333] The server sends the acquired post content to the generative AI model for analysis. The input is the message data stored on the server, and the output is the analysis result returned from the generative AI model. In concrete terms, the server sends the message "Today I will create a report" as a prompt to the generative AI model, and receives the analysis result.

[1334] Step 4:

[1335] Based on the analysis results from the generative AI model, the server visualizes to other team members the information that the relevant member is writing a report. The input is the analysis results from the generative AI model, and the output is the information displayed in the chat tool. In concrete terms, the server automatically posts to the chat tool, "Mr. / Ms. XX is writing a report."

[1336] Step 5:

[1337] The generative AI model takes into account the content of past posts and the skill sets of members, and generates advice on creating reports. The input is the content of past posts and the skill sets of members, and the output is the generated advice. In concrete terms, the generative AI model generates advice such as "When creating a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports," and returns this advice to the server.

[1338] Step 6:

[1339] The emotion engine recognizes the user's emotions and transmits that information to the generative AI model. The input is the user's emotional state, and the output is the emotional information sent to the generative AI model. Specifically, the emotion engine determines that the user is feeling stressed based on the content of the user's posts and facial expression recognition data, and sends that information to the generative AI model.

[1340] Step 7:

[1341] The generative AI model adjusts advice to the user based on information from the emotion engine. The input is emotion information from the emotion engine, and the output is adjusted advice. In concrete terms, the generative AI model generates gentle advice such as, "When writing a report, it is efficient to first consider the overall structure. Don't push yourself, and take breaks as you go along," and returns this advice to the server.

[1342] (Application example 1)

[1343] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1344] In order to improve work efficiency in factories, it is important to facilitate communication between workers and robots and to provide appropriate information about the work situation and necessary advice. However, with conventional systems, it was difficult to provide advice that took into account the emotional state of the worker, which limited the improvement of work efficiency. In addition, it was difficult for workers to give instructions to the robot and the robot's work status was not sufficiently visualized, making it difficult to manage the progress of work.

[1345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1346] In this invention, the server includes a means for inputting a post made via a communication tool to the generating AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for letting the user know what work other members are doing and what advice to give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot via a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, thereby improving work efficiency and facilitating progress management.

[1347] A "communication tool" is software or applications that allow users to exchange information through text or voice.

[1348] "Generative AI" is an artificial intelligence model that analyzes input data and generates appropriate output.

[1349] "Visualization" refers to displaying data or information visually so that users can easily understand it.

[1350] An "emotion engine" is a system that recognizes the user's emotional state and provides appropriate feedback based on that information.

[1351] A "robot" is a mechanical device that performs tasks automatically according to programmed instructions.

[1352] A "worker" is a human worker who performs a specific task in a factory.

[1353] An "instruction" is an order to perform a particular action or task.

[1354] "Analysis" refers to examining the input data in detail and understanding its meaning and structure.

[1355] "Advice" is advice or a suggestion intended to encourage appropriate action or decision in a particular situation.

[1356] "Progress management" is a management activity that monitors the progress of work or projects and keeps them moving forward as planned.

[1357] The system for implementing this invention includes a communication tool, a generation AI, an emotion engine, a robot, and a worker's terminal. The specific operation of each element and its cooperation will be described below.

[1358] System Configuration

[1359] 1. Communication tools:

[1360] Software or applications that allow workers to exchange information through text or voice.

[1361] Workers use chat tools to give instructions to the robot.

[1362] 2. Generation AI:

[1363] It uses generative AI models such as the OpenAI API to analyze chat content as it is typed.

[1364] Based on the analysis results, appropriate work instructions are issued to the robot.

[1365] Automatically generate work status and necessary advice.

[1366] 3. Emotion Engine:

[1367] Use emotion recognition libraries such as EmotionEngine to recognize the emotional state of workers.

[1368] Tailor the advice provided by the generative AI based on the perceived emotional state.

[1369] 4. Robots:

[1370] It is a mechanical device that performs tasks automatically according to programmed instructions.

[1371] It receives instructions from the generative AI and carries out the appropriate tasks.

[1372] 5. Worker terminal:

[1373] These are devices used by workers, such as smartphones and tablets.

[1374] Instructions are given to the robot through a communication tool and advice is received from the generating AI.

[1375] System Operation

[1376] The server includes a means for inputting posts from the communication tool to the generating AI, a means for visualizing the work that team members are currently doing and the advice they will need in the future, a means for letting the user know what work other members are doing and what advice they should give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot using a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, improving work efficiency and facilitating progress management.

[1377] Examples

[1378] When a worker types "I'm going to write a report today" into the chat tool, the generative AI analyzes the content and instructs the robot to support the report writing. At the same time, the emotion engine recognizes the worker's emotional state and provides advice including encouraging words such as "Do your best! You can do it!" if the worker is feeling stressed.

[1379] Examples of prompt statements

[1380] Please analyze the following chat and provide your suggestions:

[1381] I'm going to create a report today.

[1382] In this way, the work efficiency in the factory can be improved.

[1383] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1384] Step 1:

[1385] A user inputs text into a chat tool using a terminal. For example, the user inputs "Today I will be writing a report." This text becomes input data into the system.

[1386] Step 2:

[1387] The server receives text entered from the chat tool. It generates a prompt sentence to send the received text to the generative AI model. This prompt sentence becomes the input data for the generative AI model to analyze.

[1388] Step 3:

[1389] The server sends a prompt to the generative AI model, which analyzes the text. The generative AI model analyzes the input text and generates work content and advice. The analysis results become the system's output data.

[1390] Step 4:

[1391] The server receives the analysis results from the generative AI model, which include action content (creation of a report) and advice. The server visualizes this information and shares it with other team members.

[1392] Step 5:

[1393] The server uses an emotion engine to recognize the user's emotional state. Based on chat content and past data, it determines whether the user is feeling stressed. This emotional state becomes the input data for the system.

[1394] Step 6:

[1395] The server receives the emotional state from the emotion engine and adjusts the advice from the generative AI model. For example, if the user is feeling stressed, it adds words of encouragement to the advice. This adjusted advice becomes the output data of the system.

[1396] Step 7:

[1397] The server sends the adjusted advice to the user's device, and the user receives the advice from the generative AI model and the adjustments made by the emotion engine through the device.

[1398] Step 8:

[1399] The user issues instructions to the robot using a terminal. The server receives the instructions and sends them to the generative AI model. The generative AI model analyzes the instructions and generates instructions for the robot to perform the appropriate tasks.

[1400] Step 9:

[1401] The server receives instructions from the generative AI model and sends them to the robot, which then follows the instructions to perform a task, such as writing a report. The progress of this task becomes the output data of the system.

[1402] Step 10:

[1403] The server receives the progress of the robot's work and makes it visible to the user through a chat tool. The user can then check the robot's progress through their device.

[1404] Example 2

[1405] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1406] In remote work and distributed teams, it is difficult to grasp the work situation and emotional state of team members in real time and provide appropriate advice and communication. In particular, communication between members tends to be lacking, which can lead to a decrease in work efficiency and team unity.

[1407] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1408] In this invention, the server includes a means for a user to start a task on a terminal, a means for the terminal to transmit task data to the server, a means for the server to analyze the data received, a means for the server to generate advice using a generative AI model, a means for the server to analyze and visualize an emotional state using an emotion engine, a means for the server to transmit the advice and emotional information generated by the server to the terminal, and a means for displaying the information received by the terminal to the user. This makes it possible to grasp the task status and emotional state of team members in real time and provide appropriate advice and communication.

[1409] "User" refers to an individual or group that performs work using the system.

[1410] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[1411] "Server" refers to a central processing unit that receives, analyzes, generates, and transmits data.

[1412] "Task data" refers to information related to the task a user is performing (e.g., programming code or emotion data).

[1413] A "generative AI model" refers to an artificial intelligence model that analyzes a user's work data and generates appropriate advice.

[1414] An "emotion engine" refers to software or hardware for analyzing a user's emotional state and visualizing the results.

[1415] "Advice" refers to suggestions for improvement or comments that the generative AI model generates based on the user's work data.

[1416] "Emotion information" refers to information about the user's emotional state analyzed by the emotion engine.

[1417] "Visualization" refers to displaying analyzed data in a form that is easy for users to understand.

[1418] "Communication tools" refers to software or platforms for sharing information among team members.

[1419] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1420] This invention is a system for grasping the work situation and emotional state of team members in real time in remote work and distributed teams, and providing appropriate advice and communication. Below, we will show how to specifically implement this system.

[1421] 1. Generate the system program

[1422] This system combines a generative AI model and an emotion engine to grasp the work situation and emotional state of team members in real time and provide necessary advice and emotional information. For example, OpenAI's GPT-4 is used as the generative AI model, and the emotion engine uses Microsoft's Azure Cognitive Services Emotion API.

[1423] 2. Explain the program's processing in natural language

[1424] The server hosts a generative AI model and an emotion engine. The generative AI model analyzes the user's work data (e.g., programming code) and generates appropriate advice. The emotion engine analyzes the user's emotional state (e.g., facial expressions and voice) and visualizes the results.

[1425] The terminals are PCs or smartphones used by users, and send data to the server. Specifically, the terminals send the user's task data and emotion data to the server.

[1426] The server analyzes the data it receives and uses a generative AI model to generate advice. For example, if a member is working on a program, the server analyzes the member's code and generates advice on how to improve the code or identify potential bugs.

[1427] The emotion engine also analyzes the user's emotional state and visualizes the results on the chat tool. For example, if the emotion engine determines that the user is happy, the server will display "Member A is happy" on the chat tool.

[1428] 3. Add examples to your description

[1429] As a concrete example, consider the following scenario.

[1430] Scenario 1: Programming task advice

[1431] User A is programming on a device. User A's device sends the code he is working on to the server. The server analyzes the code using a generative AI model (GPT-4) and generates advice such as the following:

[1432] "This part of the code is inefficient and can be improved as follows."

[1433] "There may be a bug in this line. Details below."

[1434] Example prompt:

[1435] Analyze User A's code and generate advice pointing out areas for improvement and potential bugs.

[1436] Code: [User A's code]

[1437] Scenario 2: Visualizing Emotional States

[1438] User B is working on a device. The emotion engine analyzes User B's facial expression and determines that User B is happy. The server displays this information on the chat tool.

[1439] "Member B is happy."

[1440] Example prompt:

[1441] Analyze User B's emotional state and visualize it on the chat tool.

[1442] Emotional state: Joy

[1443] This system enables users to carry out their work efficiently and to understand the emotional state of other members and communicate appropriately.

[1444] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1445] Step 1:

[1446] The user begins working at the terminal.

[1447] Input: The device used by the user (PC or smartphone)

[1448] Output: Working data (programming code and emotion data)

[1449] Specific operation: A user starts programming or other work on his / her PC or smartphone. For example, User A opens Visual Studio Code and starts programming.

[1450] Step 2:

[1451] The terminal transmits the work data to the server.

[1452] Input: User-generated task data (programming code and emotion data)

[1453] Output: The work data sent to the server.

[1454] Specific operation: User A's device transmits the programming code written in real time and facial expression data captured by the webcam to the server.

[1455] Step 3:

[1456] The server parses the received data.

[1457] Input: Working data sent from the terminal (programming code and emotion data)

[1458] Output: Analysis results (code structure and content, emotional state)

[1459] Specific operation: The server analyzes the received programming code and understands the structure and content of the code. It also analyzes User A’s facial expressions from the webcam footage to understand his / her emotional state.

[1460] Step 4:

[1461] The server generates advice using a generative AI model.

[1462] Input: Parsed programming code

[1463] Output: Generated advice

[1464] Specific operation: The server uses a generative AI model (GPT-4) to generate advice for User A's programming code. For example, it generates specific advice such as "This loop is inefficient, so it would be better to use a list comprehension."

[1465] Step 5:

[1466] The server uses an emotion engine to analyze and visualize the emotional state.

[1467] Input: Parsed emotion data

[1468] Output: Visualized emotion information

[1469] Specific operation: The server uses the emotion engine (Emotion API) to determine that User A is happy. The server visualizes this information on the chat tool as "Member A is happy."

[1470] Step 6:

[1471] The server transmits the generated advice and emotion information to the terminal.

[1472] Input: Generated advice and visualized emotion information

[1473] Output: Advice and sentiment sent to the device

[1474] Specific operation: The server sends the generated advice and emotion information to the terminal of user A. For example, the server sends advice such as "This loop is inefficient, so it would be better to use a list comprehension" and emotion information such as "Member A is happy."

[1475] Step 7:

[1476] The terminal displays the received information to the user.

[1477] Input: Advice and sentiment information sent from the server

[1478] Output: Advice and emotion information displayed to the user

[1479] Specific operation: User A's device displays advice on Visual Studio Code and emotional information on the chat tool. For example, "This loop is inefficient, so it would be better to use a list comprehension" is displayed in the Visual Studio Code editor, and "Member A is happy" is displayed in the chat tool.

[1480] (Application example 2)

[1481] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1482] In conventional teamwork, it is difficult to grasp the working status of team members and necessary advice in real time, which often hinders efficient communication. In addition, there is no way to grasp the emotional state of team members, making it difficult to communicate appropriately. This can lead to a decrease in work efficiency and friction within the team.

[1483] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1484] In this invention, the server includes a means for inputting posts from a communication tool to a generation AI, a means for visualizing the work that team members are currently doing and advice that they will need in the future, a means for letting users know what work other members are doing and what advice they should give, a means for recognizing and visualizing the emotional state of users in real time, and a means for generating and displaying necessary advice based on the work situation. This makes it possible to grasp the work situation and emotional state of team members in real time and provide appropriate advice.

[1485] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[1486] "Generative AI" is an artificial intelligence system that automatically generates advice and information based on input data.

[1487] "Visualization" is a means of visually displaying data or information to make it easier to understand.

[1488] "User's emotional state" refers to the emotions and psychological state that the user is currently feeling.

[1489] "Real-time" refers to processing and display occurring immediately, without delay.

[1490] "Work status" refers to the progress and content of the work that team members are currently doing.

[1491] "Advice" means advice or a suggestion given to improve a task or solve a problem.

[1492] A "display" is a means of visually presenting information to a user.

[1493] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The server uses a generative AI model to automatically generate the work status and necessary advice of team members, and has the function of recognizing and visualizing the emotional state of the user in real time. The terminal is a device such as smart glasses or a smartphone, and serves to provide this information to the user. The user receives advice and feedback of the emotional state in real time through these devices.

[1494] The server uses the following software and hardware:

[1495] OpenCV: A library for image processing

[1496] EmotionRecognizer: Software for Recognizing a User's Emotional State

[1497] CodeAnalyzer: software to analyze the work of team members

[1498] AIAdvisor: An artificial intelligence system for generating needed advice

[1499] The terminals are devices such as smart glasses or smartphones, which are equipped with cameras and displays. The cameras capture images of the user and send them to a server. The server analyzes the images and recognizes the user's emotional state and work situation. The recognized information is converted into appropriate advice using a generative AI model and displayed on the terminal's display.

[1500] As a concrete example, consider a scenario in which an employee working in a factory wears smart glasses and receives advice in real time based on the work situation and emotional state. While the employee is working, the camera in the smart glasses captures video and sends it to a server. The server analyzes the video and recognizes the employee's emotional state. At the same time, it analyzes the progress of the work the employee is doing and generates the necessary advice. This information is displayed in real time on the display of the smart glasses.

[1501] Example of a prompt to be input to a generative AI model:

[1502] "Employee A puts on smart glasses, which analyze the code he is working on and gives him advice. His emotional state is also recognized in real time and shared with his team members."

[1503] In this way, it is possible to improve the work efficiency and communication of team members.

[1504] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1505] Step 1:

[1506] The camera on the device (smart glasses) captures an image of the user.

[1507] Input: Real-time video of the user

[1508] Output: Captured video data

[1509] Specific operation: The camera in the smart glasses photographs the user's face and work environment and captures the video data.

[1510] Step 2:

[1511] The video data acquired by the terminal is transmitted to the server.

[1512] Input: Acquired video data

[1513] Output: Video data sent to the server

[1514] Specific operation: The smart glasses upload video data to the server using communication methods such as Wi-Fi or Bluetooth.

[1515] Step 3:

[1516] The server receives the video data and analyzes the user's emotional state using an emotion recognition engine (EmotionRecognizer).

[1517] Input: Video data sent to the server

[1518] Output: Parsed emotional state data

[1519] Specific operation: The server inputs the video data into EmotionRecognizer, which estimates the user's emotional state from their facial expressions and movements.

[1520] Step 4:

[1521] The server uses a code analysis engine (CodeAnalyzer) to analyze the user's work.

[1522] Input: Data about what the user is working on (e.g. code files)

[1523] Output: Analyzed work status data

[1524] What it does: The server inputs the user's work (e.g., program code) into CodeAnalyzer and evaluates the code's quality and progress.

[1525] Step 5:

[1526] The server uses a generative AI model (AIAdvisor) to generate the necessary advice based on the analyzed emotional state data and work situation data.

[1527] Input: Emotional state data, work situation data

[1528] Output: Generated advice

[1529] Specific operation: The server inputs emotional state data and task situation data into the AIAdvisor and generates appropriate advice.

[1530] Step 6:

[1531] The server transmits the generated advice and emotional state to the terminal.

[1532] Input: Generated advice, emotional state data

[1533] Output: Advice and emotional state data sent to the device

[1534] Specific operation: The server uses a communication means to transmit the generated advice and emotional state data to the terminal.

[1535] Step 7:

[1536] The terminal displays the received advice and emotional state to the user.

[1537] Input: Received advice, emotional state data

[1538] Output: Advice and emotional state displayed to the user

[1539] Specific operation: The smart glasses display visually displays the received advice and emotional state to the user.

[1540] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1541] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1542] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1543] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1544] [Fourth embodiment]

[1545] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1546] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1547] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1548] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1549] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1550] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1551] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1552] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1553] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1554] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1555] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1556] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1557] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1558] "Example 1"

[1559] The embodiment of the present invention uses a chat tool that is commonly used as a communication tool. The content posted on the chat tool is input to the generation AI, which analyzes it. Based on the analysis results, the work that team members are currently doing and the advice they will need in the future are visualized. Specifically, if a member posts on the chat tool, "I will be writing a report today," the generation AI analyzes this information and visualizes to other members the information that the member is writing a report. In addition, the generation AI takes into account the content of past posts and the members' skill sets, etc., and generates advice on writing reports, which it presents to team members.

[1560] "Example 2"

[1561] In another embodiment of the present invention, the generation AI automatically generates the work status of team members and necessary advice. Specifically, the generation AI grasps the work status of team members in real time and automatically generates necessary advice. For example, when a member is performing programming work, the generation AI analyzes the member's code and generates advice pointing out improvements to the code and possible bugs. This enables team members to proceed with their work more efficiently.

[1562] The process flow of each embodiment will be described below.

[1563] "Example 1"

[1564] Step 1: A team member posts their work on a chat tool. For example, they post, "Today, we will be creating a report."

[1565] Step 2: The generation AI analyzes the content posted in the chat tool. Through this analysis, it learns that a member is writing a report.

[1566] Step 3: Based on the analysis results, the generative AI visualizes the work status of team members. Specifically, it displays information such as "Member A is writing a report" to other members on the chat tool.

[1567] Step 4: The generation AI considers past posts and the skill sets of team members, etc., and generates advice on report creation. The generated advice is presented to team members. "Form Example 2"

[1568] Step 1: Team members start working. In this example, we assume that a team member starts programming work.

[1569] Step 2: The generative AI grasps the member's work status in real time. Specifically, it analyzes the code that the member is writing.

[1570] Step 3: Based on the results of the code analysis, the generative AI generates advice pointing out areas for improvement and potential bugs.

[1571] Step 4: The generative AI presents the generated advice to team members, enabling them to work more efficiently.

[1572] Example 1

[1573] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1574] In remote work and distributed teams, it is difficult to know in real time what team members are currently doing and what advice they need. In addition, there is a lack of ways for each member to share with other members what work they are doing, making efficient communication and cooperation difficult. There is a need to improve this situation and improve the productivity of the entire team.

[1575] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1576] In this invention, the server includes a means for a user to post to the communication tool, a means for the server to acquire the posted content, a means for the server to transmit the posted content to the generative AI model, a means for the generative AI model to analyze the posted content, a means for the generative AI model to return the analysis result to the server, a means for the server to display the analysis result on the communication tool, and a means for the user to check the analysis result. This makes it possible to visualize the work currently being done by team members and any advice they need in real time, enabling efficient communication and cooperation.

[1577] "User" means an individual or organization that uses the system to post to a communication tool.

[1578] A "communication tool" is any software or platform that a user uses to send or receive messages.

[1579] The "server" is a computer system that acquires user posted content, transmits it to the generative AI model, and displays the analysis results in the communication tool.

[1580] A "generative AI model" is an artificial intelligence model that analyzes the content posted by users and generates information about work status and necessary advice.

[1581] "Posted content" refers to messages or information that a user inputs into a communication tool and sends.

[1582] "Analysis results" refer to the information and advice obtained by the generative AI model when it analyzes the posted content.

[1583] "Visualization" refers to displaying the analysis results in a form that is easy for other team members to understand.

[1584] "Remote work" is a way of working in which work is carried out from a remote location without going to a physical office.

[1585] A "prompt sentence" is an instruction sentence input to a generative AI model to perform analysis.

[1586] The present invention provides a system for improving the efficiency of communication in remote work and distributed teams, and visualizing the work status of team members and necessary advice in real time. A specific embodiment of the system will be described below.

[1587] Hardware and Software Used

[1588] Hardware: Servers, terminals (PCs, smartphones, etc.)

[1589] Software: communication tools (e.g. Slack, Microsoft Teams), generative AI models (e.g. GPT-4)

[1590] System Configuration

[1591] 1. A user posts to a communication tool

[1592] A user logs into a communication tool, enters a message about work, and sends it.

[1593] Example: A user posts, "Today I will be working on a report."

[1594] 2. The server retrieves the post

[1595] The server periodically checks for and retrieves new posted content using the communication tool's API.

[1596] Example: A server calls the Slack API to get new messages.

[1597] 3. The server sends the post to the generative AI model

[1598] The server sends the acquired post content to the generative AI model as a prompt sentence.

[1599] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[1600] 4. The generative AI model analyzes the post

[1601] The generative AI model analyzes the received prompt and identifies the user's current task.

[1602] Example: A generative AI model extracts the keyword "report writing" and determines that the user is writing a report.

[1603] 5. The generative AI model returns the analysis results to the server

[1604] The generative AI model generates analysis results and sends them back to the server.

[1605] Example: A generative AI model generates information that "the user is creating a report" and "advice on creating the report," and returns this to the server.

[1606] 6. The server displays the analysis results in the communication tool.

[1607] The server displays the analysis results received from the generative AI model on the communication tool.

[1608] Example: The server posts a message to a communication tool saying, "A user is creating a report. When creating the report, it may be a good idea to refer to past project reports."

[1609] 7. The user checks the analysis results

[1610] The user checks the analysis results and advice displayed on the communication tool.

[1611] Example: A user reviews "report writing advice" and modifies their actions as necessary.

[1612] The system provides real-time visibility into what team members are currently doing and what advice they need, enabling efficient communication and collaboration.

[1613] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1614] Step 1:

[1615] A user posts to a communication tool.

[1616] Specific behavior: A user logs into a communications tool, types and sends a work-related message.

[1617] Input: A message typed by the user (e.g. "Today I will be writing a report").

[1618] Output: The message posted to the conversations tool.

[1619] Step 2:

[1620] The server retrieves the post.

[1621] Specific operation: The server uses the communication tool's API to periodically check for and retrieve new post content.

[1622] Input: The message posted in the conversations tool.

[1623] Output: The post retrieved by the server.

[1624] Step 3:

[1625] The server sends the post to the generative AI model.

[1626] Specific operation: The server sends the post content obtained as a prompt to the generative AI model.

[1627] Input: The post content retrieved by the server.

[1628] Output: The prompt sent to the generative AI model.

[1629] Example prompt: "A user posts, 'Today I'm going to write a report.' Analyze this information and make it visible to other members that the member is writing a report. Also, generate advice on writing the report."

[1630] Step 4:

[1631] A generative AI model analyzes the content of the post.

[1632] Specific behavior: The generative AI model analyzes the received prompt and identifies the user's current task.

[1633] Input: The prompt sent to the generative AI model.

[1634] Output: Analysis results (user actions and advice).

[1635] Step 5:

[1636] The generative AI model returns the analysis results to the server.

[1637] Specific operation: The generative AI model generates analysis results and sends them back to the server.

[1638] Input: Analysis results from a generative AI model.

[1639] Output: Analysis results sent back to the server.

[1640] Step 6:

[1641] The server displays the analysis results on the communication tool.

[1642] Specific operation: The server displays the analysis results received from the generative AI model on the communication tool.

[1643] Input: The analysis result sent back to the server.

[1644] Output: Analysis results displayed in a communication tool.

[1645] Example: Post a message in a communication tool saying, "A user is creating a report. When creating the report, it would be a good idea to refer to past project reports."

[1646] Step 7:

[1647] The user checks the analysis results.

[1648] Specific action: The user checks the analysis results and advice displayed on the communication tool.

[1649] Input: Analysis results displayed in communication tool.

[1650] Output: Analysis results and advice reviewed by the user.

[1651] (Application example 1)

[1652] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1653] With conventional work management systems in factories, it was difficult to grasp the working status of each machine in real time, and it was not possible to provide efficient work methods or points of caution appropriately. In addition, sharing of work status and provision of advice between team members was often done manually, making efficient work management difficult.

[1654] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for inputting a post on a communication tool to the generating AI, a means for visualizing the work currently being performed by team members and advice that will be required in the future, a means for making it possible to know what work other members are performing and what advice should be given, a means for visualizing the work status of machines operating in a factory in real time, and a means for providing efficient work methods and points of caution using the generating AI. This makes it possible to grasp the work status in a factory in real time and provide efficient work methods and points of caution appropriately.

[1655] A "communication tool" is software or an application that enables the exchange of information between users.

[1656] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate appropriate output.

[1657] "Visualization" is the process of displaying data or information visually so that it can be easily understood by users.

[1658] "Machinery operating in a factory" refers to automated equipment and robots used in a factory to carry out manufacturing operations.

[1659] "Real-time" refers to a state in which data and information are processed immediately and provided without delay.

[1660] An "efficient work method" is the optimal procedure or method for completing a task with the least amount of time and effort.

[1661] "Points to note" are matters or points that require special attention when performing work.

[1662] As a form for implementing this invention, a system will be constructed that visualizes the working status of machines operating in a factory in real time and uses generative AI to provide efficient work methods and points to note.

[1663] System Configuration

[1664] 1. Hardware:

[1665] Machines operating in factories (robots, etc.)

[1666] Administrator's PC or tablet

[1667] 2. Software:

[1668] Communication tools (e.g. Slack)

[1669] Generative AI (e.g. OpenAI GPT-3)

[1670] Data processing and calculation

[1671] Data Acquisition

[1672] The server receives real-time data on the operation status of machines operating in the factory, collected from the machine's sensors and log files.

[1673] 4. Data Analysis

[1674] The server posts the acquired work status data to the communication tool. The generation AI analyzes the posted content and understands the work status.

[1675] Visualization and Advice

[1676] The server visualizes the work status in real time based on the analysis results of the Generative AI. The Generative AI also automatically generates efficient work methods and points to note, taking into account past data and the skillset of the machine, and provides these to managers and other machines.

[1677] Examples

[1678] For example, if a post is made on a communication tool within a factory saying, "Robot A has started assembling parts," the generative AI would analyze this information and generate advice such as, "Robot A is assembling parts. According to past data, efficiency can be improved by paying attention to the order in which the parts are installed."

[1679] Examples of prompt statements

[1680] Please analyze the following tasks and provide your advice.

[1681] Task: Robot A starts assembling parts.

[1682] In this way, it is possible to grasp the work status within the factory in real time and provide efficient work methods and points of caution appropriately.

[1683] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1684] Step 1:

[1685] The server obtains data on the operation status from the machines operating in the factory in real time. The input is data from the machine's sensors and log files, and the output is the obtained operation status data. Specifically, the server collects data such as temperature, pressure, and operating status from the machine's sensors.

[1686] Step 2:

[1687] The server posts the acquired work status data to the communication tool. The input is the work status data acquired in step 1, and the output is the content to be posted to the communication tool. Specifically, the server converts the data into text format and posts it using the API of the communication tool.

[1688] Step 3:

[1689] The generation AI analyzes the content posted on the communication tool. The input is the content posted on the communication tool, and the output is the analysis result. In concrete terms, the generation AI analyzes the posted content using natural language processing technology and grasps the work status.

[1690] Step 4:

[1691] The server visualizes the work status in real time based on the analysis results of the generative AI. The input is the analysis results of the generative AI, and the output is the visualized work status. Specifically, the server displays the analysis results on a dashboard or monitor.

[1692] Step 5:

[1693] Generative AI automatically generates efficient work methods and points to note by taking into account past data and the skillset of the machine. The input is past data and the skillset of the machine, and the output is efficient work methods and points to note. In concrete terms, generative AI retrieves past work history from the database and generates the optimal work method.

[1694] Step 6:

[1695] The server provides the efficient work methods and cautions generated by the generative AI to managers and other machines. The input is the advice generated by the generative AI, and the output is a notification to the manager or other machines. Specifically, the server converts the advice into text format and notifies the user through a communication tool or dashboard.

[1696] Example 2

[1697] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1698] In remote work and distributed teams, it is difficult to grasp the work status of team members in real time and provide appropriate advice. In particular, it is difficult for other members to understand what work each member is doing and what kind of advice is needed, which can hinder efficient work progress and quality improvement.

[1699] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1700] In this invention, the server includes means for collecting task data from the terminals of the team members, means for analyzing the collected data, means for generating advice based on the analysis results, means for notifying the terminals of each member of the generated advice, and means for letting each member know what tasks the other members are performing and what advice to give them. This makes it possible to grasp the task status of team members in real time and provide appropriate advice.

[1701] "Team members" refers to two or more individuals who come together to work on a project or task.

[1702] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[1703] "Work Data" means information related to work being performed by a Team Member, including code, logs, notes, etc.

[1704] "Means for collecting" refers to a method or device for acquiring work data from a terminal and transmitting it to a server.

[1705] "Means for analyzing" refers to methods and devices for analyzing and evaluating collected work data.

[1706] "Means for generating advice" refers to a method or device for suggesting improvements or modifications based on the results of the analysis.

[1707] The "means for notifying" refers to a method or device for transmitting the generated advice to the terminals of the team members.

[1708] "Visualization means" refers to a method or device for visually displaying what work other members are doing and what advice is needed.

[1709] "Generative AI" refers to artificial intelligence that uses machine learning models to analyze data and automatically generate advice.

[1710] "Communication tools" refers to software or platforms that allow team members to share information and stay in touch.

[1711] "Remote work" refers to a way of working in which work is carried out via the internet from a physically distant location.

[1712] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1713] The present invention is a system for grasping the working status of team members in real time and automatically generating necessary advice. A specific embodiment of this system will be described below.

[1714] 1. Generate the system program

[1715] The server uses a generative AI model to understand the work status of team members in real time and generate a program that automatically generates the necessary advice. This program is implemented using Python and utilizes machine learning libraries such as TensorFlow and PyTorch.

[1716] 2. Explain the program's processing in natural language

[1717] The server collects data from each team member's device to monitor the work status of team members in real time. This data includes information from the IDE (integrated development environment) and editor (e.g. Visual Studio Code, PyCharm) used by the team members. The device periodically sends the work data to the server.

[1718] The server analyzes the collected data and inputs it into the generative AI model. The generative AI model evaluates the work status of each member based on the collected data and generates necessary advice. For example, if a member is working on programming, the generative AI model analyzes the member's code and generates advice pointing out areas to improve the code and possible bugs. This advice is sent to the member's device in real time.

[1719] 3. Add concrete examples

[1720] As a concrete example, let's say a member is developing a web application in Python. As the member writes code using Visual Studio Code, the server collects the code in real time and inputs it into the generative AI model. The generative AI model evaluates the quality of the code and generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the scope of the variable."

[1721] Example prompt:

[1722] "Analyze this code, written in Python, and generate suggestions for improvements or potential bugs."

[1723] In this way, generative AI models help team members work more efficiently and improve quality.

[1724] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1725] Step 1: Data collection

[1726] The server collects work data from each team member's device. The device obtains the code and work log in real time from the IDE or editor (e.g., Visual Studio Code, PyCharm) used by the team member and sends it to the server. The input is the code and work log sent from the device, and the output is the work data stored on the server.

[1727] Step 2: Data analysis

[1728] The server analyzes the collected work data. Specifically, it runs Python scripts and uses machine learning libraries such as TensorFlow and PyTorch to evaluate the quality of the code and possible bugs. The input is the work data stored on the server, and the output is the analysis results, for example checking function redundancy and variable scope.

[1729] Step 3: Advice Generation

[1730] The server uses a generative AI model to generate advice based on the analysis results. The generative AI model takes the analysis results as input and outputs advice including specific improvements and correction methods. For example, it generates advice such as "This function is redundant. Consider refactoring it" or "There may be a bug in this part. Check the variable scope."

[1731] Step 4: Advice Notification

[1732] The server notifies each member's terminal of the generated advice. The terminal receives the notification and displays it as a pop-up message to the user. The input is the generated advice, and the output is the advice displayed to the user. The user checks the displayed advice and modifies the code.

[1733] In this way, generative AI models help team members work more efficiently and improve quality.

[1734] (Application example 2)

[1735] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1736] Conventional communication tools and generative AI systems can visualize the work status of team members and necessary advice, but they have the problem that it is difficult to provide specific advice in real time to improve work efficiency in the factory. In addition, there is a lack of systems that can monitor the work status in the factory in real time and suggest optimal operation patterns and improvements. This has led to problems such as a decrease in work efficiency in the factory and a lack of improvement in productivity.

[1737] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1738] In this invention, the server includes a means for inputting posts from the communication tool to the generation AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for making it possible to know what work other members are doing and what advice should be given, a means for monitoring the work situation in the factory in real time and automatically generating advice for improving work efficiency, and a means for displaying the generated advice. This makes it possible to provide specific advice for improving work efficiency in the factory in real time.

[1739] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[1740] "Generative AI" is a system that uses artificial intelligence technology to analyze data and automatically generate necessary information and advice.

[1741] "Visualization" is the process of displaying data or information visually and making it easier to understand.

[1742] "Real-time" refers to processing ongoing events and situations immediately and providing results.

[1743] "Work efficiency" is an index that indicates how much work can be effectively performed within a certain amount of time.

[1744] "Advice" is guidance or a suggestion given to address a particular situation or problem.

[1745] "Surveillance" is the act of continuously observing a specific object or situation and collecting necessary information.

[1746] A "behavior pattern" refers to a series of actions or steps in a specific task or process.

[1747] "Areas for improvement" are specific proposals or methods for identifying current problems or inefficiencies and making them better.

[1748] "Display" refers to showing information or data visually, often using a device such as a screen or display.

[1749] A system for implementing the present invention provides specific advice in real time to improve work efficiency in a factory. The system includes a camera, a generative AI model, and a display device.

[1750] First, a camera (e.g., Logitech C920) captures the work situation in the factory in real time. The captured video is sent to a server. The server analyzes the video using a generative AI model (e.g., a model built with TensorFlow or PyTorch). The generative AI model analyzes the work situation in the video and automatically generates optimal operation patterns and areas for improvement.

[1751] The generated advice is displayed on a display device (e.g., a monitor or smart glasses), allowing workers in the factory to receive the advice in real time and improve their work efficiency.

[1752] As a concrete example, consider a case where a robot is tightening screws in a factory. The generative AI model analyzes how and at what speed the screws are tightened, and suggests the optimal tightening method and speed. For example, by inputting a prompt such as "Analyze the current work situation and suggest the optimal tightening method and speed for the screws" into the generative AI model, specific advice can be obtained.

[1753] This system makes it possible to provide specific advice in real time to improve work efficiency within the factory.

[1754] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1755] Step 1:

[1756] Cameras capture the work situation inside the factory in real time.

[1757] Input: Work status in the factory

[1758] Output: Captured video data

[1759] Specific operation: A camera (e.g., Logitech C920) captures work being done in the factory and sends the video data to a server.

[1760] Step 2:

[1761] The server receives the captured video data and inputs it into the generative AI model.

[1762] Input: Captured video data

[1763] Output: Data input to the generative AI model

[1764] How it works: The server receives the video data and preprocesses it for input into a generative AI model (e.g., a model built with TensorFlow or PyTorch).

[1765] Step 3:

[1766] A generative AI model analyzes the video data and evaluates the work situation.

[1767] Input: Preprocessed video data

[1768] Output: Evaluation result of the work situation

[1769] How it works: A generative AI model analyzes video data to evaluate the progress and efficiency of a task, for example analysing how and how fast a screw is tightened.

[1770] Step 4:

[1771] The generative AI model automatically generates optimal movement patterns and areas for improvement.

[1772] Input: Evaluation result of work status

[1773] Output: Optimal operation patterns and advice on areas for improvement

[1774] Specific behavior: The generative AI model generates optimal behavior patterns and improvements based on the evaluation results. For example, it uses a prompt sentence such as, "Analyze the current work situation and suggest the optimal way and speed to tighten the screws."

[1775] Step 5:

[1776] The server displays the generated advice on a display device.

[1777] Input: Optimal behavior patterns and suggestions for improvement

[1778] Output: Advice displayed on your display device

[1779] Specific operation: The server receives the generated advice and displays it on a display device (e.g., a monitor or smart glasses). The worker can check the advice in real time to improve work efficiency.

[1780] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1781] "Example 1"

[1782] One embodiment of the present invention is a system that combines an emotion engine. This system recognizes the user's emotions and adjusts advice based on the user's emotional state. Specifically, if the user feels stressed, the emotion engine communicates this information to the generation AI. The generation AI then adjusts the advice to the user based on this information. For example, the system provides advice to a user who is stressed that is kinder or includes encouraging words.

[1783] "Example 2"

[1784] In addition, the emotion engine visualizes the user's emotional state to other team members. This enables team members to understand the emotional state of other members and communicate appropriately. Specifically, the emotion engine visualizes the user's emotional state on a chat tool. For example, if a user feels happy, this information is displayed on the chat tool as "Member A is happy." This allows other members to know that Member A is happy and to communicate based on that information.

[1785] The process flow of each embodiment will be described below.

[1786] "Example 1"

[1787] Step 1: Users communicate using a chat tool.

[1788] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[1789] Step 3: The emotion engine communicates the recognized emotional state to the generative AI.

[1790] Step 4: The generative AI tailors the advice to the user based on their emotional state.

[1791] "Example 2"

[1792] Step 1: Users communicate using a chat tool.

[1793] Step 2: The emotion engine recognizes the user's emotions. This emotion recognition is done from the user's text and / or voice input.

[1794] Step 3: The emotional state recognized by the emotion engine is visualized on the chat tool.

[1795] Step 4: Other team members see the visualized emotional state and communicate based on it.

[1796] Example 1

[1797] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1798] In remote work and distributed teams, it is difficult to grasp the work status of team members and the advice they need in real time. It is also difficult to provide appropriate advice that takes into account the emotional state of members. This can lead to reduced communication efficiency and affect the productivity of the entire team.

[1799] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for inputting a post of a communication tool to a generative AI model, a means for the generative AI model to analyze the posted content, a means for visualizing the work currently being performed by team members and advice that will be required in the future based on the analysis result, and a means for recognizing the user's emotions and adjusting advice based on the emotional state. This makes it possible to grasp the work status and necessary advice of team members in real time and provide appropriate advice according to the emotional state.

[1800] A "communication tool" is software or a platform that enables users to send and receive messages.

[1801] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing to analyze text data and generate appropriate responses or advice.

[1802] "Posted content" refers to messages and text data sent by users on communication tools.

[1803] "Analysis" is the process by which the generative AI model understands the content of a post and extracts meaning from it.

[1804] "Visualization" is the process of visually displaying information based on analysis results so that users can easily understand it.

[1805] An “emotion engine” is software or algorithms for recognizing a user’s emotional state and providing that information to a generative AI model.

[1806] "Advice" is advice or suggestions that the generative AI model provides to the user based on the analysis results and emotional state.

[1807] "Remote work" refers to working in which work is carried out via the Internet from a physically distant location.

[1808] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1809] This invention is a system for streamlining communication in remote work and distributed teams, and understanding the work status of team members and necessary advice in real time. This system is realized by combining communication tools, generative AI models, and emotion engines.

[1810] Hardware and Software Used

[1811] Hardware: Servers, terminals (PCs, smartphones)

[1812] Software: communication tools (e.g. Slack, Microsoft Teams), generative AI models (e.g. GPT-4), emotion engines

[1813] System Configuration

[1814] 1. User Submissions

[1815] A user uses a terminal to post a message on a communication tool. For example, the user posts, "I will be writing a report today."

[1816] 2. Acquiring posted content

[1817] The server uses the communication tool's API to obtain user posts in real time.

[1818] 3. Analysis of posted content

[1819] The server sends the acquired post content to the generative AI model for analysis. The generative AI model understands the post content and recognizes that the user is creating a report.

[1820] 4. Information Visualization

[1821] Based on the analysis results from the generative AI model, the server makes the information that the relevant member is writing a report visible to other team members. Specifically, it displays "Mr. / Ms. XX is writing a report" on the chat tool.

[1822] 5. Generating Advice

[1823] The generative AI model takes into account past posts and the skill sets of team members to generate advice on report creation. The server presents the advice to team members.

[1824] 6. Use of Emotion Engines

[1825] The emotion engine recognizes the user's emotions and conveys that information to the generative AI model. For example, if the emotion engine determines that the user is feeling stressed, it will send that information to the generative AI model.

[1826] The generative AI model uses information from the emotion engine to tailor its advice to users, for example providing gentler or more encouraging advice to a user who is feeling stressed.

[1827] Examples

[1828] User submission examples

[1829] I will be writing a report today.

[1830] Example prompts for generative AI models

[1831] "A user posts, 'Today I'm going to create a report.' Use this information to make it visible to other members that the user is creating a report, and generate advice on creating the report."

[1832] Advice examples

[1833] "When preparing a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports."

[1834] Example of using the emotion engine

[1835] The emotion engine detects the user's stress and passes that information to the generative AI model.

[1836] The generative AI model generates gentle advice such as, "When writing a report, it is most efficient to first consider the overall structure. Take your time and take breaks as you go," and returns this advice to the server.

[1837] In this way, the system can analyze the content posted by users, visualize the work status and necessary advice of team members in real time, and provide appropriate advice according to their emotional state.

[1838] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1839] Step 1:

[1840] A user uses a terminal to post a message on a communication tool. The input is the user's message, and the output is the message sent to the communication tool. A specific example of this is when a user accesses a chat tool from a PC or smartphone, types "I will be writing a report today," and sends it.

[1841] Step 2:

[1842] The server uses the API of the communication tool to obtain user posts in real time. The input is the message data obtained from the communication tool, and the output is the message data stored on the server. Specifically, the server calls the Slack API and obtains new messages from a specific channel.

[1843] Step 3:

[1844] The server sends the acquired post content to the generative AI model for analysis. The input is the message data stored on the server, and the output is the analysis result returned from the generative AI model. In concrete terms, the server sends the message "Today I will create a report" as a prompt to the generative AI model, and receives the analysis result.

[1845] Step 4:

[1846] Based on the analysis results from the generative AI model, the server visualizes to other team members the information that the relevant member is writing a report. The input is the analysis results from the generative AI model, and the output is the information displayed in the chat tool. In concrete terms, the server automatically posts to the chat tool, "Mr. / Ms. XX is writing a report."

[1847] Step 5:

[1848] The generative AI model takes into account the content of past posts and the skill sets of members, and generates advice on creating reports. The input is the content of past posts and the skill sets of members, and the output is the generated advice. In concrete terms, the generative AI model generates advice such as "When creating a report, it is efficient to first consider the overall structure. It is also a good idea to refer to past reports," and returns this advice to the server.

[1849] Step 6:

[1850] The emotion engine recognizes the user's emotions and transmits that information to the generative AI model. The input is the user's emotional state, and the output is the emotional information sent to the generative AI model. Specifically, the emotion engine determines that the user is feeling stressed based on the content of the user's posts and facial expression recognition data, and sends that information to the generative AI model.

[1851] Step 7:

[1852] The generative AI model adjusts advice to the user based on information from the emotion engine. The input is emotion information from the emotion engine, and the output is adjusted advice. In concrete terms, the generative AI model generates gentle advice such as, "When writing a report, it is efficient to first consider the overall structure. Don't push yourself, and take breaks as you go along," and returns this advice to the server.

[1853] (Application example 1)

[1854] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1855] In order to improve work efficiency in factories, it is important to facilitate communication between workers and robots and to provide appropriate information about the work situation and necessary advice. However, with conventional systems, it was difficult to provide advice that took into account the emotional state of the worker, which limited the improvement of work efficiency. In addition, it was difficult for workers to give instructions to the robot and the robot's work status was not sufficiently visualized, making it difficult to manage the progress of work.

[1856] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1857] In this invention, the server includes a means for inputting a post made via a communication tool to the generating AI, a means for visualizing the work currently being done by team members and advice that will be needed in the future, a means for letting the user know what work other members are doing and what advice to give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot via a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, thereby improving work efficiency and facilitating progress management.

[1858] A "communication tool" is software or applications that allow users to exchange information through text or voice.

[1859] "Generative AI" is an artificial intelligence model that analyzes input data and generates appropriate output.

[1860] "Visualization" refers to displaying data or information visually so that users can easily understand it.

[1861] An "emotion engine" is a system that recognizes the user's emotional state and provides appropriate feedback based on that information.

[1862] A "robot" is a mechanical device that performs tasks automatically according to programmed instructions.

[1863] A "worker" is a human worker who performs a specific task in a factory.

[1864] An "instruction" is an order to perform a particular action or task.

[1865] "Analysis" refers to examining the input data in detail and understanding its meaning and structure.

[1866] "Advice" is advice or a suggestion intended to encourage appropriate action or decision in a particular situation.

[1867] "Progress management" is a management activity that monitors the progress of work or projects and keeps them moving forward as planned.

[1868] The system for implementing this invention includes a communication tool, a generation AI, an emotion engine, a robot, and a worker's terminal. The specific operation of each element and its cooperation will be described below.

[1869] System Configuration

[1870] 1. Communication tools:

[1871] Software or applications that allow workers to exchange information through text or voice.

[1872] Workers use chat tools to give instructions to the robot.

[1873] 2. Generation AI:

[1874] It uses generative AI models such as the OpenAI API to analyze chat content as it is typed.

[1875] Based on the analysis results, appropriate work instructions are issued to the robot.

[1876] Automatically generate work status and necessary advice.

[1877] 3. Emotion Engine:

[1878] Use emotion recognition libraries such as EmotionEngine to recognize the emotional state of workers.

[1879] Tailor the advice provided by the generative AI based on the perceived emotional state.

[1880] 4. Robots:

[1881] It is a mechanical device that performs tasks automatically according to programmed instructions.

[1882] It receives instructions from the generative AI and carries out the appropriate tasks.

[1883] 5. Worker terminal:

[1884] These are devices used by workers, such as smartphones and tablets.

[1885] Instructions are given to the robot through a communication tool and advice is received from the generating AI.

[1886] System Operation

[1887] The server includes a means for inputting posts from the communication tool to the generating AI, a means for visualizing the work that team members are currently doing and the advice they will need in the future, a means for letting the user know what work other members are doing and what advice they should give, a means for recognizing the user's emotional state using an emotion engine and adjusting advice based on the emotional state, a means for being installed in a robot used in a factory to facilitate communication between the worker and the robot, and a means for the worker to give instructions to the robot using a chat tool, for the generating AI to analyze the instructions and have the robot perform the appropriate work. This makes it possible to provide appropriate advice that takes into account the emotional state of the worker, improving work efficiency and facilitating progress management.

[1888] Examples

[1889] When a worker types "I'm going to write a report today" into the chat tool, the generative AI analyzes the content and instructs the robot to support the report writing. At the same time, the emotion engine recognizes the worker's emotional state and provides advice including encouraging words such as "Do your best! You can do it!" if the worker is feeling stressed.

[1890] Examples of prompt statements

[1891] Please analyze the following chat and provide your suggestions:

[1892] I'm going to create a report today.

[1893] In this way, the work efficiency in the factory can be improved.

[1894] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1895] Step 1:

[1896] A user inputs text into a chat tool using a terminal. For example, the user inputs "Today I will be writing a report." This text becomes input data into the system.

[1897] Step 2:

[1898] The server receives text entered from the chat tool. It generates a prompt sentence to send the received text to the generative AI model. This prompt sentence becomes the input data for the generative AI model to analyze.

[1899] Step 3:

[1900] The server sends a prompt to the generative AI model, which analyzes the text. The generative AI model analyzes the input text and generates work content and advice. The analysis results become the system's output data.

[1901] Step 4:

[1902] The server receives the analysis results from the generative AI model, which include action content (creation of a report) and advice. The server visualizes this information and shares it with other team members.

[1903] Step 5:

[1904] The server uses an emotion engine to recognize the user's emotional state. Based on chat content and past data, it determines whether the user is feeling stressed. This emotional state becomes the input data for the system.

[1905] Step 6:

[1906] The server receives the emotional state from the emotion engine and adjusts the advice from the generative AI model. For example, if the user is feeling stressed, it adds words of encouragement to the advice. This adjusted advice becomes the output data of the system.

[1907] Step 7:

[1908] The server sends the adjusted advice to the user's device, and the user receives the advice from the generative AI model and the adjustments made by the emotion engine through the device.

[1909] Step 8:

[1910] The user issues instructions to the robot using a terminal. The server receives the instructions and sends them to the generative AI model. The generative AI model analyzes the instructions and generates instructions for the robot to perform the appropriate tasks.

[1911] Step 9:

[1912] The server receives instructions from the generative AI model and sends them to the robot, which then follows the instructions to perform a task, such as writing a report. The progress of this task becomes the output data of the system.

[1913] Step 10:

[1914] The server receives the progress of the robot's work and makes it visible to the user through a chat tool. The user can then check the robot's progress through their device.

[1915] Example 2

[1916] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1917] In remote work and distributed teams, it is difficult to grasp the work situation and emotional state of team members in real time and provide appropriate advice and communication. In particular, communication between members tends to be lacking, which can lead to a decrease in work efficiency and team unity.

[1918] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1919] In this invention, the server includes a means for a user to start a task on a terminal, a means for the terminal to transmit task data to the server, a means for the server to analyze the data received, a means for the server to generate advice using a generative AI model, a means for the server to analyze and visualize an emotional state using an emotion engine, a means for the server to transmit the advice and emotional information generated by the server to the terminal, and a means for displaying the information received by the terminal to the user. This makes it possible to grasp the task status and emotional state of team members in real time and provide appropriate advice and communication.

[1920] "User" refers to an individual or group that performs work using the system.

[1921] "Terminal" refers to electronic devices such as computers and smartphones used by users.

[1922] "Server" refers to a central processing unit that receives, analyzes, generates, and transmits data.

[1923] "Task data" refers to information related to the task a user is performing (e.g., programming code or emotion data).

[1924] A "generative AI model" refers to an artificial intelligence model that analyzes a user's work data and generates appropriate advice.

[1925] An "emotion engine" refers to software or hardware for analyzing a user's emotional state and visualizing the results.

[1926] "Advice" refers to suggestions for improvement or comments that the generative AI model generates based on the user's work data.

[1927] "Emotion information" refers to information about the user's emotional state analyzed by the emotion engine.

[1928] "Visualization" refers to displaying analyzed data in a form that is easy for users to understand.

[1929] "Communication tools" refers to software or platforms for sharing information among team members.

[1930] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1931] This invention is a system for grasping the work situation and emotional state of team members in real time in remote work and distributed teams, and providing appropriate advice and communication. Below, we will show how to specifically implement this system.

[1932] 1. Generate the system program

[1933] This system combines a generative AI model and an emotion engine to grasp the work situation and emotional state of team members in real time and provide necessary advice and emotional information. For example, OpenAI's GPT-4 is used as the generative AI model, and the emotion engine uses Microsoft's Azure Cognitive Services Emotion API.

[1934] 2. Explain the program's processing in natural language

[1935] The server hosts a generative AI model and an emotion engine. The generative AI model analyzes the user's work data (e.g., programming code) and generates appropriate advice. The emotion engine analyzes the user's emotional state (e.g., facial expressions and voice) and visualizes the results.

[1936] The terminals are PCs or smartphones used by users, and send data to the server. Specifically, the terminals send the user's task data and emotion data to the server.

[1937] The server analyzes the data it receives and uses a generative AI model to generate advice. For example, if a member is working on a program, the server analyzes the member's code and generates advice on how to improve the code or identify potential bugs.

[1938] The emotion engine also analyzes the user's emotional state and visualizes the results on the chat tool. For example, if the emotion engine determines that the user is happy, the server will display "Member A is happy" on the chat tool.

[1939] 3. Add examples to your description

[1940] As a concrete example, consider the following scenario.

[1941] Scenario 1: Programming task advice

[1942] User A is programming on a device. User A's device sends the code he is working on to the server. The server analyzes the code using a generative AI model (GPT-4) and generates advice such as the following:

[1943] "This part of the code is inefficient and can be improved as follows."

[1944] "There may be a bug in this line. Details below."

[1945] Example prompt:

[1946] Analyze User A's code and generate advice pointing out areas for improvement and potential bugs.

[1947] Code: [User A's code]

[1948] Scenario 2: Visualizing Emotional States

[1949] User B is working on a device. The emotion engine analyzes User B's facial expression and determines that User B is happy. The server displays this information on the chat tool.

[1950] "Member B is happy."

[1951] Example prompt:

[1952] Analyze User B's emotional state and visualize it on the chat tool.

[1953] Emotional state: Joy

[1954] This system enables users to efficiently carry out their own tasks and to understand the emotional states of other members and communicate appropriately.

[1955] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1956] Step 1:

[1957] The user begins working at the terminal.

[1958] Input: The device used by the user (PC or smartphone)

[1959] Output: Working data (programming code and emotion data)

[1960] Specific operation: A user starts programming or other work on his / her PC or smartphone. For example, User A opens Visual Studio Code and starts programming.

[1961] Step 2:

[1962] The terminal transmits the work data to the server.

[1963] Input: User-generated task data (programming code and emotion data)

[1964] Output: The work data sent to the server.

[1965] Specific operation: User A's device transmits the programming code written in real time and facial expression data captured by the webcam to the server.

[1966] Step 3:

[1967] The server parses the received data.

[1968] Input: Working data sent from the terminal (programming code and emotion data)

[1969] Output: Analysis results (code structure and content, emotional state)

[1970] Specific operation: The server analyzes the received programming code and understands the structure and content of the code. It also analyzes User A’s facial expressions from the webcam footage to understand his / her emotional state.

[1971] Step 4:

[1972] The server generates advice using a generative AI model.

[1973] Input: Parsed programming code

[1974] Output: Generated advice

[1975] Specific operation: The server uses a generative AI model (GPT-4) to generate advice for User A's programming code. For example, it generates specific advice such as "This loop is inefficient, so it would be better to use a list comprehension."

[1976] Step 5:

[1977] The server uses an emotion engine to analyze and visualize the emotional state.

[1978] Input: Parsed emotion data

[1979] Output: Visualized emotion information

[1980] Specific operation: The server uses the emotion engine (Emotion API) to determine that User A is happy. The server visualizes this information on the chat tool as "Member A is happy."

[1981] Step 6:

[1982] The server transmits the generated advice and emotion information to the terminal.

[1983] Input: Generated advice and visualized emotion information

[1984] Output: Advice and sentiment sent to the device

[1985] Specific operation: The server sends the generated advice and emotion information to the terminal of user A. For example, the server sends advice such as "This loop is inefficient, so it would be better to use a list comprehension" and emotion information such as "Member A is happy."

[1986] Step 7:

[1987] The terminal displays the received information to the user.

[1988] Input: Advice and sentiment information sent from the server

[1989] Output: Advice and emotion information displayed to the user

[1990] Specific operation: User A's device displays advice on Visual Studio Code and emotional information on the chat tool. For example, "This loop is inefficient, so it would be better to use a list comprehension" is displayed in the Visual Studio Code editor, and "Member A is happy" is displayed in the chat tool.

[1991] (Application example 2)

[1992] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1993] In conventional teamwork, it is difficult to grasp the working status of team members and necessary advice in real time, which often hinders efficient communication. In addition, there is no way to grasp the emotional state of team members, making it difficult to communicate appropriately. This can lead to a decrease in work efficiency and friction within the team.

[1994] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1995] In this invention, the server includes a means for inputting posts from a communication tool to a generation AI, a means for visualizing the work that team members are currently doing and advice that they will need in the future, a means for letting users know what work other members are doing and what advice they should give, a means for recognizing and visualizing the emotional state of users in real time, and a means for generating and displaying necessary advice based on the work situation. This makes it possible to grasp the work situation and emotional state of team members in real time and provide appropriate advice.

[1996] "Communication tools" are software or platforms that allow team members to share information and stay in touch.

[1997] "Generative AI" is an artificial intelligence system that automatically generates advice and information based on input data.

[1998] "Visualization" is a means of visually displaying data or information to make it easier to understand.

[1999] "User's emotional state" refers to the emotions and psychological state that the user is currently feeling.

[2000] "Real-time" refers to processing and display occurring immediately, without delay.

[2001] "Work status" refers to the progress and content of the work that team members are currently doing.

[2002] "Advice" means advice or a suggestion given to improve a task or solve a problem.

[2003] A "display" is a means of visually presenting information to a user.

[2004] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The server uses a generative AI model to automatically generate the work status and necessary advice of team members, and has the function of recognizing and visualizing the emotional state of the user in real time. The terminal is a device such as smart glasses or a smartphone, and serves to provide this information to the user. The user receives advice and feedback of the emotional state in real time through these devices.

[2005] The server uses the following software and hardware:

[2006] OpenCV: A library for image processing

[2007] EmotionRecognizer: Software for Recognizing a User's Emotional State

[2008] CodeAnalyzer: software to analyze the work of team members

[2009] AIAdvisor: An artificial intelligence system for generating needed advice

[2010] The terminals are devices such as smart glasse...

Claims

1. A means for acquiring messages posted on a communication tool used for sharing information among a plurality of users and for each user to send and receive messages; A means for inputting a prompt sentence including the content of the acquired message into a generative AI model, thereby causing the generative AI model to share the content of the message with other members and generate advice for the message; A means for presenting information generated by the generative AI model to each user through the communication tool; A system including:

2. A means for acquiring work data indicating the work status of each team member from the terminal of each team member; A means for analyzing the acquired operation data; A means for generating advice regarding work content using a generative AI model based on an analysis result of the work data; A means for notifying each member of the generated advice to their terminal; A system including:

3. A means for acquiring video data capturing a work situation; A means for evaluating the progress and efficiency of the work based on the acquired video data; A means for generating advice indicating an optimal operation pattern or an improvement point using a generating AI model based on an evaluation result of the progress and efficiency of the work; A means for presenting the generated advice to a worker; A system including:

Images