System

Abstract

To provide a system.SOLUTION: A system includes: means that makes the details of an evaluation interview with a person to be evaluated carried out by an evaluator into a text; means that acquires data of the emotions of the person to be evaluated; means that inputs, to generative AI, the details of the evaluation interview that has been made into the text and the emotion data; and means that provides feedback about the evaluation interview from the generative AI to the evaluator.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, there is no opportunity for managers to receive feedback on the content of the evaluation interviews they conduct with their employees, so they have to reflect on the interviews themselves to see if the content was appropriate or if there are any areas for improvement. [Means for solving the problem]

[0005] To solve this problem, we record the contents of evaluation interviews conducted by managers with their employees as audio logs and convert the audio logs into text.The text contents of the evaluation interviews are then input into a generative AI, and feedback is received from the generative AI.Furthermore, the feedback from the generative AI is provided to managers to encourage improvements in the quality of the evaluation interviews. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention is a system that records evaluation interviews conducted by managers with their team members as audio logs. This system has the function of recording the contents of the evaluation interviews in real time and converting the audio data into text data. Specifically, the system converts the audio data into text using voice recognition technology.

[0029] "Example 2"

[0030] Furthermore, it has the function of inputting the textual content of the evaluation interview into a generative AI and receiving feedback from the generative AI. The generative AI generates feedback such as good and bad points based on the content of the evaluation interview. Specifically, it analyzes the text data using natural language processing technology and evaluates the quality of the evaluation interview.

[0031] "Example 3"

[0032] Furthermore, the system also has a function that provides feedback from the generative AI to managers and encourages them to improve the quality of evaluation interviews. Specifically, it has a notification system that notifies managers of feedback from the generative AI. This allows managers to objectively understand the quality of their own evaluation interviews and identify areas for improvement.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Managers initiate evaluation interviews with members. These evaluation interviews are conducted in the form of direct dialogue between managers and members.

[0036] Step 2: Once the assessment interview begins, the system automatically records the interview as an audio log in real time, capturing the entire interview.

[0037] Step 3: After the evaluation interview is completed, the system converts the recorded audio log into text using voice recognition technology.

[0038] "Example 2"

[0039] Step 1: The system inputs the text of the evaluation interview into the generative AI. This input is done automatically, without human intervention.

[0040] Step 2: The generative AI analyzes the input text data and generates feedback on good and bad points based on the content of the evaluation interview. This analysis and feedback generation is carried out using natural language processing technology.

[0041] "Example 3"

[0042] Step 1: Once the generative AI feedback is generated, the system provides it to the manager via a notification system, where the manager can view the feedback on their device.

[0043] Step 2: Based on the feedback provided, managers can objectively assess the quality of their own evaluation interviews and identify areas for improvement, which will promote improvements in the quality of evaluation interviews.

[0044] Example 1

[0045] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] Conventional methods for recording evaluation interviews have had the problem of making it difficult to accurately record the interview content and analyze it later. In addition, there are limited means of obtaining feedback to improve the quality of the interviews, making it difficult for managers to identify areas for improvement in the interviews.

[0047] The identification process by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes a means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, a means for receiving audio data in real time and converting it into text data using speech recognition technology, a means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, and a means for saving the generated text data in a database. This makes it possible to accurately record the contents of the evaluation interviews and analyze them in detail later. Furthermore, specific improvements to improve the quality of the interviews can be identified through feedback from the generative AI.

[0048] A "management position" is someone in a position within an organization who evaluates and guides members.

[0049] A "member" is someone in an organization who is in a position to receive guidance and evaluation from management.

[0050] An "evaluation interview" is an interview conducted by a manager with a member of staff to evaluate their work and provide feedback.

[0051] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0052] "Text conversion" is the process of converting audio data into text data.

[0053] "Generative AI" is an artificial intelligence system that generates feedback and analytical results based on input data.

[0054] "Feedback" refers to evaluation information such as good points and areas for improvement that is provided by generative AI based on the content of the evaluation interview.

[0055] A "database" is an information system for storing and managing text data and feedback.

[0056] "Voice recognition technology" is a technology that analyzes voice data and converts it into text data.

[0057] "Real-time" refers to processing occurring immediately while the assessment interview is in progress.

[0058] This invention is a system that records evaluation interviews between managers and their employees as audio logs, converts the audio data into text data, and inputs it into a generative AI to obtain feedback. A specific embodiment of this system is described below.

[0059] Server Roles

[0060] The server receives the voice data from the evaluation interview in real time and converts it into text data using speech recognition technology. Specifically, it uses speech recognition software such as Google® Cloud Speech-to-Text API. The converted text data is stored in a database such as MySQL® or PostgreSQL.

[0061] Device Role

[0062] The device provides an interface for managers to start recording when they begin an evaluation interview. A dedicated recording application is used to send the audio data being recorded to the server in real time. After the interview ends, the recording is stopped and the text data is retrieved from the server and displayed.

[0063] User Roles

[0064] The user (management) presses the recording button on the device to start the evaluation interview. During the interview, the user can check the text data in real time to confirm whether specific keywords and phrases were accurately recorded. After the interview is over, the user stops recording and reviews the generated text data.

[0065] Specific examples

[0066] Example 1: Recording and transcribing evaluation interviews

[0067] Scenario: Manager A conducts an evaluation interview with member B.

[0068] Hardware used: Manager A's smartphone

[0069] Software used: Dedicated recording application, Google Cloud Speech-to-Text API, MySQL database

[0070] 1. Manager A launches the dedicated recording application on his smartphone and presses the record button.

[0071] 2. The smartphone records the interview audio and transmits it to the server in real time.

[0072] 3. The server converts the audio data into text using the Google Cloud Speech-to-Text API.

[0073] 4. The server stores the converted text data in a MySQL database.

[0074] 5. After the interview, Manager A stops the recording and checks the text data generated by the smartphone application.

[0075] Example prompts to be input to the generative AI model

[0076] Prompt: "Describe a system that records managerial interviews with employees and converts the audio data into text in real time. Include the names of the specific hardware and software."

[0077] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[0078] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0079] Step 1:

[0080] Subject: User

[0081] The user presses the recording button on the terminal when starting the evaluation interview.

[0082] Input: User operation (pressing the record button)

[0083] Output: Trigger to start recording

[0084] What happens: The user taps the record button in the smartphone or tablet application to start the recording process.

[0085] Step 2:

[0086] Subject: Terminal

[0087] The terminal detects that the recording button has been pressed and starts recording.

[0088] Input: Trigger to start recording

[0089] Output: Audio data being recorded

[0090] Specific operation: Captures audio data using the device's microphone and prepares to send it to the server in real time.

[0091] Step 3:

[0092] Subject: Terminal

[0093] The terminal transmits the audio data being recorded to the server in real time.

[0094] Input: Audio data being recorded

[0095] Output: Sending audio data to the server

[0096] Specific behavior: Recorded audio data is sent to the server using WebSocket or HTTP streaming.

[0097] Step 4:

[0098] Subject: Server

[0099] The server receives the voice data transmitted from the terminal in real time.

[0100] Input: Audio data from the device

[0101] Output: Temporarily saves received audio data

[0102] Specific operation: Receives audio data via an HTTP request and temporarily stores it in the server's local storage or cloud storage.

[0103] Step 5:

[0104] Subject: Server

[0105] The server converts the received voice data into text data using voice recognition technology.

[0106] Input: Received audio data

[0107] Output: Converted text data

[0108] What it does: Uses the Google Cloud Speech-to-Text API to create an API request to convert audio data to text and send the audio data.

[0109] Step 6:

[0110] Subject: Server

[0111] The server stores the converted text data in a database.

[0112] Input: Converted text data

[0113] Output: Text data stored in a database

[0114] Specific operation: Executes an INSERT query in a database such as MySQL or PostgreSQL and saves text data.

[0115] Step 7:

[0116] Subject: User

[0117] The user presses the stop recording button after the interview is over.

[0118] Input: User action (pressing the stop recording button)

[0119] Output: Trigger to stop recording

[0120] What happens: The recording process ends when the user taps the stop recording button in the smartphone or tablet application.

[0121] Step 8:

[0122] Subject: Terminal

[0123] The terminal detects that the recording stop button has been pressed and stops recording.

[0124] Input: Trigger to stop recording

[0125] Output: End of recording

[0126] Specific Action: Ends the recording process and stops sending data to the server.

[0127] Step 9:

[0128] Subject: Terminal

[0129] The terminal displays the text data obtained from the server to the user.

[0130] Input: Text data from the server

[0131] Output: Text data that is displayed to the user

[0132] Specific behavior: Renders text data in the application's UI and displays it for the user to see.

[0133] Step 10:

[0134] Subject: User

[0135] The user reviews the generated text data and adds corrections or comments as necessary.

[0136] Input: The displayed text data

[0137] Output: Text data with corrections and comments added

[0138] Specific Behavior: Use the application's text editor function to add corrections and comments to text data.

[0139] (Application example 1)

[0140] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0141] The conventional evaluation interview system had the problem that it was difficult for managers to accurately record the contents of interviews with team members and analyze them later. There was also a lack of a way to efficiently record work instructions and reports within the factory and convert them into text data in real time. This made it difficult to manage work progress and quality, and there was a risk of overall productivity decreasing.

[0142] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0143] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the textual contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for recording work instructions and reports within the factory as audio logs and converting them into text data in real time, and means for storing the converted text data in the cloud so that it can be searched and analyzed later. This makes it possible to accurately record the contents of evaluation interviews and receive feedback from the generative AI, as well as efficiently record work instructions and reports within the factory and convert them into text data in real time.

[0144] A "management position" is a position within an organization that is responsible for supervising and evaluating the work of its members.

[0145] "Members" refer to employees or staff members who are responsible for specific tasks within an organization.

[0146] An "evaluation interview" is a dialogue-style interview in which managers evaluate their team members' work performance and results and provide feedback.

[0147] An "audio log" is a digital file that records audio data such as evaluation interviews and work instructions.

[0148] "Text conversion" is the process of converting audio data into text data.

[0149] "Generative AI" is an artificial intelligence system that generates feedback and suggestions based on input data.

[0150] "Feedback" refers to evaluations and information on areas for improvement provided by generative AI based on the content of evaluation interviews and work instructions.

[0151] "Inside the factory" refers to the interior of the facility where the product is manufactured or processed.

[0152] "Work instructions" are information that instructs specific work content and procedures within a factory.

[0153] "Report" is information used to communicate the progress and results of work.

[0154] "Real-time" refers to data being processed and transformed as it is generated.

[0155] "Cloud" refers to a collection of remote servers provided over the internet for data storage and processing.

[0156] "Searching" is the process of locating specific information from stored data.

[0157] "Analysis" is the process of examining collected data in detail to find meaning and trends.

[0158] A system for implementing this invention is configured as follows: First, a manager uses smart glasses or a device with a microphone to record the contents of an evaluation interview with a member of staff as an audio log. This allows audio data from the evaluation interview to be collected in real time.

[0159] The collected voice data is then converted to text using a Python program with the Google Cloud Speech-to-Text API. This text data is then input into a generative AI system, which generates feedback based on the content of the evaluation interview. The generated feedback is provided to managers to encourage them to improve the quality of the evaluation interviews.

[0160] Furthermore, work instructions and reports within the factory are also recorded as voice logs and converted into text data in real time. This text data is stored in the cloud and can be searched and analyzed later. This allows for efficient progress management and quality control of work.

[0161] The hardware used includes smart glasses, a device with a microphone, a PC or a cloud server, and the software used is Python, the Google Cloud Speech-to-Text API, audio-related libraries, and wave.

[0162] For example, when a worker in a factory reports, "Let's proceed to the next step," the voice is recorded by smart glasses and converted in real time into text data that says, "Let's proceed to the next step." This data is stored in the cloud and can be used later by managers to check the progress of work.

[0163] An example prompt might look like this:

[0164] "Your task is to develop a system that records work instructions and reports in the factory as voice logs and converts them into text data in real time. This will be done using smart glasses and the Google Cloud Speech-to-Text API to convert the voice data into text."

[0165] The above is an embodiment of the present invention.

[0166] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0167] Step 1:

[0168] The user wears the smart glasses and conducts evaluation interviews and gives instructions and reports on work in the factory. The microphone in the smart glasses collects voice data in real time. The input is the user's voice, and the output is voice data.

[0169] Step 2:

[0170] The terminal (smart glasses) temporarily stores collected voice data. The stored voice data is used for subsequent processing. The input is voice data, and the output is the temporarily stored voice data.

[0171] Step 3:

[0172] The device sends the temporarily saved voice data to the cloud server. The cloud server receives the voice data and starts the voice recognition process. The input is the temporarily saved voice data, and the output is the voice data sent to the cloud server.

[0173] Step 4:

[0174] The cloud server uses the Google Cloud Speech-to-Text API to convert voice data into text data. The input is voice data and the output is text data. Specifically, the voice data is sent to the API and the returned text data is received.

[0175] Step 5:

[0176] The cloud server inputs text data into the generative AI and generates feedback. The input is text data, and the output is feedback from the generative AI. Specifically, the text data is input into the AI ​​model and the generated feedback is received.

[0177] Step 6:

[0178] The cloud server provides the generated feedback to managers, who then receive the feedback and use it to improve the quality of the evaluation interviews. The input is the feedback from the generative AI, and the output is the feedback provided to managers.

[0179] Step 7:

[0180] The cloud server stores text data of work instructions and reports within the factory in the cloud. The stored data can be searched and analyzed later. The input is text data, and the output is data stored in the cloud.

[0181] Step 8:

[0182] Managers and other users search and analyze data stored in the cloud. This allows for efficient progress management and quality control of work. The input is data stored in the cloud, and the output is the search and analysis results. Specifically, the system searches for the necessary information from a database on the cloud and analyzes the data using analysis tools.

[0183] Example 2

[0184] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0185] In traditional appraisal interview systems, recording interview content and generating feedback is often done manually, which takes time and effort. Furthermore, issues include inconsistent quality of feedback and not clearly indicating areas for improvement in appraisal interviews. This makes it difficult to improve the quality of appraisal interviews and can hinder smooth communication between managers and employees.

[0186] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for recording the contents of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for generating a prompt sentence to input the converted contents of the evaluation interview into the generative AI, means for receiving feedback from the generative AI, and means for returning the received feedback to the user. This makes it possible to automatically analyze the contents of the evaluation interview and quickly provide high-quality feedback.

[0187] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[0188] "Texting" is the process of converting audio logs into written data.

[0189] "Generative AI" is an artificial intelligence model that analyzes input data and generates feedback or other outputs.

[0190] A "prompt sentence" is an input sentence that instructs a generative AI to perform a specific analysis or generation.

[0191] "Feedback" refers to evaluation information such as good and bad points that is generated by generative AI based on the content of the evaluation interview.

[0192] "Users" are system users such as managers and members who receive feedback from evaluation interviews.

[0193] MODE FOR CARRYING OUT THE INVENTION

[0194] The present invention relates to a system for efficiently analyzing the content of an evaluation interview and providing high-quality feedback. Specific embodiments of this system will be described below.

[0195] Hardware and software used

[0196] Hardware: Server (e.g., virtual server for cloud services)

[0197] Software: Generative AI models (e.g., natural language processing models), natural language processing libraries (e.g., spaCy)

[0198] System configuration

[0199] The system consists of the following major components:

[0200] 1. Recording and transcribing audio logs

[0201] The user records the contents of the evaluation interview as an audio log, for example, by using a recording device to collect audio data during the evaluation interview.

[0202] The server receives the recorded voice log and converts it into text data using voice recognition technology. Specifically, it uses a voice recognition API to convert the voice log into text.

[0203] 2. Prompt generation

[0204] The server generates a prompt to be input into the generative AI model based on the textual content of the evaluation interview. For example, it creates a prompt such as, "Please analyze the content of the following evaluation interview and provide feedback on the good and bad points."

[0205] 3. Input to the generative AI model and analysis

[0206] The server sends the generated prompt and text data to the generative AI model, which then analyzes the text data using natural language processing technology and generates feedback based on the content of the evaluation interview.

[0207] 4. Receiving and Providing Feedback

[0208] The server receives the feedback generated by the generative AI model and returns it to the user, for example, by formatting the feedback in HTML and displaying it in the user's browser.

[0209] Specific examples

[0210] For example, a user saves the contents of an evaluation interview in a file called "evaluationinterview.txt" and drags and drops that file into the system's upload form. The server receives the uploaded "evaluationinterview.txt" and reads the text data into memory. The server then generates a prompt message saying, "Please analyze the contents of the evaluation interview below and provide feedback on the good and bad points," and sends it to the generative AI model.

[0211] The generative AI model analyzes the received prompt and text data and generates feedback such as, "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems." The server receives this feedback and sends it back to the user.

[0212] Prompt Sentence Examples

[0213] Please analyze the following evaluation interview and provide feedback on what was good and what was bad. The evaluation interview content is as follows: "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems."

[0214] In this way, the system can efficiently analyze the content of the evaluation interview and provide high-quality feedback quickly.

[0215] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0216] Step 1:

[0217] The user records the contents of the evaluation interview as an audio log. For example, during the evaluation interview, audio data is collected using a recording device. The input is the audio data, and the output is the recorded audio log.

[0218] Step 2:

[0219] The server receives audio logs uploaded by users. The input is the audio logs, and the output is the audio data stored on the server.

[0220] Step 3:

[0221] The server uses speech recognition technology to convert the voice log into text data. Specifically, it uses a speech recognition API to analyze the voice log and generate text data. The input is the voice log, and the output is text data.

[0222] Step 4:

[0223] The server generates a prompt to be input to the generative AI model based on the textual content of the evaluation interview. For example, it creates a prompt such as, "Please analyze the content of the following evaluation interview and provide feedback on the good and bad points." The input is text data, and the output is a prompt.

[0224] Step 5:

[0225] The server sends the generated prompt sentence and text data to the generative AI model. Specifically, it sends the data using an API request. The input is the prompt sentence and text data, and the output is a request to the generative AI model.

[0226] Step 6:

[0227] The generative AI model analyzes the received prompt and text data. It uses natural language processing technology to understand the content of the text data and extracts the good and bad points of the evaluation interview. The input is the prompt and text data, and the output is feedback.

[0228] Step 7:

[0229] A generative AI model generates feedback based on the analysis results. For example, it generates feedback such as, "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems." The input is the analysis results, and the output is the feedback.

[0230] Step 8:

[0231] The server receives the feedback generated from the generative AI model. The feedback is sent back to the server through an API response. The input is the feedback, and the output is the feedback stored on the server.

[0232] Step 9:

[0233] The server sends the received feedback back to the user, for example, formatting the feedback in HTML and displaying it in the user's browser. The input is the feedback, and the output is the feedback displayed in the user's browser.

[0234] (Application example 2)

[0235] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0236] In conventional evaluation interview systems, recording interview content and generating feedback is done manually, which takes time and effort, and there are problems with inconsistent evaluation quality. Another issue is that the feedback is vague and lacks specificity, making it difficult for the members being evaluated to clearly understand areas for improvement. Furthermore, creating evaluation reports is time-consuming, placing a heavy burden on managers.

[0237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0238] In this invention, the server includes a means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, a means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, and a means for automatically generating an evaluation report based on the feedback from the generative AI. This makes it possible to efficiently record the contents of evaluation interviews and automatically generate high-quality feedback. Furthermore, the automatic generation of evaluation reports reduces the burden on managers and improves the quality of evaluations.

[0239] A "management position" is a position within an organization that supervises and evaluates the work of members.

[0240] A "member" is an employee who is responsible for a specific task within an organization.

[0241] An "evaluation interview" is a conversation in which a manager evaluates a member's work performance and provides feedback.

[0242] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0243] "Transcription" refers to the process of converting audio logs into written data.

[0244] "Generative AI" refers to artificial intelligence that automatically generates feedback and reports based on input data.

[0245] "Feedback" refers to evaluation information such as good and bad points provided by generative AI based on the content of the evaluation interview.

[0246] An "evaluation report" is a summary document of evaluations that is automatically generated based on feedback from generative AI.

[0247] The system for implementing this invention efficiently records the content of evaluation interviews conducted by managers with their team members, generates high-quality feedback, and automatically generates evaluation reports. A specific embodiment of this system will be described below.

[0248] System configuration

[0249] 1. Speech Recognition Module

[0250] The server uses a speech recognition module to record the contents of the evaluation interview as an audio log. This module has the function of converting voice data into text data. Specifically, it uses a speech recognition library and Google's speech recognition API.

[0251] 2. Natural Language Processing Module

[0252] The server uses a natural language processing (NLP) module to analyze the textual content of the evaluation interview. This module has the function of analyzing the text data and evaluating the quality of the evaluation. Specifically, it uses the pipeline function of the transformers library and utilizes the Japanese BERT model.

[0253] 3. Generative AI Module

[0254] The server uses generative AI to generate feedback based on the data analyzed by the natural language processing module. This generative AI automatically generates feedback on the positive and negative points based on the input data.

[0255] 4. Report Generation Module

[0256] The server automatically generates an evaluation report based on feedback from the generative AI, which is provided to managers and used to encourage them to improve the quality of evaluation interviews.

[0257] System Operation

[0258] 1. Recording and transcribing audio logs

[0259] When a manager conducts an evaluation interview with a member, the contents are recorded as an audio log, which is then converted into text data by a speech recognition module.

[0260] 2. Text Data Analysis

[0261] The transcript of the interview is then analyzed by a natural language processing module, which evaluates the quality of the evaluation and extracts key points.

[0262] 3. Generate feedback

[0263] Based on the analysis results, the generative AI module generates feedback, including good and bad points.

[0264] 4. Automatic generation of evaluation reports

[0265] Based on the generated feedback, the report generation module automatically generates an evaluation report, which is provided to management to encourage improvement of the quality of evaluation interviews.

[0266] Specific examples

[0267] For example, by inputting the following prompt sentence into a generative AI model, specific feedback can be obtained.

[0268] Contents of the evaluation interview:

[0269] Worker A works quickly and produces high quality work.

[0270] However, there appears to be a lack of communication.

[0271] In the future, it will be necessary to strengthen cooperation with the team.

[0272] Generate feedback.

[0273] By inputting this prompt into a generative AI model, specific feedback can be obtained and an evaluation report can be automatically generated, allowing managers to conduct evaluation interviews efficiently and provide high-quality feedback.

[0274] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0275] Step 1:

[0276] The server records the contents of the evaluation interviews between managers and employees as audio logs. The audio logs are collected in real time during the evaluation interviews and saved as audio files. The input is the audio data of the evaluation interviews, and the output is an audio file.

[0277] Step 2:

[0278] The server uses a speech recognition module to convert the recorded voice log into text data. Specifically, it uses a speech recognition library to call Google's speech recognition API and converts the voice data into text data. The input is an audio file and the output is text data.

[0279] Step 3:

[0280] The server analyzes the textual content of the evaluation interview using a natural language processing (NLP) module. It uses the pipeline function of the transformers library to apply the Japanese BERT model to analyze the text data. The input is the text data, and the output is the analysis results.

[0281] Step 4:

[0282] The server uses a generative AI module to generate feedback based on the analysis results. The generative AI automatically generates feedback on good and bad points based on the input analysis results. The input is the analysis results, and the output is feedback data.

[0283] Step 5:

[0284] The server automatically generates an evaluation report based on the generated feedback. A report generation module is used to format the feedback data into an evaluation report. The input is the feedback data, and the output is the evaluation report.

[0285] Step 6:

[0286] The server provides the generated evaluation report to the manager, who then checks the evaluation report and uses it to improve the quality of the evaluation interview. The input is the evaluation report, and the output is provided to the manager.

[0287] Example 3

[0288] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0289] Conventional evaluation interview systems lacked a means to effectively provide feedback on the content of evaluation interviews conducted by managers, making it difficult to obtain specific advice or areas for improvement to improve the quality of evaluation interviews.In addition, there was a lack of a means to provide feedback to managers quickly and reliably, which sometimes delayed improvements to evaluation interviews.

[0290] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0291] In this invention, the server includes means for recording the content of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the text content of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for providing the generated feedback to the manager via a notification system, means for providing an interface for inputting the content of the evaluation interview, means for transmitting the content of the evaluation interview to the server using a secure communication protocol, means for generating feedback using a generative AI model, means for transmitting the generated feedback to the notification system, and means for notifying the feedback to the manager via the notification system. This enables the content of the evaluation interview to be effectively fed back, and enables managers to quickly and reliably receive specific advice and areas for improvement to improve the quality of the evaluation interview.

[0292] A "management position" is someone in a position within an organization who evaluates and guides members.

[0293] "Members" are employees and staff within an organization who are guided and evaluated by management.

[0294] An "evaluation interview" is an interview in which a manager evaluates a member's work performance and behavior and provides feedback.

[0295] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[0296] "Texting" is the process of converting audio logs into written data.

[0297] "Generative AI" is an artificial intelligence model that generates feedback and advice based on input data.

[0298] "Feedback" refers to specific advice and areas for improvement provided by the generative AI based on the content of the evaluation interview.

[0299] A "notification system" is a system for providing generated feedback to management.

[0300] The "interface" is a user interface for inputting the contents of the evaluation interview.

[0301] A "secure communication protocol" is a communication protocol for securely transmitting data.

[0302] "Server" is a computer system that uses generative AI models to generate feedback and provide it to managers via a notification system.

[0303] The present invention is a system for providing feedback to improve the quality of evaluation interviews conducted by managers. A specific embodiment of this system will be described below.

[0304] First, the user (manager) enters the details of the evaluation interview into a dedicated application or web form. For example, they write details about their subordinate's performance and areas for improvement. An example of a specific prompt sentence might be, "Subordinate A is good at managing the progress of projects, but feels that he has issues with communication with his team members. As an area for future improvement, I have suggested holding regular meetings."

[0305] Next, the device (user's PC or smartphone) sends the entered contents of the evaluation interview to the server. At this time, the data is sent using a secure communication protocol (e.g., HTTPS).

[0306] The server inputs the content of the received evaluation interview into a generative AI model (for example, GPT-4 (registered trademark) from OpenAI (registered trademark)). The generative AI model generates feedback based on the input content. This feedback includes specific advice and improvements to improve the quality of the evaluation interview.

[0307] The generated feedback is sent from the server to a notification system, which delivers the feedback to the user, for example, by email or using a notification function of a dedicated application.

[0308] Finally, the terminal displays the feedback received from the notification system to the user, who receives the notification and checks for specific advice on how to improve the quality of the evaluation interview.

[0309] This system enables managers to effectively provide feedback on the content of the evaluation interviews, and quickly and reliably obtain specific advice and points for improvement to improve the quality of the evaluation interviews. The flow of the identification process in the third embodiment will be described with reference to FIG. 15.

[0310] Step 1:

[0311] The user inputs the details of the evaluation interview.

[0312] The user (manager) enters the details of the evaluation interview using a dedicated application or web form. For example, the user might enter, "My subordinate, Mr. A, is good at managing the progress of projects, but I feel he has issues with communication with his team members. I have suggested that he hold regular meetings as an area for improvement in the future." The input data is saved in text format.

[0313] Step 2:

[0314] The terminal sends the input information to the server.

[0315] The device (user's PC or smartphone) sends the entered content of the evaluation interview to the server by pressing the "Send" button. At this time, the data is sent using a secure communication protocol (e.g., HTTPS). The input data is sent to the server in text format.

[0316] Step 3:

[0317] The server generates feedback using a generative AI model.

[0318] The server inputs the content of the received evaluation interview into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model analyzes the input text data and generates feedback including specific advice and improvements to improve the quality of the evaluation interview. The output data is the feedback in text format.

[0319] Step 4:

[0320] The server sends the generated feedback to a notification system.

[0321] The server sends the generated feedback to a notification system, which delivers the feedback to the user by, for example, email or a notification function of a dedicated application. The output data is the feedback in text format.

[0322] Step 5:

[0323] The terminal notifies the user of the feedback through a notification system.

[0324] The terminal (user's smartphone or PC) displays the feedback received from the notification system to the user. The user receives the notification and checks specific advice to improve the quality of the evaluation interview. The output data is the feedback in text format.

[0325] (Application example 3)

[0326] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0327] Conventional evaluation interview systems have the problem that it is difficult to objectively evaluate the quality of the evaluation interviews conducted by managers and identify areas for improvement. Furthermore, there is a lack of means to evaluate the performance of machines used in factory work and notify managers of this feedback, making it difficult to improve work efficiency and provide appropriate guidance. To solve these issues, a system is needed that improves the quality of evaluation interviews and properly evaluates the performance of machines in factories and provides feedback to managers.

[0328] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0329] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for notifying the managers of the feedback from the generative AI, and means for evaluating the performance of machines operating in the factory and notifying the managers of the feedback. This makes it possible to objectively grasp the quality of the evaluation interviews and identify areas for improvement, as well as to appropriately evaluate the performance of machines in the factory and provide feedback to the managers.

[0330] A "management position" is a position within an organization that is responsible for supervising and evaluating the work of its members.

[0331] "Members" are employees who work within an organization under the direction of a manager.

[0332] An "evaluation interview" is an interview conducted by a manager to evaluate the work of a member and provide feedback.

[0333] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0334] "Texting" is the process of converting audio logs into written data.

[0335] "Generative AI" is an artificial intelligence system that generates feedback and evaluations based on input data.

[0336] "Feedback" refers to the evaluation and information on areas for improvement provided by generative AI based on the content of the evaluation interview and the machine's performance.

[0337] "Notification" refers to the act of informing management of feedback from generative AI.

[0338] "Machines performing work in factories" are automated devices that perform work such as production and assembly in factories.

[0339] "Performance" refers to the efficiency and quality of operation of machines that perform work within a factory.

[0340] A system for carrying out this invention has the following configuration: The server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for notifying managers of the feedback from the generative AI, and means for evaluating the performance of machines that perform work in a factory and notifying managers of the feedback.

[0341] Hardware and software used

[0342] Hardware: Factory robots, servers, managerial devices (PCs and smartphones)

[0343] Software: Python, generative AI model (e.g., OpenAI GPT-3 (registered trademark)), mail server (e.g., SMTP)

[0344] Data processing and calculation

[0345] 1. Recording and Transcription of Audio Logs:

[0346] The server records the contents of the evaluation interview as an audio log.

[0347] Use speech recognition software (e.g., Google Speech-to-Text API) to convert audio logs into text.

[0348] 2. Getting input and feedback for generative AI models:

[0349] The server inputs the textual content of the evaluation interview into a generative AI model.

[0350] A generative AI model generates feedback based on the input text.

[0351] Example: Prompt: "Please rate the performance of robot ID: 123"

[0352] 3. Feedback Notification:

[0353] The server notifies management of feedback from the generative AI model.

[0354] For notifications, a mail server is used to send feedback to the manager's email address.

[0355] Specific examples

[0356] For example, when evaluating the performance of robot ID: 123 working in a factory, the server performs the following process.

[0357] 1. Collect robot work data and send the prompt "Please evaluate the performance of robot ID: 123" to the generative AI model.

[0358] 2. The generative AI model evaluates the robot's work efficiency and the quality of its movements, and generates feedback such as, "Robot ID: 123's work efficiency is 90%. The area for improvement is optimizing its movement speed."

[0359] 3. The server sends this feedback to the manager's email address, and the manager can use the feedback to identify areas for improvement in the robot's behavior and work efficiency.

[0360] In this way, it becomes possible to objectively grasp the quality of the evaluation interview and identify areas for improvement, as well as to properly evaluate the performance of machines in the factory and provide feedback to management.

[0361] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0362] Step 1:

[0363] The server records the contents of evaluation interviews conducted by managers with members as audio logs.

[0364] Input: Audio data from the evaluation interview

[0365] Output: Audio log

[0366] Specific operation: When an evaluation interview is conducted, the server uses the audio recording function to record the contents of the interview as audio data.

[0367] Step 2:

[0368] The server converts the recorded audio logs into text.

[0369] Input: Audio log

[0370] Output: Text data

[0371] Specific operation: The server uses voice recognition software (e.g., Google Speech-to-Text API) to convert the voice log into text data.

[0372] Step 3:

[0373] The server inputs the textual content of the evaluation interview into a generative AI model and receives feedback.

[0374] Input: Text data

[0375] Output: Feedback from the generative AI model

[0376] Specific operation: The server sends the text data to a generative AI model (e.g., OpenAI GPT-3) and inputs the prompt "Please generate feedback based on the content of the evaluation interview." The generative AI model generates feedback and sends it back to the server.

[0377] Step 4:

[0378] The server notifies management of feedback from the generative AI model.

[0379] Input: Feedback from a generative AI model

[0380] Output: Notification to management (e.g. email)

[0381] Specific operation: The server uses a mail server (e.g., SMTP) to send feedback from the generative AI model to the manager's email address.

[0382] Step 5:

[0383] The server evaluates the performance of machines operating within the factory and communicates the feedback to management.

[0384] Input: Machine operation data

[0385] Output: Feedback from the generative AI model, notification to management

[0386] How it works: The server collects work data from machines in the factory and sends a prompt to the generative AI model: "Please rate the performance of robot ID: 123." The generative AI model generates feedback and sends it back to the server. The server then sends this feedback to the manager's email address.

[0387] In this way, it becomes possible to objectively grasp the quality of the evaluation interview and identify areas for improvement, as well as to properly evaluate the performance of machines in the factory and provide feedback to management.

[0388] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0389] "Example 1"

[0390] One embodiment of the present invention provides a system including an emotion engine that recognizes a user's emotions during an evaluation interview. The emotion engine recognizes emotions from the user's voice, facial expressions, gestures, etc., and inputs the emotion data into a generative AI. Specifically, when the user expresses emotions such as joy, anger, surprise, or sadness, the emotion engine recognizes them and provides the information to the generative AI.

[0391] "Example 2"

[0392] The generative AI generates feedback based on the content of the evaluation interview and the user's emotional data. Specifically, if the user expresses joy, it determines from the content of the evaluation interview what that joy is about and generates the result as feedback. Also, if the user expresses anger, it determines from the content of the evaluation interview what that anger is about and generates the result as feedback.

[0393] "Example 3"

[0394] Feedback from the generative AI is provided to managers, who can view it on their own devices. This feedback includes not only good and bad points based on the content of the evaluation interview, but also information based on the user's emotional data. This allows managers to gain a deeper understanding of the quality of the evaluation interview and identify areas for improvement.

[0395] The processing flow of each embodiment will be described below.

[0396] "Example 1"

[0397] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[0398] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[0399] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[0400] "Example 2"

[0401] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[0402] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[0403] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[0404] Step 4: Feedback from the generative AI is provided to management, who reviews the feedback.

[0405] "Example 3"

[0406] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[0407] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[0408] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[0409] Step 4: Feedback from the generative AI is provided to management, who reviews the feedback.

[0410] Step 5: Managers take action based on the feedback to improve the quality of the evaluation interview.

[0411] Example 1

[0412] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0413] Conventional evaluation interview systems have difficulty not only recording the interview content but also recognizing the user's emotions during the interview in real time and providing feedback based on that. Furthermore, in order to improve the quality of feedback using generative AI, it is necessary to appropriately utilize the user's emotional data, but there was no established method for doing so.

[0414] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for recording the content of an evaluation interview conducted by a manager with a member as an audio log, a means for converting the audio log into text, a means for inputting the text content of the evaluation interview into a generative AI and receiving feedback from the generative AI, a means for recognizing the user's emotions, a means for inputting the recognized emotion data into the generative AI, and a means for providing the user with feedback from the generative AI. This enables high-quality feedback that utilizes not only the content of the evaluation interview but also the user's emotion data.

[0415] A "management position" is someone in a position within an organization who evaluates and guides members.

[0416] A "member" is someone in an organization who is in a position to receive guidance and evaluation from management.

[0417] An "evaluation interview" is a dialogue between a manager and a member of staff to evaluate their work and provide feedback.

[0418] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0419] "Text conversion" is the process of converting audio data into text data.

[0420] "Generative AI" is artificial intelligence that generates feedback and advice based on input data.

[0421] "Feedback" refers to evaluations and advice provided by generative AI based on the content of the evaluation interview and the user's emotional data.

[0422] "User" refers to the manager or member participating in the evaluation interview.

[0423] "Emotion data" is information about emotions recognized from the user's voice, facial expressions, gestures, etc.

[0424] "Emotion recognition" is the process of identifying emotions from a user's voice, facial expressions, gestures, etc.

[0425] A "server" is a computer system that processes voice and emotional data and generates feedback using generative AI.

[0426] A "terminal" is a device that records audio and video of the evaluation interview and transmits the data to a server.

[0427] MODE FOR CARRYING OUT THE INVENTION

[0428] This invention is a system that records evaluation interviews conducted by managers with their employees as audio logs, converts the contents into text, and further recognizes the user's emotions and inputs them into a generative AI.The purpose of this system is for the generative AI to provide appropriate feedback based on the contents of the evaluation interview and the user's emotional data.

[0429] Hardware and software used

[0430] Hardware: microphones, cameras, servers, devices (PCs and smartphones)

[0431] Software: Speech recognition software (e.g., Google Cloud Speech-to-Text), emotion recognition software, generative AI (e.g., OpenAI GPT-4)

[0432] Specific operation of the system

[0433] 1. Recording audio data

[0434] The user starts the evaluation interview.

[0435] The device records audio data in real time through a microphone.

[0436] The device receives a trigger to start recording and begins streaming audio data.

[0437] 2. Sending audio data to the server

[0438] The device divides the recorded audio data into chunks of a fixed buffer size.

[0439] The terminal transmits the divided audio data to the server.

[0440] The server temporarily stores the received audio data.

[0441] 3. Converting audio data into text

[0442] The server calls the speech recognition software (Google Cloud Speech-to-Text).

[0443] The server inputs the stored voice data into voice recognition software.

[0444] The server receives the text data returned from the speech recognition software.

[0445] The server stores the text data in a database.

[0446] 4. Capturing User Emotions

[0447] The device captures the user's facial expressions and gestures in real time through the camera.

[0448] The device divides the captured video data into a fixed number of frames.

[0449] The terminal transmits the divided video data to the server.

[0450] 5. Sending emotion data to the server

[0451] The terminal transmits the captured video data to the server.

[0452] The server temporarily stores the received video data.

[0453] 6. Emotional Data Recognition

[0454] The server invokes the emotion recognition software.

[0455] The server inputs the stored video data into emotion recognition software.

[0456] The server receives the emotion data returned from the emotion recognition software.

[0457] The server stores the emotion data in a database.

[0458] 7. Data input to generative AI and feedback generation

[0459] The server integrates the text and emotion data from the audio data.

[0460] The server inputs the integrated data into the generative AI (OpenAI GPT-4).

[0461] Generative AI generates feedback based on the input data.

[0462] The server provides the generated feedback to the user.

[0463] Specific examples

[0464] Evaluation interview scenario

[0465] A user says, "This month's project went very well."

[0466] The terminal records this speech and stores the audio data in a buffer.

[0467] The terminal transmits the audio data stored in the buffer to the server.

[0468] The server calls Google Cloud Speech-to-Text to convert the audio data into text data.

[0469] The server stores the text data "This month's project went very well" in the database.

[0470] The device captures the user's facial expressions through a camera and stores the video data in a buffer.

[0471] The terminal transmits the video data stored in the buffer to the server.

[0472] The server calls emotion recognition software and converts the video data into emotion data.

[0473] The server stores the emotion data of "joy" in a database.

[0474] The server combines the text data "This month's project went very well" with the emotion data "joy."

[0475] The server inputs the integrated data into OpenAI GPT-4.

[0476] The generative AI generates feedback such as, "That's a great result. Let's try to do the same on your next project."

[0477] The server provides the generated feedback to the user.

[0478] Example prompts for generative AI models

[0479] "If the user is expressing joy, generate appropriate feedback."

[0480] "If the user is showing signs of anger, generate advice on how to respond calmly."

[0481] In this way, the system efficiently records the contents of the evaluation interview, recognizes the user's emotions in real time, and provides appropriate feedback.

[0482] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0483] Step 1:

[0484] Recording audio data

[0485] The user starts the evaluation interview.

[0486] The device records audio data in real time through a microphone.

[0487] The device receives a trigger to start recording and begins streaming audio data.

[0488] Input: User's voice

[0489] Output: Recorded audio data

[0490] Step 2:

[0491] Sending audio data to the server

[0492] The device divides the recorded audio data into chunks of a fixed buffer size.

[0493] The terminal transmits the divided audio data to the server.

[0494] The server temporarily stores the received audio data.

[0495] Input: Recorded audio data

[0496] Output: Audio data sent to the server

[0497] Step 3:

[0498] Converting audio data to text

[0499] The server calls the speech recognition software (Google Cloud Speech-to-Text).

[0500] The server inputs the stored voice data into voice recognition software.

[0501] The server receives the text data returned from the speech recognition software.

[0502] The server stores the text data in a database.

[0503] Input: Audio data stored on the server

[0504] Output: Text data

[0505] Step 4:

[0506] Capturing User Emotions

[0507] The device captures the user's facial expressions and gestures in real time through the camera.

[0508] The device divides the captured video data into a fixed number of frames.

[0509] The terminal transmits the divided video data to the server.

[0510] Input: User facial expressions and gestures

[0511] Output: Captured video data

[0512] Step 5:

[0513] Sending emotion data to the server

[0514] The terminal transmits the captured video data to the server.

[0515] The server temporarily stores the received video data.

[0516] Input: Captured video data

[0517] Output: Video data sent to the server

[0518] Step 6:

[0519] Emotional Data Recognition

[0520] The server invokes the emotion recognition software.

[0521] The server inputs the stored video data into emotion recognition software.

[0522] The server receives the emotion data returned from the emotion recognition software.

[0523] The server stores the emotion data in a database.

[0524] Input: Video data stored on the server

[0525] Output: Emotion data

[0526] Step 7:

[0527] Data input to generative AI and feedback generation

[0528] The server integrates the text and emotion data from the audio data.

[0529] The server inputs the integrated data into the generative AI (OpenAI GPT-4).

[0530] Generative AI generates feedback based on the input data.

[0531] The server provides the generated feedback to the user.

[0532] Input: Text data and emotion data

[0533] Output: Feedback from the generative AI

[0534] In this way, the system efficiently records the contents of the evaluation interview, recognizes the user's emotions in real time, and provides appropriate feedback.

[0535] (Application example 1)

[0536] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0537] Conventional evaluation interview systems could record and convert interview content into text, but they were unable to recognize the user's emotions during the interview and generate appropriate feedback based on that. As a result, the quality of the interviews was not consistent, and communication between managers and team members was sometimes not smooth. Furthermore, communication between workers and robots in factories was not able to take into account the emotions of the workers, which led to issues with work efficiency and reducing worker stress.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for recording the contents of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the converted contents of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for recognizing emotions from the user's voice, facial expressions, and gestures and generating emotion data, and means for inputting the generated emotion data into the generative AI and generating an appropriate response. This improves the quality of the evaluation interview and facilitates communication between workers and robots in the factory, thereby improving work efficiency and reducing worker stress.

[0539] A "management position" is a position within an organization that is responsible for supervising members and conducting evaluation interviews.

[0540] "Members" are employees within an organization who are guided and evaluated by managers.

[0541] An "appraisal interview" is a conversation in which a manager evaluates a member's performance and behavior and provides feedback.

[0542] An "audio log" is audio data that is recorded from an evaluation interview and saved in digital format.

[0543] "Text conversion" is the process of converting audio data into text data.

[0544] "Generative AI" refers to artificial intelligence that generates feedback and responses based on input data.

[0545] "Feedback" refers to the evaluation and areas for improvement provided by generative AI based on the content of the evaluation interview.

[0546] "Emotion data" refers to emotional information recognized from the user's voice, facial expressions, and gestures.

[0547] "Appropriate response" refers to the optimal reaction or behavior toward the user that the generative AI generates based on emotional data.

[0548] The "system" refers to a set of devices and software that includes the above means and is used to improve the quality of evaluation interviews.

[0549] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings.

[0550] System configuration

[0551] The system consists of the following main components:

[0552] 1. Audio Log Recording Method:

[0553] Hardware: Microphone

[0554] Software: Audio recording application

[0555] Function: Records the contents of evaluation interviews between managers and members in real time and saves them as audio data.

[0556] 2. Text conversion method:

[0557] Software: Google Cloud Speech-to-Text API

[0558] Function: Converts recorded audio data into text data.

[0559] 3. Emotion recognition means:

[0560] Hardware: Camera, microphone

[0561] Software: Microsoft® Azure® Emotion API

[0562] Function: Recognizes emotions from the user's voice, facial expressions, and gestures, and generates emotion data.

[0563] 4. Generative AI means:

[0564] Software: OpenAI GPT-4

[0565] Function: Inputs the textual content and emotional data of the evaluation interview and generates appropriate feedback and responses.

[0566] System Operation

[0567] 1. Audio Log Transcript:

[0568] When managers conduct evaluation interviews with members, a microphone records the audio.

[0569] The recorded audio data is saved as an audio log.

[0570] 2. Converting audio data to text:

[0571] The recorded audio data is converted into text data using the Google Cloud Speech-to-Text API.

[0572] The text data stores the contents of the evaluation interview as text information.

[0573] 3. Emotion recognition:

[0574] The camera and microphone capture the user's facial expressions, voice, and gestures.

[0575] Using the Microsoft Azure Emotion API, we recognize user emotions and generate emotional data.

[0576] 4. Feedback generation using generative AI:

[0577] The textual content and emotional data from the evaluation interview are input into OpenAI GPT-4.

[0578] Generative AI generates appropriate feedback and responses based on input data.

[0579] Specific examples

[0580] For example, a communication support system between workers and robots in a factory works as follows:

[0581] 1. The worker instructs, "Please take this part to the next station."

[0582] 2. The microphone records the audio, and the Google Cloud Speech-to-Text API converts the audio to text.

[0583] 3. Cameras and microphones capture the worker's facial expressions and voice, and the Microsoft Azure Emotion API recognizes fatigue.

[0584] 4. Input the emotion data (fatigue) into OpenAI GPT-4, and the robot responds, "Got it. Please take a short break."

[0585] Example prompts to input to the generative AI model:

[0586] A worker commands, "Please move this part to the next station." The worker sounds tired. How should the robot respond?

[0587] This system will improve the quality of evaluation interviews and facilitate communication between workers and robots within the factory, thereby improving work efficiency and reducing worker stress.

[0588] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0589] Step 1:

[0590] When a manager conducts an evaluation interview with a member, the device's microphone records the audio. The input is the conversation between the manager and the member, and the output is audio data. The device saves this audio data as an audio log.

[0591] Step 2:

[0592] The server sends the saved voice data to the Google Cloud Speech-to-Text API. The input is voice data and the output is text data. The server converts the voice data into text data and saves the contents of the evaluation interview as text information.

[0593] Step 3:

[0594] The device's camera and microphone capture the user's facial expressions, voice, and gestures. The input is the user's facial expressions, voice, and gestures, and the output is emotion data. The device sends this data to the Microsoft Azure Emotion API, which recognizes the user's emotions and generates emotion data.

[0595] Step 4:

[0596] The server inputs the textual content of the evaluation interview and emotional data into OpenAI GPT-4. The input is text data and emotional data, and the output is generated feedback and responses. The server inputs this data into the generative AI to generate appropriate feedback and responses.

[0597] Step 5:

[0598] The server provides the generated feedback and responses to the manager. The input is the generated feedback and responses, and the output is a notification to the manager. The server notifies the manager of the feedback and responses to improve the quality of the appraisal interview.

[0599] Step 6:

[0600] In the communication between workers and robots in a factory, when a worker gives instructions to a robot, the microphone on the terminal records the voice. The input is the worker's voice instructions, and the output is voice data. The terminal saves this voice data as an audio log.

[0601] Step 7:

[0602] The server sends the saved voice data to the Google Cloud Speech-to-Text API. The input is voice data and the output is text data. The server converts the voice data into text data and saves the worker's instructions as text information.

[0603] Step 8:

[0604] The device's camera and microphone capture the worker's facial expressions, voice, and gestures. The input is the worker's facial expressions, voice, and gestures, and the output is emotion data. The device sends this data to the Microsoft Azure Emotion API, which recognizes the worker's emotions and generates emotion data.

[0605] Step 9:

[0606] The server inputs the textual instructions and emotional data into OpenAI GPT-4. The input is text data and emotional data, and the output is the generated response. The server inputs this data into the generative AI to generate an appropriate response.

[0607] Step 10:

[0608] The server provides the generated response to the robot. The input is the generated response, and the output is instructions to the robot. The server notifies the robot of the appropriate response based on the worker's instructions.

[0609] Example 2

[0610] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0611] With conventional evaluation interview systems, it was difficult to properly record and analyze the contents of evaluation interviews, making it difficult to obtain specific feedback to improve the quality of evaluations. Furthermore, there was a lack of means to provide feedback that took the user's emotions into consideration, making it impossible to maximize the effectiveness of evaluation interviews. This resulted in a decline in the quality of evaluation interviews, hindering the growth and motivation of members.

[0612] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0613] In this invention, the server includes means for recording the content of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the converted content of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for the generative AI to generate feedback based on the content of the evaluation interview and user emotion data, and means for providing the generated feedback to the user. This makes it possible to accurately record and analyze the content of the evaluation interview and provide specific feedback that takes the user's emotions into consideration. This makes it possible to improve the quality of the evaluation interview and promote member growth and motivation.

[0614] A "management position" is a position within an organization that is responsible for evaluating and guiding members.

[0615] A "member" is an individual employee within an organization who receives guidance and evaluation from management.

[0616] An "evaluation interview" is a conversation between a manager and a member of staff to evaluate their performance and provide feedback.

[0617] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0618] "Transcription" refers to the process of converting audio logs into written data.

[0619] "Generative AI" refers to artificial intelligence that generates feedback and analytical results based on input data.

[0620] "Feedback" refers to the evaluation and areas for improvement provided by generative AI based on the content of the evaluation interview.

[0621] "Users" are the members and managers who receive feedback from evaluation interviews.

[0622] "Emotion data" refers to data that indicates the user's emotional state, and includes emotions such as joy and anger.

[0623] The "server" is a computer system that records the contents of evaluation interviews and transmits the data to the generative AI.

[0624] This invention is a system that records the contents of an evaluation interview as an audio log, converts it into text, and generates feedback using generative AI. Specific embodiments of this system are described below.

[0625] Hardware and software used

[0626] Hardware: Servers (e.g., general-purpose servers), terminals (e.g., personal computers, smartphones)

[0627] Software: Speech recognition software (e.g., speech recognition APIs), natural language processing libraries (e.g., natural language processing libraries), generative AI models (e.g., generative AI models)

[0628] System Operation

[0629] Subject: Server

[0630] The server records the contents of evaluation interviews conducted by managers with employees as audio logs. The audio logs are converted into text data using a speech recognition API. The converted text data is input into a generative AI model.

[0631] Subject: Generative AI

[0632] The generative AI analyzes the input text data and generates feedback based on the content of the evaluation interview. The generative AI uses a natural language processing library to tokenize the text data and perform grammatical and semantic analysis. It also generates feedback based on the user's emotional data.

[0633] Subject: Server

[0634] The server provides the feedback received from the generative AI to the user, who can then check the feedback using their device.

[0635] Specific examples

[0636] Contents of the evaluation interview

[0637] Evaluator: Your contribution to this project was enormous, especially your efforts in data analysis.

[0638] Applicant: Thank you. However, I felt there was a lack of communication during the project.

[0639] User emotion data

[0640] Delight: Show delight when someone says, "Your efforts on the data analysis part were great."

[0641] Anger: Showing anger when told there was a lack of communication.

[0642] Example prompts for generative AI models

[0643] Evaluation interview: "Your contribution to this project was enormous. Your efforts in data analysis were particularly impressive."

[0644] User emotion data: "Joy"

[0645] Feedback example

[0646] Feedback: "You seem pleased that your data analysis efforts are being recognized. This is a very good point. However, you also pointed out a lack of communication. If you pay more attention to this in the future, you could receive a better evaluation."

[0647] In this way, the server records the contents of the evaluation interview as an audio log and converts it into text. The generative AI analyzes the text data and generates feedback based on the user's emotional data. The server provides the generated feedback to the user, who can then check it using their device.

[0648] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0649] Step 1:

[0650] The server records the contents of the evaluation interviews conducted by managers with members as audio logs.

[0651] Specific operation: The server receives the audio data of the evaluation interview and saves it as an audio log.

[0652] Input: Audio data from the evaluation interview

[0653] Output: Audio log

[0654] Step 2:

[0655] The server converts the audio log into text data.

[0656] Specific operation: The server calls the speech recognition API and converts the voice log into text data.

[0657] Input: Audio log

[0658] Output: Text data

[0659] Step 3:

[0660] The server inputs the textual content of the evaluation interview into a generative AI model.

[0661] Specific operation: The server sends the text data to the API of the generative AI model and waits for the analysis results.

[0662] Input: Text data

[0663] Output: Input to a generative AI model

[0664] Step 4:

[0665] The generative AI analyzes text data and generates feedback based on the content of the evaluation interview.

[0666] How it works: Generative AI uses natural language processing libraries to tokenize text data, perform grammatical and semantic analysis, and generate feedback based on user sentiment data.

[0667] Input: Text data, user emotion data

[0668] Output: Feedback

[0669] Step 5:

[0670] The server provides the user with the feedback received from the generative AI.

[0671] Specific behavior: The server sends the feedback to the user's device so that the user can check it.

[0672] Input: Feedback

[0673] Output: Feedback to the user's terminal

[0674] Step 6:

[0675] The user uses the terminal to check the feedback.

[0676] Specific actions: The user views the feedback on the device and understands the content of the evaluation interview.

[0677] Input: Feedback

[0678] Output: User understanding

[0679] (Application example 2)

[0680] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0681] Conventional appraisal interview systems have had problems with providing appropriate feedback on the content of appraisal interviews, particularly with a lack of feedback that takes into account the user's emotions. Furthermore, there is a lack of means to provide specific improvements to improve the quality of appraisal interviews. This has resulted in limited effectiveness of appraisal interviews, and the quality of communication between managers and employees has not improved.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0683] In this invention, the server includes means for recording the content of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the converted content of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for acquiring user emotion data and inputting it together with the content of the evaluation interview into the generative AI, and means for providing feedback from the generative AI to the user. This enables feedback that takes into account the content of the evaluation interview and the user's emotions, and makes it possible to provide specific improvements that will improve the quality of the evaluation interview.

[0684] A "management position" is a position within an organization that involves supervising the work of members and conducting evaluation interviews.

[0685] "Members" are employees who work within an organization under the direction of a manager.

[0686] An "evaluation interview" is an interactive meeting in which managers evaluate team members' performance and provide feedback.

[0687] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0688] "Texting" is the process of converting audio logs into written data.

[0689] "Generative AI" is an artificial intelligence system that generates feedback based on input data.

[0690] "Feedback" refers to evaluation information such as good and bad points provided by generative AI based on the content of the evaluation interview.

[0691] "User" refers to the manager or member participating in the evaluation interview.

[0692] "Emotion data" is data that indicates the emotional state of the user, and includes emotions such as joy and anger.

[0693] The "server" is a computer system that records the contents of evaluation interviews, converts them into text, and inputs them into the generative AI.

[0694] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings.

[0695] First, the system records the content of the evaluation interview conducted by the manager with the employee as an audio log. The audio log is captured using a smartphone or a dedicated recording device. Next, speech recognition software is used to convert the captured audio log into text. Specifically, speech recognition services such as Google Speech-to-Text API or IBM Watson (registered trademark) Speech to Text can be used.

[0696] The textual content of the evaluation interview is input into a generative AI. Natural language processing models such as OpenAI's GPT-3 and GPT-4 are used as the generative AI. The generative AI analyzes the input text data and generates feedback such as good and bad points based on the content of the evaluation interview.

[0697] Furthermore, emotion recognition software is used to obtain user emotional data. For example, emotion recognition technology from Microsoft Azure's Emotion API can be used. The emotional data, along with the content of the evaluation interview, is input into generative AI and used to improve the quality of feedback.

[0698] Feedback from the generative AI is provided to the user. The feedback can be displayed on a smartphone or computer screen, or read aloud. This allows users to receive specific feedback based on the content of the evaluation interview and emotional data.

[0699] As a concrete example, the following prompt sentence could be input to a generative AI:

[0700] Example prompt sentence:

[0701] Please generate feedback on the positive and negative points based on the content of the evaluation interview below.

[0702] Contents of the evaluation interview:

[0703] Robot A is fast but lacks precision. Robot B is highly accurate but slow.

[0704] User sentiment data:

[0705] Joy: Robot A's speed, Anger: Robot B's slowness

[0706] feedback:

[0707] In this way, feedback that takes into account the content of the evaluation interview and the user's emotional data can be generated and provided to the user. This system can improve the quality of evaluation interviews and communication between managers and employees.

[0708] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0709] Step 1:

[0710] Managers conduct evaluation interviews and record the contents as audio logs.

[0711] Input: Audio from the evaluation interview

[0712] Output: Audio log

[0713] How it works: Managers record the appraisal interview using a smartphone or a dedicated recording device, and the recorded audio data is saved as an audio log.

[0714] Step 2:

[0715] The server transcribes the audio log into text.

[0716] Input: Audio log

[0717] Output: Text data

[0718] How it works: The server uses a speech recognition service such as Google Speech-to-Text API or IBM Watson Speech to Text to convert the audio log into text data, which is then passed on to the next processing step.

[0719] Step 3:

[0720] The server acquires the user's emotion data.

[0721] Input: User's facial expressions and voice during the evaluation interview

[0722] Output: Emotion data

[0723] How it works: The server uses emotion recognition technology from Microsoft Azure's Emotion API to obtain emotional data from the user's facial expressions and voice during the evaluation interview, and passes the obtained emotional data to the next processing step.

[0724] Step 4:

[0725] The server inputs text data and emotional data into the generative AI and receives feedback.

[0726] Input: Text data, emotion data

[0727] Output: Feedback

[0728] How it works: The server combines text and emotion data into a prompt and inputs it into a generative AI such as OpenAI's GPT-3 or GPT-4. The generative AI analyzes the input data and generates feedback such as good and bad points based on the content of the evaluation interview.

[0729] Step 5:

[0730] The server provides the generated feedback to the user.

[0731] Input: Feedback

[0732] Output: Feedback that is displayed to the user

[0733] Specific operation: The server displays the generated feedback on the screen of a smartphone or PC. It also provides a function to read the feedback aloud if necessary. This allows the user to receive specific feedback based on the content of the evaluation interview and emotional data.

[0734] Example 3

[0735] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0736] Conventional evaluation interview systems have the problem that it is difficult for managers to objectively evaluate the quality of the interview and identify areas for improvement. In addition, they do not provide feedback that takes into account the user's emotional data, making it impossible to deeply understand the quality of the interview. This makes it difficult to improve the quality of evaluation interviews.

[0737] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0738] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI model and receiving feedback from the generative AI model, means for collecting user emotion data and inputting the emotion data into the generative AI model, means for notifying the manager's terminal of the feedback from the generative AI model, and means for the manager to check the feedback on his or her own terminal. This enables the manager to objectively grasp the quality of the evaluation interviews and identify specific areas for improvement that take into account the user's emotion data.

[0739] A "management position" is a position within an organization that is responsible for evaluating and guiding members.

[0740] "Members" are employees and staff within an organization who are guided and evaluated by management.

[0741] An "evaluation interview" is an interview conducted by a manager with a member of staff to evaluate their work progress and performance.

[0742] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[0743] "Texting" is the process of converting audio logs into written data.

[0744] A "generative AI model" is an artificial intelligence model that generates feedback and suggestions based on input data.

[0745] "Feedback" refers to evaluation information such as good and bad points provided by the generative AI model based on the content of the evaluation interview.

[0746] "User emotion data" is data that indicates the user's emotional state during the evaluation interview, and is acquired using voice analysis and facial expression recognition technology.

[0747] "Notification" is the process of informing managers of feedback from generative AI models on their devices.

[0748] "Device" refers to the device used by managers to review feedback, including PCs, tablets, smartphones, etc.

[0749] This invention is a system for managers to improve the quality of performance evaluation interviews, and provides feedback using a generative AI model. Specific embodiments of this system are described below.

[0750] Hardware and software used

[0751] Hardware: Servers, management devices (PCs, tablets, smartphones, etc.)

[0752] Software: Generative AI models (e.g., OpenAI's GPT-4), notification systems, messaging platforms, databases (e.g., MySQL)

[0753] Data processing and calculation

[0754] 1. Collecting user evaluation interview data

[0755] When a user undergoes an evaluation interview, the content of the interview and the user's emotional data are collected. The emotional data is obtained using, for example, voice analysis and facial expression recognition technology. This data is collected in real time and sent from the device to the server.

[0756] 2. Data storage by the server

[0757] The server stores the received evaluation interview data and emotion data in a database, using, for example, MySQL. The stored data includes the interview text content, the user's emotion score, the interview date and time, etc.

[0758] 3. Server generates and sends prompts to the AI ​​model

[0759] The server generates prompts based on the stored evaluation interview data to be sent to the generative AI model. The prompts include the interview content and emotional data. The generated prompts are sent to the generative AI model (e.g., OpenAI's GPT-4).

[0760] 4. Feedback generation using generative AI models

[0761] The generative AI model generates feedback for the evaluation interview based on the prompts received. The feedback includes good and bad points about the interview and information based on the user's emotional data. The generated feedback is sent back to the server.

[0762] 5. Receiving and storing feedback by the server

[0763] The server receives the feedback sent back by the generative AI model and stores it in a database, which includes specific advice and areas for improvement based on the interview content.

[0764] 6. Server-Based Feedback Notification

[0765] The server then notifies the manager of the saved feedback via a notification system, such as a messaging platform, which includes a summary of the feedback and a link to the details.

[0766] 7. Displaying feedback via device

[0767] The manager's device receives notifications from the server and displays the feedback, allowing the manager to review the feedback on their own device and receive specific advice on how to improve the quality of the appraisal interview.

[0768] Specific examples

[0769] Evaluation interview: "We discussed the progress of the project. The users seemed unhappy with the delays."

[0770] Feedback from a generative AI model: "Your explanation of the project's progress was clear, but your response to user complaints was insufficient. Next time, pay more attention to user feelings and provide specific solutions."

[0771] Prompt Sentence Examples

[0772] "Generate feedback based on the content of the evaluation interview and user sentiment data. Content of the evaluation interview: 'We discussed the progress of the project. The user seemed unhappy with the delays in the project.'"

[0773] This system allows managers to gain a deeper understanding of the quality of the evaluation interviews and identify areas for improvement. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0774] Step 1:

[0775] When a user undergoes an evaluation interview, the device records the interview contents as an audio log and collects the user's emotional data. The emotional data is obtained using voice analysis and facial expression recognition technology. The input is the audio data of the evaluation interview and the user's emotional data, and the output is a record of these data.

[0776] Step 2:

[0777] The terminal transmits the recorded voice log and emotion data to the server. The input is the voice log and emotion data, and the output is data transmission to the server.

[0778] Step 3:

[0779] The server converts the received audio logs into text. It uses speech recognition software to convert the audio data into text. The input is the audio log, and the output is the transcribed content of the evaluation interview.

[0780] Step 4:

[0781] The server stores the textual content of the evaluation interview and the emotional data in a database. For example, MySQL is used as the database. The input is the textual content of the evaluation interview and the emotional data, and the output is saved in the database.

[0782] Step 5:

[0783] The server generates prompts to be sent to the generative AI model based on the stored evaluation interview data and emotion data. The prompts include the interview content and emotion data. The input is the textual content of the evaluation interview and emotion data, and the output is the generated prompt.

[0784] Step 6:

[0785] The server sends the generated prompt to the generative AI model. The input is the generated prompt, and the output is the prompt sent to the generative AI model.

[0786] Step 7:

[0787] The generative AI model generates feedback for the evaluation interview based on the prompts received. The feedback includes information based on the positive and negative aspects of the interview and the user's emotional data. The input is the prompt, and the output is the generated feedback.

[0788] Step 8:

[0789] The server receives the feedback sent back from the generative AI model and stores it in a database. The input is the generated feedback and the output is the storage in the database.

[0790] Step 9:

[0791] The server notifies the manager's terminal of the saved feedback. The notification system uses, for example, a messaging platform. The input is the generated feedback, and the output is the notification to the manager's terminal.

[0792] Step 10:

[0793] The manager's device receives notifications from the server and displays the feedback. The manager can check the feedback on their own device and get specific advice to improve the quality of the appraisal interview. The input is the notification from the server, and the output is the display of the feedback.

[0794] (Application example 3)

[0795] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0796] Conventional evaluation interview systems have the problem that it is difficult for managers to objectively evaluate the quality of the interviews and identify areas for improvement. Furthermore, because feedback is not provided that takes into account the worker's emotional data, there is also the problem that the quality of the interviews cannot be sufficiently improved. Furthermore, because feedback is not provided in real time, there is also the problem that it is difficult to make rapid improvements.

[0797] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0798] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for recognizing the worker's emotional data, means for notifying the manager of the feedback from the generative AI, and means for the manager to check the feedback from the generative AI on his or her own device. This enables the manager to objectively grasp the quality of the evaluation interviews and receive feedback in real time that takes into account the worker's emotional data.

[0799] A "management position" is a position within an organization that is responsible for guiding and evaluating members.

[0800] "Members" are employees within an organization who are guided and evaluated by managers.

[0801] An "evaluation interview" is an interview in which a manager evaluates a member's work performance and behavior and provides feedback.

[0802] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0803] "Transcription" refers to the process of converting audio logs into written data.

[0804] "Generative AI" refers to artificial intelligence that generates feedback and information based on input data.

[0805] "Feedback" refers to evaluations and areas for improvement provided by generative AI based on the content of the evaluation interview and emotional data.

[0806] "Emotion data" refers to the emotional state of a worker that is recognized and recorded as data.

[0807] "Notification" refers to the process of informing management of feedback from generative AI.

[0808] A "device" is any electronic device, such as a smartphone or tablet, that a manager uses to view feedback.

[0809] A system for implementing this invention is configured as follows. First, the contents of evaluation interviews conducted by managers with their employees are recorded as audio logs. Next, speech recognition software is used to convert these audio logs into text. Specifically, a speech recognition service such as the Google Cloud Speech-to-Text API can be used.

[0810] The textual content of the evaluation interview is input into a generative AI. This generative AI uses an advanced natural language processing model, such as OpenAI's GPT-4. The generative AI generates feedback based on the input text data. This feedback includes good and bad points based on the content of the evaluation interview, as well as information based on the worker's emotional data.

[0811] Emotion data is acquired using software that recognizes emotions from the worker's facial expressions and voice, such as Microsoft Azure's Emotion API or an emotion recognition system that combines OpenCV and Dlib.

[0812] The generated feedback is provided to management through a notification system, which may include an application for displaying the feedback on a device such as a smartphone or tablet, or may send notifications in real time using a messaging platform, for example.

[0813] For example, the following prompts can be fed to a generative AI model:

[0814] Example prompt sentence:

[0815] Conduct a performance evaluation of Worker A and generate feedback based on the following data:

[0816] Working speed: 85%

[0817] Working accuracy: 90%

[0818] Emotion data: Smile 70%, Surprise 10%, Anger 5%, Sadness 15%

[0819] feedback:

[0820] Pros: Very fast working speed and excellent accuracy.

[0821] Areas for improvement: Stress management is needed as emotions of surprise and anger are observed during the work.

[0822] In this way, managers can objectively grasp the quality of the evaluation interviews and receive real-time feedback that takes into account the workers' emotional data, which will promote improvements in the quality of evaluation interviews and contribute to improving workers' performance.

[0823] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0824] Step 1:

[0825] The contents of evaluation interviews conducted by managers with members are recorded as audio logs.

[0826] Input: Audio data from the evaluation interview

[0827] Output: Audio log file

[0828] Specific operation: The device (e.g., smartphone or tablet) records the audio of the evaluation interview and saves it as an audio log file.

[0829] Step 2:

[0830] Use voice recognition software to transcribe audio logs into text.

[0831] Input: Audio log file

[0832] Output: Text data

[0833] Specific operation: The server calls a speech recognition service such as Google Cloud Speech-to-Text API to convert the audio log file into text data.

[0834] Step 3:

[0835] The textual content of the evaluation interview is input into the generative AI.

[0836] Input: Text data

[0837] Output: Input data for generative AI

[0838] Specific operation: The server converts the text data into a format for sending to a generative AI (e.g., OpenAI's GPT-4).

[0839] Step 4:

[0840] Generative AI generates feedback.

[0841] Input: Input data for generative AI

[0842] Output: Feedback data

[0843] Specific operation: The generative AI generates feedback on the content of the evaluation interview based on the input text data.

[0844] Step 5:

[0845] Recognize worker emotion data.

[0846] Input: Facial expression and voice data of the worker

[0847] Output: Emotion data

[0848] Specific operation: The device uses Microsoft Azure's Emotion API, OpenCV, and Dlib to recognize emotions from the worker's facial expressions and voice and save them as emotion data.

[0849] Step 6:

[0850] Generative AI generates feedback that takes emotional data into account.

[0851] Input: Feedback data, emotion data

[0852] Output: Final feedback data

[0853] Specific operation: The server inputs feedback data and emotion data into the generative AI, and generates final feedback that takes the emotion data into consideration.

[0854] Step 7:

[0855] Provide generated feedback to management through a notification system.

[0856] Input: Final feedback data

[0857] Output: Feedback displayed on the manager's device

[0858] What it does: The server uses a messaging platform to send the final feedback data to the manager's smartphone or tablet in real time.

[0859] Step 8:

[0860] Managers view feedback on their own devices.

[0861] Input: Feedback displayed on the manager's device

[0862] Output: Managerial understanding and action

[0863] What it does: Managers can view feedback on their smartphones or tablets to help improve the quality of evaluation interviews.

[0864] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0865] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0866] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0867] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0868] [Second embodiment]

[0869] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0870] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0871] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0872] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0873] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0874] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0875] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0876] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0877] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0878] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0879] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0880] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0881] "Example 1"

[0882] One embodiment of the present invention is a system that records evaluation interviews conducted by managers with their team members as audio logs. This system has the function of recording the contents of the evaluation interviews in real time and converting the audio data into text data. Specifically, the system converts the audio data into text using voice recognition technology.

[0883] "Example 2"

[0884] Furthermore, it has the function of inputting the textual content of the evaluation interview into a generative AI and receiving feedback from the generative AI. The generative AI generates feedback such as good and bad points based on the content of the evaluation interview. Specifically, it analyzes the text data using natural language processing technology and evaluates the quality of the evaluation interview.

[0885] "Example 3"

[0886] Furthermore, the system also has a function that provides feedback from the generative AI to managers and encourages them to improve the quality of evaluation interviews. Specifically, it has a notification system that notifies managers of feedback from the generative AI. This allows managers to objectively understand the quality of their own evaluation interviews and identify areas for improvement.

[0887] The processing flow of each embodiment will be described below.

[0888] "Example 1"

[0889] Step 1: Managers initiate evaluation interviews with members. These evaluation interviews are conducted in the form of direct dialogue between managers and members.

[0890] Step 2: Once the assessment interview begins, the system automatically records the interview as an audio log in real time, capturing the entire interview.

[0891] Step 3: After the evaluation interview is completed, the system converts the recorded audio log into text using voice recognition technology.

[0892] "Example 2"

[0893] Step 1: The system inputs the text of the evaluation interview into the generative AI. This input is done automatically, without human intervention.

[0894] Step 2: The generative AI analyzes the input text data and generates feedback on good and bad points based on the content of the evaluation interview. This analysis and feedback generation is carried out using natural language processing technology.

[0895] "Example 3"

[0896] Step 1: Once the generative AI feedback is generated, the system provides it to the manager via a notification system, where the manager can view the feedback on their device.

[0897] Step 2: Based on the feedback provided, managers can objectively assess the quality of their own evaluation interviews and identify areas for improvement, which will promote improvements in the quality of evaluation interviews.

[0898] Example 1

[0899] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0900] Conventional methods for recording evaluation interviews have had the problem of making it difficult to accurately record the interview content and analyze it later. In addition, there are limited means of obtaining feedback to improve the quality of the interviews, making it difficult for managers to identify areas for improvement in the interviews.

[0901] The identification process by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes a means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, a means for receiving audio data in real time and converting it into text data using speech recognition technology, a means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, and a means for saving the generated text data in a database. This makes it possible to accurately record the contents of the evaluation interviews and analyze them in detail later. Furthermore, specific improvements to improve the quality of the interviews can be identified through feedback from the generative AI.

[0902] A "management position" is someone in a position within an organization who evaluates and guides members.

[0903] A "member" is someone in an organization who is in a position to receive guidance and evaluation from management.

[0904] An "evaluation interview" is an interview conducted by a manager with a member of staff to evaluate their work and provide feedback.

[0905] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[0906] "Text conversion" is the process of converting audio data into text data.

[0907] "Generative AI" is an artificial intelligence system that generates feedback and analytical results based on input data.

[0908] "Feedback" refers to evaluation information such as good points and areas for improvement that is provided by generative AI based on the content of the evaluation interview.

[0909] A "database" is an information system for storing and managing text data and feedback.

[0910] "Voice recognition technology" is a technology that analyzes voice data and converts it into text data.

[0911] "Real-time" refers to processing occurring immediately while the assessment interview is in progress.

[0912] This invention is a system that records evaluation interviews between managers and their employees as audio logs, converts the audio data into text data, and inputs it into a generative AI to obtain feedback. A specific embodiment of this system is described below.

[0913] Server Roles

[0914] The server receives the voice data from the evaluation interview in real time and converts it into text data using speech recognition technology. Specifically, it uses speech recognition software such as Google Cloud Speech-to-Text API. The converted text data is then stored in a database such as MySQL or PostgreSQL.

[0915] Device Role

[0916] The device provides an interface for managers to start recording when they begin an evaluation interview. A dedicated recording application is used to send the audio data being recorded to the server in real time. After the interview ends, the recording is stopped and the text data is retrieved from the server and displayed.

[0917] User Roles

[0918] The user (management) presses the recording button on the device to start the evaluation interview. During the interview, the user can check the text data in real time to confirm whether specific keywords and phrases were accurately recorded. After the interview is over, the user stops recording and reviews the generated text data.

[0919] Specific examples

[0920] Example 1: Recording and transcribing evaluation interviews

[0921] Scenario: Manager A conducts an evaluation interview with member B.

[0922] Hardware used: Manager A's smartphone

[0923] Software used: Dedicated recording application, Google Cloud Speech-to-Text API, MySQL database

[0924] 1. Manager A launches the dedicated recording application on his smartphone and presses the record button.

[0925] 2. The smartphone records the interview audio and transmits it to the server in real time.

[0926] 3. The server converts the audio data into text using the Google Cloud Speech-to-Text API.

[0927] 4. The server stores the converted text data in a MySQL database.

[0928] 5. After the interview, Manager A stops the recording and checks the text data generated by the smartphone application.

[0929] Example prompts to be input to the generative AI model

[0930] Prompt: "Describe a system that records managerial interviews with employees and converts the audio data into text in real time. Include the names of the specific hardware and software."

[0931] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[0932] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0933] Step 1:

[0934] Subject: User

[0935] The user presses the recording button on the terminal when starting the evaluation interview.

[0936] Input: User operation (pressing the record button)

[0937] Output: Trigger to start recording

[0938] What happens: The user taps the record button in the smartphone or tablet application to start the recording process.

[0939] Step 2:

[0940] Subject: Terminal

[0941] The terminal detects that the recording button has been pressed and starts recording.

[0942] Input: Trigger to start recording

[0943] Output: Audio data being recorded

[0944] Specific operation: Captures audio data using the device's microphone and prepares to send it to the server in real time.

[0945] Step 3:

[0946] Subject: Terminal

[0947] The terminal transmits the audio data being recorded to the server in real time.

[0948] Input: Audio data being recorded

[0949] Output: Sending audio data to the server

[0950] Specific behavior: Recorded audio data is sent to the server using WebSocket or HTTP streaming.

[0951] Step 4:

[0952] Subject: Server

[0953] The server receives the voice data transmitted from the terminal in real time.

[0954] Input: Audio data from the device

[0955] Output: Temporarily saves received audio data

[0956] Specific operation: Receives audio data via an HTTP request and temporarily stores it in the server's local storage or cloud storage.

[0957] Step 5:

[0958] Subject: Server

[0959] The server converts the received voice data into text data using voice recognition technology.

[0960] Input: Received audio data

[0961] Output: Converted text data

[0962] What it does: Uses the Google Cloud Speech-to-Text API to create an API request to convert audio data to text and send the audio data.

[0963] Step 6:

[0964] Subject: Server

[0965] The server stores the converted text data in a database.

[0966] Input: Converted text data

[0967] Output: Text data stored in a database

[0968] Specific operation: Executes an INSERT query in a database such as MySQL or PostgreSQL and saves text data.

[0969] Step 7:

[0970] Subject: User

[0971] The user presses the stop recording button after the interview is over.

[0972] Input: User action (pressing the stop recording button)

[0973] Output: Trigger to stop recording

[0974] What happens: The recording process ends when the user taps the stop recording button in the smartphone or tablet application.

[0975] Step 8:

[0976] Subject: Terminal

[0977] The terminal detects that the recording stop button has been pressed and stops recording.

[0978] Input: Trigger to stop recording

[0979] Output: End of recording

[0980] Specific Action: Ends the recording process and stops sending data to the server.

[0981] Step 9:

[0982] Subject: Terminal

[0983] The terminal displays the text data obtained from the server to the user.

[0984] Input: Text data from the server

[0985] Output: Text data that is displayed to the user

[0986] Specific behavior: Renders text data in the application's UI and displays it for the user to see.

[0987] Step 10:

[0988] Subject: User

[0989] The user reviews the generated text data and adds corrections or comments as necessary.

[0990] Input: The displayed text data

[0991] Output: Text data with corrections and comments added

[0992] Specific Behavior: Use the application's text editor function to add corrections and comments to text data.

[0993] (Application example 1)

[0994] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0995] The conventional evaluation interview system had the problem that it was difficult for managers to accurately record the contents of interviews with team members and analyze them later. There was also a lack of a way to efficiently record work instructions and reports within the factory and convert them into text data in real time. This made it difficult to manage work progress and quality, and there was a risk of overall productivity decreasing.

[0996] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0997] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the textual contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for recording work instructions and reports within the factory as audio logs and converting them into text data in real time, and means for storing the converted text data in the cloud so that it can be searched and analyzed later. This makes it possible to accurately record the contents of evaluation interviews and receive feedback from the generative AI, as well as efficiently record work instructions and reports within the factory and convert them into text data in real time.

[0998] A "management position" is a position within an organization that is responsible for supervising and evaluating the work of its members.

[0999] "Members" refer to employees or staff members who are responsible for specific tasks within an organization.

[1000] An "evaluation interview" is a dialogue-style interview in which managers evaluate their team members' work performance and results and provide feedback.

[1001] An "audio log" is a digital file that records audio data such as evaluation interviews and work instructions.

[1002] "Text conversion" is the process of converting audio data into text data.

[1003] "Generative AI" is an artificial intelligence system that generates feedback and suggestions based on input data.

[1004] "Feedback" refers to evaluations and information on areas for improvement provided by generative AI based on the content of evaluation interviews and work instructions.

[1005] "Inside the factory" refers to the interior of the facility where the product is manufactured or processed.

[1006] "Work instructions" are information that instructs specific work content and procedures within a factory.

[1007] "Report" is information used to communicate the progress and results of work.

[1008] "Real-time" refers to data being processed and transformed as it is generated.

[1009] "Cloud" refers to a collection of remote servers provided over the internet for data storage and processing.

[1010] "Searching" is the process of locating specific information from stored data.

[1011] "Analysis" is the process of examining collected data in detail to find meaning and trends.

[1012] A system for implementing this invention is configured as follows: First, a manager uses smart glasses or a device with a microphone to record the contents of an evaluation interview with a member of staff as an audio log. This allows audio data from the evaluation interview to be collected in real time.

[1013] The collected voice data is then converted to text using a Python program with the Google Cloud Speech-to-Text API. This text data is then input into a generative AI system, which generates feedback based on the content of the interview. This feedback is then provided to managers to encourage them to improve the quality of the interviews.

[1014] Furthermore, work instructions and reports within the factory are also recorded as voice logs and converted into text data in real time. This text data is stored in the cloud and can be searched and analyzed later. This allows for efficient progress management and quality control of work.

[1015] The hardware used includes smart glasses, a device with a microphone, a PC or a cloud server, and the software used is Python, the Google Cloud Speech-to-Text API, audio-related libraries, and wave.

[1016] For example, when a worker in a factory reports, "Let's proceed to the next process," the voice is recorded by smart glasses and converted in real time into text data that says, "Let's proceed to the next process." This data is stored in the cloud and can be used later by managers to check the progress of work.

[1017] An example prompt might look like this:

[1018] "Develop a system that records work instructions and reports in the factory as voice logs and converts them into text data in real time. Use smart glasses and use the Google Cloud Speech-to-Text API to convert the voice data into text."

[1019] The above is an embodiment of the present invention.

[1020] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1021] Step 1:

[1022] The user wears the smart glasses and conducts evaluation interviews and gives instructions and reports on work in the factory. The microphone in the smart glasses collects voice data in real time. The input is the user's voice, and the output is voice data.

[1023] Step 2:

[1024] The terminal (smart glasses) temporarily stores collected voice data. The stored voice data is used for subsequent processing. The input is voice data, and the output is the temporarily stored voice data.

[1025] Step 3:

[1026] The device sends the temporarily saved voice data to the cloud server. The cloud server receives the voice data and starts the voice recognition process. The input is the temporarily saved voice data, and the output is the voice data sent to the cloud server.

[1027] Step 4:

[1028] The cloud server uses the Google Cloud Speech-to-Text API to convert voice data into text data. The input is voice data and the output is text data. Specifically, the voice data is sent to the API and the returned text data is received.

[1029] Step 5:

[1030] The cloud server inputs text data into the generative AI and generates feedback. The input is text data, and the output is feedback from the generative AI. Specifically, the text data is input into the AI ​​model and the generated feedback is received.

[1031] Step 6:

[1032] The cloud server provides the generated feedback to managers, who then receive the feedback and use it to improve the quality of the evaluation interviews. The input is the feedback from the generative AI, and the output is the feedback provided to managers.

[1033] Step 7:

[1034] The cloud server stores text data of work instructions and reports within the factory in the cloud. The stored data can be searched and analyzed later. The input is text data, and the output is data stored in the cloud.

[1035] Step 8:

[1036] Managers and other users search and analyze data stored in the cloud. This allows for efficient progress management and quality control of work. The input is data stored in the cloud, and the output is the search and analysis results. Specifically, the system searches for the necessary information from a database on the cloud and analyzes the data using analysis tools.

[1037] Example 2

[1038] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1039] In traditional appraisal interview systems, recording interview content and generating feedback is often done manually, which takes time and effort. Furthermore, issues include inconsistent quality of feedback and not clearly indicating areas for improvement in appraisal interviews. This makes it difficult to improve the quality of appraisal interviews and can hinder smooth communication between managers and employees.

[1040] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for recording the contents of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for generating a prompt sentence to input the converted contents of the evaluation interview into the generative AI, means for receiving feedback from the generative AI, and means for returning the received feedback to the user. This makes it possible to automatically analyze the contents of the evaluation interview and quickly provide high-quality feedback.

[1041] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[1042] "Texting" is the process of converting audio logs into written data.

[1043] "Generative AI" is an artificial intelligence model that analyzes input data and generates feedback or other outputs.

[1044] A "prompt sentence" is an input sentence that instructs a generative AI to perform a specific analysis or generation.

[1045] "Feedback" refers to evaluation information such as good and bad points that is generated by generative AI based on the content of the evaluation interview.

[1046] "Users" are system users such as managers and members who receive feedback from evaluation interviews.

[1047] MODE FOR CARRYING OUT THE INVENTION

[1048] The present invention relates to a system for efficiently analyzing the content of an evaluation interview and providing high-quality feedback. Specific embodiments of this system will be described below.

[1049] Hardware and software used

[1050] Hardware: Server (e.g., virtual server for cloud services)

[1051] Software: Generative AI models (e.g., natural language processing models), natural language processing libraries (e.g., spaCy)

[1052] System configuration

[1053] The system consists of the following major components:

[1054] 1. Recording and transcribing audio logs

[1055] The user records the contents of the evaluation interview as an audio log, for example, by using a recording device to collect audio data during the evaluation interview.

[1056] The server receives the recorded voice log and converts it into text data using voice recognition technology. Specifically, it uses a voice recognition API to convert the voice log into text.

[1057] 2. Prompt generation

[1058] The server generates a prompt to be input into the generative AI model based on the textual content of the evaluation interview. For example, it creates a prompt such as, "Please analyze the content of the following evaluation interview and provide feedback on the good and bad points."

[1059] 3. Input to the generative AI model and analysis

[1060] The server sends the generated prompt and text data to the generative AI model, which then analyzes the text data using natural language processing technology and generates feedback based on the content of the evaluation interview.

[1061] 4. Receiving and Providing Feedback

[1062] The server receives the feedback generated by the generative AI model and returns it to the user, for example, by formatting the feedback in HTML and displaying it in the user's browser.

[1063] Specific examples

[1064] For example, a user saves the contents of an evaluation interview in a file called "evaluationinterview.txt" and drags and drops that file into the system's upload form. The server receives the uploaded "evaluationinterview.txt" and reads the text data into memory. The server then generates a prompt message saying, "Please analyze the contents of the evaluation interview below and provide feedback on the good and bad points," and sends it to the generative AI model.

[1065] The generative AI model analyzes the received prompt and text data and generates feedback such as, "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems." The server receives this feedback and sends it back to the user.

[1066] Prompt Sentence Examples

[1067] Please analyze the following evaluation interview and provide feedback on what was good and what was bad. The evaluation interview content is as follows: "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems."

[1068] In this way, the system can efficiently analyze the content of the evaluation interview and provide high-quality feedback quickly.

[1069] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1070] Step 1:

[1071] The user records the contents of the evaluation interview as an audio log. For example, during the evaluation interview, audio data is collected using a recording device. The input is the audio data, and the output is the recorded audio log.

[1072] Step 2:

[1073] The server receives audio logs uploaded by users. The input is the audio logs, and the output is the audio data stored on the server.

[1074] Step 3:

[1075] The server uses speech recognition technology to convert the voice log into text data. Specifically, it uses a speech recognition API to analyze the voice log and generate text data. The input is the voice log, and the output is text data.

[1076] Step 4:

[1077] The server generates a prompt to be input to the generative AI model based on the textual content of the evaluation interview. For example, it creates a prompt such as, "Please analyze the content of the following evaluation interview and provide feedback on the good and bad points." The input is text data, and the output is a prompt.

[1078] Step 5:

[1079] The server sends the generated prompt sentence and text data to the generative AI model. Specifically, it sends the data using an API request. The input is the prompt sentence and text data, and the output is a request to the generative AI model.

[1080] Step 6:

[1081] The generative AI model analyzes the received prompt and text data. It uses natural language processing technology to understand the content of the text data and extracts the good and bad points of the evaluation interview. The input is the prompt and text data, and the output is feedback.

[1082] Step 7:

[1083] A generative AI model generates feedback based on the analysis results. For example, it generates feedback such as, "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems." The input is the analysis results, and the output is the feedback.

[1084] Step 8:

[1085] The server receives the feedback generated from the generative AI model. The feedback is sent back to the server through an API response. The input is the feedback, and the output is the feedback stored on the server.

[1086] Step 9:

[1087] The server sends the received feedback back to the user, for example, formatting the feedback in HTML and displaying it in the user's browser. The input is the feedback, and the output is the feedback displayed in the user's browser.

[1088] (Application example 2)

[1089] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1090] In conventional evaluation interview systems, recording interview content and generating feedback is done manually, which takes time and effort, and there are problems with inconsistent evaluation quality. Another issue is that the feedback is vague and lacks specificity, making it difficult for the members being evaluated to clearly understand areas for improvement. Furthermore, creating evaluation reports is time-consuming, placing a heavy burden on managers.

[1091] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1092] In this invention, the server includes a means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, a means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, and a means for automatically generating an evaluation report based on the feedback from the generative AI. This makes it possible to efficiently record the contents of evaluation interviews and automatically generate high-quality feedback. Furthermore, the automatic generation of evaluation reports reduces the burden on managers and improves the quality of evaluations.

[1093] A "management position" is a position within an organization that supervises and evaluates the work of members.

[1094] A "member" is an employee who is responsible for a specific task within an organization.

[1095] An "evaluation interview" is a conversation in which a manager evaluates a member's work performance and provides feedback.

[1096] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1097] "Transcription" refers to the process of converting audio logs into written data.

[1098] "Generative AI" refers to artificial intelligence that automatically generates feedback and reports based on input data.

[1099] "Feedback" refers to evaluation information such as good and bad points provided by generative AI based on the content of the evaluation interview.

[1100] An "evaluation report" is a summary document of evaluations that is automatically generated based on feedback from generative AI.

[1101] The system for implementing this invention efficiently records the content of evaluation interviews conducted by managers with their team members, generates high-quality feedback, and automatically generates evaluation reports. A specific embodiment of this system will be described below.

[1102] System configuration

[1103] 1. Speech Recognition Module

[1104] The server uses a speech recognition module to record the contents of the evaluation interview as an audio log. This module has the function of converting voice data into text data. Specifically, it uses a speech recognition library and Google's speech recognition API.

[1105] 2. Natural Language Processing Module

[1106] The server uses a natural language processing (NLP) module to analyze the textual content of the evaluation interview. This module has the function of analyzing the text data and evaluating the quality of the evaluation. Specifically, it uses the pipeline function of the transformers library and utilizes the Japanese BERT model.

[1107] 3. Generative AI Module

[1108] The server uses generative AI to generate feedback based on the data analyzed by the natural language processing module. This generative AI automatically generates feedback on the positive and negative points based on the input data.

[1109] 4. Report Generation Module

[1110] The server automatically generates an evaluation report based on feedback from the generative AI, which is provided to managers and used to encourage them to improve the quality of evaluation interviews.

[1111] System Operation

[1112] 1. Recording and transcribing audio logs

[1113] When a manager conducts an evaluation interview with a member, the contents are recorded as an audio log, which is then converted into text data by a speech recognition module.

[1114] 2. Text Data Analysis

[1115] The transcript of the interview is then analyzed by a natural language processing module, which evaluates the quality of the evaluation and extracts key points.

[1116] 3. Generate feedback

[1117] Based on the analysis results, the generative AI module generates feedback, including good and bad points.

[1118] 4. Automatic generation of evaluation reports

[1119] Based on the generated feedback, the report generation module automatically generates an evaluation report, which is provided to management to encourage improvement of the quality of evaluation interviews.

[1120] Specific examples

[1121] For example, by inputting the following prompt sentence into a generative AI model, specific feedback can be obtained.

[1122] Contents of the evaluation interview:

[1123] Worker A works quickly and produces high quality work.

[1124] However, there appears to be a lack of communication.

[1125] In the future, it will be necessary to strengthen cooperation with the team.

[1126] Generate feedback.

[1127] By inputting this prompt into a generative AI model, specific feedback can be obtained and an evaluation report can be automatically generated, allowing managers to conduct evaluation interviews efficiently and provide high-quality feedback.

[1128] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1129] Step 1:

[1130] The server records the contents of the evaluation interviews between managers and employees as audio logs. The audio logs are collected in real time during the evaluation interviews and saved as audio files. The input is the audio data of the evaluation interviews, and the output is an audio file.

[1131] Step 2:

[1132] The server uses a speech recognition module to convert the recorded voice log into text data. Specifically, it uses a speech recognition library to call Google's speech recognition API and converts the voice data into text data. The input is an audio file and the output is text data.

[1133] Step 3:

[1134] The server analyzes the textual content of the evaluation interview using a natural language processing (NLP) module. It uses the pipeline function of the transformers library to apply the Japanese BERT model to analyze the text data. The input is the text data, and the output is the analysis results.

[1135] Step 4:

[1136] The server uses a generative AI module to generate feedback based on the analysis results. The generative AI automatically generates feedback on good and bad points based on the input analysis results. The input is the analysis results, and the output is feedback data.

[1137] Step 5:

[1138] The server automatically generates an evaluation report based on the generated feedback. A report generation module is used to format the feedback data into an evaluation report. The input is the feedback data, and the output is the evaluation report.

[1139] Step 6:

[1140] The server provides the generated evaluation report to the manager, who then checks the evaluation report and uses it to improve the quality of the evaluation interview. The input is the evaluation report, and the output is provided to the manager.

[1141] Example 3

[1142] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1143] Conventional evaluation interview systems lacked a means to effectively provide feedback on the content of evaluation interviews conducted by managers, making it difficult to obtain specific advice or areas for improvement to improve the quality of evaluation interviews.In addition, there was a lack of a means to provide feedback to managers quickly and reliably, which sometimes delayed improvements to evaluation interviews.

[1144] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1145] In this invention, the server includes means for recording the content of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the text content of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for providing the generated feedback to the manager via a notification system, means for providing an interface for inputting the content of the evaluation interview, means for transmitting the content of the evaluation interview to the server using a secure communication protocol, means for generating feedback using a generative AI model, means for transmitting the generated feedback to the notification system, and means for notifying the feedback to the manager via the notification system. This enables the content of the evaluation interview to be effectively fed back, and enables managers to quickly and reliably receive specific advice and areas for improvement to improve the quality of the evaluation interview.

[1146] A "management position" is someone in a position within an organization who evaluates and guides members.

[1147] "Members" are employees and staff within an organization who are guided and evaluated by management.

[1148] An "evaluation interview" is an interview in which a manager evaluates a member's work performance and behavior and provides feedback.

[1149] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[1150] "Texting" is the process of converting audio logs into written data.

[1151] "Generative AI" is an artificial intelligence model that generates feedback and advice based on input data.

[1152] "Feedback" refers to specific advice and areas for improvement provided by the generative AI based on the content of the evaluation interview.

[1153] A "notification system" is a system for providing generated feedback to management.

[1154] The "interface" is a user interface for inputting the contents of the evaluation interview.

[1155] A "secure communication protocol" is a communication protocol for securely transmitting data.

[1156] "Server" is a computer system that uses generative AI models to generate feedback and provide it to managers via a notification system.

[1157] The present invention is a system for providing feedback to improve the quality of evaluation interviews conducted by managers. A specific embodiment of this system will be described below.

[1158] First, the user (manager) enters the details of the evaluation interview into a dedicated application or web form. For example, they write details about their subordinate's performance and areas for improvement. An example of a specific prompt sentence might be, "Subordinate A is good at managing the progress of projects, but feels that he has issues with communication with his team members. As an area for future improvement, I have suggested holding regular meetings."

[1159] Next, the device (user's PC or smartphone) sends the entered contents of the evaluation interview to the server. At this time, the data is sent using a secure communication protocol (e.g., HTTPS).

[1160] The server inputs the content of the received evaluation interview into a generative AI model (e.g., OpenAI's GPT-4), which generates feedback based on the input content. This feedback includes specific advice and improvements to improve the quality of the evaluation interview.

[1161] The generated feedback is sent from the server to a notification system, which delivers the feedback to the user, for example, by email or using a notification function of a dedicated application.

[1162] Finally, the terminal displays the feedback received from the notification system to the user, who receives the notification and checks for specific advice on how to improve the quality of the evaluation interview.

[1163] This system enables managers to effectively provide feedback on the content of the evaluation interviews and quickly and reliably obtain specific advice and points for improvement to improve the quality of the evaluation interviews.The flow of the identification process in Example 3 will be described with reference to Figure 15.

[1164] Step 1:

[1165] The user inputs the details of the evaluation interview.

[1166] The user (manager) enters the details of the evaluation interview using a dedicated application or web form. For example, the user might enter, "My subordinate, Mr. A, is good at managing the progress of projects, but I feel he has issues with communication with his team members. I have suggested that he hold regular meetings as an area for improvement in the future." The input data is saved in text format.

[1167] Step 2:

[1168] The terminal sends the input information to the server.

[1169] The device (user's PC or smartphone) sends the entered content of the evaluation interview to the server by pressing the "Send" button. At this time, the data is sent using a secure communication protocol (e.g., HTTPS). The input data is sent to the server in text format.

[1170] Step 3:

[1171] The server generates feedback using a generative AI model.

[1172] The server inputs the content of the received evaluation interview into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model analyzes the input text data and generates feedback including specific advice and improvements to improve the quality of the evaluation interview. The output data is the feedback in text format.

[1173] Step 4:

[1174] The server sends the generated feedback to a notification system.

[1175] The server sends the generated feedback to a notification system, which delivers the feedback to the user by, for example, email or a notification function of a dedicated application. The output data is the feedback in text format.

[1176] Step 5:

[1177] The terminal notifies the user of the feedback through a notification system.

[1178] The terminal (user's smartphone or PC) displays the feedback received from the notification system to the user. The user receives the notification and checks specific advice to improve the quality of the evaluation interview. The output data is the feedback in text format.

[1179] (Application example 3)

[1180] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1181] Conventional evaluation interview systems have the problem that it is difficult to objectively evaluate the quality of the evaluation interviews conducted by managers and identify areas for improvement. Furthermore, there is a lack of means to evaluate the performance of machines used in factory work and notify managers of this feedback, making it difficult to improve work efficiency and provide appropriate guidance. To solve these issues, a system is needed that improves the quality of evaluation interviews and properly evaluates the performance of machines in factories and provides feedback to managers.

[1182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1183] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for notifying the managers of the feedback from the generative AI, and means for evaluating the performance of machines operating in the factory and notifying the managers of the feedback. This makes it possible to objectively grasp the quality of the evaluation interviews and identify areas for improvement, as well as to appropriately evaluate the performance of machines in the factory and provide feedback to the managers.

[1184] A "management position" is a position within an organization that is responsible for supervising and evaluating the work of its members.

[1185] "Members" are employees who work within an organization under the direction of a manager.

[1186] An "evaluation interview" is an interview conducted by a manager to evaluate the work of a member and provide feedback.

[1187] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1188] "Texting" is the process of converting audio logs into written data.

[1189] "Generative AI" is an artificial intelligence system that generates feedback and evaluations based on input data.

[1190] "Feedback" refers to the evaluation and information on areas for improvement provided by generative AI based on the content of the evaluation interview and the machine's performance.

[1191] "Notification" refers to the act of informing management of feedback from generative AI.

[1192] "Machines performing work in factories" are automated devices that perform work such as production and assembly in factories.

[1193] "Performance" refers to the efficiency and quality of operation of machines that perform work within a factory.

[1194] A system for carrying out this invention has the following configuration: The server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for notifying managers of the feedback from the generative AI, and means for evaluating the performance of machines that perform work in a factory and notifying managers of the feedback.

[1195] Hardware and software used

[1196] Hardware: Factory robots, servers, managerial devices (PCs and smartphones)

[1197] Software: Python, generative AI models (e.g., OpenAI GPT-3), mail servers (e.g., SMTP)

[1198] Data processing and calculation

[1199] 1. Recording and Transcription of Audio Logs:

[1200] The server records the contents of the evaluation interview as an audio log.

[1201] Use speech recognition software (e.g., Google Speech-to-Text API) to convert audio logs into text.

[1202] 2. Getting input and feedback for generative AI models:

[1203] The server inputs the textual content of the evaluation interview into a generative AI model.

[1204] A generative AI model generates feedback based on the input text.

[1205] Example: Prompt: "Please rate the performance of robot ID: 123"

[1206] 3. Feedback Notification:

[1207] The server notifies management of feedback from the generative AI model.

[1208] For notifications, a mail server is used to send feedback to the manager's email address.

[1209] Specific examples

[1210] For example, when evaluating the performance of robot ID: 123 working in a factory, the server performs the following process.

[1211] 1. Collect robot work data and send the prompt "Please evaluate the performance of robot ID: 123" to the generative AI model.

[1212] 2. The generative AI model evaluates the robot's work efficiency and the quality of its movements, and generates feedback such as, "Robot ID: 123's work efficiency is 90%. The area for improvement is optimizing its movement speed."

[1213] 3. The server sends this feedback to the manager's email address, and the manager can use the feedback to identify areas for improvement in the robot's behavior and work efficiency.

[1214] In this way, it becomes possible to objectively grasp the quality of the evaluation interview and identify areas for improvement, as well as to properly evaluate the performance of machines in the factory and provide feedback to management.

[1215] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1216] Step 1:

[1217] The server records the contents of evaluation interviews conducted by managers with members as audio logs.

[1218] Input: Audio data from the evaluation interview

[1219] Output: Audio log

[1220] Specific operation: When an evaluation interview is conducted, the server uses the audio recording function to record the contents of the interview as audio data.

[1221] Step 2:

[1222] The server converts the recorded audio logs into text.

[1223] Input: Audio log

[1224] Output: Text data

[1225] Specific operation: The server uses voice recognition software (e.g., Google Speech-to-Text API) to convert the voice log into text data.

[1226] Step 3:

[1227] The server inputs the textual content of the evaluation interview into a generative AI model and receives feedback.

[1228] Input: Text data

[1229] Output: Feedback from the generative AI model

[1230] Specific operation: The server sends the text data to a generative AI model (e.g., OpenAI GPT-3) and inputs the prompt "Please generate feedback based on the content of the evaluation interview." The generative AI model generates feedback and sends it back to the server.

[1231] Step 4:

[1232] The server notifies management of feedback from the generative AI model.

[1233] Input: Feedback from a generative AI model

[1234] Output: Notification to management (e.g. email)

[1235] Specific operation: The server uses a mail server (e.g., SMTP) to send feedback from the generative AI model to the manager's email address.

[1236] Step 5:

[1237] The server evaluates the performance of machines operating within the factory and communicates the feedback to management.

[1238] Input: Machine operation data

[1239] Output: Feedback from the generative AI model, notification to management

[1240] How it works: The server collects work data from machines in the factory and sends a prompt to the generative AI model: "Please rate the performance of robot ID: 123." The generative AI model generates feedback and sends it back to the server. The server then sends this feedback to the manager's email address.

[1241] In this way, it becomes possible to objectively grasp the quality of the evaluation interview and identify areas for improvement, as well as to properly evaluate the performance of machines in the factory and provide feedback to management.

[1242] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1243] "Example 1"

[1244] One embodiment of the present invention provides a system including an emotion engine that recognizes a user's emotions during an evaluation interview. The emotion engine recognizes emotions from the user's voice, facial expressions, gestures, etc., and inputs the emotion data into a generative AI. Specifically, when the user expresses emotions such as joy, anger, surprise, or sadness, the emotion engine recognizes them and provides the information to the generative AI.

[1245] "Example 2"

[1246] The generative AI generates feedback based on the content of the evaluation interview and the user's emotional data. Specifically, if the user expresses joy, it determines from the content of the evaluation interview what that joy is about and generates the result as feedback. Also, if the user expresses anger, it determines from the content of the evaluation interview what that anger is about and generates the result as feedback.

[1247] "Example 3"

[1248] Feedback from the generative AI is provided to managers, who can view it on their own devices. This feedback includes not only good and bad points based on the content of the evaluation interview, but also information based on the user's emotional data. This allows managers to gain a deeper understanding of the quality of the evaluation interview and identify areas for improvement.

[1249] The processing flow of each embodiment will be described below.

[1250] "Example 1"

[1251] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[1252] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[1253] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[1254] "Example 2"

[1255] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[1256] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[1257] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[1258] Step 4: Feedback from the generative AI is provided to management, who reviews the feedback.

[1259] "Example 3"

[1260] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[1261] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[1262] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[1263] Step 4: Feedback from the generative AI is provided to management, who reviews the feedback.

[1264] Step 5: Managers take action based on the feedback to improve the quality of the evaluation interview.

[1265] Example 1

[1266] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1267] Conventional evaluation interview systems have difficulty not only recording the interview content but also recognizing the user's emotions during the interview in real time and providing feedback based on that. Furthermore, in order to improve the quality of feedback using generative AI, it is necessary to appropriately utilize the user's emotional data, but there was no established method for doing so.

[1268] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for recording the content of an evaluation interview conducted by a manager with a member as an audio log, a means for converting the audio log into text, a means for inputting the text content of the evaluation interview into a generative AI and receiving feedback from the generative AI, a means for recognizing the user's emotions, a means for inputting the recognized emotion data into the generative AI, and a means for providing the user with feedback from the generative AI. This enables high-quality feedback that utilizes not only the content of the evaluation interview but also the user's emotion data.

[1269] A "management position" is someone in a position within an organization who evaluates and guides members.

[1270] A "member" is someone in an organization who is in a position to receive guidance and evaluation from management.

[1271] An "evaluation interview" is a dialogue between a manager and a member of staff to evaluate their work and provide feedback.

[1272] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1273] "Text conversion" is the process of converting audio data into text data.

[1274] "Generative AI" is artificial intelligence that generates feedback and advice based on input data.

[1275] "Feedback" refers to evaluations and advice provided by generative AI based on the content of the evaluation interview and the user's emotional data.

[1276] "User" refers to the manager or member participating in the evaluation interview.

[1277] "Emotion data" is information about emotions recognized from the user's voice, facial expressions, gestures, etc.

[1278] "Emotion recognition" is the process of identifying emotions from a user's voice, facial expressions, gestures, etc.

[1279] A "server" is a computer system that processes voice and emotional data and generates feedback using generative AI.

[1280] A "terminal" is a device that records audio and video of the evaluation interview and transmits the data to a server.

[1281] MODE FOR CARRYING OUT THE INVENTION

[1282] This invention is a system that records evaluation interviews conducted by managers with their employees as audio logs, converts the contents into text, and further recognizes the user's emotions and inputs them into a generative AI.The purpose of this system is for the generative AI to provide appropriate feedback based on the contents of the evaluation interview and the user's emotional data.

[1283] Hardware and software used

[1284] Hardware: microphones, cameras, servers, devices (PCs and smartphones)

[1285] Software: Speech recognition software (e.g., Google Cloud Speech-to-Text), emotion recognition software, generative AI (e.g., OpenAI GPT-4)

[1286] Specific operation of the system

[1287] 1. Recording audio data

[1288] The user starts the evaluation interview.

[1289] The device records audio data in real time through a microphone.

[1290] The device receives a trigger to start recording and begins streaming audio data.

[1291] 2. Sending audio data to the server

[1292] The device divides the recorded audio data into chunks of a fixed buffer size.

[1293] The terminal transmits the divided audio data to the server.

[1294] The server temporarily stores the received audio data.

[1295] 3. Converting audio data into text

[1296] The server calls the speech recognition software (Google Cloud Speech-to-Text).

[1297] The server inputs the stored voice data into voice recognition software.

[1298] The server receives the text data returned from the speech recognition software.

[1299] The server stores the text data in a database.

[1300] 4. Capturing User Emotions

[1301] The device captures the user's facial expressions and gestures in real time through the camera.

[1302] The device divides the captured video data into a fixed number of frames.

[1303] The terminal transmits the divided video data to the server.

[1304] 5. Sending emotion data to the server

[1305] The terminal transmits the captured video data to the server.

[1306] The server temporarily stores the received video data.

[1307] 6. Emotional Data Recognition

[1308] The server invokes the emotion recognition software.

[1309] The server inputs the stored video data into emotion recognition software.

[1310] The server receives the emotion data returned from the emotion recognition software.

[1311] The server stores the emotion data in a database.

[1312] 7. Data input to generative AI and feedback generation

[1313] The server integrates the text and emotion data from the audio data.

[1314] The server inputs the integrated data into the generative AI (OpenAI GPT-4).

[1315] Generative AI generates feedback based on the input data.

[1316] The server provides the generated feedback to the user.

[1317] Specific examples

[1318] Evaluation interview scenario

[1319] A user says, "This month's project went very well."

[1320] The terminal records this speech and stores the audio data in a buffer.

[1321] The terminal transmits the audio data stored in the buffer to the server.

[1322] The server calls Google Cloud Speech-to-Text to convert the audio data into text data.

[1323] The server stores the text data "This month's project went very well" in the database.

[1324] The device captures the user's facial expressions through a camera and stores the video data in a buffer.

[1325] The terminal transmits the video data stored in the buffer to the server.

[1326] The server calls emotion recognition software and converts the video data into emotion data.

[1327] The server stores the emotion data of "joy" in a database.

[1328] The server combines the text data "This month's project went very well" with the emotion data "joy."

[1329] The server inputs the integrated data into OpenAI GPT-4.

[1330] The generative AI generates feedback such as, "That's a great result. Let's try to do the same on your next project."

[1331] The server provides the generated feedback to the user.

[1332] Example prompts for generative AI models

[1333] "If the user is expressing joy, generate appropriate feedback."

[1334] "If the user is showing signs of anger, generate advice on how to respond calmly."

[1335] In this way, the system efficiently records the contents of the evaluation interview, recognizes the user's emotions in real time, and provides appropriate feedback.

[1336] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1337] Step 1:

[1338] Recording audio data

[1339] The user starts the evaluation interview.

[1340] The device records audio data in real time through a microphone.

[1341] The device receives a trigger to start recording and begins streaming audio data.

[1342] Input: User's voice

[1343] Output: Recorded audio data

[1344] Step 2:

[1345] Sending audio data to the server

[1346] The device divides the recorded audio data into chunks of a fixed buffer size.

[1347] The terminal transmits the divided audio data to the server.

[1348] The server temporarily stores the received audio data.

[1349] Input: Recorded audio data

[1350] Output: Audio data sent to the server

[1351] Step 3:

[1352] Converting audio data to text

[1353] The server calls the speech recognition software (Google Cloud Speech-to-Text).

[1354] The server inputs the stored voice data into voice recognition software.

[1355] The server receives the text data returned from the speech recognition software.

[1356] The server stores the text data in a database.

[1357] Input: Audio data stored on the server

[1358] Output: Text data

[1359] Step 4:

[1360] Capturing User Emotions

[1361] The device captures the user's facial expressions and gestures in real time through the camera.

[1362] The device divides the captured video data into a fixed number of frames.

[1363] The terminal transmits the divided video data to the server.

[1364] Input: User facial expressions and gestures

[1365] Output: Captured video data

[1366] Step 5:

[1367] Sending emotion data to the server

[1368] The terminal transmits the captured video data to the server.

[1369] The server temporarily stores the received video data.

[1370] Input: Captured video data

[1371] Output: Video data sent to the server

[1372] Step 6:

[1373] Emotional Data Recognition

[1374] The server invokes the emotion recognition software.

[1375] The server inputs the stored video data into emotion recognition software.

[1376] The server receives the emotion data returned from the emotion recognition software.

[1377] The server stores the emotion data in a database.

[1378] Input: Video data stored on the server

[1379] Output: Emotion data

[1380] Step 7:

[1381] Data input to generative AI and feedback generation

[1382] The server integrates the text and emotion data from the audio data.

[1383] The server inputs the integrated data into the generative AI (OpenAI GPT-4).

[1384] Generative AI generates feedback based on the input data.

[1385] The server provides the generated feedback to the user.

[1386] Input: Text data and emotion data

[1387] Output: Feedback from the generative AI

[1388] In this way, the system efficiently records the contents of the evaluation interview, recognizes the user's emotions in real time, and provides appropriate feedback.

[1389] (Application example 1)

[1390] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1391] Conventional evaluation interview systems could record and convert interview content into text, but they were unable to recognize the user's emotions during the interview and generate appropriate feedback based on that. As a result, the quality of the interviews was not consistent, and communication between managers and team members was sometimes not smooth. Furthermore, communication between workers and robots in factories was not able to take into account the emotions of the workers, which led to issues with work efficiency and reducing worker stress.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for recording the contents of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the converted contents of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for recognizing emotions from the user's voice, facial expressions, and gestures and generating emotion data, and means for inputting the generated emotion data into the generative AI and generating an appropriate response. This improves the quality of the evaluation interview and facilitates communication between workers and robots in the factory, thereby improving work efficiency and reducing worker stress.

[1393] A "management position" is a position within an organization that is responsible for supervising members and conducting evaluation interviews.

[1394] "Members" are employees within an organization who are guided and evaluated by managers.

[1395] An "appraisal interview" is a conversation in which a manager evaluates a member's performance and behavior and provides feedback.

[1396] An "audio log" is audio data that is recorded from an evaluation interview and saved in digital format.

[1397] "Text conversion" is the process of converting audio data into text data.

[1398] "Generative AI" refers to artificial intelligence that generates feedback and responses based on input data.

[1399] "Feedback" refers to the evaluation and areas for improvement provided by generative AI based on the content of the evaluation interview.

[1400] "Emotion data" refers to emotional information recognized from the user's voice, facial expressions, and gestures.

[1401] "Appropriate response" refers to the optimal reaction or behavior toward the user that the generative AI generates based on emotional data.

[1402] The "system" refers to a set of devices and software that includes the above means and is used to improve the quality of evaluation interviews.

[1403] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings.

[1404] System configuration

[1405] The system consists of the following main components:

[1406] 1. Audio Log Recording Method:

[1407] Hardware: Microphone

[1408] Software: Audio recording application

[1409] Function: Records the contents of evaluation interviews between managers and members in real time and saves them as audio data.

[1410] 2. Text conversion method:

[1411] Software: Google Cloud Speech-to-Text API

[1412] Function: Converts recorded audio data into text data.

[1413] 3. Emotion recognition means:

[1414] Hardware: Camera, microphone

[1415] Software: Microsoft Azure Emotion API

[1416] Function: Recognizes emotions from the user's voice, facial expressions, and gestures, and generates emotion data.

[1417] 4. Generative AI means:

[1418] Software: OpenAI GPT-4

[1419] Function: Inputs the textual content and emotional data of the evaluation interview and generates appropriate feedback and responses.

[1420] System Operation

[1421] 1. Audio Log Transcript:

[1422] When managers conduct evaluation interviews with members, a microphone records the audio.

[1423] The recorded audio data is saved as an audio log.

[1424] 2. Converting audio data to text:

[1425] The recorded audio data is converted into text data using the Google Cloud Speech-to-Text API.

[1426] The text data stores the contents of the evaluation interview as text information.

[1427] 3. Emotion recognition:

[1428] The camera and microphone capture the user's facial expressions, voice, and gestures.

[1429] Using the Microsoft Azure Emotion API, we recognize user emotions and generate emotional data.

[1430] 4. Feedback generation using generative AI:

[1431] The textual content and emotional data from the evaluation interview are input into OpenAI GPT-4.

[1432] Generative AI generates appropriate feedback and responses based on input data.

[1433] Specific examples

[1434] For example, a communication support system between workers and robots in a factory works as follows:

[1435] 1. The worker instructs, "Please take this part to the next station."

[1436] 2. The microphone records the audio, and the Google Cloud Speech-to-Text API converts the audio to text.

[1437] 3. Cameras and microphones capture the worker's facial expressions and voice, and the Microsoft Azure Emotion API recognizes fatigue.

[1438] 4. Input the emotion data (fatigue) into OpenAI GPT-4, and the robot responds, "Got it. Please take a short break."

[1439] Example prompts to input to the generative AI model:

[1440] A worker commands, "Please move this part to the next station." The worker sounds tired. How should the robot respond?

[1441] This system will improve the quality of evaluation interviews and facilitate communication between workers and robots within the factory, thereby improving work efficiency and reducing worker stress.

[1442] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1443] Step 1:

[1444] When a manager conducts an evaluation interview with a member, the device's microphone records the audio. The input is the conversation between the manager and the member, and the output is audio data. The device saves this audio data as an audio log.

[1445] Step 2:

[1446] The server sends the saved voice data to the Google Cloud Speech-to-Text API. The input is voice data and the output is text data. The server converts the voice data into text data and saves the contents of the evaluation interview as text information.

[1447] Step 3:

[1448] The device's camera and microphone capture the user's facial expressions, voice, and gestures. The input is the user's facial expressions, voice, and gestures, and the output is emotion data. The device sends this data to the Microsoft Azure Emotion API, which recognizes the user's emotions and generates emotion data.

[1449] Step 4:

[1450] The server inputs the textual content of the evaluation interview and emotional data into OpenAI GPT-4. The input is text data and emotional data, and the output is generated feedback and responses. The server inputs this data into the generative AI to generate appropriate feedback and responses.

[1451] Step 5:

[1452] The server provides the generated feedback and responses to the manager. The input is the generated feedback and responses, and the output is a notification to the manager. The server notifies the manager of the feedback and responses to improve the quality of the appraisal interview.

[1453] Step 6:

[1454] In the communication between workers and robots in a factory, when a worker gives instructions to a robot, the microphone on the terminal records the voice. The input is the worker's voice instructions, and the output is voice data. The terminal saves this voice data as an audio log.

[1455] Step 7:

[1456] The server sends the saved voice data to the Google Cloud Speech-to-Text API. The input is voice data and the output is text data. The server converts the voice data into text data and saves the worker's instructions as text information.

[1457] Step 8:

[1458] The device's camera and microphone capture the worker's facial expressions, voice, and gestures. The input is the worker's facial expressions, voice, and gestures, and the output is emotion data. The device sends this data to the Microsoft Azure Emotion API, which recognizes the worker's emotions and generates emotion data.

[1459] Step 9:

[1460] The server inputs the textual instructions and emotional data into OpenAI GPT-4. The input is text data and emotional data, and the output is the generated response. The server inputs this data into the generative AI to generate an appropriate response.

[1461] Step 10:

[1462] The server provides the generated response to the robot. The input is the generated response, and the output is instructions to the robot. The server notifies the robot of the appropriate response based on the worker's instructions.

[1463] Example 2

[1464] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1465] With conventional evaluation interview systems, it was difficult to properly record and analyze the contents of evaluation interviews, making it difficult to obtain specific feedback to improve the quality of evaluations. Furthermore, there was a lack of means to provide feedback that took the user's emotions into consideration, making it impossible to maximize the effectiveness of evaluation interviews. This resulted in a decline in the quality of evaluation interviews, hindering the growth and motivation of members.

[1466] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1467] In this invention, the server includes means for recording the content of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the converted content of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for the generative AI to generate feedback based on the content of the evaluation interview and user emotion data, and means for providing the generated feedback to the user. This makes it possible to accurately record and analyze the content of the evaluation interview and provide specific feedback that takes the user's emotions into consideration. This makes it possible to improve the quality of the evaluation interview and promote member growth and motivation.

[1468] A "management position" is a position within an organization that is responsible for evaluating and guiding members.

[1469] A "member" is an individual employee within an organization who receives guidance and evaluation from management.

[1470] An "evaluation interview" is a conversation between a manager and a member of staff to evaluate their performance and provide feedback.

[1471] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1472] "Transcription" refers to the process of converting audio logs into written data.

[1473] "Generative AI" refers to artificial intelligence that generates feedback and analytical results based on input data.

[1474] "Feedback" refers to the evaluation and areas for improvement provided by generative AI based on the content of the evaluation interview.

[1475] "Users" are the members and managers who receive feedback from evaluation interviews.

[1476] "Emotion data" refers to data that indicates the user's emotional state, and includes emotions such as joy and anger.

[1477] The "server" is a computer system that records the contents of evaluation interviews and transmits the data to the generative AI.

[1478] This invention is a system that records the contents of an evaluation interview as an audio log, converts it into text, and generates feedback using generative AI. Specific embodiments of this system are described below.

[1479] Hardware and software used

[1480] Hardware: Servers (e.g., general-purpose servers), terminals (e.g., personal computers, smartphones)

[1481] Software: Speech recognition software (e.g., speech recognition APIs), natural language processing libraries (e.g., natural language processing libraries), generative AI models (e.g., generative AI models)

[1482] System Operation

[1483] Subject: Server

[1484] The server records the contents of evaluation interviews conducted by managers with employees as audio logs. The audio logs are converted into text data using a speech recognition API. The converted text data is input into a generative AI model.

[1485] Subject: Generative AI

[1486] The generative AI analyzes the input text data and generates feedback based on the content of the evaluation interview. The generative AI uses a natural language processing library to tokenize the text data and perform grammatical and semantic analysis. It also generates feedback based on the user's emotional data.

[1487] Subject: Server

[1488] The server provides the feedback received from the generative AI to the user, who can then check the feedback using their device.

[1489] Specific examples

[1490] Contents of the evaluation interview

[1491] Evaluator: Your contribution to this project was enormous, especially your efforts in data analysis.

[1492] Applicant: Thank you. However, I felt there was a lack of communication during the project.

[1493] User emotion data

[1494] Delight: Show delight when someone says, "Your efforts on the data analysis part were great."

[1495] Anger: Showing anger when told there was a lack of communication.

[1496] Example prompts for generative AI models

[1497] Evaluation interview: "Your contribution to this project was enormous. Your efforts in data analysis were particularly impressive."

[1498] User emotion data: "Joy"

[1499] Feedback example

[1500] Feedback: "You seem pleased that your data analysis efforts are being recognized. This is a very good point. However, you also pointed out a lack of communication. If you pay more attention to this in the future, you could receive a better evaluation."

[1501] In this way, the server records the contents of the evaluation interview as an audio log and converts it into text. The generative AI analyzes the text data and generates feedback based on the user's emotional data. The server provides the generated feedback to the user, who can then check it using their device.

[1502] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1503] Step 1:

[1504] The server records the contents of the evaluation interviews conducted by managers with members as audio logs.

[1505] Specific operation: The server receives the audio data of the evaluation interview and saves it as an audio log.

[1506] Input: Audio data from the evaluation interview

[1507] Output: Audio log

[1508] Step 2:

[1509] The server converts the audio log into text data.

[1510] Specific operation: The server calls the speech recognition API and converts the voice log into text data.

[1511] Input: Audio log

[1512] Output: Text data

[1513] Step 3:

[1514] The server inputs the textual content of the evaluation interview into a generative AI model.

[1515] Specific operation: The server sends the text data to the API of the generative AI model and waits for the analysis results.

[1516] Input: Text data

[1517] Output: Input to a generative AI model

[1518] Step 4:

[1519] The generative AI analyzes text data and generates feedback based on the content of the evaluation interview.

[1520] How it works: Generative AI uses natural language processing libraries to tokenize text data, perform grammatical and semantic analysis, and generate feedback based on user sentiment data.

[1521] Input: Text data, user emotion data

[1522] Output: Feedback

[1523] Step 5:

[1524] The server provides the user with the feedback received from the generative AI.

[1525] Specific behavior: The server sends the feedback to the user's device so that the user can check it.

[1526] Input: Feedback

[1527] Output: Feedback to the user's terminal

[1528] Step 6:

[1529] The user uses the terminal to check the feedback.

[1530] Specific actions: The user views the feedback on the device and understands the content of the evaluation interview.

[1531] Input: Feedback

[1532] Output: User understanding

[1533] (Application example 2)

[1534] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1535] Conventional appraisal interview systems have had problems with providing appropriate feedback on the content of appraisal interviews, particularly with a lack of feedback that takes into account the user's emotions. Furthermore, there is a lack of means to provide specific improvements to improve the quality of appraisal interviews. This has resulted in limited effectiveness of appraisal interviews, and the quality of communication between managers and employees has not improved.

[1536] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1537] In this invention, the server includes means for recording the content of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the converted content of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for acquiring user emotion data and inputting it together with the content of the evaluation interview into the generative AI, and means for providing feedback from the generative AI to the user. This enables feedback that takes into account the content of the evaluation interview and the user's emotions, and makes it possible to provide specific improvements that will improve the quality of the evaluation interview.

[1538] A "management position" is a position within an organization that involves supervising the work of members and conducting evaluation interviews.

[1539] "Members" are employees who work within an organization under the direction of a manager.

[1540] An "evaluation interview" is an interactive meeting in which managers evaluate team members' performance and provide feedback.

[1541] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1542] "Texting" is the process of converting audio logs into written data.

[1543] "Generative AI" is an artificial intelligence system that generates feedback based on input data.

[1544] "Feedback" refers to evaluation information such as good and bad points provided by generative AI based on the content of the evaluation interview.

[1545] "User" refers to the manager or member participating in the evaluation interview.

[1546] "Emotion data" is data that indicates the emotional state of the user, and includes emotions such as joy and anger.

[1547] The "server" is a computer system that records the contents of evaluation interviews, converts them into text, and inputs them into the generative AI.

[1548] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings.

[1549] First, the system records the contents of evaluation interviews conducted by managers with their employees as audio logs. The audio logs are captured using a smartphone or a dedicated recording device. Next, speech recognition software is used to convert the captured audio logs into text. Specifically, speech recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text can be used.

[1550] The textual content of the evaluation interview is input into a generative AI. Natural language processing models such as OpenAI's GPT-3 and GPT-4 are used as the generative AI. The generative AI analyzes the input text data and generates feedback such as good and bad points based on the content of the evaluation interview.

[1551] Furthermore, emotion recognition software is used to obtain user emotional data. For example, emotion recognition technology from Microsoft Azure's Emotion API can be used. The emotional data, along with the content of the evaluation interview, is input into generative AI and used to improve the quality of feedback.

[1552] Feedback from the generative AI is provided to the user. The feedback can be displayed on a smartphone or computer screen, or read aloud. This allows users to receive specific feedback based on the content of the evaluation interview and emotional data.

[1553] As a concrete example, the following prompt sentence could be input to a generative AI:

[1554] Example prompt sentence:

[1555] Please generate feedback on the positive and negative points based on the content of the evaluation interview below.

[1556] Contents of the evaluation interview:

[1557] Robot A is fast but lacks precision. Robot B is highly accurate but slow.

[1558] User sentiment data:

[1559] Joy: Robot A's speed, Anger: Robot B's slowness

[1560] feedback:

[1561] In this way, feedback that takes into account the content of the evaluation interview and the user's emotional data can be generated and provided to the user. This system can improve the quality of evaluation interviews and communication between managers and employees.

[1562] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1563] Step 1:

[1564] Managers conduct evaluation interviews and record the contents as audio logs.

[1565] Input: Audio from the evaluation interview

[1566] Output: Audio log

[1567] How it works: Managers record the appraisal interview using a smartphone or a dedicated recording device, and the recorded audio data is saved as an audio log.

[1568] Step 2:

[1569] The server transcribes the audio log into text.

[1570] Input: Audio log

[1571] Output: Text data

[1572] How it works: The server uses a speech recognition service such as Google Speech-to-Text API or IBM Watson Speech to Text to convert the audio log into text data, which is then passed on to the next processing step.

[1573] Step 3:

[1574] The server acquires the user's emotion data.

[1575] Input: User's facial expressions and voice during the evaluation interview

[1576] Output: Emotion data

[1577] How it works: The server uses emotion recognition technology from Microsoft Azure's Emotion API to obtain emotional data from the user's facial expressions and voice during the evaluation interview, and passes the obtained emotional data to the next processing step.

[1578] Step 4:

[1579] The server inputs text data and emotional data into the generative AI and receives feedback.

[1580] Input: Text data, emotion data

[1581] Output: Feedback

[1582] How it works: The server combines text and emotion data into a prompt and inputs it into a generative AI such as OpenAI's GPT-3 or GPT-4. The generative AI analyzes the input data and generates feedback such as good and bad points based on the content of the evaluation interview.

[1583] Step 5:

[1584] The server provides the generated feedback to the user.

[1585] Input: Feedback

[1586] Output: Feedback that is displayed to the user

[1587] Specific operation: The server displays the generated feedback on the screen of a smartphone or PC. It also provides a function to read the feedback aloud if necessary. This allows the user to receive specific feedback based on the content of the evaluation interview and emotional data.

[1588] Example 3

[1589] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1590] Conventional evaluation interview systems have the problem that it is difficult for managers to objectively evaluate the quality of the interview and identify areas for improvement. In addition, they do not provide feedback that takes into account the user's emotional data, making it impossible to deeply understand the quality of the interview. This makes it difficult to improve the quality of evaluation interviews.

[1591] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1592] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI model and receiving feedback from the generative AI model, means for collecting user emotion data and inputting the emotion data into the generative AI model, means for notifying the manager's terminal of the feedback from the generative AI model, and means for the manager to check the feedback on his or her own terminal. This enables the manager to objectively grasp the quality of the evaluation interviews and identify specific areas for improvement that take into account the user's emotion data.

[1593] A "management position" is a position within an organization that is responsible for evaluating and guiding members.

[1594] "Members" are employees and staff within an organization who are guided and evaluated by management.

[1595] An "evaluation interview" is an interview conducted by a manager with a member of staff to evaluate their work progress and performance.

[1596] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[1597] "Texting" is the process of converting audio logs into written data.

[1598] A "generative AI model" is an artificial intelligence model that generates feedback and suggestions based on input data.

[1599] "Feedback" refers to evaluation information such as good and bad points provided by the generative AI model based on the content of the evaluation interview.

[1600] "User emotion data" is data that indicates the user's emotional state during the evaluation interview, and is acquired using voice analysis and facial expression recognition technology.

[1601] "Notification" is the process of informing managers of feedback from generative AI models on their devices.

[1602] "Device" refers to the device used by managers to review feedback, including PCs, tablets, smartphones, etc.

[1603] This invention is a system for managers to improve the quality of performance evaluation interviews, and provides feedback using a generative AI model. Specific embodiments of this system are described below.

[1604] Hardware and software used

[1605] Hardware: Servers, management devices (PCs, tablets, smartphones, etc.)

[1606] Software: Generative AI models (e.g., OpenAI's GPT-4), notification systems, messaging platforms, databases (e.g., MySQL)

[1607] Data processing and calculation

[1608] 1. Collecting user evaluation interview data

[1609] When a user undergoes an evaluation interview, the content of the interview and the user's emotional data are collected. The emotional data is obtained using, for example, voice analysis and facial expression recognition technology. This data is collected in real time and sent from the device to the server.

[1610] 2. Data storage by the server

[1611] The server stores the received evaluation interview data and emotion data in a database, using, for example, MySQL. The stored data includes the interview text content, the user's emotion score, the interview date and time, etc.

[1612] 3. Server generates and sends prompts to the AI ​​model

[1613] The server generates prompts based on the stored evaluation interview data to be sent to the generative AI model. The prompts include the interview content and emotional data. The generated prompts are sent to the generative AI model (e.g., OpenAI's GPT-4).

[1614] 4. Feedback generation using generative AI models

[1615] The generative AI model generates feedback for the evaluation interview based on the prompts received. The feedback includes good and bad points about the interview and information based on the user's emotional data. The generated feedback is sent back to the server.

[1616] 5. Receiving and storing feedback by the server

[1617] The server receives the feedback sent back by the generative AI model and stores it in a database, which includes specific advice and areas for improvement based on the interview content.

[1618] 6. Server-Based Feedback Notification

[1619] The server then notifies the manager of the saved feedback via a notification system, such as a messaging platform, which includes a summary of the feedback and a link to the details.

[1620] 7. Displaying feedback via device

[1621] The manager's device receives notifications from the server and displays the feedback, allowing the manager to review the feedback on their own device and receive specific advice on how to improve the quality of the appraisal interview.

[1622] Specific examples

[1623] Evaluation interview: "We discussed the progress of the project. The users seemed unhappy with the delays."

[1624] Feedback from a generative AI model: "Your explanation of the project's progress was clear, but your response to user complaints was insufficient. Next time, pay more attention to user feelings and provide concrete solutions."

[1625] Prompt Sentence Examples

[1626] "Generate feedback based on the content of the evaluation interview and user sentiment data. Content of the evaluation interview: 'We discussed the progress of the project. The user seemed unhappy with the delays in the project.'"

[1627] This system allows managers to gain a deeper understanding of the quality of the evaluation interviews and identify areas for improvement. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1628] Step 1:

[1629] When a user undergoes an evaluation interview, the device records the interview contents as an audio log and collects the user's emotional data. The emotional data is obtained using voice analysis and facial expression recognition technology. The input is the audio data of the evaluation interview and the user's emotional data, and the output is a record of these data.

[1630] Step 2:

[1631] The terminal transmits the recorded voice log and emotion data to the server. The input is the voice log and emotion data, and the output is data transmission to the server.

[1632] Step 3:

[1633] The server converts the received audio logs into text. It uses speech recognition software to convert the audio data into text. The input is the audio log, and the output is the transcribed content of the evaluation interview.

[1634] Step 4:

[1635] The server stores the textual content of the evaluation interview and the emotional data in a database. For example, MySQL is used as the database. The input is the textual content of the evaluation interview and the emotional data, and the output is saved in the database.

[1636] Step 5:

[1637] The server generates prompts to be sent to the generative AI model based on the stored evaluation interview data and emotion data. The prompts include the interview content and emotion data. The input is the textual content of the evaluation interview and emotion data, and the output is the generated prompt.

[1638] Step 6:

[1639] The server sends the generated prompt to the generative AI model. The input is the generated prompt, and the output is the prompt sent to the generative AI model.

[1640] Step 7:

[1641] The generative AI model generates feedback for the evaluation interview based on the prompts received. The feedback includes information based on the positive and negative aspects of the interview and the user's emotional data. The input is the prompt, and the output is the generated feedback.

[1642] Step 8:

[1643] The server receives the feedback sent back from the generative AI model and stores it in a database. The input is the generated feedback and the output is the storage in the database.

[1644] Step 9:

[1645] The server notifies the manager's terminal of the saved feedback. The notification system uses, for example, a messaging platform. The input is the generated feedback, and the output is the notification to the manager's terminal.

[1646] Step 10:

[1647] The manager's device receives notifications from the server and displays the feedback. The manager can check the feedback on their own device and get specific advice to improve the quality of the appraisal interview. The input is the notification from the server, and the output is the display of the feedback.

[1648] (Application example 3)

[1649] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1650] Conventional evaluation interview systems have the problem that it is difficult for managers to objectively evaluate the quality of the interviews and identify areas for improvement. Furthermore, because feedback is not provided that takes into account the worker's emotional data, there is also the problem that the quality of the interviews cannot be sufficiently improved. Furthermore, because feedback is not provided in real time, there is also the problem that it is difficult to make rapid improvements.

[1651] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1652] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for recognizing the worker's emotional data, means for notifying the manager of the feedback from the generative AI, and means for the manager to check the feedback from the generative AI on his or her own device. This enables the manager to objectively grasp the quality of the evaluation interviews and receive feedback in real time that takes into account the worker's emotional data.

[1653] A "management position" is a position within an organization that is responsible for guiding and evaluating members.

[1654] "Members" are employees within an organization who are guided and evaluated by managers.

[1655] An "evaluation interview" is an interview in which a manager evaluates a member's work performance and behavior and provides feedback.

[1656] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1657] "Transcription" refers to the process of converting audio logs into written data.

[1658] "Generative AI" refers to artificial intelligence that generates feedback and information based on input data.

[1659] "Feedback" refers to evaluations and areas for improvement provided by generative AI based on the content of the evaluation interview and emotional data.

[1660] "Emotion data" refers to the emotional state of a worker that is recognized and recorded as data.

[1661] "Notification" refers to the process of informing management of feedback from generative AI.

[1662] A "device" is any electronic device, such as a smartphone or tablet, that a manager uses to view feedback.

[1663] A system for implementing this invention is configured as follows. First, the contents of evaluation interviews conducted by managers with their employees are recorded as audio logs. Next, speech recognition software is used to convert these audio logs into text. Specifically, a speech recognition service such as the Google Cloud Speech-to-Text API can be used.

[1664] The textual content of the evaluation interview is input into a generative AI. This generative AI uses an advanced natural language processing model, such as OpenAI's GPT-4. The generative AI generates feedback based on the input text data. This feedback includes good and bad points based on the content of the evaluation interview, as well as information based on the worker's emotional data.

[1665] Emotion data is acquired using software that recognizes emotions from the worker's facial expressions and voice, such as Microsoft Azure's Emotion API or an emotion recognition system that combines OpenCV and Dlib.

[1666] The generated feedback is provided to management through a notification system, which may include an application for displaying the feedback on a device such as a smartphone or tablet, or may send notifications in real time using a messaging platform, for example.

[1667] For example, the following prompts can be fed to a generative AI model:

[1668] Example prompt sentence:

[1669] Conduct a performance evaluation of Worker A and generate feedback based on the following data:

[1670] Working speed: 85%

[1671] Working accuracy: 90%

[1672] Emotion data: Smile 70%, Surprise 10%, Anger 5%, Sadness 15%

[1673] feedback:

[1674] Pros: Very fast working speed and excellent accuracy.

[1675] Areas for improvement: Stress management is needed as emotions of surprise and anger are observed during the work.

[1676] In this way, managers can objectively grasp the quality of the evaluation interviews and receive real-time feedback that takes into account the workers' emotional data, which will promote improvements in the quality of evaluation interviews and contribute to improving workers' performance.

[1677] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1678] Step 1:

[1679] The contents of evaluation interviews conducted by managers with members are recorded as audio logs.

[1680] Input: Audio data from the evaluation interview

[1681] Output: Audio log file

[1682] Specific operation: The device (e.g., smartphone or tablet) records the audio of the evaluation interview and saves it as an audio log file.

[1683] Step 2:

[1684] Use voice recognition software to transcribe audio logs into text.

[1685] Input: Audio log file

[1686] Output: Text data

[1687] Specific operation: The server calls a speech recognition service such as Google Cloud Speech-to-Text API to convert the audio log file into text data.

[1688] Step 3:

[1689] The textual content of the evaluation interview is input into the generative AI.

[1690] Input: Text data

[1691] Output: Input data for generative AI

[1692] Specific operation: The server converts the text data into a format for sending to a generative AI (e.g., OpenAI's GPT-4).

[1693] Step 4:

[1694] Generative AI generates feedback.

[1695] Input: Input data for generative AI

[1696] Output: Feedback data

[1697] Specific operation: The generative AI generates feedback on the content of the evaluation interview based on the input text data.

[1698] Step 5:

[1699] Recognize worker emotion data.

[1700] Input: Facial expression and voice data of the worker

[1701] Output: Emotion data

[1702] Specific operation: The device uses Microsoft Azure's Emotion API, OpenCV, and Dlib to recognize emotions from the worker's facial expressions and voice and save them as emotion data.

[1703] Step 6:

[1704] Generative AI generates feedback that takes emotional data into account.

[1705] Input: Feedback data, emotion data

[1706] Output: Final feedback data

[1707] Specific operation: The server inputs feedback data and emotion data into the generative AI, and generates final feedback that takes the emotion data into consideration.

[1708] Step 7:

[1709] Provide generated feedback to management through a notification system.

[1710] Input: Final feedback data

[1711] Output: Feedback displayed on the manager's device

[1712] What it does: The server uses a messaging platform to send the final feedback data to the manager's smartphone or tablet in real time.

[1713] Step 8:

[1714] Managers view feedback on their own devices.

[1715] Input: Feedback displayed on the manager's device

[1716] Output: Managerial understanding and action

[1717] What it does: Managers can view feedback on their smartphones or tablets to help improve the quality of evaluation interviews.

[1718] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1719] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1720] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1721] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1722] [Third embodiment]

[1723] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1724] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1725] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1726] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1727] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1728] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1729] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1730] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1731] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1732] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1733] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1734] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1735] "Example 1"

[1736] One embodiment of the present invention is a system that records evaluation interviews conducted by managers with their team members as audio logs. This system has the function of recording the contents of the evaluation interviews in real time and converting the audio data into text data. Specifically, the system converts the audio data into text using voice recognition technology.

[1737] "Example 2"

[1738] Furthermore, it also has the function of inputting the contents of the evaluation interview converted into text into the generative AI and receiving feedback from the generative AI. The generative AI then identifies good and bad points based on the contents of the evaluation interview.

[1739] Specifically, the system analyzes text data using natural language processing technology to evaluate the quality of the evaluation interview.

[1740] "Example 3"

[1741] Furthermore, the system also has a function that provides feedback from the generative AI to managers and encourages them to improve the quality of evaluation interviews. Specifically, it has a notification system that notifies managers of feedback from the generative AI. This allows managers to objectively understand the quality of their own evaluation interviews and identify areas for improvement.

[1742] The processing flow of each embodiment will be described below.

[1743] "Example 1"

[1744] Step 1: Managers initiate evaluation interviews with members. These evaluation interviews are conducted in the form of direct dialogue between managers and members.

[1745] Step 2: Once the assessment interview begins, the system automatically records the interview as an audio log in real time, capturing the entire interview.

[1746] Step 3: After the evaluation interview is completed, the system converts the recorded audio log into text using voice recognition technology.

[1747] "Example 2"

[1748] Step 1: The system inputs the text of the evaluation interview into the generative AI. This input is done automatically, without human intervention.

[1749] Step 2: The generative AI analyzes the input text data and generates feedback on good and bad points based on the content of the evaluation interview. This analysis and feedback generation is carried out using natural language processing technology.

[1750] "Example 3"

[1751] Step 1: Once the generative AI feedback is generated, the system provides it to the manager via a notification system, where the manager can view the feedback on their device.

[1752] Step 2: Based on the feedback provided, managers can objectively assess the quality of their own evaluation interviews and identify areas for improvement, which will promote improvements in the quality of evaluation interviews.

[1753] Example 1

[1754] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1755] Conventional methods for recording evaluation interviews have had the problem of making it difficult to accurately record the interview content and analyze it later. In addition, there are limited means of obtaining feedback to improve the quality of the interviews, making it difficult for managers to identify areas for improvement in the interviews.

[1756] The identification process by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes a means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, a means for receiving audio data in real time and converting it into text data using speech recognition technology, a means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, and a means for saving the generated text data in a database. This makes it possible to accurately record the contents of the evaluation interviews and analyze them in detail later. Furthermore, specific improvements to improve the quality of the interviews can be identified through feedback from the generative AI.

[1757] A "management position" is someone in a position within an organization who evaluates and guides members.

[1758] A "member" is someone in an organization who is in a position to receive guidance and evaluation from management.

[1759] An "evaluation interview" is an interview conducted by a manager with a member of staff to evaluate their work and provide feedback.

[1760] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1761] "Text conversion" is the process of converting audio data into text data.

[1762] "Generative AI" is an artificial intelligence system that generates feedback and analytical results based on input data.

[1763] "Feedback" refers to evaluation information such as good points and areas for improvement that is provided by generative AI based on the content of the evaluation interview.

[1764] A "database" is an information system for storing and managing text data and feedback.

[1765] "Voice recognition technology" is a technology that analyzes voice data and converts it into text data.

[1766] "Real-time" refers to processing occurring immediately while the assessment interview is in progress.

[1767] This invention is a system that records evaluation interviews between managers and their employees as audio logs, converts the audio data into text data, and inputs it into a generative AI to obtain feedback. A specific embodiment of this system is described below.

[1768] Server Roles

[1769] The server receives the voice data from the evaluation interview in real time and converts it into text data using speech recognition technology. Specifically, it uses speech recognition software such as Google Cloud Speech-to-Text API. The converted text data is then stored in a database such as MySQL or PostgreSQL.

[1770] Device Role

[1771] The device provides an interface for managers to start recording when they begin an evaluation interview. A dedicated recording application is used to send the audio data being recorded to the server in real time. After the interview ends, the recording is stopped and the text data is retrieved from the server and displayed.

[1772] User Roles

[1773] The user (management) presses the recording button on the device to start the evaluation interview. During the interview, the user can check the text data in real time to confirm whether specific keywords and phrases were accurately recorded. After the interview is over, the user stops recording and reviews the generated text data.

[1774] Specific examples

[1775] Example 1: Recording and transcribing evaluation interviews

[1776] Scenario: Manager A conducts an evaluation interview with member B.

[1777] Hardware used: Manager A's smartphone

[1778] Software used: Dedicated recording application, Google Cloud Speech-to-Text API, MySQL database

[1779] 1. Manager A launches the dedicated recording application on his smartphone and presses the record button.

[1780] 2. The smartphone records the interview audio and transmits it to the server in real time.

[1781] 3. The server converts the audio data into text using the Google Cloud Speech-to-Text API.

[1782] 4. The server stores the converted text data in a MySQL database.

[1783] 5. After the interview, Manager A stops the recording and checks the text data generated by the smartphone application.

[1784] Example prompts to input to the generative AI model

[1785] Prompt: "Describe a system that records managerial interviews with employees and converts the audio data into text in real time. Include the names of the specific hardware and software."

[1786] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[1787] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1788] Step 1:

[1789] Subject: User

[1790] The user presses the recording button on the terminal when starting the evaluation interview.

[1791] Input: User operation (pressing the record button)

[1792] Output: Trigger to start recording

[1793] What happens: The user taps the record button in the smartphone or tablet application to start the recording process.

[1794] Step 2:

[1795] Subject: Terminal

[1796] The terminal detects that the recording button has been pressed and starts recording.

[1797] Input: Trigger to start recording

[1798] Output: Audio data being recorded

[1799] Specific operation: Captures audio data using the device's microphone and prepares to send it to the server in real time.

[1800] Step 3:

[1801] Subject: Terminal

[1802] The terminal transmits the audio data being recorded to the server in real time.

[1803] Input: Audio data being recorded

[1804] Output: Sending audio data to the server

[1805] Specific behavior: Recorded audio data is sent to the server using WebSocket or HTTP streaming.

[1806] Step 4:

[1807] Subject: Server

[1808] The server receives the voice data transmitted from the terminal in real time.

[1809] Input: Audio data from the device

[1810] Output: Temporarily saves received audio data

[1811] Specific operation: Receives audio data via an HTTP request and temporarily stores it in the server's local storage or cloud storage.

[1812] Step 5:

[1813] Subject: Server

[1814] The server converts the received voice data into text data using voice recognition technology.

[1815] Input: Received audio data

[1816] Output: Converted text data

[1817] What it does: Uses the Google Cloud Speech-to-Text API to create an API request to convert audio data to text and send the audio data.

[1818] Step 6:

[1819] Subject: Server

[1820] The server stores the converted text data in a database.

[1821] Input: Converted text data

[1822] Output: Text data stored in a database

[1823] Specific operation: Executes an INSERT query in a database such as MySQL or PostgreSQL and saves text data.

[1824] Step 7:

[1825] Subject: User

[1826] The user presses the stop recording button after the interview is over.

[1827] Input: User action (pressing the stop recording button)

[1828] Output: Trigger to stop recording

[1829] What happens: The recording process ends when the user taps the stop recording button in the smartphone or tablet application.

[1830] Step 8:

[1831] Subject: Terminal

[1832] The terminal detects that the recording stop button has been pressed and stops recording.

[1833] Input: Trigger to stop recording

[1834] Output: End of recording

[1835] Specific Action: Ends the recording process and stops sending data to the server.

[1836] Step 9:

[1837] Subject: Terminal

[1838] The terminal displays the text data obtained from the server to the user.

[1839] Input: Text data from the server

[1840] Output: Text data that is displayed to the user

[1841] Specific behavior: Renders text data in the application's UI and displays it for the user to see.

[1842] Step 10:

[1843] Subject: User

[1844] The user reviews the generated text data and adds corrections or comments as necessary.

[1845] Input: The displayed text data

[1846] Output: Text data with corrections and comments added

[1847] Specific Behavior: Use the application's text editor function to add corrections and comments to text data.

[1848] (Application example 1)

[1849] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1850] The conventional evaluation interview system had the problem that it was difficult for managers to accurately record the contents of interviews with team members and analyze them later. There was also a lack of a way to efficiently record work instructions and reports within the factory and convert them into text data in real time. This made it difficult to manage work progress and quality, and there was a risk of overall productivity decreasing.

[1851] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1852] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the textual contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for recording work instructions and reports within the factory as audio logs and converting them into text data in real time, and means for storing the converted text data in the cloud so that it can be searched and analyzed later. This makes it possible to accurately record the contents of evaluation interviews and receive feedback from the generative AI, as well as efficiently record work instructions and reports within the factory and convert them into text data in real time.

[1853] A "management position" is a position within an organization that is responsible for supervising and evaluating the work of its members.

[1854] "Members" refer to employees or staff members who are responsible for specific tasks within an organization.

[1855] An "evaluation interview" is a dialogue-style interview in which managers evaluate their team members' work performance and results and provide feedback.

[1856] An "audio log" is a digital file that records audio data such as evaluation interviews and work instructions.

[1857] "Text conversion" is the process of converting audio data into text data.

[1858] "Generative AI" is an artificial intelligence system that generates feedback and suggestions based on input data.

[1859] "Feedback" refers to evaluations and information on areas for improvement provided by generative AI based on the content of evaluation interviews and work instructions.

[1860] "Inside the factory" refers to the interior of the facility where the product is manufactured or processed.

[1861] "Work instructions" are information that instructs specific work content and procedures within a factory.

[1862] "Report" is information used to communicate the progress and results of work.

[1863] "Real-time" refers to data being processed and transformed as it is generated.

[1864] "Cloud" refers to a collection of remote servers provided over the internet for data storage and processing.

[1865] "Searching" is the process of locating specific information from stored data.

[1866] "Analysis" is the process of examining collected data in detail to find meaning and trends.

[1867] A system for implementing this invention is configured as follows: First, a manager uses smart glasses or a device with a microphone to record the contents of an evaluation interview with a member of staff as an audio log. This allows audio data from the evaluation interview to be collected in real time.

[1868] The collected voice data is then converted to text using a Python program with the Google Cloud Speech-to-Text API. This text data is then input into a generative AI system, which generates feedback based on the content of the interview. This feedback is then provided to managers to encourage them to improve the quality of the interviews.

[1869] Furthermore, work instructions and reports within the factory are also recorded as voice logs and converted into text data in real time. This text data is stored in the cloud and can be searched and analyzed later. This allows for efficient progress management and quality control of work.

[1870] The hardware used includes smart glasses, a device with a microphone, a PC or a cloud server, and the software used is Python, the Google Cloud Speech-to-Text API, audio-related libraries, and wave.

[1871] For example, when a worker in a factory reports, "Let's proceed to the next process," the voice is recorded by smart glasses and converted in real time into text data that says, "Let's proceed to the next process." This data is stored in the cloud and can be used later by managers to check the progress of work.

[1872] An example prompt might look like this:

[1873] "Develop a system that records work instructions and reports in the factory as voice logs and converts them into text data in real time. Use smart glasses and use the Google Cloud Speech-to-Text API to convert the voice data into text."

[1874] The above is an embodiment of the present invention.

[1875] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1876] Step 1:

[1877] The user wears the smart glasses and conducts evaluation interviews and gives instructions and reports on work in the factory. The microphone in the smart glasses collects voice data in real time. The input is the user's voice, and the output is voice data.

[1878] Step 2:

[1879] The terminal (smart glasses) temporarily stores collected voice data. The stored voice data is used for subsequent processing. The input is voice data, and the output is the temporarily stored voice data.

[1880] Step 3:

[1881] The device sends the temporarily saved voice data to the cloud server. The cloud server receives the voice data and starts the voice recognition process. The input is the temporarily saved voice data, and the output is the voice data sent to the cloud server.

[1882] Step 4:

[1883] The cloud server uses the Google Cloud Speech-to-Text API to convert voice data into text data. The input is voice data and the output is text data. Specifically, the voice data is sent to the API and the returned text data is received.

[1884] Step 5:

[1885] The cloud server inputs text data into the generative AI and generates feedback. The input is text data, and the output is feedback from the generative AI. Specifically, the text data is input into the AI ​​model and the generated feedback is received.

[1886] Step 6:

[1887] The cloud server provides the generated feedback to managers, who then receive the feedback and use it to improve the quality of the evaluation interviews. The input is the feedback from the generative AI, and the output is the feedback provided to managers.

[1888] Step 7:

[1889] The cloud server stores text data of work instructions and reports within the factory in the cloud. The stored data can be searched and analyzed later. The input is text data, and the output is data stored in the cloud.

[1890] Step 8:

[1891] Managers and other users search and analyze data stored in the cloud. This allows for efficient progress management and quality control of work. The input is data stored in the cloud, and the output is the search and analysis results. Specifically, the system searches for the necessary information from a database on the cloud and analyzes the data using analysis tools.

[1892] Example 2

[1893] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1894] In traditional appraisal interview systems, recording interview content and generating feedback is often done manually, which takes time and effort. Furthermore, issues include inconsistent quality of feedback and not clearly indicating areas for improvement in appraisal interviews. This makes it difficult to improve the quality of appraisal interviews and can hinder smooth communication between managers and employees.

[1895] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for recording the contents of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for generating a prompt sentence to input the converted contents of the evaluation interview into the generative AI, means for receiving feedback from the generative AI, and means for returning the received feedback to the user. This makes it possible to automatically analyze the contents of the evaluation interview and quickly provide high-quality feedback.

[1896] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[1897] "Texting" is the process of converting audio logs into written data.

[1898] "Generative AI" is an artificial intelligence model that analyzes input data and generates feedback or other outputs.

[1899] A "prompt sentence" is an input sentence that instructs a generative AI to perform a specific analysis or generation.

[1900] "Feedback" refers to evaluation information such as good and bad points that is generated by generative AI based on the content of the evaluation interview.

[1901] "Users" are system users such as managers and members who receive feedback from evaluation interviews.

[1902] MODE FOR CARRYING OUT THE INVENTION

[1903] The present invention relates to a system for efficiently analyzing the content of an evaluation interview and providing high-quality feedback. Specific embodiments of this system will be described below.

[1904] Hardware and software used

[1905] Hardware: Server (e.g., virtual server for cloud services)

[1906] Software: Generative AI models (e.g., natural language processing models), natural language processing libraries (e.g., spaCy)

[1907] System configuration

[1908] The system consists of the following major components:

[1909] 1. Recording and transcribing audio logs

[1910] The user records the contents of the evaluation interview as an audio log, for example, by using a recording device to collect audio data during the evaluation interview.

[1911] The server receives the recorded voice log and converts it into text data using voice recognition technology. Specifically, it uses a voice recognition API to convert the voice log into text.

[1912] 2. Prompt generation

[1913] The server generates a prompt to be input into the generative AI model based on the textual content of the evaluation interview. For example, it creates a prompt such as, "Please analyze the content of the following evaluation interview and provide feedback on the good and bad points."

[1914] 3. Input to the generative AI model and analysis

[1915] The server sends the generated prompt and text data to the generative AI model, which then analyzes the text data using natural language processing technology and generates feedback based on the content of the evaluation interview.

[1916] 4. Receiving and Providing Feedback

[1917] The server receives the feedback generated by the generative AI model and returns it to the user, for example, by formatting the feedback in HTML and displaying it in the user's browser.

[1918] Specific examples

[1919] For example, a user saves the contents of an evaluation interview in a file called "evaluationinterview.txt" and drags and drops that file into the system's upload form. The server receives the uploaded "evaluationinterview.txt" and reads the text data into memory. The server then generates a prompt message saying, "Please analyze the contents of the evaluation interview below and provide feedback on the good and bad points," and sends it to the generative AI model.

[1920] The generative AI model analyzes the received prompt and text data and generates feedback such as, "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems." The server receives this feedback and sends it back to the user.

[1921] Prompt Sentence Examples

[1922] Please analyze the following evaluation interview and provide feedback on what was good and what was bad. The evaluation interview content is as follows: "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems."

[1923] In this way, the system can efficiently analyze the content of the evaluation interview and provide high-quality feedback quickly.

[1924] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1925] Step 1:

[1926] The user records the contents of the evaluation interview as an audio log. For example, during the evaluation interview, audio data is collected using a recording device. The input is the audio data, and the output is the recorded audio log.

[1927] Step 2:

[1928] The server receives audio logs uploaded by users. The input is the audio logs, and the output is the audio data stored on the server.

[1929] Step 3:

[1930] The server uses speech recognition technology to convert the voice log into text data. Specifically, it uses a speech recognition API to analyze the voice log and generate text data. The input is the voice log, and the output is text data.

[1931] Step 4:

[1932] The server generates a prompt to be input to the generative AI model based on the textual content of the evaluation interview. For example, it creates a prompt such as, "Please analyze the content of the following evaluation interview and provide feedback on the good and bad points." The input is text data, and the output is a prompt.

[1933] Step 5:

[1934] The server sends the generated prompt sentence and text data to the generative AI model. Specifically, it sends the data using an API request. The input is the prompt sentence and text data, and the output is a request to the generative AI model.

[1935] Step 6:

[1936] The generative AI model analyzes the received prompt and text data. It uses natural language processing technology to understand the content of the text data and extracts the good and bad points of the evaluation interview. The input is the prompt and text data, and the output is feedback.

[1937] Step 7:

[1938] A generative AI model generates feedback based on the analysis results. For example, it generates feedback such as, "In this project, the whole team worked together to achieve the goal, but a lack of communication caused some problems." The input is the analysis results, and the output is the feedback.

[1939] Step 8:

[1940] The server receives the feedback generated from the generative AI model. The feedback is sent back to the server through an API response. The input is the feedback, and the output is the feedback stored on the server.

[1941] Step 9:

[1942] The server sends the received feedback back to the user, for example, formatting the feedback in HTML and displaying it in the user's browser. The input is the feedback, and the output is the feedback displayed in the user's browser.

[1943] (Application example 2)

[1944] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1945] In conventional evaluation interview systems, recording interview content and generating feedback is done manually, which takes time and effort, and there are problems with inconsistent evaluation quality. Another issue is that the feedback is vague and lacks specificity, making it difficult for the members being evaluated to clearly understand areas for improvement. Furthermore, creating evaluation reports is time-consuming, placing a heavy burden on managers.

[1946] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1947] In this invention, the server includes a means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, a means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, and a means for automatically generating an evaluation report based on the feedback from the generative AI. This makes it possible to efficiently record the contents of evaluation interviews and automatically generate high-quality feedback. Furthermore, the automatic generation of evaluation reports reduces the burden on managers and improves the quality of evaluations.

[1948] A "management position" is a position within an organization that supervises and evaluates the work of members.

[1949] A "member" is an employee who is responsible for a specific task within an organization.

[1950] An "evaluation interview" is a conversation in which a manager evaluates a member's work performance and provides feedback.

[1951] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[1952] "Transcription" refers to the process of converting audio logs into written data.

[1953] "Generative AI" refers to artificial intelligence that automatically generates feedback and reports based on input data.

[1954] "Feedback" refers to evaluation information such as good and bad points provided by generative AI based on the content of the evaluation interview.

[1955] An "evaluation report" is a summary document of evaluations that is automatically generated based on feedback from generative AI.

[1956] The system for implementing this invention efficiently records the content of evaluation interviews conducted by managers with their team members, generates high-quality feedback, and automatically generates evaluation reports. A specific embodiment of this system will be described below.

[1957] System configuration

[1958] 1. Speech Recognition Module

[1959] The server uses a speech recognition module to record the contents of the evaluation interview as an audio log. This module has the function of converting voice data into text data. Specifically, it uses a speech recognition library and Google's speech recognition API.

[1960] 2. Natural Language Processing Module

[1961] The server uses a natural language processing (NLP) module to analyze the textual content of the evaluation interview. This module has the function of analyzing the text data and evaluating the quality of the evaluation. Specifically, it uses the pipeline function of the transformers library and utilizes the Japanese BERT model.

[1962] 3. Generative AI Module

[1963] The server uses generative AI to generate feedback based on the data analyzed by the natural language processing module. This generative AI automatically generates feedback on the positive and negative points based on the input data.

[1964] 4. Report Generation Module

[1965] The server automatically generates an evaluation report based on feedback from the generative AI, which is provided to managers and used to encourage them to improve the quality of evaluation interviews.

[1966] System Operation

[1967] 1. Recording and transcribing audio logs

[1968] When a manager conducts an evaluation interview with a member, the contents are recorded as an audio log, which is then converted into text data by a speech recognition module.

[1969] 2. Text Data Analysis

[1970] The transcript of the interview is then analyzed by a natural language processing module, which evaluates the quality of the evaluation and extracts key points.

[1971] 3. Generate feedback

[1972] Based on the analysis results, the generative AI module generates feedback, including good and bad points.

[1973] 4. Automatic generation of evaluation reports

[1974] Based on the generated feedback, the report generation module automatically generates an evaluation report, which is provided to management to encourage improvement of the quality of evaluation interviews.

[1975] Specific examples

[1976] For example, by inputting the following prompt sentence into a generative AI model, specific feedback can be obtained.

[1977] Contents of the evaluation interview:

[1978] Worker A works quickly and produces high quality work.

[1979] However, there appears to be a lack of communication.

[1980] In the future, it will be necessary to strengthen cooperation with the team.

[1981] Generate feedback.

[1982] By inputting this prompt into a generative AI model, specific feedback can be obtained and an evaluation report can be automatically generated, allowing managers to conduct evaluation interviews efficiently and provide high-quality feedback.

[1983] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1984] Step 1:

[1985] The server records the contents of the evaluation interviews between managers and employees as audio logs. The audio logs are collected in real time during the evaluation interviews and saved as audio files. The input is the audio data of the evaluation interviews, and the output is an audio file.

[1986] Step 2:

[1987] The server uses a speech recognition module to convert the recorded voice log into text data. Specifically, it uses a speech recognition library to call Google's speech recognition API and converts the voice data into text data. The input is an audio file and the output is text data.

[1988] Step 3:

[1989] The server analyzes the textual content of the evaluation interview using a natural language processing (NLP) module. It uses the pipeline function of the transformers library to apply the Japanese BERT model to analyze the text data. The input is the text data, and the output is the analysis results.

[1990] Step 4:

[1991] The server uses a generative AI module to generate feedback based on the analysis results. The generative AI automatically generates feedback on good and bad points based on the input analysis results. The input is the analysis results, and the output is feedback data.

[1992] Step 5:

[1993] The server automatically generates an evaluation report based on the generated feedback. A report generation module is used to format the feedback data into an evaluation report. The input is the feedback data, and the output is the evaluation report.

[1994] Step 6:

[1995] The server provides the generated evaluation report to the manager, who then checks the evaluation report and uses it to improve the quality of the evaluation interview. The input is the evaluation report, and the output is provided to the manager.

[1996] Example 3

[1997] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1998] Conventional evaluation interview systems lacked a means to effectively provide feedback on the content of evaluation interviews conducted by managers, making it difficult to obtain specific advice or areas for improvement to improve the quality of evaluation interviews.In addition, there was a lack of a means to provide feedback to managers quickly and reliably, which sometimes delayed improvements to evaluation interviews.

[1999] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[2000] In this invention, the server includes means for recording the content of an evaluation interview conducted by a manager with a member as an audio log and converting the audio log into text, means for inputting the text content of the evaluation interview into a generative AI and receiving feedback from the generative AI, means for providing the generated feedback to the manager via a notification system, means for providing an interface for inputting the content of the evaluation interview, means for transmitting the content of the evaluation interview to the server using a secure communication protocol, means for generating feedback using a generative AI model, means for transmitting the generated feedback to the notification system, and means for notifying the feedback to the manager via the notification system. This enables the content of the evaluation interview to be effectively fed back, and enables managers to quickly and reliably receive specific advice and areas for improvement to improve the quality of the evaluation interview.

[2001] A "management position" is someone in a position within an organization who evaluates and guides members.

[2002] "Members" are employees and staff within an organization who are guided and evaluated by management.

[2003] An "evaluation interview" is an interview in which a manager evaluates a member's work performance and behavior and provides feedback.

[2004] An "audio log" is data in which the contents of an evaluation interview are recorded in audio format.

[2005] "Texting" is the process of converting audio logs into written data.

[2006] "Generative AI" is an artificial intelligence model that generates feedback and advice based on input data.

[2007] "Feedback" refers to specific advice and areas for improvement provided by the generative AI based on the content of the evaluation interview.

[2008] A "notification system" is a system for providing generated feedback to management.

[2009] The "interface" is a user interface for inputting the contents of the evaluation interview.

[2010] A "secure communication protocol" is a communication protocol for securely transmitting data.

[2011] "Server" is a computer system that uses generative AI models to generate feedback and provide it to managers via a notification system.

[2012] The present invention is a system for providing feedback to improve the quality of evaluation interviews conducted by managers. A specific embodiment of this system will be described below.

[2013] First, the user (manager) enters the details of the evaluation interview into a dedicated application or web form. For example, they write details about their subordinate's performance and areas for improvement. An example of a specific prompt sentence might be, "Subordinate A is good at managing the progress of projects, but feels that he has issues with communication with his team members. As an area for future improvement, I have suggested holding regular meetings."

[2014] Next, the device (user's PC or smartphone) sends the entered contents of the evaluation interview to the server. At this time, the data is sent using a secure communication protocol (e.g., HTTPS).

[2015] The server inputs the content of the received evaluation interview into a generative AI model (e.g., OpenAI's GPT-4), which generates feedback based on the input content. This feedback includes specific advice and improvements to improve the quality of the evaluation interview.

[2016] The generated feedback is sent from the server to a notification system, which delivers the feedback to the user, for example, by email or using a notification function of a dedicated application.

[2017] Finally, the terminal displays the feedback received from the notification system to the user, who receives the notification and checks for specific advice on how to improve the quality of the evaluation interview.

[2018] This system enables managers to effectively provide feedback on the content of the evaluation interviews, and quickly and reliably obtain specific advice and points for improvement to improve the quality of the evaluation interviews. The flow of the identification process in the third embodiment will be described with reference to FIG. 15.

[2019] Step 1:

[2020] The user inputs the details of the evaluation interview.

[2021] The user (manager) enters the details of the evaluation interview using a dedicated application or web form. For example, the user might enter, "My subordinate, Mr. A, is good at managing the progress of projects, but I feel he has issues with communication with his team members. I have suggested that he hold regular meetings as an area for improvement in the future." The input data is saved in text format.

[2022] Step 2:

[2023] The terminal sends the input information to the server.

[2024] The device (user's PC or smartphone) sends the entered content of the evaluation interview to the server by pressing the "Send" button. At this time, the data is sent using a secure communication protocol (e.g., HTTPS). The input data is sent to the server in text format.

[2025] Step 3:

[2026] The server generates feedback using a generative AI model.

[2027] The server inputs the content of the received evaluation interview into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model analyzes the input text data and generates feedback including specific advice and improvements to improve the quality of the evaluation interview. The output data is the feedback in text format.

[2028] Step 4:

[2029] The server sends the generated feedback to a notification system.

[2030] The server sends the generated feedback to a notification system, which delivers the feedback to the user by, for example, email or a notification function of a dedicated application. The output data is the feedback in text format.

[2031] Step 5:

[2032] The terminal notifies the user of the feedback through a notification system.

[2033] The terminal (user's smartphone or PC) displays the feedback received from the notification system to the user. The user receives the notification and checks specific advice to improve the quality of the evaluation interview. The output data is the feedback in text format.

[2034] (Application example 3)

[2035] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2036] Conventional evaluation interview systems have the problem that it is difficult to objectively evaluate the quality of the evaluation interviews conducted by managers and identify areas for improvement. Furthermore, there is a lack of means to evaluate the performance of machines used in factory work and notify managers of this feedback, making it difficult to improve work efficiency and provide appropriate guidance. To solve these issues, a system is needed that improves the quality of evaluation interviews and properly evaluates the performance of machines in factories and provides feedback to managers.

[2037] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[2038] In this invention, the server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for notifying the managers of the feedback from the generative AI, and means for evaluating the performance of machines operating in the factory and notifying the managers of the feedback. This makes it possible to objectively grasp the quality of the evaluation interviews and identify areas for improvement, as well as to appropriately evaluate the performance of machines in the factory and provide feedback to the managers.

[2039] A "management position" is a position within an organization that is responsible for supervising and evaluating the work of its members.

[2040] "Members" are employees who work within an organization under the direction of a manager.

[2041] An "evaluation interview" is an interview conducted by a manager to evaluate the work of a member and provide feedback.

[2042] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[2043] "Texting" is the process of converting audio logs into written data.

[2044] "Generative AI" is an artificial intelligence system that generates feedback and evaluations based on input data.

[2045] "Feedback" refers to the evaluation and information on areas for improvement provided by generative AI based on the content of the evaluation interview and the machine's performance.

[2046] "Notification" refers to the act of informing management of feedback from generative AI.

[2047] "Machines performing work in factories" are automated devices that perform work such as production and assembly in factories.

[2048] "Performance" refers to the efficiency and quality of operation of machines that perform work within a factory.

[2049] A system for carrying out this invention has the following configuration: The server includes means for recording the contents of evaluation interviews conducted by managers with members as audio logs and converting the audio logs into text, means for inputting the converted contents of the evaluation interviews into a generative AI and receiving feedback from the generative AI, means for notifying managers of the feedback from the generative AI, and means for evaluating the performance of machines that perform work in a factory and notifying managers of the feedback.

[2050] Hardware and software used

[2051] Hardware: Factory robots, servers, managerial devices (PCs and smartphones)

[2052] Software: Python, generative AI models (e.g., OpenAI GPT-3), mail servers (e.g., SMTP)

[2053] Data processing and calculation

[2054] 1. Recording and Transcription of Audio Logs:

[2055] The server records the contents of the evaluation interview as an audio log.

[2056] Use speech recognition software (e.g., Google Speech-to-Text API) to convert audio logs into text.

[2057] 2. Getting input and feedback for generative AI models:

[2058] The server inputs the textual content of the evaluation interview into a generative AI model.

[2059] A generative AI model generates feedback based on the input text.

[2060] Example: Prompt: "Please rate the performance of robot ID: 123"

[2061] 3. Feedback Notification:

[2062] The server notifies management of feedback from the generative AI model.

[2063] For notifications, a mail server is used to send feedback to the manager's email address.

[2064] Specific examples

[2065] For example, when evaluating the performance of robot ID: 123 working in a factory, the server performs the following process.

[2066] 1. Collect robot work data and send the prompt "Please evaluate the performance of robot ID: 123" to the generative AI model.

[2067] 2. The generative AI model evaluates the robot's work efficiency and the quality of its movements, and generates feedback such as, "Robot ID: 123's work efficiency is 90%. The area for improvement is optimizing its movement speed."

[2068] 3. The server sends this feedback to the manager's email address, and the manager can use the feedback to identify areas for improvement in the robot's behavior and work efficiency.

[2069] In this way, it becomes possible to objectively grasp the quality of the evaluation interview and identify areas for improvement, as well as to properly evaluate the performance of machines in the factory and provide feedback to management.

[2070] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[2071] Step 1:

[2072] The server records the contents of evaluation interviews conducted by managers with members as audio logs.

[2073] Input: Audio data from the evaluation interview

[2074] Output: Audio log

[2075] Specific operation: When an evaluation interview is conducted, the server uses the audio recording function to record the contents of the interview as audio data.

[2076] Step 2:

[2077] The server converts the recorded audio logs into text.

[2078] Input: Audio log

[2079] Output: Text data

[2080] Specific operation: The server uses voice recognition software (e.g., Google Speech-to-Text API) to convert the voice log into text data.

[2081] Step 3:

[2082] The server inputs the textual content of the evaluation interview into a generative AI model and receives feedback.

[2083] Input: Text data

[2084] Output: Feedback from the generative AI model

[2085] Specific operation: The server sends the text data to a generative AI model (e.g., OpenAI GPT-3) and inputs the prompt "Please generate feedback based on the content of the evaluation interview." The generative AI model generates feedback and sends it back to the server.

[2086] Step 4:

[2087] The server notifies management of feedback from the generative AI model.

[2088] Input: Feedback from a generative AI model

[2089] Output: Notification to management (e.g. email)

[2090] Specific operation: The server uses a mail server (e.g., SMTP) to send feedback from the generative AI model to the manager's email address.

[2091] Step 5:

[2092] The server evaluates the performance of machines operating within the factory and communicates the feedback to management.

[2093] Input: Machine operation data

[2094] Output: Feedback from the generative AI model, notification to management

[2095] How it works: The server collects work data from machines in the factory and sends a prompt to the generative AI model: "Please rate the performance of robot ID: 123." The generative AI model generates feedback and sends it back to the server. The server then sends this feedback to the manager's email address.

[2096] In this way, it becomes possible to objectively grasp the quality of the evaluation interview and identify areas for improvement, as well as to properly evaluate the performance of machines in the factory and provide feedback to management.

[2097] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2098] "Example 1"

[2099] One embodiment of the present invention provides a system including an emotion engine that recognizes a user's emotions during an evaluation interview. The emotion engine recognizes emotions from the user's voice, facial expressions, gestures, etc., and inputs the emotion data into a generative AI. Specifically, when the user expresses emotions such as joy, anger, surprise, or sadness, the emotion engine recognizes them and provides the information to the generative AI.

[2100] "Example 2"

[2101] The generative AI generates feedback based on the content of the evaluation interview and the user's emotional data. Specifically, if the user expresses joy, it determines from the content of the evaluation interview what that joy is about and generates the result as feedback. Also, if the user expresses anger, it determines from the content of the evaluation interview what that anger is about and generates the result as feedback.

[2102] "Example 3"

[2103] Feedback from the generative AI is provided to managers, who can view it on their own devices. This feedback includes not only good and bad points based on the content of the evaluation interview, but also information based on the user's emotional data. This allows managers to gain a deeper understanding of the quality of the evaluation interview and identify areas for improvement.

[2104] The processing flow of each embodiment will be described below.

[2105] "Example 1"

[2106] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[2107] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[2108] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[2109] "Example 2"

[2110] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[2111] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[2112] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[2113] Step 4: Feedback from the generative AI is provided to management, who reviews the feedback.

[2114] "Example 3"

[2115] Step 1: The evaluation interview begins, and the user's voice, facial expressions, gestures, etc. are input into the emotion engine.

[2116] Step 2: The emotion engine recognizes the user's emotions from the input information and inputs that emotion data into the generative AI.

[2117] Step 3: The generative AI generates feedback based on the content of the evaluation interview and emotional data.

[2118] Step 4: Feedback from the generative AI is provided to management, who reviews the feedback.

[2119] Step 5: Managers take action based on the feedback to improve the quality of the evaluation interview.

[2120] Example 1

[2121] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2122] Conventional evaluation interview systems have difficulty not only recording the interview content but also recognizing the user's emotions during the interview in real time and providing feedback based on that. Furthermore, in order to improve the quality of feedback using generative AI, it is necessary to appropriately utilize the user's emotional data, but there was no established method for doing so.

[2123] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for recording the content of an evaluation interview conducted by a manager with a member as an audio log, a means for converting the audio log into text, a means for inputting the text content of the evaluation interview into a generative AI and receiving feedback from the generative AI, a means for recognizing the user's emotions, a means for inputting the recognized emotion data into the generative AI, and a means for providing the user with feedback from the generative AI. This enables high-quality feedback that utilizes not only the content of the evaluation interview but also the user's emotion data.

[2124] A "management position" is someone in a position within an organization who evaluates and guides members.

[2125] A "member" is someone in an organization who is in a position to receive guidance and evaluation from management.

[2126] An "evaluation interview" is a dialogue between a manager and a member of staff to evaluate their work and provide feedback.

[2127] An "audio log" is a recording of the contents of an evaluation interview as audio data.

[2128] "Text conversion" is the process of converting audio data into text data.

[2129] "Generative AI" is artificial intelligence that generates feedback and advice based on input data.

[2130] "Feedback" refers to evaluations and advice provided by generative AI based on the content of the evaluation interview and the user's emotional data.

[2131] "User" refers to the manager or member participating in the evaluation interview.

[2132] "Emotion data" is information about emotions recognized from the user's voice, facial expressions, gestures, etc.

[2133] "Emotion recognition" is the process of identifying emotions from a user's voice, facial expressions, gestures, etc.

[2134] A "server" is a computer system that processes voice and emotional data and generates feedback using generative AI.

[2135] A "terminal" is a device that records audio and video of the evaluation interview and transmits th...

Claims

1. A means for converting the contents of the evaluation interview conducted by the evaluator with the evaluatee into text; A means for acquiring emotion data of the person to be evaluated; A means for inputting the contents of the evaluation interview converted into text and the acquired emotion data into a generative AI; A means for providing feedback of the evaluation interview from the generative AI to the evaluator; A system including:

2. The generation system AI generates the feedback including information on good and bad points of the evaluation interview based on a prompt sentence corresponding to the content of the evaluation interview and the emotion data. The system of claim 1 .

3. A means for notifying a terminal of the evaluator of the feedback from the generative AI; and a means for the evaluator to confirm the feedback on the terminal. The system of claim 1 .

Images