System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a plurality of types of data about a job applicant; means for converting voice data of the plurality of types of data into text data; means for integrating the plurality of types of data and the text data; means for automatically generating a letter of recommendation based on the integrated data; means for providing an interface for displaying and modifying the automatically generated letter of recommendation; and means for storing and transmitting the modified letter of recommendation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When recruitment agencies introduce job seekers to companies, the problem is that writing letters of recommendation takes a lot of time and effort. Even experienced career consultants need 30 minutes to an hour to write a letter of recommendation for each candidate. As a result, as the number of job seekers increases, it becomes difficult to allocate much time to writing letters of recommendation, resulting in less time for interviews with job seekers. Furthermore, due to concerns about the handling of personal information and confidentiality, it is difficult to use public AI services. Therefore, there is a need to streamline the process of writing letters of recommendation and ensure that personal information is handled safely. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for receiving multiple forms of data related to a job seeker, a means for using speech recognition technology to convert the received voice data into text data, and a means for integrating the text data with a resume or CV. The system further includes a means for using a generative AI model to automatically generate a recommendation letter based on the integrated data, and a means for providing an interface for displaying and modifying the automatically generated recommendation letter. The system also includes a means for saving and transmitting the final modified recommendation letter. This significantly reduces the time required to create a recommendation letter and ensures the safe handling of personal information.

[0006] "Job seeker data" includes information about job seekers used in recruitment activities, such as resumes, CVs, and interview audio data.

[0007] "Receiving means" refers to the function or device for inputting job seeker data into a server or system, including uploading and collecting data.

[0008] "Speech recognition technology" is a technology for analyzing voice data and converting it into corresponding text data, specifically using AI models and voice recognition software.

[0009] "Text data" refers to data converted using speech recognition technology, in which spoken content is expressed in text form.

[0010] "Integration means" refers to a function or device for combining multiple forms of job applicant data into a single data set.

[0011] A "generative AI model" refers to an artificial intelligence that is trained to automatically generate documents and reports based on input data.

[0012] A "letter of recommendation" is a document that introduces a job seeker's skills and experience to a company, and is prepared by a recruitment agency for the purpose of recommending a candidate.

[0013] An "interface" is something that provides a means for a user to interact with a system and view and modify information, and typically refers to a graphical user interface (GUI).

[0014] "Storage means" refers to a function or device that safely stores generated or modified recommendations in a database or file system.

[0015] "Transmission means" means a function or device for transmitting stored testimonials to companies, including email, file transfer protocol (FTP), etc. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] MODE FOR CARRYING OUT THE INVENTION

[0038] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[0039] System configuration

[0040] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[0041] Program Overview

[0042] The system receives a job seeker's resume, CV, and audio recording of the interview, and automatically generates a recommendation letter based on these. The generated recommendation letter is then finalized after user confirmation and correction, and sent to the company.

[0043] Operation of each method

[0044] Receiving data

[0045] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio (WAV or MP3 format) to the system. The server receives these data and saves the files in the appropriate format. This process ensures that the necessary information is captured in the system.

[0046] Text data conversion using voice recognition technology

[0047] The server uses voice recognition technology to convert the uploaded interview audio data into text data. Specifically, it uses an AI voice recognition model to convert the audio data into text format, which will be used in further processes.

[0048] Data Integration

[0049] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[0050] Automatic generation of letters of recommendation

[0051] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[0052] Reviewing and correcting letters of recommendation

[0053] The user can check the generated draft recommendation letter and, if necessary, can edit the content to create the final recommendation letter. After the edits are reflected, the server saves the final recommendation letter.

[0054] Save and send letters of recommendation

[0055] The server stores the final recommendation in the appropriate format (e.g. PDF) and then sends it to the company if required, ensuring data security and confidentiality.

[0056] Specific examples

[0057] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives this data and transcribes the audio data. Next, it integrates each data and generates a draft recommendation letter using a generative AI model. The user then reviews the draft and makes corrections. The final revised recommendation letter is saved and ready to be sent to the company.

[0058] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[0062] Step 2:

[0063] The server stores the uploaded resumes, CVs, and interview audio data. Specifically, the server places each data in a designated folder so that it can be accessed in subsequent processes.

[0064] Step 3:

[0065] The server converts the interview audio data into text data using speech recognition technology. This process uses an AI speech recognition model to convert the audio file into text format and save the converted text data.

[0066] Step 4:

[0067] The server then combines the received and converted data, specifically the resumes, CVs, and text data, into a single dataset that is ready to be fed into a generative AI model.

[0068] Step 5:

[0069] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[0070] Step 6:

[0071] The server displays the generated draft recommendation letter through an interface provided to the user, who can use this interface to review the recommendation letter and make corrections as necessary.

[0072] Step 7:

[0073] After the user modifies the draft, the server saves the final version of the recommendation, which records the changes as the final modified recommendation.

[0074] Step 8:

[0075] The server converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] Creating recommendation letters, which are required when introducing job seekers to companies, is not only time-consuming and laborious, but also has the potential for inaccurate information integration and analysis. For this reason, a system for creating recommendation letters efficiently and accurately is needed. Furthermore, an environment is needed that centralizes the processes of transcribing audio data, integrating data in multiple formats, and automatically generating recommendation letters based on that data, while allowing users to easily edit and review them.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for receiving multiple types of data about the job seeker, means for using voice recognition software to convert voice data into text data, means for integrating the multiple types of data and the text data into a dataset, means for using a generative AI model to automatically generate a recommendation letter based on the integrated dataset, means for providing an interface for displaying and modifying the automatically generated recommendation letter, and means for saving, formatting, and transmitting the modified recommendation letter. This streamlines the recommendation letter creation process and enables the generation of more accurate recommendation letters. Furthermore, users can easily review the generated recommendation letter and modify it as necessary.

[0081] The "means for receiving data" refers to a means for uploading and storing multiple forms of data related to job seekers, such as resumes, CVs, and audio data, into the system.

[0082] "Means for converting voice data into text data" means means for converting uploaded voice data into text format using voice recognition software.

[0083] The "means of integrating data" refers to a means of combining resumes, CVs, and text data converted from audio data into a single data set.

[0084] A "generative AI model" is an artificial intelligence model that automatically generates recommendation letters based on an integrated dataset.

[0085] The "means for providing an interface" refers to a means for providing a screen for a user to display the generated recommendation letter and to modify it as necessary.

[0086] "Means for saving and sending the recommendation letter" refers to the means for saving the final recommendation letter in an appropriate format (e.g., PDF) and sending it to the company.

[0087] A "prompt" is a specific statement that instructs the generative AI model and provides the information necessary to generate a recommendation letter.

[0088] MODE FOR CARRYING OUT THE INVENTION

[0089] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[0090] System configuration

[0091] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[0092] Operation of each method

[0093] Receiving data

[0094] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio data (WAV or MP3 format) to the system. The server receives this data and saves each file in the appropriate folder. This process ensures that the necessary information is captured in the system.

[0095] Text data conversion using voice recognition technology

[0096] The server converts the stored interview voice data into text data using speech recognition software (e.g., Google Speech-to-Text API). The converted text data is temporarily stored and used for subsequent processing.

[0097] Data Integration

[0098] The server combines the resumes, CVs, and converted text data, and stores this combined dataset in JSON format, which is then fed into a generative AI model.

[0099] Automatic generation of letters of recommendation

[0100] The generative AI model automatically generates recommendation letters based on the integrated dataset. The recommendation letters generated using specific prompts reflect the job seeker's skills and experience and include suggestions for the company. An example of a prompt is, "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience."

[0101] Reviewing and correcting letters of recommendation

[0102] The user can review the generated recommendation letter through a web interface, which provides editing functionality to revise the content of the recommendation letter as needed and finalize it.

[0103] Save and send letters of recommendation

[0104] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. A function is implemented to automatically send the saved recommendation letter to the specified company's email address.

[0105] Specific examples

[0106] For example, if a user uploads applicant A's resume (PDF format), CV (Word format), and interview audio data (MP3 format) to the system, the server receives these data and converts the audio data into text using an AI speech recognition model. The server then integrates the resume, CV, and text data and generates a draft letter of recommendation using a generative AI model. The user reviews the draft and makes any necessary revisions. The revised letter of recommendation is saved on the server and can be sent to the company in PDF format.

[0107] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[0108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0109] Step 1:

[0110] The user uploads the job seeker's resume (PDF or Word format), resume (PDF or Word format), and interview audio data (WAV or MP3 format) from their terminal to the system. These three types of files are taken in as input. The server receives the data and saves the files in the appropriate folder. The output of this step is various data files saved in the system. Specifically, the server receives the upload request, checks the files, and saves them in the appropriate directory.

[0111] Step 2:

[0112] The server retrieves the saved interview audio data and converts it into text data using speech recognition software (e.g., Google Speech-to-Text API). The input is the interview audio data file. The server sends this audio file to the speech recognition API and obtains text data. The output of this step is the text data converted from the audio data. Specifically, the server passes the audio data to the API and saves the returned text data.

[0113] Step 3:

[0114] The server integrates the resumes, CVs, and converted text data. The inputs are the various data files and text data saved in the previous step. The server integrates them into a single dataset and saves it in JSON format. The output of this step is the integrated dataset. Specifically, it converts the contents of PDF and Word files into text format and combines them into a single JSON file.

[0115] Step 4:

[0116] The generative AI model automatically generates a recommendation letter based on the integrated dataset. The integrated dataset and a prompt are used as input. An example of a specific prompt is input to the model: "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience." The output is a draft of the generated recommendation letter. Specific operations include the process by which the AI ​​model generates a recommendation letter based on the set prompt.

[0117] Step 5:

[0118] The user reviews the generated recommendation letter through a web interface. The generated draft recommendation letter is used as input. The interface provides an editing function to modify the recommendation letter content as needed. The output is the final recommendation letter that has been reviewed and modified by the user. Specific operations include the process in which the user reviews the recommendation letter content and makes modifications on the interface.

[0119] Step 6:

[0120] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. The final recommendation letter corrected by the user is used as input. A function is then implemented to automatically send the saved recommendation letter to the specified company's email address. The output is the recommendation letter sent to the company. Specific operations include converting the corrected recommendation letter to a PDF file, saving it on the server, and sending it to the company by email.

[0121] (Application example 1)

[0122] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0123] The present invention relates to a system that efficiently records the work content and progress of workers in a factory, allowing managers to quickly review and correct work reports. Conventional methods require workers to individually write out work reports, which is time-consuming and labor-intensive, and the recorded content is often inaccurate or inconsistent. Furthermore, managers must spend a lot of time reviewing and correcting the reports. The present invention aims to solve these problems and streamline the process of generating work reports.

[0124] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0125] In this invention, the server includes means for receiving data in multiple formats related to workers, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for automatically generating a work report, means for providing an interface for displaying and modifying the automatically generated work report, and means for saving and transmitting the modified work report, thereby reducing the burden on workers and enabling efficient and consistent creation of work reports.

[0126] "Workers" refers to personnel who perform practical work in a work environment such as a factory or work site.

[0127] "Voice data" refers to acoustic signals that record the statements, instructions, and reports of workers.

[0128] "Character data" refers to information in text format obtained by converting voice data.

[0129] "Integrated data" refers to an information set that brings together data in multiple formats and converted character data.

[0130] A "work report" refers to a document that records the work content, progress, problems, etc. of a worker.

[0131] "Interface" refers to the operation screens and functions that allow users to access and operate the system.

[0132] "Server" refers to a computer system that receives, processes, stores, and transmits data.

[0133] "Voice recognition technology" refers to the technology that analyzes voice data and converts it into text data.

[0134] A "generative AI model" refers to an artificial intelligence model that automatically generates the necessary documents and reports based on given input data.

[0135] "Correction" refers to checking the contents of generated reports and documents and making changes to correct any errors or omissions.

[0136] The present invention provides a system that records the work content and progress of workers in a factory and allows managers to quickly check and correct work reports. The system has the function of collecting voice reports from workers using smart glasses or other devices while they are working, and automatically generating work reports using voice recognition technology and a generative AI model.

[0137] System configuration

[0138] Worker devices: Use devices that allow voice input and video recording, such as smart glasses or headsets.

[0139] Server: Processes data using speech recognition technology and generative AI models, and manages automatically generated work reports.

[0140] Interface terminal: Provides an interface that allows managers to check and modify generated reports using a smartphone or tablet.

[0141] Program Overview and Procedures

[0142] Receiving data

[0143] The server receives the voice data (WAV or MP3 format) and work history data (JSON format) uploaded by the worker and stores this data in an appropriate format.

[0144] Text data conversion using voice recognition technology

[0145] The server converts the uploaded voice data into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text API).

[0146] Data Integration

[0147] The server integrates the work history data with the text data converted from the voice data, and this integrated data is used to generate a work report.

[0148] Automatic generation of work reports

[0149] Use a generative AI model (e.g., OpenAI's GPT-3) to automatically generate a draft working report based on the integrated data.

[0150] Check and correct work reports

[0151] The manager can check the automatically generated draft work report on their device (smartphone or tablet) and make any necessary corrections through the interface.

[0152] Save and send work reports

[0153] The final revised work report is saved on the server and sent to the relevant departments and parties as needed.

[0154] Specific examples

[0155] For example, while working, a worker wearing smart glasses may report by voice, "Starting work on ~. Progress is going well, no problems." This voice data is uploaded to a server and converted into text data using speech recognition technology. It is then integrated with work history data and input into a generative AI model. The model generates a draft work report based on prompt sentences such as the following:

[0156] Prompt Sentence Examples

[0157] Work history: XX Factory, Work content: Machine inspection, Progress: 100%, No special notes.

[0158] Converted character data: Starting work on ~. Good progress, no problems.

[0159] Using this prompt as input, a draft work report is generated, after which the manager can review the generated report on a smartphone or tablet, make any necessary corrections, save it, and send it to the relevant parties.

[0160] In this way, the system of the present invention makes the process of generating work reports more efficient, reduces the burden on workers, and improves the quality of reports.

[0161] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0162] Step 1:

[0163] The server receives the voice data and work history data uploaded by the worker. Specifically, the voice data (WAV or MP3 format) recorded by the worker using smart glasses and the work history data in JSON format are sent to the server. The server saves them in an appropriate directory. The input is the voice data and work history data, and the output is the saved voice file and work history file.

[0164] Step 2:

[0165] The server converts the received and saved voice data into text data using voice recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the voice data into text data. The input is voice data, and the output is the converted text data. This text data is required for subsequent processing.

[0166] Step 3:

[0167] The server integrates the work history data with the text data generated by speech recognition. Specifically, the server reads the work history data in JSON format and combines it with the converted text data. The input is the work history data and text data, and the output is the integrated data (JSON format). This integrated data is used to generate a work report.

[0168] Step 4:

[0169] The server uses a generative AI model to automatically generate a work report based on the integrated data. Specifically, it uses OpenAI's GPT-3 to generate a detailed work report based on the contents of the integrated data. Based on the prompt, the input is the integrated data, and the output is a draft of the work report.

[0170] Step 5:

[0171] The terminal displays the automatically generated work report created by the server and provides an interface for the user to check and modify it. Specifically, the administrator checks the contents of the generated work report via a smartphone or tablet and makes any necessary modifications. The input is a draft work report, and the output is the modified work report.

[0172] Step 6:

[0173] The server saves the revised work report and sends it to relevant parties and departments as needed. Specifically, the server saves the revised work report in PDF format or other format and sends it to the email addresses of relevant departments. The input is the revised work report, and the output is the saved report and transmission log.

[0174] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0175] MODE FOR CARRYING OUT THE INVENTION

[0176] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[0177] System configuration

[0178] The system consists of the following components:

[0179] 1. Means of receiving job seeker data

[0180] 2. Means using voice recognition technology

[0181] 3. Data Integration Methods

[0182] 4. Using Generative AI Models

[0183] 5. Interface

[0184] 6. Preservation means

[0185] 7. Means of transmission

[0186] 8. Emotion Engine

[0187] Program Overview

[0188] The system receives a job seeker's resume, CV, and interview audio data, and automatically generates a recommendation letter based on them. It also incorporates an emotion engine that recognizes the user's emotions and assists in the revision process.

[0189] Operation of each method

[0190] Receiving data

[0191] Users upload job seekers' resumes, CVs, and interview audio data to the system. The server receives these data and saves them in a designated folder, capturing the necessary data.

[0192] Text data conversion using voice recognition technology

[0193] The server converts the uploaded interview audio data into text data. Specifically, it uses an AI speech recognition model to convert the audio file into text format and saves the converted text data.

[0194] Data Integration

[0195] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[0196] Automatic generation of letters of recommendation

[0197] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[0198] Interface and Emotion Engine

[0199] The server displays the generated draft recommendation letter through an interface provided to the user. The user uses this interface to confirm the contents of the recommendation letter. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state and provides suggestions for revisions to help the user revise the recommendation letter.

[0200] Reviewing and correcting letters of recommendation

[0201] As the user edits the draft, the emotion engine provides advice based on the user's emotional state. For example, if the user is feeling unhappy, it can suggest more positive wording. The revised recommendation letter is then saved to the server in its final form.

[0202] Save and send letters of recommendation

[0203] The server then converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[0204] Specific examples

[0205] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data. Next, each data is integrated and a generative AI model is used to generate a draft recommendation letter. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final recommendation letter is then saved and sent to the company.

[0206] The above is a specific embodiment of the present invention. This system improves the efficiency of recommendation letter creation and provides flexible revision support that takes into account the user's emotions.

[0207] The processing flow will be explained below.

[0208] Step 1:

[0209] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[0210] Step 2:

[0211] The server stores the uploaded resumes, CVs, and interview audio data in designated folders, making them accessible for subsequent processing.

[0212] Step 3:

[0213] The server converts the interview audio data into text data using speech recognition technology, converts the audio file into text format using an AI speech recognition model, and saves the converted text data.

[0214] Step 4:

[0215] The server combines the resumes, CVs, and converted text data to create a structured dataset that is ready to be fed into a generative AI model.

[0216] Step 5:

[0217] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[0218] Step 6:

[0219] The server displays the generated draft recommendation letter through an interface provided to the user, who can use the interface to review and modify the recommendation letter content.

[0220] Step 7:

[0221] The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. Specifically, it uses a camera and microphone to capture facial and voice data in real time and performs emotion analysis.

[0222] Step 8:

[0223] The emotion engine provides suggestions for revising the recommendation letter on the interface based on the user's emotional state. For example, if the user is feeling dissatisfied, the engine will suggest more positive expressions to help the user make revisions smoothly.

[0224] Step 9:

[0225] The user then modifies the recommendation based on the suggestions from the emotion engine. Once the modifications are complete, the final recommendation is sent to the server.

[0226] Step 10:

[0227] The server saves the final, modified version of the recommendation, which is then recorded as a finalized recommendation.

[0228] Step 11:

[0229] The server converts the final recommendation letter into the appropriate format (e.g., PDF) and prepares it for transmission to the company. Once the recommendation letter is ready for transmission, it is securely transmitted to the company.

[0230] Example 2

[0231] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0232] When job placement agencies introduce job seekers to companies, they create recommendation letters based on multiple forms of data, including the job seeker's resume, CV, and interview audio. This process is often manual, time-consuming, and prone to typos and inconsistencies. Furthermore, there is a lack of flexible correction support that takes user sentiment into account. Therefore, there is a need for a system that streamlines the recommendation letter creation and correction process and improves its accuracy.

[0233] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0234] In this invention, the server includes means for receiving data in multiple formats related to the job seeker, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for using a generative AI model to automatically generate a letter of recommendation based on the integrated data, means for providing an interface for displaying and modifying the automatically generated letter of recommendation and for recognizing a user's emotions to support the modification process, and means for saving and transmitting the modified letter of recommendation. This enables efficient creation of letters of recommendation and flexible modification support that takes the user's emotions into consideration.

[0235] "Job seekers" refer to people who are seeking a new job.

[0236] "Means for receiving data" refers to a device or method for incorporating various data, such as resumes, CVs, and audio files, input from outside into the system.

[0237] "Audio data" refers to audio files containing what a job seeker said during an interview or other occasion.

[0238] "Means for converting into text data" refers to speech recognition technology or software for converting voice data into text format.

[0239] "Means for integrating data" refers to a device or method that combines data in multiple formats (e.g., resumes, CVs, text data converted from audio data) to create a single integrated data set.

[0240] The use of a "generative AI model" refers to the use of an artificial intelligence model to automatically generate a recommendation based on the integrated data.

[0241] "Interface" refers to the user interface through which a user interacts with a system.

[0242] "Means for recognizing emotions" refers to technology and software for analyzing and determining emotions from a user's facial expressions and voice.

[0243] "Means for storing" refers to a device or method for recording the revised recommendation as a digital file.

[0244] "Transmitting means" refers to a device or method for transferring the completed recommendation letter to another terminal such as a company.

[0245] A "letter of recommendation" is a document that describes a job seeker's background and skills in order to recommend them to a company.

[0246] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[0247] System configuration

[0248] The system consists of the following components:

[0249] 1. Means of receiving job seeker data

[0250] 2. Means using voice recognition technology

[0251] 3. Data Integration Methods

[0252] 4. Using Generative AI Models

[0253] 5. Interface

[0254] 6. Preservation means

[0255] 7. Means of transmission

[0256] 8. Emotion Engine

[0257] System Operation

[0258] Receiving data

[0259] A user uploads a job seeker's resume, CV, and interview audio data to the system. The server securely receives this data using the HTTPS protocol with SSL / TLS and saves it in a specified folder. For example, the data for job seeker A is saved in the / uploaded_data / A / directory.

[0260] Text data conversion using voice recognition technology

[0261] The server converts the uploaded interview audio data into text data. Specifically, it uses AI speech recognition services such as Google Cloud Speech-to-Text to convert the audio file into text format. The converted text data is saved in the / transcripts / A / directory.

[0262] Data Integration

[0263] The server combines the resume, CV, and converted text data. This combined data is saved in JSON format in / integrated_data / A / combined_data.json and input to the generative AI model.

[0264] Automatic generation of letters of recommendation

[0265] A generative AI model (e.g., OpenAI's GPT-4) automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. A draft of the generated recommendation letter is saved in / recommendations / A / draft.txt.

[0266] Interface and Emotion Engine

[0267] The server provides the user with an interface to display the generated draft recommendation letter. The user can then review and revise the recommendation letter through a web browser. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. For example, using Microsoft Azure's Emotion API, if the user shows a dissatisfied expression, the engine will suggest more positive revisions.

[0268] Reviewing and correcting letters of recommendation

[0269] The emotion engine provides real-time feedback as users revise their draft recommendation, which is saved in / updated_recommendations / A / final.txt.

[0270] Save and send letters of recommendation

[0271] The final recommendation letter is converted into a PDF format and sent to the company. The server securely transmits it using the SMTP protocol.

[0272] Specific examples

[0273] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data using Google Cloud Speech-to-Text. The server then combines the resume, CV, and converted text data into JSON format and generates a draft letter of recommendation using OpenAI's GPT-4. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final letter of recommendation is saved and sent to the specified company.

[0274] An example of a prompt might be, "I have uploaded Applicant A's resume and interview audio data. Please transcribe the key skills and achievements the applicant mentioned during the interview into text and draft a letter of recommendation that includes them. Also, please suggest revisions based on the user's sentiment."

[0275] This system improves the efficiency of creating recommendation letters and provides flexible revision support that takes into account the user's emotions.

[0276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0277] Step 1: Receiving Data

[0278] A user uploads a job seeker's resume, CV, and interview audio data to the system via a web interface. The server receives these data securely using SSL / TLS over the HTTPS protocol and stores them in a specified folder structure. The input data is the resume (e.g., resume.pdf), resume (e.g., cv.pdf), and audio data (e.g., interview_audio.mp3), and stores them in the / uploaded_data / {username} / directory.

[0279] Step 2: Convert audio data to text

[0280] The server converts the saved interview audio data into text data using speech recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the audio file into text format and saves the conversion result in / transcripts / {username} / transcript.txt. The input is audio data, and the output is text data.

[0281] Step 3: Integrate the data

[0282] The server combines the job seeker's resume, CV, and text data generated by speech recognition. Specifically, it compiles this data into JSON format and saves it in / integrated_data / {username} / combined_data.json. The input is the resume, CV, and text data, and the output is the combined data in JSON format.

[0283] Step 4: Auto-generate letters of recommendation

[0284] The server uses a generative AI model (e.g., OpenAI's GPT-4) based on the integrated data to automatically generate a recommendation letter. The generated draft recommendation letter is saved in / recommendations / {username} / draft.txt. The input is the integrated data JSON file, and the output is the draft recommendation letter text. Specifically, the integrated data is input as a prompt into the AI ​​model, and the resulting text is saved.

[0285] Step 5: Providing the interface and emotion engine

[0286] The server presents the generated draft recommendation letter through a user interface. The user can review and revise the recommendation letter via a web browser. An emotion engine (e.g., Microsoft Azure Emotion API) is used to analyze the user's facial expressions and voice and provide revision suggestions based on their emotional state. The input is the user's emotional data (facial expressions and voice), and the output is revision suggestions.

[0287] Step 6: Review and revise your recommendation letter

[0288] The user reviews the draft recommendation letter and makes any necessary revisions. Based on the feedback from the emotion engine, the server makes appropriate revisions. The server saves the revised recommendation letter in / updated_recommendations / {username} / final.txt. The input is the draft text and the user's revisions, and the output is the revised recommendation letter text.

[0289] Step 7: Save and submit your recommendation letter

[0290] The server converts the final recommendation letter into PDF format and sends it to the company. The input is the revised recommendation letter text and the output is a PDF file. The server securely sends the recommendation letter to the specified company using the SMTP protocol.

[0291] (Application example 2)

[0292] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0293] In brick-and-mortar stores, it is difficult to properly receive feedback from customers and efficiently and accurately generate countermeasures and messages. Furthermore, when staff manually process feedback, it takes time and the quality of responses can vary. The present invention aims to solve these problems and provide a system that improves customer service.

[0294] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0295] In this invention, the server includes means for receiving data related to customers in multiple formats, means for converting voice data from the multiple formats into text data, and means for integrating the multiple formats and the text data. This enables rapid analysis of customer feedback and automatic generation of appropriate countermeasures and messages. Furthermore, an emotion engine built into the interface helps modify countermeasures taking into account customer emotions, thereby providing consistent, high-quality customer service.

[0296] "Multiple forms of data about a job seeker" refers to different forms of data, such as a job seeker's resume, CV, and interview audio.

[0297] "Means for converting voice data into text data" refers to means for converting voice into text format using voice recognition technology.

[0298] A "data integration means" is a means for integrating multiple data formats into a single analyzable data set.

[0299] "Means for automatically generating recommendation letters" refers to means for automatically generating recommendation letters based on the integrated data using a generative AI model.

[0300] "Interface" refers to the user interface through which a user interacts with the system and reviews and modifies the generated recommendations.

[0301] The "emotion engine" is a system that analyzes a user's facial expressions and voice to recognize their emotions, and is used to assist in the recommendation letter revision process.

[0302] The "means for saving and transmitting a revised recommendation letter" refers to the means by which a user saves a revised recommendation letter in an appropriate format and transmits it to a recipient such as a company.

[0303] MODE FOR CARRYING OUT THE INVENTION

[0304] The present invention relates to a system that receives feedback from customers in a physical store and automatically generates countermeasures and messages based on that feedback. The purpose of this system is to improve customer service and increase the work efficiency of staff. Specific embodiments for implementing the present invention are described below.

[0305] System Components

[0306] 1. Data receiving means:

[0307] The server provides a means for collecting customer feedback data, specifically, receiving customer voice feedback and reviews as text data.

[0308] 2. Voice Recognition Technology:

[0309] The server uses voice recognition technology to convert the received voice data into text data, using Google Cloud's Speech-to-Text API.

[0310] 3. Data integration methods:

[0311] The server consolidates multiple forms of feedback data received from customers (audio data, text data, etc.) and converts them into a single analyzable data set.

[0312] 4. Generative AI Models:

[0313] The server uses generative AI models, such as Transformer-based generative AI, to automatically generate customer responses and messages based on the integrated data.

[0314] 5. Interface:

[0315] The server provides a user interface for the user to review and, if necessary, modify the generated message.

[0316] 6. Emotion Engine:

[0317] The interface incorporates an emotion engine that analyzes the user's emotions to provide better suggestions and advice, using, for example, a BERT-based emotion analysis model.

[0318] 7. Storage and transmission methods:

[0319] It provides a means for the user to save the modified message in an appropriate format (e.g. PDF) and send it to the desired recipients. For example, PyPDF2 is used to save the data.

[0320] Processing flow

[0321] 1. Receiving data:

[0322] When customers give feedback by voice, the user uploads the voice data to the server. Feedback by text data is also uploaded in the same way.

[0323] 2. Transcription of audio data:

[0324] The server uses Google Cloud's Speech-to-Text API to convert the audio data into text data, which is then temporarily stored.

[0325] 3. Data integration:

[0326] The server integrates the received text data and audio data and converts them into a single data set.

[0327] 4. Automatic message generation:

[0328] Based on the integrated data, the server automatically generates responses and messages using a generative AI model that takes into account customer emotions and feedback content.

[0329] 5. Check and fix in the interface:

[0330] The user can review the generated message on the interface and make any necessary edits. The emotion engine analyzes the user's facial expressions and voice to interpret their emotions and provides suggestions for editing.

[0331] 6. Storage and Transmission:

[0332] The corrected message is converted to PDF format and saved in a specified location, after which it can be sent to the customer or administrator as needed.

[0333] Specific examples

[0334] For example, if a customer provides feedback such as "The staff were very kind and helpful," the server receives this feedback as voice data and converts it into text data. If emotion analysis subsequently determines that the feedback is "joyful," the generative AI model automatically generates a message such as "Thank you for your kind words. We will continue to strive to maintain and further improve this service," and presents it to the user.

[0335] Prompt Sentence Examples

[0336] Customer feedback: 'The staff were very friendly and helpful' Based on this feedback, generate appropriate improvement suggestions and messages when the emotion is 'delighted'.

[0337] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0338] Step 1:

[0339] The server receives customer feedback data. Specifically, the customer gives voice feedback using a terminal and uploads it to the server. The input is voice data, and the output is an audio file stored on the server.

[0340] Step 2:

[0341] The server converts the received voice data into text data using Google Cloud's Speech-to-Text API. The input is voice data, speech recognition technology is applied to process the data, and the output is text data.

[0342] Step 3:

[0343] The server integrates the received text and audio data, unifying and organizing the different data formats to create an integrated dataset. The input is text and audio data, and the output is the integrated dataset.

[0344] Step 4:

[0345] The server inputs the integrated dataset into a generative AI model to automatically generate responses and messages for customers. The input is the integrated dataset, and the output is the generated message. Natural language generation is performed as a data calculation in this process.

[0346] Step 5:

[0347] The server displays the generated message on a user interface, which the user uses to review the message and make corrections if necessary. The input is the generated message, and the output is the message displayed to the user.

[0348] Step 6:

[0349] When a user edits a message, an emotion engine built into the interface analyzes the user's emotions from their facial expressions and voice and provides suggestions for editing. The input is the user's facial expression and voice data, and the output is the emotion analysis results and editing suggestions.

[0350] Step 7:

[0351] The server converts the modified message to PDF format and saves it in an appropriate location. The input is the modified message, the PDF conversion is performed as data processing, and the output is a PDF file.

[0352] Step 8:

[0353] The server sends the saved PDF file to the customer or administrator as needed. The input is the PDF file and the output is the message sent.

[0354] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0355] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0356] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0357] [Second embodiment]

[0358] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0359] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0360] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0361] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0362] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0363] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0364] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0365] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0366] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0367] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0368] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0369] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0370] MODE FOR CARRYING OUT THE INVENTION

[0371] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[0372] System configuration

[0373] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[0374] Program Overview

[0375] The system receives a job seeker's resume, CV, and audio recording of the interview, and automatically generates a recommendation letter based on these. The generated recommendation letter is then finalized after user confirmation and correction, and sent to the company.

[0376] Operation of each method

[0377] Receiving data

[0378] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio (WAV or MP3 format) to the system. The server receives these data and saves the files in the appropriate format. This process ensures that the necessary information is captured in the system.

[0379] Text data conversion using voice recognition technology

[0380] The server uses voice recognition technology to convert the uploaded interview audio data into text data. Specifically, it uses an AI voice recognition model to convert the audio data into text format, which will be used in further processes.

[0381] Data Integration

[0382] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[0383] Automatic generation of letters of recommendation

[0384] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[0385] Reviewing and correcting letters of recommendation

[0386] The user can check the generated draft recommendation letter and, if necessary, can edit the content to create the final recommendation letter. After the edits are reflected, the server saves the final recommendation letter.

[0387] Save and send letters of recommendation

[0388] The server stores the final recommendation in the appropriate format (e.g. PDF) and then sends it to the company if required, ensuring data security and confidentiality.

[0389] Specific examples

[0390] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives this data and transcribes the audio data. Next, it integrates each data and generates a draft recommendation letter using a generative AI model. The user then reviews the draft and makes corrections. The final revised recommendation letter is saved and ready to be sent to the company.

[0391] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[0392] The processing flow will be explained below.

[0393] Step 1:

[0394] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[0395] Step 2:

[0396] The server stores the uploaded resumes, CVs, and interview audio data. Specifically, the server places each data in a designated folder so that it can be accessed in subsequent processes.

[0397] Step 3:

[0398] The server converts the interview audio data into text data using speech recognition technology. This process uses an AI speech recognition model to convert the audio file into text format and save the converted text data.

[0399] Step 4:

[0400] The server then combines the received and converted data, specifically the resumes, CVs, and text data, into a single dataset that is ready to be fed into a generative AI model.

[0401] Step 5:

[0402] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[0403] Step 6:

[0404] The server displays the generated draft recommendation letter through an interface provided to the user, who can use this interface to review the recommendation letter and make corrections as necessary.

[0405] Step 7:

[0406] After the user modifies the draft, the server saves the final version of the recommendation, which records the changes as the final modified recommendation.

[0407] Step 8:

[0408] The server converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[0409] Example 1

[0410] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0411] Creating recommendation letters, which are required when introducing job seekers to companies, is not only time-consuming and laborious, but also has the potential for inaccurate information integration and analysis. For this reason, a system for creating recommendation letters efficiently and accurately is needed. Furthermore, an environment is needed that centralizes the processes of transcribing audio data, integrating data in multiple formats, and automatically generating recommendation letters based on that data, while allowing users to easily edit and review them.

[0412] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0413] In this invention, the server includes means for receiving multiple types of data about the job seeker, means for using voice recognition software to convert voice data into text data, means for integrating the multiple types of data and the text data into a dataset, means for using a generative AI model to automatically generate a recommendation letter based on the integrated dataset, means for providing an interface for displaying and modifying the automatically generated recommendation letter, and means for saving, formatting, and transmitting the modified recommendation letter. This streamlines the recommendation letter creation process and enables the generation of more accurate recommendation letters. Furthermore, users can easily review the generated recommendation letter and modify it as necessary.

[0414] The "means for receiving data" refers to a means for uploading and storing multiple forms of data related to job seekers, such as resumes, CVs, and audio data, into the system.

[0415] "Means for converting voice data into text data" means means for converting uploaded voice data into text format using voice recognition software.

[0416] The "means of integrating data" refers to a means of combining resumes, CVs, and text data converted from audio data into a single data set.

[0417] A "generative AI model" is an artificial intelligence model that automatically generates recommendation letters based on an integrated dataset.

[0418] The "means for providing an interface" refers to a means for providing a screen for a user to display the generated recommendation letter and to modify it as necessary.

[0419] "Means for saving and sending the recommendation letter" refers to the means for saving the final recommendation letter in an appropriate format (e.g., PDF) and sending it to the company.

[0420] A "prompt" is a specific statement that instructs the generative AI model and provides the information necessary to generate a recommendation letter.

[0421] MODE FOR CARRYING OUT THE INVENTION

[0422] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[0423] System configuration

[0424] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[0425] Operation of each method

[0426] Receiving data

[0427] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio data (WAV or MP3 format) to the system. The server receives this data and saves each file in the appropriate folder. This process ensures that the necessary information is captured in the system.

[0428] Text data conversion using voice recognition technology

[0429] The server converts the stored interview voice data into text data using speech recognition software (e.g., Google Speech-to-Text API). The converted text data is temporarily stored and used for subsequent processing.

[0430] Data Integration

[0431] The server combines the resumes, CVs, and converted text data, and stores this combined dataset in JSON format, which is then fed into a generative AI model.

[0432] Automatic generation of letters of recommendation

[0433] The generative AI model automatically generates recommendation letters based on the integrated dataset. The recommendation letters generated using specific prompts reflect the job seeker's skills and experience and include suggestions for the company. An example of a prompt is, "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience."

[0434] Reviewing and correcting letters of recommendation

[0435] The user can review the generated recommendation letter through a web interface, which provides editing functionality to revise the content of the recommendation letter as needed and finalize it.

[0436] Save and send letters of recommendation

[0437] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. A function is implemented to automatically send the saved recommendation letter to the specified company's email address.

[0438] Specific examples

[0439] For example, if a user uploads applicant A's resume (PDF format), CV (Word format), and interview audio data (MP3 format) to the system, the server receives these data and converts the audio data into text using an AI speech recognition model. The server then integrates the resume, CV, and text data and generates a draft letter of recommendation using a generative AI model. The user reviews the draft and makes any necessary revisions. The revised letter of recommendation is saved on the server and can be sent to the company in PDF format.

[0440] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[0441] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0442] Step 1:

[0443] The user uploads the job seeker's resume (PDF or Word format), resume (PDF or Word format), and interview audio data (WAV or MP3 format) from their terminal to the system. These three types of files are taken in as input. The server receives the data and saves the files in the appropriate folder. The output of this step is various data files saved in the system. Specifically, the server receives the upload request, checks the files, and saves them in the appropriate directory.

[0444] Step 2:

[0445] The server retrieves the saved interview audio data and converts it into text data using speech recognition software (e.g., Google Speech-to-Text API). The input is the interview audio data file. The server sends this audio file to the speech recognition API and obtains text data. The output of this step is the text data converted from the audio data. Specifically, the server passes the audio data to the API and saves the returned text data.

[0446] Step 3:

[0447] The server integrates the resumes, CVs, and converted text data. The inputs are the various data files and text data saved in the previous step. The server integrates them into a single dataset and saves it in JSON format. The output of this step is the integrated dataset. Specifically, it converts the contents of PDF and Word files into text format and combines them into a single JSON file.

[0448] Step 4:

[0449] The generative AI model automatically generates a recommendation letter based on the integrated dataset. The integrated dataset and a prompt are used as input. An example of a specific prompt is input to the model: "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience." The output is a draft of the generated recommendation letter. Specific operations include the process by which the AI ​​model generates a recommendation letter based on the set prompt.

[0450] Step 5:

[0451] The user reviews the generated recommendation letter through a web interface. The generated draft recommendation letter is used as input. The interface provides an editing function to modify the recommendation letter content as needed. The output is the final recommendation letter that has been reviewed and modified by the user. Specific operations include the process in which the user reviews the recommendation letter content and makes modifications on the interface.

[0452] Step 6:

[0453] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. The final recommendation letter corrected by the user is used as input. A function is then implemented to automatically send the saved recommendation letter to the specified company's email address. The output is the recommendation letter sent to the company. Specific operations include converting the corrected recommendation letter to a PDF file, saving it on the server, and sending it to the company by email.

[0454] (Application example 1)

[0455] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0456] The present invention relates to a system that efficiently records the work content and progress of workers in a factory, allowing managers to quickly review and correct work reports. Conventional methods require workers to individually write out work reports, which is time-consuming and labor-intensive, and the recorded content is often inaccurate or inconsistent. Furthermore, managers must spend a lot of time reviewing and correcting the reports. The present invention aims to solve these problems and streamline the process of generating work reports.

[0457] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0458] In this invention, the server includes means for receiving data in multiple formats related to workers, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for automatically generating a work report, means for providing an interface for displaying and modifying the automatically generated work report, and means for saving and transmitting the modified work report, thereby reducing the burden on workers and enabling efficient and consistent creation of work reports.

[0459] "Workers" refers to personnel who perform practical work in a work environment such as a factory or work site.

[0460] "Voice data" refers to acoustic signals that record the statements, instructions, and reports of workers.

[0461] "Character data" refers to information in text format obtained by converting voice data.

[0462] "Integrated data" refers to an information set that brings together data in multiple formats and converted character data.

[0463] A "work report" refers to a document that records the work content, progress, problems, etc. of a worker.

[0464] "Interface" refers to the operation screens and functions that allow users to access and operate the system.

[0465] "Server" refers to a computer system that receives, processes, stores, and transmits data.

[0466] "Voice recognition technology" refers to the technology that analyzes voice data and converts it into text data.

[0467] A "generative AI model" refers to an artificial intelligence model that automatically generates the necessary documents and reports based on given input data.

[0468] "Correction" refers to checking the contents of generated reports and documents and making changes to correct any errors or omissions.

[0469] The present invention provides a system that records the work content and progress of workers in a factory and allows managers to quickly check and correct work reports. The system has the function of collecting voice reports from workers using smart glasses or other devices while they are working, and automatically generating work reports using voice recognition technology and a generative AI model.

[0470] System configuration

[0471] Worker devices: Use devices that allow voice input and video recording, such as smart glasses or headsets.

[0472] Server: Processes data using speech recognition technology and generative AI models, and manages automatically generated work reports.

[0473] Interface terminal: Provides an interface that allows managers to check and modify generated reports using a smartphone or tablet.

[0474] Program Overview and Procedures

[0475] Receiving data

[0476] The server receives the voice data (WAV or MP3 format) and work history data (JSON format) uploaded by the worker and stores this data in an appropriate format.

[0477] Text data conversion using voice recognition technology

[0478] The server converts the uploaded voice data into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text API).

[0479] Data Integration

[0480] The server integrates the work history data with the text data converted from the voice data, and this integrated data is used to generate a work report.

[0481] Automatic generation of work reports

[0482] Use a generative AI model (e.g., OpenAI's GPT-3) to automatically generate a draft working report based on the integrated data.

[0483] Check and correct work reports

[0484] The manager can check the automatically generated draft work report on their device (smartphone or tablet) and make any necessary corrections through the interface.

[0485] Save and send work reports

[0486] The final revised work report is saved on the server and sent to the relevant departments and parties as needed.

[0487] Specific examples

[0488] For example, while working, a worker wearing smart glasses may report by voice, "Starting work on ~. Progress is going well, no problems." This voice data is uploaded to a server and converted into text data using speech recognition technology. It is then integrated with work history data and input into a generative AI model. The model generates a draft work report based on prompt sentences such as the following:

[0489] Prompt Sentence Examples

[0490] Work history: XX Factory, Work content: Machine inspection, Progress: 100%, No special notes.

[0491] Converted character data: Starting work on ~. Good progress, no problems.

[0492] Using this prompt as input, a draft work report is generated, after which the manager can review the generated report on a smartphone or tablet, make any necessary corrections, save it, and send it to the relevant parties.

[0493] In this way, the system of the present invention makes the process of generating work reports more efficient, reduces the burden on workers, and improves the quality of reports.

[0494] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0495] Step 1:

[0496] The server receives the voice data and work history data uploaded by the worker. Specifically, the voice data (WAV or MP3 format) recorded by the worker using smart glasses and the work history data in JSON format are sent to the server. The server saves them in an appropriate directory. The input is the voice data and work history data, and the output is the saved voice file and work history file.

[0497] Step 2:

[0498] The server converts the received and saved voice data into text data using voice recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the voice data into text data. The input is voice data, and the output is the converted text data. This text data is required for subsequent processing.

[0499] Step 3:

[0500] The server integrates the work history data with the text data generated by speech recognition. Specifically, the server reads the work history data in JSON format and combines it with the converted text data. The input is the work history data and text data, and the output is the integrated data (JSON format). This integrated data is used to generate a work report.

[0501] Step 4:

[0502] The server uses a generative AI model to automatically generate a work report based on the integrated data. Specifically, it uses OpenAI's GPT-3 to generate a detailed work report based on the contents of the integrated data. Based on the prompt, the input is the integrated data, and the output is a draft of the work report.

[0503] Step 5:

[0504] The terminal displays the automatically generated work report created by the server and provides an interface for the user to check and modify it. Specifically, the administrator checks the contents of the generated work report via a smartphone or tablet and makes any necessary modifications. The input is a draft work report, and the output is the modified work report.

[0505] Step 6:

[0506] The server saves the revised work report and sends it to relevant parties and departments as needed. Specifically, the server saves the revised work report in PDF format or other format and sends it to the email addresses of relevant departments. The input is the revised work report, and the output is the saved report and transmission log.

[0507] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0508] MODE FOR CARRYING OUT THE INVENTION

[0509] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[0510] System configuration

[0511] The system consists of the following components:

[0512] 1. Means of receiving job seeker data

[0513] 2. Means using voice recognition technology

[0514] 3. Data Integration Methods

[0515] 4. Using Generative AI Models

[0516] 5. Interface

[0517] 6. Preservation means

[0518] 7. Means of transmission

[0519] 8. Emotion Engine

[0520] Program Overview

[0521] The system receives a job seeker's resume, CV, and interview audio data, and automatically generates a recommendation letter based on them. It also incorporates an emotion engine that recognizes the user's emotions and assists in the revision process.

[0522] Operation of each method

[0523] Receiving data

[0524] Users upload job seekers' resumes, CVs, and interview audio data to the system. The server receives these data and saves them in a designated folder, capturing the necessary data.

[0525] Text data conversion using voice recognition technology

[0526] The server converts the uploaded interview audio data into text data. Specifically, it uses an AI speech recognition model to convert the audio file into text format and saves the converted text data.

[0527] Data Integration

[0528] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[0529] Automatic generation of letters of recommendation

[0530] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[0531] Interface and Emotion Engine

[0532] The server displays the generated draft recommendation letter through an interface provided to the user. The user uses this interface to confirm the contents of the recommendation letter. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state and provides suggestions for revisions to help the user revise the recommendation letter.

[0533] Reviewing and correcting letters of recommendation

[0534] As the user edits the draft, the emotion engine provides advice based on the user's emotional state. For example, if the user is feeling unhappy, it can suggest more positive wording. The revised recommendation letter is then saved to the server in its final form.

[0535] Save and send letters of recommendation

[0536] The server then converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[0537] Specific examples

[0538] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data. Next, each data is integrated and a generative AI model is used to generate a draft recommendation letter. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final recommendation letter is then saved and sent to the company.

[0539] The above is a specific embodiment of the present invention. This system improves the efficiency of recommendation letter creation and provides flexible revision support that takes into account the user's emotions.

[0540] The processing flow will be explained below.

[0541] Step 1:

[0542] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[0543] Step 2:

[0544] The server stores the uploaded resumes, CVs, and interview audio data in designated folders, making them accessible for subsequent processing.

[0545] Step 3:

[0546] The server converts the interview audio data into text data using speech recognition technology, converts the audio file into text format using an AI speech recognition model, and saves the converted text data.

[0547] Step 4:

[0548] The server combines the resumes, CVs, and converted text data to create a structured dataset that is ready to be fed into a generative AI model.

[0549] Step 5:

[0550] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[0551] Step 6:

[0552] The server displays the generated draft recommendation letter through an interface provided to the user, who can use the interface to review and modify the recommendation letter content.

[0553] Step 7:

[0554] The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. Specifically, it uses a camera and microphone to capture facial and voice data in real time and performs emotion analysis.

[0555] Step 8:

[0556] The emotion engine provides suggestions for revising the recommendation letter on the interface based on the user's emotional state. For example, if the user is feeling dissatisfied, the engine will suggest more positive expressions to help the user make revisions smoothly.

[0557] Step 9:

[0558] The user then modifies the recommendation based on the suggestions from the emotion engine. Once the modifications are complete, the final recommendation is sent to the server.

[0559] Step 10:

[0560] The server saves the final, modified version of the recommendation, which is then recorded as a finalized recommendation.

[0561] Step 11:

[0562] The server converts the final recommendation letter into the appropriate format (e.g., PDF) and prepares it for transmission to the company. Once the recommendation letter is ready for transmission, it is securely transmitted to the company.

[0563] Example 2

[0564] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0565] When job placement agencies introduce job seekers to companies, they create recommendation letters based on multiple forms of data, including the job seeker's resume, CV, and interview audio. This process is often manual, time-consuming, and prone to typos and inconsistencies. Furthermore, there is a lack of flexible correction support that takes user sentiment into account. Therefore, there is a need for a system that streamlines the recommendation letter creation and correction process and improves its accuracy.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0567] In this invention, the server includes means for receiving data in multiple formats related to the job seeker, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for using a generative AI model to automatically generate a letter of recommendation based on the integrated data, means for providing an interface for displaying and modifying the automatically generated letter of recommendation and for recognizing a user's emotions to support the modification process, and means for saving and transmitting the modified letter of recommendation. This enables efficient creation of letters of recommendation and flexible modification support that takes the user's emotions into consideration.

[0568] "Job seekers" refer to people who are seeking a new job.

[0569] "Means for receiving data" refers to a device or method for incorporating various data, such as resumes, CVs, and audio files, input from outside into the system.

[0570] "Audio data" refers to audio files containing what a job seeker said during an interview or other occasion.

[0571] "Means for converting into text data" refers to speech recognition technology or software for converting voice data into text format.

[0572] "Means for integrating data" refers to a device or method that combines data in multiple formats (e.g., resumes, CVs, text data converted from audio data) to create a single integrated data set.

[0573] The use of a "generative AI model" refers to the use of an artificial intelligence model to automatically generate a recommendation based on the integrated data.

[0574] "Interface" refers to the user interface through which a user interacts with a system.

[0575] "Means for recognizing emotions" refers to technology and software for analyzing and determining emotions from a user's facial expressions and voice.

[0576] "Means for storing" refers to a device or method for recording the revised recommendation as a digital file.

[0577] "Transmitting means" refers to a device or method for transferring the completed recommendation letter to another terminal such as a company.

[0578] A "letter of recommendation" is a document that describes a job seeker's background and skills in order to recommend them to a company.

[0579] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[0580] System configuration

[0581] The system consists of the following components:

[0582] 1. Means of receiving job seeker data

[0583] 2. Means using voice recognition technology

[0584] 3. Data Integration Methods

[0585] 4. Using Generative AI Models

[0586] 5. Interface

[0587] 6. Preservation means

[0588] 7. Means of transmission

[0589] 8. Emotion Engine

[0590] System Operation

[0591] Receiving data

[0592] A user uploads a job seeker's resume, CV, and interview audio data to the system. The server securely receives this data using the HTTPS protocol with SSL / TLS and saves it in a specified folder. For example, the data for job seeker A is saved in the / uploaded_data / A / directory.

[0593] Text data conversion using voice recognition technology

[0594] The server converts the uploaded interview audio data into text data. Specifically, it uses AI speech recognition services such as Google Cloud Speech-to-Text to convert the audio file into text format. The converted text data is saved in the / transcripts / A / directory.

[0595] Data Integration

[0596] The server combines the resume, CV, and converted text data. This combined data is saved in JSON format in / integrated_data / A / combined_data.json and input to the generative AI model.

[0597] Automatic generation of letters of recommendation

[0598] A generative AI model (e.g., OpenAI's GPT-4) automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. A draft of the generated recommendation letter is saved in / recommendations / A / draft.txt.

[0599] Interface and Emotion Engine

[0600] The server provides the user with an interface to display the generated draft recommendation letter. The user can then review and revise the recommendation letter through a web browser. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. For example, using Microsoft Azure's Emotion API, if the user shows a dissatisfied expression, the engine will suggest more positive revisions.

[0601] Reviewing and correcting letters of recommendation

[0602] The emotion engine provides real-time feedback as users revise their draft recommendation, which is saved in / updated_recommendations / A / final.txt.

[0603] Save and send letters of recommendation

[0604] The final recommendation letter is converted into a PDF format and sent to the company. The server securely transmits it using the SMTP protocol.

[0605] Specific examples

[0606] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data using Google Cloud Speech-to-Text. The server then combines the resume, CV, and converted text data into JSON format and generates a draft letter of recommendation using OpenAI's GPT-4. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final letter of recommendation is saved and sent to the specified company.

[0607] An example of a prompt might be, "I have uploaded Applicant A's resume and interview audio data. Please transcribe the key skills and achievements the applicant mentioned during the interview into text and draft a letter of recommendation that includes them. Also, please suggest revisions based on the user's sentiment."

[0608] This system improves the efficiency of creating recommendation letters and provides flexible revision support that takes into account the user's emotions.

[0609] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0610] Step 1: Receiving Data

[0611] A user uploads a job seeker's resume, CV, and interview audio data to the system via a web interface. The server receives these data securely using SSL / TLS over the HTTPS protocol and stores them in a specified folder structure. The input data is the resume (e.g., resume.pdf), resume (e.g., cv.pdf), and audio data (e.g., interview_audio.mp3), and stores them in the / uploaded_data / {username} / directory.

[0612] Step 2: Convert audio data to text

[0613] The server converts the saved interview audio data into text data using speech recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the audio file into text format and saves the conversion result in / transcripts / {username} / transcript.txt. The input is audio data, and the output is text data.

[0614] Step 3: Integrate the data

[0615] The server combines the job seeker's resume, CV, and text data generated by speech recognition. Specifically, it compiles this data into JSON format and saves it in / integrated_data / {username} / combined_data.json. The input is the resume, CV, and text data, and the output is the combined data in JSON format.

[0616] Step 4: Auto-generate letters of recommendation

[0617] The server uses a generative AI model (e.g., OpenAI's GPT-4) based on the integrated data to automatically generate a recommendation letter. The generated draft recommendation letter is saved in / recommendations / {username} / draft.txt. The input is the integrated data JSON file, and the output is the draft recommendation letter text. Specifically, the integrated data is input as a prompt into the AI ​​model, and the resulting text is saved.

[0618] Step 5: Providing the interface and emotion engine

[0619] The server presents the generated draft recommendation letter through a user interface. The user can review and revise the recommendation letter via a web browser. An emotion engine (e.g., Microsoft Azure Emotion API) is used to analyze the user's facial expressions and voice and provide revision suggestions based on their emotional state. The input is the user's emotional data (facial expressions and voice), and the output is revision suggestions.

[0620] Step 6: Review and revise your recommendation letter

[0621] The user reviews the draft recommendation letter and makes any necessary revisions. Based on the feedback from the emotion engine, the server makes appropriate revisions. The server saves the revised recommendation letter in / updated_recommendations / {username} / final.txt. The input is the draft text and the user's revisions, and the output is the revised recommendation letter text.

[0622] Step 7: Save and submit your recommendation letter

[0623] The server converts the final recommendation letter into PDF format and sends it to the company. The input is the revised recommendation letter text and the output is a PDF file. The server securely sends the recommendation letter to the specified company using the SMTP protocol.

[0624] (Application example 2)

[0625] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0626] In brick-and-mortar stores, it is difficult to properly receive feedback from customers and efficiently and accurately generate countermeasures and messages. Furthermore, when staff manually process feedback, it takes time and the quality of responses can vary. The present invention aims to solve these problems and provide a system that improves customer service.

[0627] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0628] In this invention, the server includes means for receiving data related to customers in multiple formats, means for converting voice data from the multiple formats into text data, and means for integrating the multiple formats and the text data. This enables rapid analysis of customer feedback and automatic generation of appropriate countermeasures and messages. Furthermore, an emotion engine built into the interface helps modify countermeasures taking into account customer emotions, thereby providing consistent, high-quality customer service.

[0629] "Multiple forms of data about a job seeker" refers to different forms of data, such as a job seeker's resume, CV, and interview audio.

[0630] "Means for converting voice data into text data" refers to means for converting voice into text format using voice recognition technology.

[0631] A "data integration means" is a means for integrating multiple data formats into a single analyzable data set.

[0632] "Means for automatically generating recommendation letters" refers to means for automatically generating recommendation letters based on the integrated data using a generative AI model.

[0633] "Interface" refers to the user interface through which a user interacts with the system and reviews and modifies the generated recommendations.

[0634] The "emotion engine" is a system that analyzes a user's facial expressions and voice to recognize their emotions, and is used to assist in the recommendation letter revision process.

[0635] The "means for saving and transmitting a revised recommendation letter" refers to the means by which a user saves a revised recommendation letter in an appropriate format and transmits it to a recipient such as a company.

[0636] MODE FOR CARRYING OUT THE INVENTION

[0637] The present invention relates to a system that receives feedback from customers in a physical store and automatically generates countermeasures and messages based on that feedback. The purpose of this system is to improve customer service and increase the work efficiency of staff. Specific embodiments for implementing the present invention are described below.

[0638] System Components

[0639] 1. Data receiving means:

[0640] The server provides a means for collecting customer feedback data, specifically, receiving customer voice feedback and reviews as text data.

[0641] 2. Voice Recognition Technology:

[0642] The server uses voice recognition technology to convert the received voice data into text data, using Google Cloud's Speech-to-Text API.

[0643] 3. Data integration methods:

[0644] The server consolidates multiple forms of feedback data received from customers (audio data, text data, etc.) and converts them into a single analyzable data set.

[0645] 4. Generative AI Models:

[0646] The server uses generative AI models, such as Transformer-based generative AI, to automatically generate customer responses and messages based on the integrated data.

[0647] 5. Interface:

[0648] The server provides a user interface for the user to review and, if necessary, modify the generated message.

[0649] 6. Emotion Engine:

[0650] The interface incorporates an emotion engine that analyzes the user's emotions to provide better suggestions and advice, using, for example, a BERT-based emotion analysis model.

[0651] 7. Storage and transmission methods:

[0652] It provides a means for the user to save the modified message in an appropriate format (e.g. PDF) and send it to the desired recipients. For example, PyPDF2 is used to save the data.

[0653] Processing flow

[0654] 1. Receiving data:

[0655] When customers give feedback by voice, the user uploads the voice data to the server. Feedback by text data is also uploaded in the same way.

[0656] 2. Transcription of audio data:

[0657] The server uses Google Cloud's Speech-to-Text API to convert the audio data into text data, which is then temporarily stored.

[0658] 3. Data integration:

[0659] The server integrates the received text data and audio data and converts them into a single data set.

[0660] 4. Automatic message generation:

[0661] Based on the integrated data, the server automatically generates responses and messages using a generative AI model that takes into account customer emotions and feedback content.

[0662] 5. Check and fix in the interface:

[0663] The user can review the generated message on the interface and make any necessary edits. The emotion engine analyzes the user's facial expressions and voice to interpret their emotions and provides suggestions for editing.

[0664] 6. Storage and Transmission:

[0665] The corrected message is converted to PDF format and saved in a specified location, after which it can be sent to the customer or administrator as needed.

[0666] Specific examples

[0667] For example, if a customer provides feedback such as "The staff were very kind and helpful," the server receives this feedback as voice data and converts it into text data. If emotion analysis subsequently determines that the feedback is "joyful," the generative AI model automatically generates a message such as "Thank you for your kind words. We will continue to strive to maintain and further improve this service," and presents it to the user.

[0668] Prompt Sentence Examples

[0669] Customer feedback: 'The staff were very friendly and helpful' Based on this feedback, generate appropriate improvement suggestions and messages when the emotion is 'delighted'.

[0670] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0671] Step 1:

[0672] The server receives customer feedback data. Specifically, the customer gives voice feedback using a terminal and uploads it to the server. The input is voice data, and the output is an audio file stored on the server.

[0673] Step 2:

[0674] The server converts the received voice data into text data using Google Cloud's Speech-to-Text API. The input is voice data, speech recognition technology is applied to process the data, and the output is text data.

[0675] Step 3:

[0676] The server integrates the received text and audio data, unifying and organizing the different data formats to create an integrated dataset. The input is text and audio data, and the output is the integrated dataset.

[0677] Step 4:

[0678] The server inputs the integrated dataset into a generative AI model to automatically generate responses and messages for customers. The input is the integrated dataset, and the output is the generated message. Natural language generation is performed as a data calculation in this process.

[0679] Step 5:

[0680] The server displays the generated message on a user interface, which the user uses to review the message and make corrections if necessary. The input is the generated message, and the output is the message displayed to the user.

[0681] Step 6:

[0682] When a user edits a message, an emotion engine built into the interface analyzes the user's emotions from their facial expressions and voice and provides suggestions for editing. The input is the user's facial expression and voice data, and the output is the emotion analysis results and editing suggestions.

[0683] Step 7:

[0684] The server converts the modified message to PDF format and saves it in an appropriate location. The input is the modified message, the PDF conversion is performed as data processing, and the output is a PDF file.

[0685] Step 8:

[0686] The server sends the saved PDF file to the customer or administrator as needed. The input is the PDF file and the output is the message sent.

[0687] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0688] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0689] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0690] [Third embodiment]

[0691] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0692] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0693] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0694] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0695] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0696] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0697] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0698] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0699] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0700] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0701] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0702] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0703] MODE FOR CARRYING OUT THE INVENTION

[0704] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[0705] System configuration

[0706] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[0707] Program Overview

[0708] The system receives a job seeker's resume, CV, and audio recording of the interview, and automatically generates a recommendation letter based on these. The generated recommendation letter is then finalized after user confirmation and correction, and sent to the company.

[0709] Operation of each method

[0710] Receiving data

[0711] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio (WAV or MP3 format) to the system. The server receives these data and saves the files in the appropriate format. This process ensures that the necessary information is captured in the system.

[0712] Text data conversion using voice recognition technology

[0713] The server uses voice recognition technology to convert the uploaded interview audio data into text data. Specifically, it uses an AI voice recognition model to convert the audio data into text format, which will be used in further processes.

[0714] Data Integration

[0715] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[0716] Automatic generation of letters of recommendation

[0717] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[0718] Reviewing and correcting letters of recommendation

[0719] The user can check the generated draft recommendation letter and, if necessary, can edit the content to create the final recommendation letter. After the edits are reflected, the server saves the final recommendation letter.

[0720] Save and send letters of recommendation

[0721] The server stores the final recommendation in the appropriate format (e.g. PDF) and then sends it to the company if required, ensuring data security and confidentiality.

[0722] Specific examples

[0723] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives this data and transcribes the audio data. Next, it integrates each data and generates a draft recommendation letter using a generative AI model. The user then reviews the draft and makes corrections. The final revised recommendation letter is saved and ready to be sent to the company.

[0724] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[0725] The processing flow will be explained below.

[0726] Step 1:

[0727] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[0728] Step 2:

[0729] The server stores the uploaded resumes, CVs, and interview audio data. Specifically, the server places each data in a designated folder so that it can be accessed in subsequent processes.

[0730] Step 3:

[0731] The server converts the interview audio data into text data using speech recognition technology. This process uses an AI speech recognition model to convert the audio file into text format and save the converted text data.

[0732] Step 4:

[0733] The server then combines the received and converted data, specifically the resumes, CVs, and text data, into a single dataset that is ready to be fed into a generative AI model.

[0734] Step 5:

[0735] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[0736] Step 6:

[0737] The server displays the generated draft recommendation letter through an interface provided to the user, who can use this interface to review the recommendation letter and make corrections as necessary.

[0738] Step 7:

[0739] After the user modifies the draft, the server saves the final version of the recommendation, which records the changes as the final modified recommendation.

[0740] Step 8:

[0741] The server converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[0742] Example 1

[0743] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0744] Creating recommendation letters, which are required when introducing job seekers to companies, is not only time-consuming and laborious, but also has the potential for inaccurate information integration and analysis. For this reason, a system for creating recommendation letters efficiently and accurately is needed. Furthermore, an environment is needed that centralizes the processes of transcribing audio data, integrating data in multiple formats, and automatically generating recommendation letters based on that data, while allowing users to easily edit and review them.

[0745] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0746] In this invention, the server includes means for receiving multiple types of data about the job seeker, means for using voice recognition software to convert voice data into text data, means for integrating the multiple types of data and the text data into a dataset, means for using a generative AI model to automatically generate a recommendation letter based on the integrated dataset, means for providing an interface for displaying and modifying the automatically generated recommendation letter, and means for saving, formatting, and transmitting the modified recommendation letter. This streamlines the recommendation letter creation process and enables the generation of more accurate recommendation letters. Furthermore, users can easily review the generated recommendation letter and modify it as necessary.

[0747] The "means for receiving data" refers to a means for uploading and storing multiple forms of data related to job seekers, such as resumes, CVs, and audio data, into the system.

[0748] "Means for converting voice data into text data" means means for converting uploaded voice data into text format using voice recognition software.

[0749] The "means of integrating data" refers to a means of combining resumes, CVs, and text data converted from audio data into a single data set.

[0750] A "generative AI model" is an artificial intelligence model that automatically generates recommendation letters based on an integrated dataset.

[0751] The "means for providing an interface" refers to a means for providing a screen for a user to display the generated recommendation letter and to modify it as necessary.

[0752] "Means for saving and sending the recommendation letter" refers to the means for saving the final recommendation letter in an appropriate format (e.g., PDF) and sending it to the company.

[0753] A "prompt" is a specific statement that instructs the generative AI model and provides the information necessary to generate a recommendation letter.

[0754] MODE FOR CARRYING OUT THE INVENTION

[0755] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[0756] System configuration

[0757] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[0758] Operation of each method

[0759] Receiving data

[0760] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio data (WAV or MP3 format) to the system. The server receives this data and saves each file in the appropriate folder. This process ensures that the necessary information is captured in the system.

[0761] Text data conversion using voice recognition technology

[0762] The server converts the stored interview voice data into text data using speech recognition software (e.g., Google Speech-to-Text API). The converted text data is temporarily stored and used for subsequent processing.

[0763] Data Integration

[0764] The server combines the resumes, CVs, and converted text data, and stores this combined dataset in JSON format, which is then fed into a generative AI model.

[0765] Automatic generation of letters of recommendation

[0766] The generative AI model automatically generates recommendation letters based on the integrated dataset. The recommendation letters generated using specific prompts reflect the job seeker's skills and experience and include suggestions for the company. An example of a prompt is, "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience."

[0767] Reviewing and correcting letters of recommendation

[0768] The user can review the generated recommendation letter through a web interface, which provides editing functionality to revise the content of the recommendation letter as needed and finalize it.

[0769] Save and send letters of recommendation

[0770] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. A function is implemented to automatically send the saved recommendation letter to the specified company's email address.

[0771] Specific examples

[0772] For example, if a user uploads applicant A's resume (PDF format), CV (Word format), and interview audio data (MP3 format) to the system, the server receives these data and converts the audio data into text using an AI speech recognition model. The server then integrates the resume, CV, and text data and generates a draft letter of recommendation using a generative AI model. The user reviews the draft and makes any necessary revisions. The revised letter of recommendation is saved on the server and can be sent to the company in PDF format.

[0773] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[0774] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0775] Step 1:

[0776] The user uploads the job seeker's resume (PDF or Word format), resume (PDF or Word format), and interview audio data (WAV or MP3 format) from their terminal to the system. These three types of files are taken in as input. The server receives the data and saves the files in the appropriate folder. The output of this step is various data files saved in the system. Specifically, the server receives the upload request, checks the files, and saves them in the appropriate directory.

[0777] Step 2:

[0778] The server retrieves the saved interview audio data and converts it into text data using speech recognition software (e.g., Google Speech-to-Text API). The input is the interview audio data file. The server sends this audio file to the speech recognition API and obtains text data. The output of this step is the text data converted from the audio data. Specifically, the server passes the audio data to the API and saves the returned text data.

[0779] Step 3:

[0780] The server integrates the resumes, CVs, and converted text data. The inputs are the various data files and text data saved in the previous step. The server integrates them into a single dataset and saves it in JSON format. The output of this step is the integrated dataset. Specifically, it converts the contents of PDF and Word files into text format and combines them into a single JSON file.

[0781] Step 4:

[0782] The generative AI model automatically generates a recommendation letter based on the integrated dataset. The integrated dataset and a prompt are used as input. An example of a specific prompt is input to the model: "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience." The output is a draft of the generated recommendation letter. Specific operations include the process by which the AI ​​model generates a recommendation letter based on the set prompt.

[0783] Step 5:

[0784] The user reviews the generated recommendation letter through a web interface. The generated draft recommendation letter is used as input. The interface provides an editing function to modify the recommendation letter content as needed. The output is the final recommendation letter that has been reviewed and modified by the user. Specific operations include the process in which the user reviews the recommendation letter content and makes modifications on the interface.

[0785] Step 6:

[0786] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. The final recommendation letter corrected by the user is used as input. A function is then implemented to automatically send the saved recommendation letter to the specified company's email address. The output is the recommendation letter sent to the company. Specific operations include converting the corrected recommendation letter to a PDF file, saving it on the server, and sending it to the company by email.

[0787] (Application example 1)

[0788] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0789] The present invention relates to a system that efficiently records the work content and progress of workers in a factory, allowing managers to quickly review and correct work reports. Conventional methods require workers to individually write out work reports, which is time-consuming and labor-intensive, and the recorded content is often inaccurate or inconsistent. Furthermore, managers must spend a lot of time reviewing and correcting the reports. The present invention aims to solve these problems and streamline the process of generating work reports.

[0790] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0791] In this invention, the server includes means for receiving data in multiple formats related to workers, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for automatically generating a work report, means for providing an interface for displaying and modifying the automatically generated work report, and means for saving and transmitting the modified work report, thereby reducing the burden on workers and enabling efficient and consistent creation of work reports.

[0792] "Workers" refers to personnel who perform practical work in a work environment such as a factory or work site.

[0793] "Voice data" refers to acoustic signals that record the statements, instructions, and reports of workers.

[0794] "Character data" refers to information in text format obtained by converting voice data.

[0795] "Integrated data" refers to an information set that brings together data in multiple formats and converted character data.

[0796] A "work report" refers to a document that records the work content, progress, problems, etc. of a worker.

[0797] "Interface" refers to the operation screens and functions that allow users to access and operate the system.

[0798] "Server" refers to a computer system that receives, processes, stores, and transmits data.

[0799] "Voice recognition technology" refers to the technology that analyzes voice data and converts it into text data.

[0800] A "generative AI model" refers to an artificial intelligence model that automatically generates the necessary documents and reports based on given input data.

[0801] "Correction" refers to checking the contents of generated reports and documents and making changes to correct any errors or omissions.

[0802] The present invention provides a system that records the work content and progress of workers in a factory and allows managers to quickly check and correct work reports. The system has the function of collecting voice reports from workers using smart glasses or other devices while they are working, and automatically generating work reports using voice recognition technology and a generative AI model.

[0803] System configuration

[0804] Worker devices: Use devices that allow voice input and video recording, such as smart glasses or headsets.

[0805] Server: Processes data using speech recognition technology and generative AI models, and manages automatically generated work reports.

[0806] Interface terminal: Provides an interface that allows managers to check and modify generated reports using a smartphone or tablet.

[0807] Program Overview and Procedures

[0808] Receiving data

[0809] The server receives the voice data (WAV or MP3 format) and work history data (JSON format) uploaded by the worker and stores this data in an appropriate format.

[0810] Text data conversion using voice recognition technology

[0811] The server converts the uploaded voice data into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text API).

[0812] Data Integration

[0813] The server integrates the work history data with the text data converted from the voice data, and this integrated data is used to generate a work report.

[0814] Automatic generation of work reports

[0815] Use a generative AI model (e.g., OpenAI's GPT-3) to automatically generate a draft working report based on the integrated data.

[0816] Check and correct work reports

[0817] The manager can check the automatically generated draft work report on their device (smartphone or tablet) and make any necessary corrections through the interface.

[0818] Save and send work reports

[0819] The final revised work report is saved on the server and sent to the relevant departments and parties as needed.

[0820] Specific examples

[0821] For example, while working, a worker wearing smart glasses may report by voice, "Starting work on ~. Progress is going well, no problems." This voice data is uploaded to a server and converted into text data using speech recognition technology. It is then integrated with work history data and input into a generative AI model. The model generates a draft work report based on prompt sentences such as the following:

[0822] Prompt Sentence Examples

[0823] Work history: XX Factory, Work content: Machine inspection, Progress: 100%, No special notes.

[0824] Converted character data: Starting work on ~. Good progress, no problems.

[0825] Using this prompt as input, a draft work report is generated, after which the manager can review the generated report on a smartphone or tablet, make any necessary corrections, save it, and send it to the relevant parties.

[0826] In this way, the system of the present invention makes the process of generating work reports more efficient, reduces the burden on workers, and improves the quality of reports.

[0827] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0828] Step 1:

[0829] The server receives the voice data and work history data uploaded by the worker. Specifically, the voice data (WAV or MP3 format) recorded by the worker using smart glasses and the work history data in JSON format are sent to the server. The server saves them in an appropriate directory. The input is the voice data and work history data, and the output is the saved voice file and work history file.

[0830] Step 2:

[0831] The server converts the received and saved voice data into text data using voice recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the voice data into text data. The input is voice data, and the output is the converted text data. This text data is required for subsequent processing.

[0832] Step 3:

[0833] The server integrates the work history data with the text data generated by speech recognition. Specifically, the server reads the work history data in JSON format and combines it with the converted text data. The input is the work history data and text data, and the output is the integrated data (JSON format). This integrated data is used to generate a work report.

[0834] Step 4:

[0835] The server uses a generative AI model to automatically generate a work report based on the integrated data. Specifically, it uses OpenAI's GPT-3 to generate a detailed work report based on the contents of the integrated data. Based on the prompt, the input is the integrated data, and the output is a draft of the work report.

[0836] Step 5:

[0837] The terminal displays the automatically generated work report created by the server and provides an interface for the user to check and modify it. Specifically, the administrator checks the contents of the generated work report via a smartphone or tablet and makes any necessary modifications. The input is a draft work report, and the output is the modified work report.

[0838] Step 6:

[0839] The server saves the revised work report and sends it to relevant parties and departments as needed. Specifically, the server saves the revised work report in PDF format or other format and sends it to the email addresses of relevant departments. The input is the revised work report, and the output is the saved report and transmission log.

[0840] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0841] MODE FOR CARRYING OUT THE INVENTION

[0842] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[0843] System configuration

[0844] The system consists of the following components:

[0845] 1. Means of receiving job seeker data

[0846] 2. Means using voice recognition technology

[0847] 3. Data Integration Methods

[0848] 4. Using Generative AI Models

[0849] 5. Interface

[0850] 6. Preservation means

[0851] 7. Means of transmission

[0852] 8. Emotion Engine

[0853] Program Overview

[0854] The system receives a job seeker's resume, CV, and interview audio data, and automatically generates a recommendation letter based on them. It also incorporates an emotion engine that recognizes the user's emotions and assists in the revision process.

[0855] Operation of each method

[0856] Receiving data

[0857] Users upload job seekers' resumes, CVs, and interview audio data to the system. The server receives these data and saves them in a designated folder, capturing the necessary data.

[0858] Text data conversion using voice recognition technology

[0859] The server converts the uploaded interview audio data into text data. Specifically, it uses an AI speech recognition model to convert the audio file into text format and saves the converted text data.

[0860] Data Integration

[0861] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[0862] Automatic generation of letters of recommendation

[0863] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[0864] Interface and Emotion Engine

[0865] The server displays the generated draft recommendation letter through an interface provided to the user. The user uses this interface to confirm the contents of the recommendation letter. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state and provides suggestions for revisions to help the user revise the recommendation letter.

[0866] Reviewing and correcting letters of recommendation

[0867] As the user edits the draft, the emotion engine provides advice based on the user's emotional state. For example, if the user is feeling unhappy, it can suggest more positive wording. The revised recommendation letter is then saved to the server in its final form.

[0868] Save and send letters of recommendation

[0869] The server then converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[0870] Specific examples

[0871] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data. Next, each data is integrated and a generative AI model is used to generate a draft recommendation letter. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final recommendation letter is then saved and sent to the company.

[0872] The above is a specific embodiment of the present invention. This system improves the efficiency of recommendation letter creation and provides flexible revision support that takes into account the user's emotions.

[0873] The processing flow will be explained below.

[0874] Step 1:

[0875] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[0876] Step 2:

[0877] The server stores the uploaded resumes, CVs, and interview audio data in designated folders, making them accessible for subsequent processing.

[0878] Step 3:

[0879] The server converts the interview audio data into text data using speech recognition technology, converts the audio file into text format using an AI speech recognition model, and saves the converted text data.

[0880] Step 4:

[0881] The server combines the resumes, CVs, and converted text data to create a structured dataset that is ready to be fed into a generative AI model.

[0882] Step 5:

[0883] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[0884] Step 6:

[0885] The server displays the generated draft recommendation letter through an interface provided to the user, who can use the interface to review and modify the recommendation letter content.

[0886] Step 7:

[0887] The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. Specifically, it uses a camera and microphone to capture facial and voice data in real time and performs emotion analysis.

[0888] Step 8:

[0889] The emotion engine provides suggestions for revising the recommendation letter on the interface based on the user's emotional state. For example, if the user is feeling dissatisfied, the engine will suggest more positive expressions to help the user make revisions smoothly.

[0890] Step 9:

[0891] The user then modifies the recommendation based on the suggestions from the emotion engine. Once the modifications are complete, the final recommendation is sent to the server.

[0892] Step 10:

[0893] The server saves the final, modified version of the recommendation, which is then recorded as a finalized recommendation.

[0894] Step 11:

[0895] The server converts the final recommendation letter into the appropriate format (e.g., PDF) and prepares it for transmission to the company. Once the recommendation letter is ready for transmission, it is securely transmitted to the company.

[0896] Example 2

[0897] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0898] When job placement agencies introduce job seekers to companies, they create recommendation letters based on multiple forms of data, including the job seeker's resume, CV, and interview audio. This process is often manual, time-consuming, and prone to typos and inconsistencies. Furthermore, there is a lack of flexible correction support that takes user sentiment into account. Therefore, there is a need for a system that streamlines the recommendation letter creation and correction process and improves its accuracy.

[0899] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0900] In this invention, the server includes means for receiving data in multiple formats related to the job seeker, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for using a generative AI model to automatically generate a letter of recommendation based on the integrated data, means for providing an interface for displaying and modifying the automatically generated letter of recommendation and for recognizing a user's emotions to support the modification process, and means for saving and transmitting the modified letter of recommendation. This enables efficient creation of letters of recommendation and flexible modification support that takes the user's emotions into consideration.

[0901] "Job seekers" refer to people who are seeking a new job.

[0902] "Means for receiving data" refers to a device or method for incorporating various data, such as resumes, CVs, and audio files, input from outside into the system.

[0903] "Audio data" refers to audio files containing what a job seeker said during an interview or other occasion.

[0904] "Means for converting into text data" refers to speech recognition technology or software for converting voice data into text format.

[0905] "Means for integrating data" refers to a device or method that combines data in multiple formats (e.g., resumes, CVs, text data converted from audio data) to create a single integrated data set.

[0906] The use of a "generative AI model" refers to the use of an artificial intelligence model to automatically generate a recommendation based on the integrated data.

[0907] "Interface" refers to the user interface through which a user interacts with a system.

[0908] "Means for recognizing emotions" refers to technology and software for analyzing and determining emotions from a user's facial expressions and voice.

[0909] "Means for storing" refers to a device or method for recording the revised recommendation as a digital file.

[0910] "Transmitting means" refers to a device or method for transferring the completed recommendation letter to another terminal such as a company.

[0911] A "letter of recommendation" is a document that describes a job seeker's background and skills in order to recommend them to a company.

[0912] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[0913] System configuration

[0914] The system consists of the following components:

[0915] 1. Means of receiving job seeker data

[0916] 2. Means using voice recognition technology

[0917] 3. Data Integration Methods

[0918] 4. Using Generative AI Models

[0919] 5. Interface

[0920] 6. Preservation means

[0921] 7. Means of transmission

[0922] 8. Emotion Engine

[0923] System Operation

[0924] Receiving data

[0925] A user uploads a job seeker's resume, CV, and interview audio data to the system. The server securely receives this data using the HTTPS protocol with SSL / TLS and saves it in a specified folder. For example, the data for job seeker A is saved in the / uploaded_data / A / directory.

[0926] Text data conversion using voice recognition technology

[0927] The server converts the uploaded interview audio data into text data. Specifically, it uses AI speech recognition services such as Google Cloud Speech-to-Text to convert the audio file into text format. The converted text data is saved in the / transcripts / A / directory.

[0928] Data Integration

[0929] The server combines the resume, CV, and converted text data. This combined data is saved in JSON format in / integrated_data / A / combined_data.json and input to the generative AI model.

[0930] Automatic generation of letters of recommendation

[0931] A generative AI model (e.g., OpenAI's GPT-4) automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. A draft of the generated recommendation letter is saved in / recommendations / A / draft.txt.

[0932] Interface and Emotion Engine

[0933] The server provides the user with an interface to display the generated draft recommendation letter. The user can then review and revise the recommendation letter through a web browser. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. For example, using Microsoft Azure's Emotion API, if the user shows a dissatisfied expression, the engine will suggest more positive revisions.

[0934] Reviewing and correcting letters of recommendation

[0935] The emotion engine provides real-time feedback as users revise their draft recommendation, which is saved in / updated_recommendations / A / final.txt.

[0936] Save and send letters of recommendation

[0937] The final recommendation letter is converted into a PDF format and sent to the company. The server securely transmits it using the SMTP protocol.

[0938] Specific examples

[0939] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data using Google Cloud Speech-to-Text. The server then combines the resume, CV, and converted text data into JSON format and generates a draft letter of recommendation using OpenAI's GPT-4. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final letter of recommendation is saved and sent to the specified company.

[0940] An example of a prompt might be, "I have uploaded Applicant A's resume and interview audio data. Please transcribe the key skills and achievements the applicant mentioned during the interview into text and draft a letter of recommendation that includes them. Also, please suggest revisions based on the user's sentiment."

[0941] This system improves the efficiency of creating recommendation letters and provides flexible revision support that takes into account the user's emotions.

[0942] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0943] Step 1: Receiving Data

[0944] A user uploads a job seeker's resume, CV, and interview audio data to the system via a web interface. The server receives these data securely using SSL / TLS over the HTTPS protocol and stores them in a specified folder structure. The input data is the resume (e.g., resume.pdf), resume (e.g., cv.pdf), and audio data (e.g., interview_audio.mp3), and stores them in the / uploaded_data / {username} / directory.

[0945] Step 2: Convert audio data to text

[0946] The server converts the saved interview audio data into text data using speech recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the audio file into text format and saves the conversion result in / transcripts / {username} / transcript.txt. The input is audio data, and the output is text data.

[0947] Step 3: Integrate the data

[0948] The server combines the job seeker's resume, CV, and text data generated by speech recognition. Specifically, it compiles this data into JSON format and saves it in / integrated_data / {username} / combined_data.json. The input is the resume, CV, and text data, and the output is the combined data in JSON format.

[0949] Step 4: Auto-generate letters of recommendation

[0950] The server uses a generative AI model (e.g., OpenAI's GPT-4) based on the integrated data to automatically generate a recommendation letter. The generated draft recommendation letter is saved in / recommendations / {username} / draft.txt. The input is the integrated data JSON file, and the output is the draft recommendation letter text. Specifically, the integrated data is input as a prompt into the AI ​​model, and the resulting text is saved.

[0951] Step 5: Providing the interface and emotion engine

[0952] The server presents the generated draft recommendation letter through a user interface. The user can review and revise the recommendation letter via a web browser. An emotion engine (e.g., Microsoft Azure Emotion API) is used to analyze the user's facial expressions and voice and provide revision suggestions based on their emotional state. The input is the user's emotional data (facial expressions and voice), and the output is revision suggestions.

[0953] Step 6: Review and revise your recommendation letter

[0954] The user reviews the draft recommendation letter and makes any necessary revisions. Based on the feedback from the emotion engine, the server makes appropriate revisions. The server saves the revised recommendation letter in / updated_recommendations / {username} / final.txt. The input is the draft text and the user's revisions, and the output is the revised recommendation letter text.

[0955] Step 7: Save and submit your recommendation letter

[0956] The server converts the final recommendation letter into PDF format and sends it to the company. The input is the revised recommendation letter text and the output is a PDF file. The server securely sends the recommendation letter to the specified company using the SMTP protocol.

[0957] (Application example 2)

[0958] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0959] In brick-and-mortar stores, it is difficult to properly receive feedback from customers and efficiently and accurately generate countermeasures and messages. Furthermore, when staff manually process feedback, it takes time and the quality of responses can vary. The present invention aims to solve these problems and provide a system that improves customer service.

[0960] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0961] In this invention, the server includes means for receiving data related to customers in multiple formats, means for converting voice data from the multiple formats into text data, and means for integrating the multiple formats and the text data. This enables rapid analysis of customer feedback and automatic generation of appropriate countermeasures and messages. Furthermore, an emotion engine built into the interface helps modify countermeasures taking into account customer emotions, thereby providing consistent, high-quality customer service.

[0962] "Multiple forms of data about a job seeker" refers to different forms of data, such as a job seeker's resume, CV, and interview audio.

[0963] "Means for converting voice data into text data" refers to means for converting voice into text format using voice recognition technology.

[0964] A "data integration means" is a means for integrating multiple data formats into a single analyzable data set.

[0965] "Means for automatically generating recommendation letters" refers to means for automatically generating recommendation letters based on the integrated data using a generative AI model.

[0966] "Interface" refers to the user interface through which a user interacts with the system and reviews and modifies the generated recommendations.

[0967] The "emotion engine" is a system that analyzes a user's facial expressions and voice to recognize their emotions, and is used to assist in the recommendation letter revision process.

[0968] The "means for saving and transmitting a revised recommendation letter" refers to the means by which a user saves a revised recommendation letter in an appropriate format and transmits it to a recipient such as a company.

[0969] MODE FOR CARRYING OUT THE INVENTION

[0970] The present invention relates to a system that receives feedback from customers in a physical store and automatically generates countermeasures and messages based on that feedback. The purpose of this system is to improve customer service and increase the work efficiency of staff. Specific embodiments for implementing the present invention are described below.

[0971] System Components

[0972] 1. Data receiving means:

[0973] The server provides a means for collecting customer feedback data, specifically, receiving customer voice feedback and reviews as text data.

[0974] 2. Voice Recognition Technology:

[0975] The server uses voice recognition technology to convert the received voice data into text data, using Google Cloud's Speech-to-Text API.

[0976] 3. Data integration methods:

[0977] The server consolidates multiple forms of feedback data received from customers (audio data, text data, etc.) and converts them into a single analyzable data set.

[0978] 4. Generative AI Models:

[0979] The server uses generative AI models, such as Transformer-based generative AI, to automatically generate customer responses and messages based on the integrated data.

[0980] 5. Interface:

[0981] The server provides a user interface for the user to review and, if necessary, modify the generated message.

[0982] 6. Emotion Engine:

[0983] The interface incorporates an emotion engine that analyzes the user's emotions to provide better suggestions and advice, using, for example, a BERT-based emotion analysis model.

[0984] 7. Storage and transmission methods:

[0985] It provides a means for the user to save the modified message in an appropriate format (e.g. PDF) and send it to the desired recipients. For example, PyPDF2 is used to save the data.

[0986] Processing flow

[0987] 1. Receiving data:

[0988] When customers give feedback by voice, the user uploads the voice data to the server. Feedback by text data is also uploaded in the same way.

[0989] 2. Transcription of audio data:

[0990] The server uses Google Cloud's Speech-to-Text API to convert the audio data into text data, which is then temporarily stored.

[0991] 3. Data integration:

[0992] The server integrates the received text data and audio data and converts them into a single data set.

[0993] 4. Automatic message generation:

[0994] Based on the integrated data, the server automatically generates responses and messages using a generative AI model that takes into account customer emotions and feedback content.

[0995] 5. Check and fix in the interface:

[0996] The user can review the generated message on the interface and make any necessary edits. The emotion engine analyzes the user's facial expressions and voice to interpret their emotions and provides suggestions for editing.

[0997] 6. Storage and Transmission:

[0998] The corrected message is converted to PDF format and saved in a specified location, after which it can be sent to the customer or administrator as needed.

[0999] Specific examples

[1000] For example, if a customer provides feedback such as "The staff were very kind and helpful," the server receives this feedback as voice data and converts it into text data. If emotion analysis subsequently determines that the feedback is "joyful," the generative AI model automatically generates a message such as "Thank you for your kind words. We will continue to strive to maintain and further improve this service," and presents it to the user.

[1001] Prompt Sentence Examples

[1002] Customer feedback: 'The staff were very friendly and helpful' Based on this feedback, generate appropriate improvement suggestions and messages when the emotion is 'delighted'.

[1003] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1004] Step 1:

[1005] The server receives customer feedback data. Specifically, the customer gives voice feedback using a terminal and uploads it to the server. The input is voice data, and the output is an audio file stored on the server.

[1006] Step 2:

[1007] The server converts the received voice data into text data using Google Cloud's Speech-to-Text API. The input is voice data, speech recognition technology is applied to process the data, and the output is text data.

[1008] Step 3:

[1009] The server integrates the received text and audio data, unifying and organizing the different data formats to create an integrated dataset. The input is text and audio data, and the output is the integrated dataset.

[1010] Step 4:

[1011] The server inputs the integrated dataset into a generative AI model to automatically generate responses and messages for customers. The input is the integrated dataset, and the output is the generated message. Natural language generation is performed as a data calculation in this process.

[1012] Step 5:

[1013] The server displays the generated message on a user interface, which the user uses to review the message and make corrections if necessary. The input is the generated message, and the output is the message displayed to the user.

[1014] Step 6:

[1015] When a user edits a message, an emotion engine built into the interface analyzes the user's emotions from their facial expressions and voice and provides suggestions for editing. The input is the user's facial expression and voice data, and the output is the emotion analysis results and editing suggestions.

[1016] Step 7:

[1017] The server converts the modified message to PDF format and saves it in an appropriate location. The input is the modified message, the PDF conversion is performed as data processing, and the output is a PDF file.

[1018] Step 8:

[1019] The server sends the saved PDF file to the customer or administrator as needed. The input is the PDF file and the output is the message sent.

[1020] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1021] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1022] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1023] [Fourth embodiment]

[1024] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1025] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1027] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1028] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1029] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1031] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1032] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1033] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1035] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1037] MODE FOR CARRYING OUT THE INVENTION

[1038] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[1039] System configuration

[1040] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[1041] Program Overview

[1042] The system receives a job seeker's resume, CV, and audio recording of the interview, and automatically generates a recommendation letter based on these. The generated recommendation letter is then finalized after user confirmation and correction, and sent to the company.

[1043] Operation of each method

[1044] Receiving data

[1045] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio (WAV or MP3 format) to the system. The server receives these data and saves the files in the appropriate format. This process ensures that the necessary information is captured in the system.

[1046] Text data conversion using voice recognition technology

[1047] The server uses voice recognition technology to convert the uploaded interview audio data into text data. Specifically, it uses an AI voice recognition model to convert the audio data into text format, which will be used in further processes.

[1048] Data Integration

[1049] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[1050] Automatic generation of letters of recommendation

[1051] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[1052] Reviewing and correcting letters of recommendation

[1053] The user can check the generated draft recommendation letter and, if necessary, can edit the content to create the final recommendation letter. After the edits are reflected, the server saves the final recommendation letter.

[1054] Save and send letters of recommendation

[1055] The server stores the final recommendation in the appropriate format (e.g. PDF) and then sends it to the company if required, ensuring data security and confidentiality.

[1056] Specific examples

[1057] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives this data and transcribes the audio data. Next, it integrates each data and generates a draft recommendation letter using a generative AI model. The user then reviews the draft and makes corrections. The final revised recommendation letter is saved and ready to be sent to the company.

[1058] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[1059] The processing flow will be explained below.

[1060] Step 1:

[1061] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[1062] Step 2:

[1063] The server stores the uploaded resumes, CVs, and interview audio data. Specifically, the server places each data in a designated folder so that it can be accessed in subsequent processes.

[1064] Step 3:

[1065] The server converts the interview audio data into text data using speech recognition technology. This process uses an AI speech recognition model to convert the audio file into text format and save the converted text data.

[1066] Step 4:

[1067] The server then combines the received and converted data, specifically the resumes, CVs, and text data, into a single dataset that is ready to be fed into a generative AI model.

[1068] Step 5:

[1069] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[1070] Step 6:

[1071] The server displays the generated draft recommendation letter through an interface provided to the user, who can use this interface to review the recommendation letter and make corrections as necessary.

[1072] Step 7:

[1073] After the user modifies the draft, the server saves the final version of the recommendation, which records the changes as the final modified recommendation.

[1074] Step 8:

[1075] The server converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[1076] Example 1

[1077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1078] Creating recommendation letters, which are required when introducing job seekers to companies, is not only time-consuming and laborious, but also has the potential for inaccurate information integration and analysis. For this reason, a system for creating recommendation letters efficiently and accurately is needed. Furthermore, an environment is needed that centralizes the processes of transcribing audio data, integrating data in multiple formats, and automatically generating recommendation letters based on that data, while allowing users to easily edit and review them.

[1079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1080] In this invention, the server includes means for receiving multiple types of data about the job seeker, means for using voice recognition software to convert voice data into text data, means for integrating the multiple types of data and the text data into a dataset, means for using a generative AI model to automatically generate a recommendation letter based on the integrated dataset, means for providing an interface for displaying and modifying the automatically generated recommendation letter, and means for saving, formatting, and transmitting the modified recommendation letter. This streamlines the recommendation letter creation process and enables the generation of more accurate recommendation letters. Furthermore, users can easily review the generated recommendation letter and modify it as necessary.

[1081] The "means for receiving data" refers to a means for uploading and storing multiple forms of data related to job seekers, such as resumes, CVs, and audio data, into the system.

[1082] "Means for converting voice data into text data" means means for converting uploaded voice data into text format using voice recognition software.

[1083] The "means of integrating data" refers to a means of combining resumes, CVs, and text data converted from audio data into a single data set.

[1084] A "generative AI model" is an artificial intelligence model that automatically generates recommendation letters based on an integrated dataset.

[1085] The "means for providing an interface" refers to a means for providing a screen for a user to display the generated recommendation letter and to modify it as necessary.

[1086] "Means for saving and sending the recommendation letter" refers to the means for saving the final recommendation letter in an appropriate format (e.g., PDF) and sending it to the company.

[1087] A "prompt" is a specific statement that instructs the generative AI model and provides the information necessary to generate a recommendation letter.

[1088] MODE FOR CARRYING OUT THE INVENTION

[1089] The present invention provides a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. An embodiment of the system will be described in detail below.

[1090] System configuration

[1091] The system includes a means for receiving job seeker data, a means for using voice recognition technology, a means for integrating data, a means for using a generative AI model, an interface, a means for storing data, and a means for transmitting data.

[1092] Operation of each method

[1093] Receiving data

[1094] Users upload job seekers' resumes (PDF or Word format), resumes (PDF or Word format), and interview audio data (WAV or MP3 format) to the system. The server receives this data and saves each file in the appropriate folder. This process ensures that the necessary information is captured in the system.

[1095] Text data conversion using voice recognition technology

[1096] The server converts the stored interview voice data into text data using speech recognition software (e.g., Google Speech-to-Text API). The converted text data is temporarily stored and used for subsequent processing.

[1097] Data Integration

[1098] The server combines the resumes, CVs, and converted text data, and stores this combined dataset in JSON format, which is then fed into a generative AI model.

[1099] Automatic generation of letters of recommendation

[1100] The generative AI model automatically generates recommendation letters based on the integrated dataset. The recommendation letters generated using specific prompts reflect the job seeker's skills and experience and include suggestions for the company. An example of a prompt is, "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience."

[1101] Reviewing and correcting letters of recommendation

[1102] The user can review the generated recommendation letter through a web interface, which provides editing functionality to revise the content of the recommendation letter as needed and finalize it.

[1103] Save and send letters of recommendation

[1104] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. A function is implemented to automatically send the saved recommendation letter to the specified company's email address.

[1105] Specific examples

[1106] For example, if a user uploads applicant A's resume (PDF format), CV (Word format), and interview audio data (MP3 format) to the system, the server receives these data and converts the audio data into text using an AI speech recognition model. The server then integrates the resume, CV, and text data and generates a draft letter of recommendation using a generative AI model. The user reviews the draft and makes any necessary revisions. The revised letter of recommendation is saved on the server and can be sent to the company in PDF format.

[1107] The above is a specific embodiment for carrying out the present invention. This system makes it possible to efficiently create recommendation letters and to safely handle personal information.

[1108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1109] Step 1:

[1110] The user uploads the job seeker's resume (PDF or Word format), resume (PDF or Word format), and interview audio data (WAV or MP3 format) from their terminal to the system. These three types of files are taken in as input. The server receives the data and saves the files in the appropriate folder. The output of this step is various data files saved in the system. Specifically, the server receives the upload request, checks the files, and saves them in the appropriate directory.

[1111] Step 2:

[1112] The server retrieves the saved interview audio data and converts it into text data using speech recognition software (e.g., Google Speech-to-Text API). The input is the interview audio data file. The server sends this audio file to the speech recognition API and obtains text data. The output of this step is the text data converted from the audio data. Specifically, the server passes the audio data to the API and saves the returned text data.

[1113] Step 3:

[1114] The server integrates the resumes, CVs, and converted text data. The inputs are the various data files and text data saved in the previous step. The server integrates them into a single dataset and saves it in JSON format. The output of this step is the integrated dataset. Specifically, it converts the contents of PDF and Word files into text format and combines them into a single JSON file.

[1115] Step 4:

[1116] The generative AI model automatically generates a recommendation letter based on the integrated dataset. The integrated dataset and a prompt are used as input. An example of a specific prompt is input to the model: "Based on this job seeker's resume, CV, and interview details, please generate a recommendation letter that highlights the job seeker's skills and experience." The output is a draft of the generated recommendation letter. Specific operations include the process by which the AI ​​model generates a recommendation letter based on the set prompt.

[1117] Step 5:

[1118] The user reviews the generated recommendation letter through a web interface. The generated draft recommendation letter is used as input. The interface provides an editing function to modify the recommendation letter content as needed. The output is the final recommendation letter that has been reviewed and modified by the user. Specific operations include the process in which the user reviews the recommendation letter content and makes modifications on the interface.

[1119] Step 6:

[1120] The server saves the final recommendation letter in PDF format after the user has confirmed and corrected it. The final recommendation letter corrected by the user is used as input. A function is then implemented to automatically send the saved recommendation letter to the specified company's email address. The output is the recommendation letter sent to the company. Specific operations include converting the corrected recommendation letter to a PDF file, saving it on the server, and sending it to the company by email.

[1121] (Application example 1)

[1122] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1123] The present invention relates to a system that efficiently records the work content and progress of workers in a factory, allowing managers to quickly review and correct work reports. Conventional methods require workers to individually write out work reports, which is time-consuming and labor-intensive, and the recorded content is often inaccurate or inconsistent. Furthermore, managers must spend a lot of time reviewing and correcting the reports. The present invention aims to solve these problems and streamline the process of generating work reports.

[1124] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1125] In this invention, the server includes means for receiving data in multiple formats related to workers, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for automatically generating a work report, means for providing an interface for displaying and modifying the automatically generated work report, and means for saving and transmitting the modified work report, thereby reducing the burden on workers and enabling efficient and consistent creation of work reports.

[1126] "Workers" refers to personnel who perform practical work in a work environment such as a factory or work site.

[1127] "Voice data" refers to acoustic signals that record the statements, instructions, and reports of workers.

[1128] "Character data" refers to information in text format obtained by converting voice data.

[1129] "Integrated data" refers to an information set that brings together data in multiple formats and converted character data.

[1130] A "work report" refers to a document that records the work content, progress, problems, etc. of a worker.

[1131] "Interface" refers to the operation screens and functions that allow users to access and operate the system.

[1132] "Server" refers to a computer system that receives, processes, stores, and transmits data.

[1133] "Voice recognition technology" refers to the technology that analyzes voice data and converts it into text data.

[1134] A "generative AI model" refers to an artificial intelligence model that automatically generates the necessary documents and reports based on given input data.

[1135] "Correction" refers to checking the contents of generated reports and documents and making changes to correct any errors or omissions.

[1136] The present invention provides a system that records the work content and progress of workers in a factory and allows managers to quickly check and correct work reports. The system has the function of collecting voice reports from workers using smart glasses or other devices while they are working, and automatically generating work reports using voice recognition technology and a generative AI model.

[1137] System configuration

[1138] Worker devices: Use devices that allow voice input and video recording, such as smart glasses or headsets.

[1139] Server: Processes data using speech recognition technology and generative AI models, and manages automatically generated work reports.

[1140] Interface terminal: Provides an interface that allows managers to check and modify generated reports using a smartphone or tablet.

[1141] Program Overview and Procedures

[1142] Receiving data

[1143] The server receives the voice data (WAV or MP3 format) and work history data (JSON format) uploaded by the worker and stores this data in an appropriate format.

[1144] Text data conversion using voice recognition technology

[1145] The server converts the uploaded voice data into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text API).

[1146] Data Integration

[1147] The server integrates the work history data with the text data converted from the voice data, and this integrated data is used to generate a work report.

[1148] Automatic generation of work reports

[1149] Use a generative AI model (e.g., OpenAI's GPT-3) to automatically generate a draft working report based on the integrated data.

[1150] Check and correct work reports

[1151] The manager can check the automatically generated draft work report on their device (smartphone or tablet) and make any necessary corrections through the interface.

[1152] Save and send work reports

[1153] The final revised work report is saved on the server and sent to the relevant departments and parties as needed.

[1154] Specific examples

[1155] For example, while working, a worker wearing smart glasses may report by voice, "Starting work on ~. Progress is going well, no problems." This voice data is uploaded to a server and converted into text data using speech recognition technology. It is then integrated with work history data and input into a generative AI model. The model generates a draft work report based on prompt sentences such as the following:

[1156] Prompt Sentence Examples

[1157] Work history: XX Factory, Work content: Machine inspection, Progress: 100%, No special notes.

[1158] Converted character data: Starting work on ~. Good progress, no problems.

[1159] Using this prompt as input, a draft work report is generated, after which the manager can review the generated report on a smartphone or tablet, make any necessary corrections, save it, and send it to the relevant parties.

[1160] In this way, the system of the present invention makes the process of generating work reports more efficient, reduces the burden on workers, and improves the quality of reports.

[1161] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1162] Step 1:

[1163] The server receives the voice data and work history data uploaded by the worker. Specifically, the voice data (WAV or MP3 format) recorded by the worker using smart glasses and the work history data in JSON format are sent to the server. The server saves them in an appropriate directory. The input is the voice data and work history data, and the output is the saved voice file and work history file.

[1164] Step 2:

[1165] The server converts the received and saved voice data into text data using voice recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the voice data into text data. The input is voice data, and the output is the converted text data. This text data is required for subsequent processing.

[1166] Step 3:

[1167] The server integrates the work history data with the text data generated by speech recognition. Specifically, the server reads the work history data in JSON format and combines it with the converted text data. The input is the work history data and text data, and the output is the integrated data (JSON format). This integrated data is used to generate a work report.

[1168] Step 4:

[1169] The server uses a generative AI model to automatically generate a work report based on the integrated data. Specifically, it uses OpenAI's GPT-3 to generate a detailed work report based on the contents of the integrated data. Based on the prompt, the input is the integrated data, and the output is a draft of the work report.

[1170] Step 5:

[1171] The terminal displays the automatically generated work report created by the server and provides an interface for the user to check and modify it. Specifically, the administrator checks the contents of the generated work report via a smartphone or tablet and makes any necessary modifications. The input is a draft work report, and the output is the modified work report.

[1172] Step 6:

[1173] The server saves the revised work report and sends it to relevant parties and departments as needed. Specifically, the server saves the revised work report in PDF format or other format and sends it to the email addresses of relevant departments. The input is the revised work report, and the output is the saved report and transmission log.

[1174] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1175] MODE FOR CARRYING OUT THE INVENTION

[1176] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[1177] System configuration

[1178] The system consists of the following components:

[1179] 1. Means of receiving job seeker data

[1180] 2. Means using voice recognition technology

[1181] 3. Data Integration Methods

[1182] 4. Using Generative AI Models

[1183] 5. Interface

[1184] 6. Preservation means

[1185] 7. Means of transmission

[1186] 8. Emotion Engine

[1187] Program Overview

[1188] The system receives a job seeker's resume, CV, and interview audio data, and automatically generates a recommendation letter based on them. It also incorporates an emotion engine that recognizes the user's emotions and assists in the revision process.

[1189] Operation of each method

[1190] Receiving data

[1191] Users upload job seekers' resumes, CVs, and interview audio data to the system. The server receives these data and saves them in a designated folder, capturing the necessary data.

[1192] Text data conversion using voice recognition technology

[1193] The server converts the uploaded interview audio data into text data. Specifically, it uses an AI speech recognition model to convert the audio file into text format and saves the converted text data.

[1194] Data Integration

[1195] The server combines the resume, CV, and converted text data, and this combined data is fed into a generative AI model to generate letters of recommendation.

[1196] Automatic generation of letters of recommendation

[1197] The generative AI model automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. This process results in a high-quality draft recommendation letter.

[1198] Interface and Emotion Engine

[1199] The server displays the generated draft recommendation letter through an interface provided to the user. The user uses this interface to confirm the contents of the recommendation letter. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state and provides suggestions for revisions to help the user revise the recommendation letter.

[1200] Reviewing and correcting letters of recommendation

[1201] As the user edits the draft, the emotion engine provides advice based on the user's emotional state. For example, if the user is feeling unhappy, it can suggest more positive wording. The revised recommendation letter is then saved to the server in its final form.

[1202] Save and send letters of recommendation

[1203] The server then converts the final recommendation into the appropriate format (e.g., PDF) and prepares it for transmission to the company, ensuring that the recommendation is securely delivered to the designated company.

[1204] Specific examples

[1205] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data. Next, each data is integrated and a generative AI model is used to generate a draft recommendation letter. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final recommendation letter is then saved and sent to the company.

[1206] The above is a specific embodiment of the present invention. This system improves the efficiency of recommendation letter creation and provides flexible revision support that takes into account the user's emotions.

[1207] The processing flow will be explained below.

[1208] Step 1:

[1209] The user uploads the job seeker's resume, CV, and interview audio data through the system interface, and the system receives the job seeker data.

[1210] Step 2:

[1211] The server stores the uploaded resumes, CVs, and interview audio data in designated folders, making them accessible for subsequent processing.

[1212] Step 3:

[1213] The server converts the interview audio data into text data using speech recognition technology, converts the audio file into text format using an AI speech recognition model, and saves the converted text data.

[1214] Step 4:

[1215] The server combines the resumes, CVs, and converted text data to create a structured dataset that is ready to be fed into a generative AI model.

[1216] Step 5:

[1217] The server inputs the integrated data into a generative AI model to automatically generate a draft recommendation letter, which reflects the job seeker's skills and experience based on the integrated data.

[1218] Step 6:

[1219] The server displays the generated draft recommendation letter through an interface provided to the user, who can use the interface to review and modify the recommendation letter content.

[1220] Step 7:

[1221] The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. Specifically, it uses a camera and microphone to capture facial and voice data in real time and performs emotion analysis.

[1222] Step 8:

[1223] The emotion engine provides suggestions for revising the recommendation letter on the interface based on the user's emotional state. For example, if the user is feeling dissatisfied, the engine will suggest more positive expressions to help the user make revisions smoothly.

[1224] Step 9:

[1225] The user then modifies the recommendation based on the suggestions from the emotion engine. Once the modifications are complete, the final recommendation is sent to the server.

[1226] Step 10:

[1227] The server saves the final, modified version of the recommendation, which is then recorded as a finalized recommendation.

[1228] Step 11:

[1229] The server converts the final recommendation letter into the appropriate format (e.g., PDF) and prepares it for transmission to the company. Once the recommendation letter is ready for transmission, it is securely transmitted to the company.

[1230] Example 2

[1231] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1232] When job placement agencies introduce job seekers to companies, they create recommendation letters based on multiple forms of data, including the job seeker's resume, CV, and interview audio. This process is often manual, time-consuming, and prone to typos and inconsistencies. Furthermore, there is a lack of flexible correction support that takes user sentiment into account. Therefore, there is a need for a system that streamlines the recommendation letter creation and correction process and improves its accuracy.

[1233] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1234] In this invention, the server includes means for receiving data in multiple formats related to the job seeker, means for converting voice data into text data, means for integrating the data in multiple formats and the text data, means for using a generative AI model to automatically generate a letter of recommendation based on the integrated data, means for providing an interface for displaying and modifying the automatically generated letter of recommendation and for recognizing a user's emotions to support the modification process, and means for saving and transmitting the modified letter of recommendation. This enables efficient creation of letters of recommendation and flexible modification support that takes the user's emotions into consideration.

[1235] "Job seekers" refer to people who are seeking a new job.

[1236] "Means for receiving data" refers to a device or method for incorporating various data, such as resumes, CVs, and audio files, input from outside into the system.

[1237] "Audio data" refers to audio files containing what a job seeker said during an interview or other occasion.

[1238] "Means for converting into text data" refers to speech recognition technology or software for converting voice data into text format.

[1239] "Means for integrating data" refers to a device or method that combines data in multiple formats (e.g., resumes, CVs, text data converted from audio data) to create a single integrated data set.

[1240] The use of a "generative AI model" refers to the use of an artificial intelligence model to automatically generate a recommendation based on the integrated data.

[1241] "Interface" refers to the user interface through which a user interacts with a system.

[1242] "Means for recognizing emotions" refers to technology and software for analyzing and determining emotions from a user's facial expressions and voice.

[1243] "Means for storing" refers to a device or method for recording the revised recommendation as a digital file.

[1244] "Transmitting means" refers to a device or method for transferring the completed recommendation letter to another terminal such as a company.

[1245] A "letter of recommendation" is a document that describes a job seeker's background and skills in order to recommend them to a company.

[1246] This invention relates to a system for efficiently creating letters of recommendation required when a recruitment agent introduces a job seeker to a company. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it provides advanced support for the recommendation letter revision process.

[1247] System configuration

[1248] The system consists of the following components:

[1249] 1. Means of receiving job seeker data

[1250] 2. Means using voice recognition technology

[1251] 3. Data Integration Methods

[1252] 4. Using Generative AI Models

[1253] 5. Interface

[1254] 6. Preservation means

[1255] 7. Means of transmission

[1256] 8. Emotion Engine

[1257] System Operation

[1258] Receiving data

[1259] A user uploads a job seeker's resume, CV, and interview audio data to the system. The server securely receives this data using the HTTPS protocol with SSL / TLS and saves it in a specified folder. For example, the data for job seeker A is saved in the / uploaded_data / A / directory.

[1260] Text data conversion using voice recognition technology

[1261] The server converts the uploaded interview audio data into text data. Specifically, it uses AI speech recognition services such as Google Cloud Speech-to-Text to convert the audio file into text format. The converted text data is saved in the / transcripts / A / directory.

[1262] Data Integration

[1263] The server combines the resume, CV, and converted text data. This combined data is saved in JSON format in / integrated_data / A / combined_data.json and input to the generative AI model.

[1264] Automatic generation of letters of recommendation

[1265] A generative AI model (e.g., OpenAI's GPT-4) automatically generates a recommendation letter based on the integrated data. The generated recommendation letter reflects the job seeker's skills and experience and includes suggestions for the company. A draft of the generated recommendation letter is saved in / recommendations / A / draft.txt.

[1266] Interface and Emotion Engine

[1267] The server provides the user with an interface to display the generated draft recommendation letter. The user can then review and revise the recommendation letter through a web browser. The emotion engine analyzes the user's facial expressions and voice to determine their emotional state. For example, using Microsoft Azure's Emotion API, if the user shows a dissatisfied expression, the engine will suggest more positive revisions.

[1268] Reviewing and correcting letters of recommendation

[1269] The emotion engine provides real-time feedback as users revise their draft recommendation, which is saved in / updated_recommendations / A / final.txt.

[1270] Save and send letters of recommendation

[1271] The final recommendation letter is converted into a PDF format and sent to the company. The server securely transmits it using the SMTP protocol.

[1272] Specific examples

[1273] For example, if a user uploads applicant A's resume, CV, and interview audio data to the system, the server receives these data and transcribes the audio data using Google Cloud Speech-to-Text. The server then combines the resume, CV, and converted text data into JSON format and generates a draft letter of recommendation using OpenAI's GPT-4. As the user reviews and revises the draft, the emotion engine recognizes the user's emotions and provides appropriate revision suggestions. The final letter of recommendation is saved and sent to the specified company.

[1274] An example of a prompt might be, "I have uploaded Applicant A's resume and interview audio data. Please transcribe the key skills and achievements the applicant mentioned during the interview into text and draft a letter of recommendation that includes them. Also, please suggest revisions based on the user's sentiment."

[1275] This system improves the efficiency of creating recommendation letters and provides flexible revision support that takes into account the user's emotions.

[1276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1277] Step 1: Receiving Data

[1278] A user uploads a job seeker's resume, CV, and interview audio data to the system via a web interface. The server receives these data securely using SSL / TLS over the HTTPS protocol and stores them in a specified folder structure. The input data is the resume (e.g., resume.pdf), resume (e.g., cv.pdf), and audio data (e.g., interview_audio.mp3), and stores them in the / uploaded_data / {username} / directory.

[1279] Step 2: Convert audio data to text

[1280] The server converts the saved interview audio data into text data using speech recognition technology. Specifically, it uses the Google Cloud Speech-to-Text API to convert the audio file into text format and saves the conversion result in / transcripts / {username} / transcript.txt. The input is audio data, and the output is text data.

[1281] Step 3: Integrate the data

[1282] The server combines the job seeker's resume, CV, and text data generated by speech recognition. Specifically, it compiles this data into JSON format and saves it in / integrated_data / {username} / combined_data.json. The input is the resume, CV, and text data, and the output is the combined data in JSON format.

[1283] Step 4: Auto-generate letters of recommendation

[1284] The server uses a generative AI model (e.g., OpenAI's GPT-4) based on the integrated data to automatically generate a recommendation letter. The generated draft recommendation letter is saved in / recommendations / {username} / draft.txt. The input is the integrated data JSON file, and the output is the draft recommendation letter text. Specifically, the integrated data is input as a prompt into the AI ​​model, and the resulting text is saved.

[1285] Step 5: Providing the interface and emotion engine

[1286] The server presents the generated draft recommendation letter through a user interface. The user can review and revise the recommendation letter via a web browser. An emotion engine (e.g., Microsoft Azure Emotion API) is used to analyze the user's facial expressions and voice and provide revision suggestions based on their emotional state. The input is the user's emotional data (facial expressions and voice), and the output is revision suggestions.

[1287] Step 6: Review and revise your recommendation letter

[1288] The user reviews the draft recommendation letter and makes any necessary revisions. Based on the feedback from the emotion engine, the server makes appropriate revisions. The server saves the revised recommendation letter in / updated_recommendations / {username} / final.txt. The input is the draft text and the user's revisions, and the output is the revised recommendation letter text.

[1289] Step 7: Save and submit your recommendation letter

[1290] The server converts the final recommendation letter into PDF format and sends it to the company. The input is the revised recommendation letter text and the output is a PDF file. The server securely sends the recommendation letter to the specified company using the SMTP protocol.

[1291] (Application example 2)

[1292] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1293] In brick-and-mortar stores, it is difficult to properly receive feedback from customers and efficiently and accurately generate countermeasures and messages. Furthermore, when staff manually process feedback, it takes time and the quality of responses can vary. The present invention aims to solve these problems and provide a system that improves customer service.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1295] In this invention, the server includes means for receiving data related to customers in multiple formats, means for converting voice data from the multiple formats into text data, and means for integrating the multiple formats and the text data. This enables rapid analysis of customer feedback and automatic generation of appropriate countermeasures and messages. Furthermore, an emotion engine built into the interface helps modify countermeasures taking into account customer emotions, thereby providing consistent, high-quality customer service.

[1296] "Multiple forms of data about a job seeker" refers to different forms of data, such as a job seeker's resume, CV, and interview audio.

[1297] "Means for converting voice data into text data" refers to means for converting voice into text format using voice recognition technology.

[1298] A "data integration means" is a means for integrating multiple data formats into a single analyzable data set.

[1299] "Means for automatically generating recommendation letters" refers to means for automatically generating recommendation letters based on the integrated data using a generative AI model.

[1300] "Interface" refers to the user interface through which a user interacts with the system and reviews and modifies the generated recommendations.

[1301] The "emotion engine" is a system that analyzes a user's facial expressions and voice to recognize their emotions, and is used to assist in the recommendation letter revision process.

[1302] The "means for saving and transmitting a revised recommendation letter" refers to the means by which a user saves a revised recommendation letter in an appropriate format and transmits it to a recipient such as a company.

[1303] MODE FOR CARRYING OUT THE INVENTION

[1304] The present invention relates to a system that receives feedback from customers in a physical store and automatically generates countermeasures and messages based on that feedback. The purpose of this system is to improve customer service and increase the work efficiency of staff. Specific embodiments for implementing the present invention are described below.

[1305] System Components

[1306] 1. Data receiving means:

[1307] The server provides a means for collecting customer feedback data, specifically, receiving customer voice feedback and reviews as text data.

[1308] 2. Voice Recognition Technology:

[1309] The server uses voice recognition technology to convert the received voice data into text data, using Google Cloud's Speech-to-Text API.

[1310] 3. Data integration methods:

[1311] The server consolidates multiple forms of feedback data received from customers (audio data, text data, etc.) and converts them into a single analyzable data set.

[1312] 4. Generative AI Models:

[1313] The server uses generative AI models, such as Transformer-based generative AI, to automatically generate customer responses and messages based on the integrated data.

[1314] 5. Interface:

[1315] The server provides a user interface for the user to review and, if necessary, modify the generated message.

[1316] 6. Emotion Engine:

[1317] The interface incorporates an emotion engine that analyzes the user's emotions to provide better suggestions and advice, using, for example, a BERT-based emotion analysis model.

[1318] 7. Storage and transmission methods:

[1319] It provides a means for the user to save the modified message in an appropriate format (e.g. PDF) and send it to the desired recipients. For example, PyPDF2 is used to save the data.

[1320] Processing flow

[1321] 1. Receiving data:

[1322] When customers give feedback by voice, the user uploads the voice data to the server. Feedback by text data is also uploaded in the same way.

[1323] 2. Transcription of audio data:

[1324] The server uses Google Cloud's Speech-to-Text API to convert the audio data into text data, which is then temporarily stored.

[1325] 3. Data integration:

[1326] The server integrates the received text data and audio data and converts them into a single data set.

[1327] 4. Automatic message generation:

[1328] Based on the integrated data, the server automatically generates responses and messages using a generative AI model that takes into account customer emotions and feedback content.

[1329] 5. Check and fix in the interface:

[1330] The user can review the generated message on the interface and make any necessary edits. The emotion engine analyzes the user's facial expressions and voice to interpret their emotions and provides suggestions for editing.

[1331] 6. Storage and Transmission:

[1332] The corrected message is converted to PDF format and saved in a specified location, after which it can be sent to the customer or administrator as needed.

[1333] Specific examples

[1334] For example, if a customer provides feedback such as "The staff were very kind and helpful," the server receives this feedback as voice data and converts it into text data. If emotion analysis subsequently determines that the feedback is "joyful," the generative AI model automatically generates a message such as "Thank you for your kind words. We will continue to strive to maintain and further improve this service," and presents it to the user.

[1335] Prompt Sentence Examples

[1336] Customer feedback: 'The staff were very friendly and helpful' Based on this feedback, generate appropriate improvement suggestions and messages when the emotion is 'delighted'.

[1337] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1338] Step 1:

[1339] The server receives customer feedback data. Specifically, the customer gives voice feedback using a terminal and uploads it to the server. The input is voice data, and the output is an audio file stored on the server.

[1340] Step 2:

[1341] The server converts the received voice data into text data using Google Cloud's Speech-to-Text API. The input is voice data, speech recognition technology is applied to process the data, and the output is text data.

[1342] Step 3:

[1343] The server integrates the received text and audio data, unifying and organizing the different data formats to create an integrated dataset. The input is text and audio data, and the output is the integrated dataset.

[1344] Step 4:

[1345] The server inputs the integrated dataset into a generative AI model to automatically generate responses and messages for customers. The input is the integrated dataset, and the output is the generated message. Natural language generation is performed as a data calculation in this process.

[1346] Step 5:

[1347] The server displays the generated message on a user interface, which the user uses to review the message and make corrections if necessary. The input is the generated message, and the output is the message displayed to the user.

[1348] Step 6:

[1349] When a user edits a message, an emotion engine built into the interface analyzes the user's emotions from their facial expressions and voice and provides suggestions for editing. The input is the user's facial expression and voice data, and the output is the emotion analysis results and editing suggestions.

[1350] Step 7:

[1351] The server converts the modified message to PDF format and saves it in an appropriate location. The input is the modified message, the PDF conversion is performed as data processing, and the output is a PDF file.

[1352] Step 8:

[1353] The server sends the saved PDF file to the customer or administrator as needed. The input is the PDF file and the output is the message sent.

[1354] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1355] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1356] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1357] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1358] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1359] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1360] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1361] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1362] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1363] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1364] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1365] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1366] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1367] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1368] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1369] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1370] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1371] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1372] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1373] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1374] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1375] The following is further disclosed regarding the above embodiment.

[1376] (Claim 1)

[1377] means for receiving multiple forms of data regarding job seekers;

[1378] means for converting voice data from among the plurality of data formats into character data;

[1379] means for integrating the data in the plurality of formats and the character data;

[1380] means for automatically generating a recommendation letter based on the integrated data;

[1381] means for providing an interface for viewing and modifying the automatically generated recommendation;

[1382] The system includes means for storing and transmitting the modified recommendation.

[1383] (Claim 2)

[1384] 2. The system according to claim 1, wherein a voice recognition technique is used as a means for converting the voice data into character data.

[1385] (Claim 3)

[1386] The system of claim 1, characterized in that a generative AI model is used as a means for automatically generating the recommendation letter.

[1387] "Example 1"

[1388] (Claim 1)

[1389] means for receiving multiple forms of data regarding job seekers;

[1390] means for converting voice data from among the plurality of data formats into character data;

[1391] means for aggregating the multiple types of data and the character data into a data set;

[1392] a means for using a generative AI model to automatically generate a recommendation based on the integrated dataset;

[1393] means for providing an interface for viewing and modifying the automatically generated recommendation;

[1394] and a system including means for storing, formatting and transmitting said modified recommendation letter.

[1395] (Claim 2)

[1396] 2. The system according to claim 1, wherein the means for converting the voice data into character data is voice recognition software.

[1397] (Claim 3)

[1398] 10. The system of claim 1, characterized by a generative AI model that uses prompt sentences when automatically generating the recommendation.

[1399] "Application Example 1"

[1400] (Claim 1)

[1401] means for receiving a plurality of forms of data relating to the worker;

[1402] means for converting voice data from among the plurality of data formats into character data;

[1403] means for integrating the data in the plurality of formats and the character data;

[1404] means for automatically generating a work report based on the integrated data;

[1405] means for providing an interface for viewing and modifying the automatically generated work report;

[1406] The system includes means for storing and transmitting the modified work report.

[1407] (Claim 2)

[1408] 2. The system according to claim 1, wherein a voice recognition technique is used as a means for converting the voice data into character data.

[1409] (Claim 3)

[1410] The system according to claim 1, characterized in that a generative AI model is used as a means for automatically generating the work report.

[1411] "Example 2: Combining Emotion Engines"

[1412] (Claim 1)

[1413] means for receiving multiple forms of data regarding job seekers;

[1414] means for converting voice data from among the plurality of data formats into character data;

[1415] means for integrating the data in the plurality of formats and the character data;

[1416] a means for using a generative AI model to automatically generate a recommendation letter based on the integrated data;

[1417] providing an interface for viewing and modifying the automatically generated recommendation, and means for recognizing user sentiment to assist in the modification process;

[1418] The system includes means for storing and transmitting the modified recommendation.

[1419] (Claim 2)

[1420] 2. The system according to claim 1, wherein a voice recognition technique is used as a means for converting the voice data into character data.

[1421] (Claim 3)

[1422] The system of claim 1, characterized in that a generative AI model is used as a means for automatically generating the recommendation letter.

[1423] "Application example 2 when combining emotion engines"

[1424] (Claim 1)

[1425] means for receiving multiple forms of data regarding job seekers;

[1426] means for converting voice data from among the plurality of data formats into character data;

[1427] means for integrating the data in the plurality of formats and the character data;

[1428] means for automatically generating a recommendation letter based on the integrated data;

[1429] means for providing an interface for viewing and modifying the automatically generated recommendation;

[1430] means for assisting the recommendation revision process using an emotion engine integrated into said interface;

[1431] The system includes means for storing and transmitting the modified recommendation.

[1432] (Claim 2)

[1433] 2. The system according to claim 1, wherein a voice recognition technique is used as a means for converting the voice data into character data.

[1434] (Claim 3)

[1435] The system of claim 1, characterized in that a generative AI model is used as a means for automatically generating the recommendation letter. [Explanation of symbols]

[1436] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving multiple forms of data regarding job seekers; means for converting voice data from among the plurality of data formats into character data; means for integrating the data in the plurality of formats and the character data; means for automatically generating a recommendation letter based on the integrated data; means for providing an interface for viewing and modifying the automatically generated recommendation; The system includes means for storing and transmitting the modified recommendation.

2. 2. The system according to claim 1, wherein a voice recognition technique is used as a means for converting the voice data into character data.

3. The system according to claim 1, characterized in that a generative AI model is used as a means for automatically generating the recommendation letter.

Images