System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving customer information and a conversation scenario from a user; means for generating a call list and a call schedule based on the customer information; means for automatically placing a call and executing a preset introductory talk when a customer answers; means for analyzing customer responses and selecting and executing an optimal switching talk; means for recording call content and results and generating data for analysis; and means for allowing a user to review the recorded call content and data for analysis.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional telemarketing requires manual work such as creating customer lists, making phone calls, and recording and analyzing conversations, which takes a lot of time and effort. In addition, the lack of standardized conversation scripts leads to variations in the quality of responses from each agent, reducing the efficiency of customer development. [Means for solving the problem]

[0005] The present invention provides a system that receives customer information and conversation scenarios from users, automatically makes phone calls, and conducts introductory and transitional conversations based on actual conversations with customers. This system generates a call list and schedule based on customer information, and after the call, analyzes the customer's response in real time to select the optimal transitional conversation. Furthermore, the system records the call content and results and generates analytical data based on them, allowing users to efficiently and effectively develop new customers.

[0006] "User" refers to the entity that uses this system to input customer information and automate telemarketing.

[0007] "Customer information" refers to basic information such as the name, telephone number, and company name of the customer who is the target of the telemarketing call.

[0008] A "conversation scenario" refers to a standardized talk script used in telemarketing, such as an introductory talk and transition talk.

[0009] The "call list" refers to a list of people to call that is created based on customer information entered by the user.

[0010] "Dialing schedule" refers to a plan that sets the time and order for automatic dialing based on a dialing list.

[0011] "Introductory talk" refers to a standard phrase that includes a self-introduction and proposal that is first used when a customer answers the phone.

[0012] "Switching talk" refers to standard phrases that allow appropriate responses or next actions to be taken in response to customer responses.

[0013] "Call content" refers to the entire process of the conversation carried out by the telemarketing system, and refers to the data that is recorded and analyzed.

[0014] "Analysis data" refers to data generated based on recorded conversation content for evaluating the effectiveness of talk scenarios and the success rate of appointments. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system that enables the complete automation of telemarketing operations. The specific processing and operation of the program of this system will be explained below.

[0037] System configuration

[0038] This system consists of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls to customers.

[0039] Program processing

[0040] 1. Data Entry

[0041] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[0042] 2. Data analysis and talk scenario generation

[0043] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, it generates an introductory conversation for a "new cloud service" for an IT company.

[0044] 3. Automatic call preparation

[0045] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[0046] 4. Auto-dial

[0047] The server automatically places a call based on the outgoing call list, and once the device is connected to the customer, an introductory conversation begins immediately.

[0048] 5. Signal connection and introductory talk begins

[0049] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[0050] 6. Response analysis and switching talk

[0051] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if the customer responds, "What kind of service are you talking about specifically?", the AI ​​will select the conversation, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0052] 7. Make an appointment

[0053] The server and terminal continue to talk to the customer, aiming to obtain an appointment step by step. For example, if the customer says, "Could I have some time to meet with you to explain in detail?", the server will respond, "Then, would 10:00 a.m. next Tuesday be convenient for you?"

[0054] 8. Information Records

[0055] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[0056] 9. Check the results

[0057] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[0058] Specific examples

[0059] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[0060] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[0061] The server analyzes the customer information on the list and generates an introduction talk for the "new cloud service" and the necessary switching talk.

[0062] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[0063] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[0064] Once an appointment is made, the results are recorded on the server and can be viewed by the user on the management screen.

[0065] This eliminates the need for manual work and enables efficient new customer acquisition.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching chat scenarios.

[0069] Step 2:

[0070] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[0071] Step 3:

[0072] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[0073] Step 4:

[0074] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[0075] Step 5:

[0076] The device starts a call with the customer, and when the customer answers the phone, the device sends a connection signal to the server. The server's AI system executes the introductory conversation. For example, it outputs an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[0077] Step 6:

[0078] When the server receives a customer response, the AI ​​system analyzes it in real time. The AI ​​selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service do you want specifically?", the AI ​​will select a conversation such as, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0079] Step 7:

[0080] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please give me a moment to explain in more detail?") is executed.

[0081] Step 8:

[0082] The server records all call content as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[0083] Step 9:

[0084] The user logs in to the management screen and checks the detailed list of all call results and appointments obtained. The user adjusts the scenario based on the analysis results and plans the next outbound campaign.

[0085] This series of steps automates all processes involved in telemarketing, enabling effective and efficient new customer acquisition.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] Traditional telemarketing operations required manual management of customer information, cumbersome outgoing calls, and a rapid response to customer responses. This made it difficult to efficiently acquire new customers and often resulted in wasted resources. Another problem was the significant time required to analyze customer responses and select appropriate conversation topics.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes means for receiving customer information and conversation scenarios from a user, means for generating a call list and a call schedule based on the customer information, means for automatically making a call and executing a pre-set introductory conversation when the customer answers, means for analyzing the customer's response in real time and using the generated AI model to select and execute an optimal switching conversation, means for recording the call content and results and generating analytical data, and means for the user to check the recorded call content and analytical data. This eliminates the complexity of manual work and enables efficient new customer development and rapid customer response.

[0091] "User" refers to the person who operates the system, logs in to the management screen, and uploads customer information and conversation scenarios.

[0092] "Customer information" refers to data including the customer's name, telephone number, company name, etc., and refers to important information used in telemarketing operations.

[0093] A "conversation scenario" refers to the text that constitutes the introductory talk and transition talk used when talking to a customer.

[0094] "Call List" means a list of customers created by the System to call.

[0095] "Call Schedule" refers to a schedule that sets the time periods during which the system will make calls.

[0096] "Introductory talk" refers to the initial sales pitch given to a customer when making a phone call.

[0097] "Switching talk" refers to an appropriate response talk selected based on the customer's response.

[0098] "Generated AI model" refers to a natural language processing model that is trained and generated using artificial intelligence technology.

[0099] "Real-time analysis" refers to the process of instantly analyzing customer responses and selecting an appropriate response.

[0100] "Call content" refers to all content discussed during a call with a customer.

[0101] "Analysis data" refers to data generated for analysis based on the content of a call and its results.

[0102] "Recorded call content" refers to the audio data recorded during a call and its textual equivalent.

[0103] "Administration screen" refers to the interface used by users to access and operate the system.

[0104] This invention is a system that enables the complete automation of telemarketing operations. This system is composed of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls with customers. A specific embodiment of this system will be described below.

[0105] Hardware and software used

[0106] This system uses the following hardware and software:

[0107] Devices: Used to communicate with users and customers. This includes PCs, tablets, and smartphones.

[0108] Server: Hosts the entire system, performs data analysis, and manages calls. The server implements a generative AI model (e.g., GPT-3).

[0109] Administration screen: The interface for users to operate the system. It is accessed using a web browser.

[0110] VoIP software: For example, Asterisk is used to make automated calls.

[0111] Database: We use MySQL or PostgreSQL to store customer information and call details.

[0112] Explanation of the processing flow

[0113] 1. Data Entry

[0114] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[0115] 2. Data analysis and talk scenario generation

[0116] The server analyzes the uploaded customer information and generates individually optimized introductory and transition conversation scenarios. The generative AI model used here is GPT-3, which generates optimal conversation scenarios based on customer information.

[0117] 3. Automatic call preparation

[0118] The server creates a call list and a call schedule based on the time slots specified by the user. The scheduling algorithm is implemented in Python, Java, or other languages.

[0119] 4. Auto-dial

[0120] The server automatically places a call based on a call list. When the terminal connects to the customer, an introductory conversation begins immediately. The call is made over the Internet using VoIP protocols.

[0121] 5. Signal connection and introductory talk begins

[0122] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from XX Company. We have a proposal for a new service."

[0123] 6. Response analysis and switching talk

[0124] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if a customer asks, "What kind of service is this specifically?", the AI ​​will respond with, "It's a new cloud service that will significantly improve business efficiency."

[0125] 7. Make an appointment

[0126] The server and terminal continue to talk to the customer, aiming to secure an appointment step by step. For example, if the customer requests, "Could I have some time to explain in detail?", the server will respond, "How about 10:00 a.m. next Tuesday?"

[0127] 8. Information Records

[0128] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[0129] 9. Check the results

[0130] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[0131] Specific examples

[0132] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[0133] 1. The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[0134] 2. The server analyzes the customer information on the list and generates an introduction token for the "new cloud service" and the necessary switching tokens.

[0135] 3. The server will automatically initiate a call at the specified time, and when the customer answers the call, an introductory conversation will be conducted through the terminal.

[0136] 4. When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[0137] 5. Once the appointment is made, the results are recorded on the server and the user can view them on the management screen.

[0138] Prompt Sentence Examples

[0139] Below are some example prompts to be input to the generative AI model:

[0140] Customer information:

[0141] Name: Yamada Taro

[0142] Company name: Sample Co., Ltd.

[0143] Phone number: 090-1234-5678

[0144] Conversation scenario:

[0145] Introduction talk: Proposing a new cloud service.

[0146] Switch Talk:

[0147] If you are asked about the content: What we are proposing this time is a new cloud service that will significantly improve your business efficiency.

[0148] These procedures enable the complete automation of telemarketing operations, improving the efficiency of new customer development and customer service.

[0149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0150] Step 1: Data entry

[0151] Input: The user uploads customer information and conversation scenarios from the management screen. Customer information includes the customer's name, phone number, company name, etc., and the conversation scenarios are provided in text format.

[0152] Data processing: The server analyzes the uploaded file and stores the customer information and conversation scenario in a database.

[0153] Output: Customer information and conversation scenario stored in a database.

[0154] Specific operation: A user logs in to the management screen using a PC, selects and uploads customer information (CSV file format) and conversation scenarios (text files). The server receives this and stores it in the database.

[0155] Step 2: Data analysis and talk scenario generation

[0156] Input: Customer information and conversation scenario stored in the database.

[0157] Data processing: The server analyzes customer information and optimizes conversation scenarios. A generative AI model (GPT-3) is used to generate the optimal conversation scenario for each customer.

[0158] Output: Optimized talk scenario.

[0159] Specific operation: The server runs an analysis program, analyzes customer information, and extracts the name, company name, purpose, etc. Based on this, the generative AI model creates individually optimized introductory conversations and transition conversations, which are then stored in a database.

[0160] Step 3: Prepare for automatic calls

[0161] Input: Parsed customer information and optimized conversation scenario.

[0162] Data processing: The server creates a call list and call schedule. The server sets the call schedule based on the time period specified by the user.

[0163] Output: Call list and call schedule.

[0164] Specific operation: The server uses a scheduling algorithm to generate a calling list based on customer information, creates a calling schedule according to the time period specified by the user (e.g., 9:00 AM to 6:00 PM), and saves it in the database.

[0165] Step 4: Auto-call

[0166] Input: Call list and call schedule.

[0167] Data processing: The server automatically makes a call based on the calling list, and immediately starts the introductory conversation after connecting.

[0168] Output: Calls made and introductory talks performed.

[0169] Specific operation: The server uses VoIP software (e.g., Asterisk) to automatically make a call at a specified time based on a calling list. When the customer answers, the terminal plays an introductory message: "Hello, this is XX from XX Company. We have a new service proposal for you."

[0170] Step 5: Connect the signal and start the introduction talk

[0171] Input: Call connect signal when customer answers.

[0172] Data processing: The terminal notifies the server of the customer's response signal and starts the introductory conversation. The server sends the appropriate introductory conversation to the terminal based on the conversation scenario.

[0173] Output: The introductory talk played to the customer.

[0174] Specific operation: The terminal receives a response signal from the customer and notifies the server. The server refers to the conversation scenario and sends the introductory conversation audio data to the terminal, which then plays it back to the customer.

[0175] Step 6: Response analysis and switching talk

[0176] Input: Customer response data and real-time voice data.

[0177] Data processing: The server analyzes customer responses in real time and uses a generative AI model to select the optimal switching conversation.

[0178] Output: Selected switched talk.

[0179] How it works: The device records the customer's response and sends it to the server in real time. The server uses a generative AI model to analyze the response, selects the appropriate switching conversation, and sends it to the device. The device then plays back this conversation.

[0180] Step 7: Make an appointment

[0181] Input: Customer conversation data.

[0182] Data processing: The server and terminal continue the conversation with the customer, aiming to secure an appointment. If necessary, they select additional responses based on the generated conversation scenario.

[0183] Output: The retrieved appointment information.

[0184] Specific operation: The terminal continues the conversation with the customer, and if the customer requests an appointment, the server confirms it at the date and time suggested. For example, if the customer requests, "Could I have some time to explain in detail?", the server responds, "How about next Tuesday at 10:00 AM?"

[0185] Step 8: Record information

[0186] Input: Call content and outcome data.

[0187] Data processing: The server records the call content and results and generates data for analysis.

[0188] Output: Recorded call logs and data for analysis.

[0189] Specific operation: The device records the call and transmits it to the server in real time. The server stores the call data in a database and generates analytical data based on the call results.

[0190] Step 9: Check the results

[0191] Input: Recorded call content and analytical data.

[0192] Data processing: The server organizes the recorded call content and analytical data, and makes it available for users to view on the management screen.

[0193] Output: The confirmation result displayed on the management screen.

[0194] Specific operation: The user accesses the management screen and logs in with their ID and password. The management screen displays the results of each call and details of the appointments obtained. Based on this, the user can plan their next outgoing campaign.

[0195] (Application example 1)

[0196] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0197] In conventional telemarketing, the process of creating an optimal conversation scenario for each customer, manually making the call, and then switching to the appropriate conversation based on the response was extremely time-consuming and labor-intensive. Furthermore, recording and analyzing the content of the call also required manual work, making it difficult to improve the conversation scenario effectively. This meant that improvements in sales efficiency and customer satisfaction were not fully achieved.

[0198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0199] In this invention, the server includes a means for receiving customer information and conversation scenarios from a user, a means for generating a call list and a call schedule based on the customer information, and a means for automatically generating individually optimized conversation scenarios using a generative AI model. This enables automatic outgoing calls, execution of the optimal conversation scenario for each customer, and selection and execution of optimal conversation switching in real time based on the customer's response. Furthermore, the call content and results can be recorded and analyzed to help improve the conversation scenario for future calls, thereby improving sales efficiency and customer satisfaction.

[0200] A "user" is an entity that utilizes the system to provide customer information and conversation scenarios.

[0201] "Customer Information" refers to basic data about a customer, such as the customer's name, phone number, and company name.

[0202] A "conversation scenario" is a pre-set talk script used in a call with a customer.

[0203] A "call list" is a list of customers to whom the system will automatically call.

[0204] An "outgoing schedule" is a plan that includes the dates, times, and time periods for the system to make calls.

[0205] The "introduction talk" is the basic explanation and greeting that is given first when you call a customer.

[0206] "Switch Talk" is a response script that provides an appropriate reply based on the customer's response.

[0207] "Call content" refers to all conversation data regarding interactions with customers.

[0208] "Analysis data" is data generated based on the content of the call and is useful for improving the talk scenario for future calls.

[0209] A "generative AI model" is an artificial intelligence model that automatically generates sentences and talk scripts based on customer and product information.

[0210] A "prompt sentence" is an initial input sentence that is input into a generative AI model to generate a talk scenario.

[0211] "Product information" refers to information such as the name, characteristics, and price of the product being sold.

[0212] "Follow-up" refers to additional guidance or support provided if the customer expresses interest in purchasing.

[0213] This invention is a system that enables the complete automation of telemarketing operations at mail-order sites. The specific processing and operation of the program of this system will be described below.

[0214] System configuration

[0215] The system is broadly divided into three main parts: a terminal managed by the user, a server that hosts the entire system, and a terminal for making calls.Another feature is that it uses a generative AI model to automatically generate a conversation scenario optimized for each customer based on prompt text.

[0216] Program processing and behavior

[0217] 1. Data Entry

[0218] Users log in to the management screen and upload a conversation scenario containing customer and product information to the system. Customer information includes the customer's name, phone number, company name, etc., while product information includes the product's name, features, price, etc.

[0219] 2. Data analysis and talk scenario generation

[0220] The server analyzes the uploaded customer and product information and uses a generative AI model to automatically generate an individually optimized conversation scenario. For example, the prompt might look like this: "Hello {name}, I'd like to introduce you to {product_name}. This product..." A scenario is generated based on this prompt.

[0221] 3. Automatic call preparation

[0222] The server generates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[0223] 4. Auto-dial

[0224] The server automatically makes a call based on the calling list. When the customer answers, an introductory conversation immediately begins, explaining the product and providing guidance to the customer.

[0225] 5. Response analysis and switching talk

[0226] The server's AI system analyzes customer responses in real time and selects and executes the appropriate switching conversation. For example, if a customer asks, "What kind of product specifically are you talking about?", the AI ​​will select a switching conversation such as, "What we're recommending today is a new, multi-functional microwave oven that has 30% improved heating efficiency compared to conventional models."

[0227] 6. Making appointments and following up

[0228] The server and terminal continue to talk to the customer, aiming to gradually obtain an appointment or induce them to make a purchase. For example, if the customer says, "I'll consider purchasing," the server will follow up by saying, "We'll send you detailed information."

[0229] 7. Information Records

[0230] All call content and results are recorded on the server, and administrators can check this data on the management screen.In addition, the recorded data can be used to improve future talk scenarios.

[0231] Hardware and software used

[0232] Hardware: Customer support robots for smartphones and enterprises.

[0233] software:

[0234] Twilio API (for calls)

[0235] The transformers library (for generative AI models)

[0236] JSON library (for managing data)

[0237] Specific examples

[0238] For example, if an online shopping site runs a campaign selling a new microwave oven, it might do the following:

[0239] An administrator uploads a customer list and product information to the system, and uses a generative AI model to generate a talk scenario called "Introducing a new microwave oven."

[0240] The server analyzes the customer information on the list, initiates an automatic call based on the specified time period, and when the customer answers the phone, immediately makes a "proposal for a new microwave oven."

[0241] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[0242] Once an appointment is made, the results are recorded on the server and can be viewed by the administrator on the management screen.

[0243] This eliminates the need for tedious manual work and enables efficient new customer development and sales promotion.

[0244] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0245] Step 1:

[0246] The user logs in to the management screen and uploads a conversation scenario containing customer and product information in file format to the server. The input includes the customer's name, phone number, company name, and product name, features, price, etc. The output is saved on the server. Specifically, the user uploads files such as "Customer Information.csv" and "Product Information.json."

[0247] Step 2:

[0248] The server analyzes the uploaded customer and product information and uses a generative AI model to generate an optimal conversation scenario for each customer. The server receives the customer's individual information and product information as input, and processes the data using a prompt such as, "Hello, {name}, I'd like to introduce you to {product_name}." The server generates an optimized conversation scenario for each customer as output. Specifically, the server invokes a generative AI model (e.g., GPT-3) to automatically generate a conversation scenario.

[0249] Step 3:

[0250] The server generates a calling list and a calling schedule based on customer information. As input, the customer's phone number and the specified time period information are given. As output, a calling list and a calling schedule are generated. Specifically, the server performs an operation to create a list of customers to be called and a calling schedule based on that list.

[0251] Step 4:

[0252] The server automatically makes a call based on the call list. The call list and talk scenario are given as input. The output is a call to the customer and the execution of the talk scenario. Specifically, the call is made automatically using the Twilio API, and when the customer answers, the introduction talk of the talk scenario begins.

[0253] Step 5:

[0254] The server's AI system analyzes the customer's responses in real time and selects and executes the optimal switching conversation. As input, the customer's voice data and pre-generated switching conversation data are given. As output, the appropriate switching conversation is selected and executed. Specifically, speech analysis technology is used to convert the customer's words into text, and an appropriate response is generated based on that.

[0255] Step 6:

[0256] The server and terminal continue the conversation with the customer, gradually making appointments and offering purchase guidance. The inputs are the customer's response data and the conversation scenario. The output is making appointments and offering additional follow-up guidance. Specifically, the server detects the customer's desire to purchase, and the next step is to send detailed materials or set up another follow-up call.

[0257] Step 7:

[0258] All call content and results are recorded on the server, and users can view this data on the management screen. Input includes the content of calls with customers and their results. Output includes the content of calls and their results, which are saved in a database and provided in a viewable format on the management screen. Specifically, call recording data and its transcription are saved on the server, and users can review them and use them to improve their conversation scenarios.

[0259] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0260] This invention is a system that realizes the complete automation of telemarketing operations and further optimizes the content of conversations by recognizing emotions from customer responses. The specific processing and operation of the program for this system are explained below.

[0261] System configuration

[0262] This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[0263] Program processing

[0264] 1. Data Entry

[0265] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[0266] 2. Data analysis and talk scenario generation

[0267] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, for an IT company, it generates an introductory conversation for a "new cloud service."

[0268] 3. Automatic call preparation

[0269] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[0270] 4. Auto-dial

[0271] The server automatically makes a call based on the calling list, and once the device is connected to the customer, the introductory conversation begins immediately.

[0272] 5. Signal connection and introductory talk begins

[0273] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[0274] 6. Response analysis and switching talk

[0275] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if the customer shows interest, a detailed explanation will be provided, but if they show caution, a softer approach will be taken.

[0276] 7. Sentiment Analysis and Talk Modulation

[0277] When the emotion engine analyzes the customer's emotional state, it tags it as, for example, "favorable," "cold," or "anxious." Based on these tags, the server's AI adjusts the content of the conversation and switches appropriately to carry out the conversation. For example, when asked, "What kind of service is this specifically?", if the emotion is "favorable," the conversation will be, "What we are proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0278] 8. Make an appointment

[0279] The server and terminal continue to talk to the customer and aim to make an appointment based on the analysis results of the emotion engine. If the customer shows interest in a specific action, the server and terminal will suggest, for example, "Could you please give us some time to explain in more detail?"

[0280] 9. Information Records

[0281] All call content and the analysis results of the emotion engine are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve future conversations.

[0282] 10. Check the results

[0283] Users can log in to the dashboard and see all call results, a detailed list of appointments made, and even the emotional state of the customer, providing the basis for more effective planning of the next outbound campaign.

[0284] Specific examples

[0285] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[0286] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[0287] The server analyzes the customer information on the list and generates an introductory talk about the "new cloud service" and a switching talk using an emotion engine.

[0288] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[0289] When a customer asks a specific question, the server's AI analyzes it in real time, and the emotion engine recognizes the customer's emotions and adapts the conversation to suit their needs.

[0290] Once an appointment is made, the results and emotional state are recorded on the server and can be viewed by the user on the management screen.

[0291] This eliminates the hassle of manual work and enables efficient and effective new customer acquisition through optimal emotional responses.

[0292] The processing flow will be explained below.

[0293] Step 1:

[0294] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching conversation scenarios.

[0295] Step 2:

[0296] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[0297] Step 3:

[0298] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[0299] Step 4:

[0300] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[0301] Step 5:

[0302] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[0303] Step 6:

[0304] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service are you talking about specifically?", the AI ​​might select a conversation like, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0305] Step 7:

[0306] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please spare a moment to explain in more detail?") is executed. The emotion engine analyzes the customer's emotional state in real time and adjusts the conversation content accordingly.

[0307] Step 8:

[0308] The server records all call content and the analysis results of the emotion engine as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[0309] Step 9:

[0310] The user logs in to the management screen and checks all call results, a detailed list of appointments obtained, and the emotional state of the customer based on the emotion engine. Based on this, the user can adjust the scenario and plan the next outbound campaign.

[0311] This series of steps automates all processes in telemarketing, enabling efficient and effective new customer acquisition through optimal responses based on the customer's emotional state.

[0312] Example 2

[0313] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] Conventional telemarketing systems have difficulty automating a series of customer conversations, particularly when it comes to recognizing customer emotions and providing appropriate conversations. As a result, customer service efficiency is low and results are limited. Furthermore, manual data entry and confirmation of conversation status are required, resulting in poor operational efficiency. Furthermore, improving conversation scenarios requires a significant amount of effort.

[0315] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0316] In this invention, the server includes: means for receiving customer information and conversation scenarios from a user; means for generating a call list and a call schedule based on the customer information; means for automatically making a call and, when the customer answers, executing a pre-set introductory conversation using a generation AI model; means for analyzing the customer's response and selecting and executing an optimal switching conversation using an emotion engine; means for recording the call content and results and generating analytical data based on the analysis results using the emotion engine; and means for the user to review the recorded call content and analytical data. This automates customer service and utilizes emotion recognition to deliver optimal conversations, thereby improving the efficiency and results of customer service. It also reduces manual work and enables effective improvement of conversation scenarios.

[0317] A "user" is a person or organization that logs in to the management screen and uploads customer information and conversation scenarios in order to operate the system.

[0318] "Customer information" is data that includes information such as the customer's name, telephone number, and company name.

[0319] A "conversation scenario" is text data that describes a series of topics and questions to be used in a conversation with a customer.

[0320] A "server" is a computer system that manages customer information and conversation scenarios, and performs automatic calls, response analysis, and result recording.

[0321] A "terminal" is a device that is controlled by the server and is used to make calls to customers.

[0322] A "call list" is a server-generated order or list of calls to make based on customer information.

[0323] The "calling schedule" is a schedule generated by the server that indicates the time periods during which calls are made.

[0324] A "generative AI model" is an algorithm or system that uses AI technology to automatically generate optimal conversation scenarios.

[0325] An "emotion engine" is software or algorithms that analyze customer responses to recognize their emotional state.

[0326] An "introduction talk" is a set script or message that is delivered to the customer at the beginning of the call.

[0327] "Switch talk" refers to additional talk or response to a customer's response.

[0328] "Call Records" are records of all conversations you have with customers.

[0329] "Analysis data" is generated based on the recorded content of the call and the results of emotion analysis, and is data that will be useful for improving future talk scenarios.

[0330] This invention is a system that realizes the complete automation of telemarketing operations and optimizes the content of conversations by recognizing emotions from customer responses. This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[0331] System configuration

[0332] 1. User Device

[0333] Users access the management screen and upload customer information and conversation scenarios in CSV file format. The management screen is browser-based and is built using HTML, CSS, and JavaScript.

[0334] 2. Server

[0335] The server mainly performs the following tasks:

[0336] Data reception and analysis

[0337] The server receives and analyzes the CSV file containing customer information and conversation scenarios uploaded by the user. The server uses the Python language and the Pandas library to process the data.

[0338] Talk scenario generation

[0339] It uses a generative AI model to generate optimal introductory and transition conversations based on customer information, built using Transformers, an open-source natural language processing library.

[0340] Calling Lists and Scheduling

[0341] Create calling lists and schedules based on user-specified time periods.

[0342] Real-time analysis of customer responses

[0343] Customer responses are converted to text using speech recognition software and then analyzed for sentiment using an emotion engine, powered by the Google Cloud Speech-to-Text API and a sentiment analysis library.

[0344] Call recording and analytics data generation

[0345] The content of the call and the results of the analysis by the emotion engine are recorded in a database, generating analytical data that will be useful for future improvements. MySQL is used as the database.

[0346] 3. Call terminal

[0347] The call terminal receives instructions from the server and makes a call to the customer. After the call connection is established, it plays the introductory speech generated by the generative AI model. The Google Cloud Text-to-Speech API is used for voice synthesis.

[0348] Specific examples

[0349] For example, if a user wants to run a campaign proposing a new cloud service for an IT company, the following process occurs:

[0350] The user logs in to the management screen using a browser and uploads customer information and a conversation scenario (e.g., "Proposal for a new cloud service") in CSV file format.

[0351] The server receives the CSV file and uses Pandas to extract and analyze customer information.

[0352] The server uses a generative AI model using the Transformers library to generate optimal introductory and switching talks for IT companies.

[0353] The server creates a call schedule based on the specified time, and the call terminal automatically calls the customer.

[0354] When the customer answers the phone, the server uses the Google Cloud Text-to-Speech API to play an introductory conversation, such as, "Hello, this is a sales representative from a general company. We have a proposal for a new cloud service."

[0355] The customer's response is converted to text using the Google Cloud Speech-to-Text API, and the emotion engine analyzes the text to recognize the customer's emotional state. For example, if the customer shows interest, it generates an optimal transition phrase that provides detailed explanations.

[0356] The server records the call content and sentiment analysis results in a MySQL database and generates analytical data for future campaigns.

[0357] Prompt Sentence Examples

[0358] Here are some examples of prompts for generative AI models:

[0359] "Please create a talk scenario proposing a new cloud service to an IT company. The customer information is as follows: Name: Taro Yamada, Company: Tech Co., Ltd., Phone number: 012-345-6789."

[0360] This system eliminates the hassle of manual work and enables efficient and effective new customer acquisition by providing optimal emotional responses.

[0361] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0362] Step 1:

[0363] A user logs in to the admin page

[0364] A user accesses the administration screen using a browser and enters login information. Specifically, the user enters a username and password and clicks the "Login" button. The input is the user's authentication information, and the output is the success or failure status of the login authentication.

[0365] Step 2:

[0366] The user uploads customer information and conversation scenarios

[0367] The user uploads customer information and conversation scenarios in CSV file format to the server from the management screen. Specifically, the user clicks the "Select File" button, selects the CSV file, and presses the upload button. The input is the CSV file, and the output is the customer information and conversation scenario data received by the server.

[0368] Step 3:

[0369] The server analyzes customer information

[0370] The server reads the uploaded CSV file and parses the information for each customer. It processes the data using Python and the Pandas library. Specifically, it reads the data from the CSV file, converts it to a data frame, and extracts each field (such as name, phone number, or company name). The input is the contents of the CSV file, and the output is a data frame of parsed customer information.

[0371] Step 4:

[0372] The server generates a talk scenario

[0373] The server uses a generative AI model based on customer information to generate optimal introductory and switching talks. Specifically, the Transformers library is used to input prompts into the model and obtain the generated text. The input is customer information and prompts, and the output is the text of the generated talk scenario.

[0374] Step 5:

[0375] The server creates the delivery list and schedule

[0376] The server creates a call list based on the analyzed customer information and sets a call schedule based on the time period specified by the user. Specifically, it creates a list of customer information and distributes it among the designated call time periods. The input is the analyzed customer information and the designated time period, and the output is a call list and call schedule.

[0377] Step 6:

[0378] The server automatically makes the call

[0379] The server makes calls through the call terminal at the specified time based on the call list. Specifically, it automatically dials each customer in the list. The input is the call list and the call schedule, and the output is the information on successful call connections.

[0380] Step 7:

[0381] The device initiates a call to the customer

[0382] The terminal makes a call, and when the customer answers, it sends a connection signal to the server. Specifically, the terminal sends a "connection successful" signal to the server. The input is the call connection status, and the output is the connection signal.

[0383] Step 8:

[0384] The server executes the introductory token

[0385] The server receives the connection signal and synthesizes and plays the introductory speech generated by the generative AI model. Specifically, it uses the Google Cloud Text-to-Speech API to convert the text into speech and plays it to the customer. The input is the text of the generated introductory speech, and the output is audio data.

[0386] Step 9:

[0387] The server analyzes customer responses in real time.

[0388] The customer's response is converted into text using speech recognition software, and the text is then analyzed using an emotion engine. Specifically, the voice data is converted into text using the Google Cloud Speech-to-Text API, and the emotional state is recognized using an emotion analysis library. The input is the customer's voice data, and the output is text data and the emotion analysis results.

[0389] Step 10:

[0390] The server will switch optimally and execute the conversation.

[0391] The server generates the optimal switching conversation based on the analysis results of the emotion engine, synthesizes it into voice, and plays it back. Specifically, the conversation is generated using a generative AI model and converted into voice data using the Google Cloud Text-to-Speech API. The input is the emotion analysis result and the prompt, and the output is voice data.

[0392] Step 11:

[0393] The server records the call content and results

[0394] The server records the call content and the analysis results from the emotion engine in a database. Specifically, it stores voice data, text data, and emotion analysis results in a MySQL database. The input is the call content and analysis result data, and the output is a database record.

[0395] Step 12:

[0396] The user checks the results on the admin page

[0397] The user logs in to the management screen and checks the call results, the detailed list of appointments obtained, and the customer's emotional state. Specifically, the user clicks the "Check Results" button on the management screen to display a detailed report. The input is the recorded data in the database, and the output is the display of the results.

[0398] (Application example 2)

[0399] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0400] In traditional telemarketing, operators communicate one-on-one with customers, which increases labor costs and makes it difficult to provide consistent customer service. Furthermore, it is difficult to recognize customer emotions in real time and respond optimally accordingly. This results in lower customer satisfaction and a deterioration in sales efficiency. Especially for online shopping sites, the quality of customer support directly impacts customer satisfaction, so it was necessary to automate this process and achieve efficient, high-quality support.

[0401] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0402] In this invention, the server includes means for receiving customer information and conversation scenarios from users and generating a call list and call schedule, means for analyzing customer responses using an emotion engine and selecting and executing optimal switching conversations, and means for recording call content and results and generating data for analysis, thereby enabling the automation of optimal customer support based on customer emotions.

[0403] "User" refers to the person or organization that manages the system and provides customer information and conversation scenarios.

[0404] "Customer information" refers to all information about a customer, such as the customer's name, telephone number, and company name.

[0405] A "conversation scenario" is a pre-defined conversation content that the system sets up for customers.

[0406] A "call list" is a list of phone numbers or contacts that the system generates for automatic calling.

[0407] The "calling schedule" indicates the specific date, time, and time period set for the system to make automatic calls.

[0408] "Introductory talk" refers to the first conversation the system conducts when the customer answers the phone.

[0409] "Switch talk" refers to the content of the next conversation that the system selects and executes in response to the customer's response.

[0410] An "emotion engine" is an engine that analyzes the customer's emotional state in real time and adjusts the optimal content of the conversation based on the results.

[0411] "Call content" refers to all telephone conversations with customers.

[0412] "Analysis data" refers to data generated based on recorded call content and analysis of customer responses, and is used to help improve future calls.

[0413] "System" refers to the entire collection of devices and programs for performing automatic transmission, analysis, recording, and analysis based on this invention.

[0414] This invention is a system that fully automates customer support operations for online shopping sites and provides optimal responses based on customer emotions. The system is primarily composed of a server, a smartphone application, an AI emotion engine, and a data analysis engine.

[0415] System configuration and program processing

[0416] 1. Hardware and Software Configuration

[0417] Server platform: AWS (Amazon Web Services) is used for hosting the entire system and data processing.

[0418] AI Emotion Engine: Uses Google Cloud Natural Language API to analyze customer responses and emotions in real time.

[0419] Speech recognition engine: Using the Google Cloud Speech-to-Text API, the contents of voice calls are converted into text data and made analyzable.

[0420] Smartphone Framework: Build cross-platform smartphone applications using React Native.

[0421] 2. Data processing and calculation

[0422] Data Entry and Analysis:

[0423] Users use a smartphone application to upload customer information and conversation scenarios to the server, with this data provided in text format.

[0424] The server receives customer information and conversation scenarios and generates an optimal calling list and schedule for each customer.

[0425] Sentiment analysis and talk moderation:

[0426] When a chat or voice call is initiated with a customer, the Google Cloud Natural Language API analyzes the data in real time to recognize the customer's sentiment.

[0427] The emotion engine tags emotional states such as "satisfaction," "anxiety," and "anger" and dynamically adjusts the content of the conversation based on the results.

[0428] 3. Response generation and call recording

[0429] Chat Responses:

[0430] When customers use chat, AI automatically generates initial messages and responses to questions.

[0431] For example, if a customer types, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[0432] Voice call answering:

[0433] Even during voice calls, what the customer says is converted into text data using the Google Cloud Speech-to-Text API and analyzed by the emotion engine.

[0434] For example, if a customer says, "The item I ordered hasn't arrived yet," the AI ​​will respond flexibly by saying, "We apologize for the inconvenience. We will check the situation immediately and take appropriate action."

[0435] Information Record:

[0436] All chat and voice call content, as well as analysis results, are recorded on the server and used as analytical data to help improve the service in future.

[0437] Specific examples

[0438] Prompt Sentence Examples

[0439] For example, if a customer types in the chat, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[0440] Or, if a customer says in a voice call, "My order hasn't arrived yet," the AI ​​will respond, "We apologize for the inconvenience. We will immediately check the situation and take action."

[0441] In this way, this system can significantly improve the quality of customer support by analyzing customer sentiment in real time and automatically providing the most appropriate response based on that.

[0442] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0443] Step 1:

[0444] A user uses a smartphone application to upload customer information and conversation scenarios to a server.

[0445] Input: Customer information (name, phone number, company name, etc.), conversation scenario (text format)

[0446] Data processing and data calculation: The server stores the received customer information and conversation scenario in a database.

[0447] Output: Customer information and conversation scenarios stored in a database

[0448] Step 2:

[0449] The server generates a delivery list and a delivery schedule based on the received customer information.

[0450] Input: Customer information in the database

[0451] Data processing and data calculation: The server uses algorithms to create the optimal calling list and schedule for each customer.

[0452] Output: Call list and call schedule

[0453] Step 3:

[0454] The server automatically makes calls based on a calling list and plays a pre-defined introductory conversation when the customer answers.

[0455] Input: Call list, call schedule, conversation scenario

[0456] Data processing and data calculation: When making a call, a conversation scenario is retrieved from the database and an introductory talk is selected.

[0457] Output: Play an introductory talk to the customer

[0458] Step 4:

[0459] Analyze the customer's response and select and execute the optimal switching conversation.

[0460] Input: Customer response (voice or text)

[0461] Data processing and calculation: Convert speech to text using the Google Cloud Speech-to-Text API, and perform sentiment analysis using the Google Cloud Natural Language API. Select the optimal conversation based on sentiment tags.

[0462] Output: Translated responses, sentiment tags, and optimal conversation switching

[0463] Step 5:

[0464] It uses an emotion engine to recognize the customer's emotional state and adjust the content of the conversation based on that recognition.

[0465] Input: Customer response converted to text, sentiment tag

[0466] Data processing and data calculation: The emotion engine analyzes the customer's emotional state and dynamically adjusts the content of the conversation.

[0467] Output: Adjusted speech content

[0468] Step 6:

[0469] Calls and results are recorded to generate data for analysis.

[0470] Input: Call content converted to text, analysis results

[0471] Data processing and calculation: Call content and analysis results are stored in a database, and analytical data is generated for each call session.

[0472] Output: Recorded call content and analysis results, generated data for analysis

[0473] Step 7:

[0474] The user can check the recorded call content and analysis data on the management screen.

[0475] Input: Recorded call content and analysis results, generated analysis data

[0476] Data processing and data calculation: The server retrieves the call content and analysis data from the database and displays them on the management screen.

[0477] Output: Call details and analysis data displayed on the management screen

[0478] Through these steps, the system will achieve optimal customer support automation based on customer sentiment.

[0479] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0480] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0481] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0482] [Second embodiment]

[0483] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0484] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0485] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0486] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0487] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0488] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0489] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0490] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0491] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0492] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0493] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0494] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0495] This invention is a system that enables the complete automation of telemarketing operations. The specific processing and operation of the program of this system will be explained below.

[0496] System configuration

[0497] This system consists of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls to customers.

[0498] Program processing

[0499] 1. Data Entry

[0500] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[0501] 2. Data analysis and talk scenario generation

[0502] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, it generates an introductory conversation for a "new cloud service" for an IT company.

[0503] 3. Automatic call preparation

[0504] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[0505] 4. Auto-dial

[0506] The server automatically places a call based on the outgoing call list, and once the device is connected to the customer, an introductory conversation begins immediately.

[0507] 5. Signal connection and introductory talk begins

[0508] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[0509] 6. Response analysis and switching talk

[0510] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if the customer responds, "What kind of service are you talking about specifically?", the AI ​​will select the conversation, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0511] 7. Make an appointment

[0512] The server and terminal continue to talk to the customer, aiming to obtain an appointment step by step. For example, if the customer says, "Could I have some time to meet with you to explain in detail?", the server will respond, "Then, would 10:00 a.m. next Tuesday be convenient for you?"

[0513] 8. Information Records

[0514] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[0515] 9. Check the results

[0516] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[0517] Specific examples

[0518] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[0519] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[0520] The server analyzes the customer information on the list and generates an introduction talk for the "new cloud service" and the necessary switching talk.

[0521] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[0522] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[0523] Once an appointment is made, the results are recorded on the server and can be viewed by the user on the management screen.

[0524] This eliminates the need for manual work and enables efficient new customer acquisition.

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching chat scenarios.

[0528] Step 2:

[0529] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[0530] Step 3:

[0531] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[0532] Step 4:

[0533] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[0534] Step 5:

[0535] The device starts a call with the customer, and when the customer answers the phone, the device sends a connection signal to the server. The server's AI system executes the introductory conversation. For example, it outputs an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[0536] Step 6:

[0537] When the server receives a customer response, the AI ​​system analyzes it in real time. The AI ​​selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service do you want specifically?", the AI ​​will select a conversation such as, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0538] Step 7:

[0539] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please give me a moment to explain in more detail?") is executed.

[0540] Step 8:

[0541] The server records all call content as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[0542] Step 9:

[0543] The user logs in to the management screen and checks the detailed list of all call results and appointments obtained. The user adjusts the scenario based on the analysis results and plans the next outbound campaign.

[0544] This series of steps automates all processes involved in telemarketing, enabling effective and efficient new customer acquisition.

[0545] Example 1

[0546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0547] Traditional telemarketing operations required manual management of customer information, cumbersome outgoing calls, and a rapid response to customer responses. This made it difficult to efficiently acquire new customers and often resulted in wasted resources. Another problem was the significant time required to analyze customer responses and select appropriate conversation topics.

[0548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0549] In this invention, the server includes means for receiving customer information and conversation scenarios from a user, means for generating a call list and a call schedule based on the customer information, means for automatically making a call and executing a pre-set introductory conversation when the customer answers, means for analyzing the customer's response in real time and using the generated AI model to select and execute an optimal switching conversation, means for recording the call content and results and generating analytical data, and means for the user to check the recorded call content and analytical data. This eliminates the complexity of manual work and enables efficient new customer development and rapid customer response.

[0550] "User" refers to the person who operates the system, logs in to the management screen, and uploads customer information and conversation scenarios.

[0551] "Customer information" refers to data including the customer's name, telephone number, company name, etc., and refers to important information used in telemarketing operations.

[0552] A "conversation scenario" refers to the text that constitutes the introductory talk and transition talk used when talking to a customer.

[0553] "Call List" means a list of customers created by the System to call.

[0554] "Call Schedule" refers to a schedule that sets the time periods during which the system will make calls.

[0555] "Introductory talk" refers to the initial sales pitch given to a customer when making a phone call.

[0556] "Switching talk" refers to an appropriate response talk selected based on the customer's response.

[0557] "Generated AI model" refers to a natural language processing model that is trained and generated using artificial intelligence technology.

[0558] "Real-time analysis" refers to the process of instantly analyzing customer responses and selecting an appropriate response.

[0559] "Call content" refers to all content discussed during a call with a customer.

[0560] "Analysis data" refers to data generated for analysis based on the content of a call and its results.

[0561] "Recorded call content" refers to the audio data recorded during a call and its textual equivalent.

[0562] "Administration screen" refers to the interface used by users to access and operate the system.

[0563] This invention is a system that enables the complete automation of telemarketing operations. This system is composed of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls with customers. A specific embodiment of this system will be described below.

[0564] Hardware and software used

[0565] This system uses the following hardware and software:

[0566] Devices: Used to communicate with users and customers. This includes PCs, tablets, and smartphones.

[0567] Server: Hosts the entire system, performs data analysis, and manages calls. The server implements a generative AI model (e.g., GPT-3).

[0568] Administration screen: The interface for users to operate the system. It is accessed using a web browser.

[0569] VoIP software: For example, Asterisk is used to make automated calls.

[0570] Database: We use MySQL or PostgreSQL to store customer information and call details.

[0571] Explanation of the processing flow

[0572] 1. Data Entry

[0573] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[0574] 2. Data analysis and talk scenario generation

[0575] The server analyzes the uploaded customer information and generates individually optimized introductory and transition conversation scenarios. The generative AI model used here is GPT-3, which generates optimal conversation scenarios based on customer information.

[0576] 3. Automatic call preparation

[0577] The server creates a call list and a call schedule based on the time slots specified by the user. The scheduling algorithm is implemented in Python, Java, or other languages.

[0578] 4. Auto-dial

[0579] The server automatically places a call based on a call list. When the terminal connects to the customer, an introductory conversation begins immediately. The call is made over the Internet using VoIP protocols.

[0580] 5. Signal connection and introductory talk begins

[0581] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from XX Company. We have a proposal for a new service."

[0582] 6. Response analysis and switching talk

[0583] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if a customer asks, "What kind of service is this specifically?", the AI ​​will respond with, "It's a new cloud service that will significantly improve business efficiency."

[0584] 7. Make an appointment

[0585] The server and terminal continue to talk to the customer, aiming to secure an appointment step by step. For example, if the customer requests, "Could I have some time to explain in detail?", the server will respond, "How about 10:00 a.m. next Tuesday?"

[0586] 8. Information Records

[0587] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[0588] 9. Check the results

[0589] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[0590] Specific examples

[0591] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[0592] 1. The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[0593] 2. The server analyzes the customer information on the list and generates an introduction token for the "new cloud service" and the necessary switching tokens.

[0594] 3. The server will automatically initiate a call at the specified time, and when the customer answers the call, an introductory conversation will be conducted through the terminal.

[0595] 4. When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[0596] 5. Once the appointment is made, the results are recorded on the server and the user can view them on the management screen.

[0597] Prompt Sentence Examples

[0598] Below are some example prompts to be input to the generative AI model:

[0599] Customer information:

[0600] Name: Yamada Taro

[0601] Company name: Sample Co., Ltd.

[0602] Phone number: 090-1234-5678

[0603] Conversation scenario:

[0604] Introduction talk: Proposing a new cloud service.

[0605] Switch Talk:

[0606] If you are asked about the content: What we are proposing this time is a new cloud service that will significantly improve your business efficiency.

[0607] These procedures enable the complete automation of telemarketing operations, improving the efficiency of new customer development and customer service.

[0608] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0609] Step 1: Data entry

[0610] Input: The user uploads customer information and conversation scenarios from the management screen. Customer information includes the customer's name, phone number, company name, etc., and the conversation scenarios are provided in text format.

[0611] Data processing: The server analyzes the uploaded file and stores the customer information and conversation scenario in a database.

[0612] Output: Customer information and conversation scenario stored in a database.

[0613] Specific operation: A user logs in to the management screen using a PC, selects and uploads customer information (CSV file format) and conversation scenarios (text files). The server receives this and stores it in the database.

[0614] Step 2: Data analysis and talk scenario generation

[0615] Input: Customer information and conversation scenario stored in the database.

[0616] Data processing: The server analyzes customer information and optimizes conversation scenarios. A generative AI model (GPT-3) is used to generate the optimal conversation scenario for each customer.

[0617] Output: Optimized talk scenario.

[0618] Specific operation: The server runs an analysis program, analyzes customer information, and extracts the name, company name, purpose, etc. Based on this, the generative AI model creates individually optimized introductory conversations and transition conversations, which are then stored in a database.

[0619] Step 3: Prepare for automatic calls

[0620] Input: Parsed customer information and optimized conversation scenario.

[0621] Data processing: The server creates a call list and call schedule. The server sets the call schedule based on the time period specified by the user.

[0622] Output: Call list and call schedule.

[0623] Specific operation: The server uses a scheduling algorithm to generate a calling list based on customer information, creates a calling schedule according to the time period specified by the user (e.g., 9:00 AM to 6:00 PM), and saves it in the database.

[0624] Step 4: Auto-call

[0625] Input: Call list and call schedule.

[0626] Data processing: The server automatically makes a call based on the calling list, and immediately starts the introductory conversation after connecting.

[0627] Output: Calls made and introductory talks performed.

[0628] Specific operation: The server uses VoIP software (e.g., Asterisk) to automatically make a call at a specified time based on a calling list. When the customer answers, the terminal plays an introductory message: "Hello, this is XX from XX Company. We have a new service proposal for you."

[0629] Step 5: Connect the signal and start the introduction talk

[0630] Input: Call connect signal when customer answers.

[0631] Data processing: The terminal notifies the server of the customer's response signal and starts the introductory conversation. The server sends the appropriate introductory conversation to the terminal based on the conversation scenario.

[0632] Output: The introductory talk played to the customer.

[0633] Specific operation: The terminal receives a response signal from the customer and notifies the server. The server refers to the conversation scenario and sends the introductory conversation audio data to the terminal, which then plays it back to the customer.

[0634] Step 6: Response analysis and switching talk

[0635] Input: Customer response data and real-time voice data.

[0636] Data processing: The server analyzes customer responses in real time and uses a generative AI model to select the optimal switching conversation.

[0637] Output: Selected switched talk.

[0638] How it works: The device records the customer's response and sends it to the server in real time. The server uses a generative AI model to analyze the response, selects the appropriate switching conversation, and sends it to the device. The device then plays back this conversation.

[0639] Step 7: Make an appointment

[0640] Input: Customer conversation data.

[0641] Data processing: The server and terminal continue the conversation with the customer, aiming to secure an appointment. If necessary, they select additional responses based on the generated conversation scenario.

[0642] Output: The retrieved appointment information.

[0643] Specific operation: The terminal continues the conversation with the customer, and if the customer requests an appointment, the server confirms it at the date and time suggested. For example, if the customer requests, "Could I have some time to explain in detail?", the server responds, "How about next Tuesday at 10:00 AM?"

[0644] Step 8: Record information

[0645] Input: Call content and outcome data.

[0646] Data processing: The server records the call content and results and generates data for analysis.

[0647] Output: Recorded call logs and data for analysis.

[0648] Specific operation: The device records the call and transmits it to the server in real time. The server stores the call data in a database and generates analytical data based on the call results.

[0649] Step 9: Check the results

[0650] Input: Recorded call content and analytical data.

[0651] Data processing: The server organizes the recorded call content and analytical data, and makes it available for users to view on the management screen.

[0652] Output: The confirmation result displayed on the management screen.

[0653] Specific operation: The user accesses the management screen and logs in with their ID and password. The management screen displays the results of each call and details of the appointments obtained. Based on this, the user can plan their next outgoing campaign.

[0654] (Application example 1)

[0655] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] In conventional telemarketing, the process of creating an optimal conversation scenario for each customer, manually making the call, and then switching to the appropriate conversation based on the response was extremely time-consuming and labor-intensive. Furthermore, recording and analyzing the content of the call also required manual work, making it difficult to improve the conversation scenario effectively. This meant that improvements in sales efficiency and customer satisfaction were not fully achieved.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0658] In this invention, the server includes a means for receiving customer information and conversation scenarios from a user, a means for generating a call list and a call schedule based on the customer information, and a means for automatically generating individually optimized conversation scenarios using a generative AI model. This enables automatic outgoing calls, execution of the optimal conversation scenario for each customer, and selection and execution of optimal conversation switching in real time based on the customer's response. Furthermore, the call content and results can be recorded and analyzed to help improve the conversation scenario for future calls, thereby improving sales efficiency and customer satisfaction.

[0659] A "user" is an entity that utilizes the system to provide customer information and conversation scenarios.

[0660] "Customer Information" refers to basic data about a customer, such as the customer's name, phone number, and company name.

[0661] A "conversation scenario" is a pre-set talk script used in a call with a customer.

[0662] A "call list" is a list of customers to whom the system will automatically call.

[0663] An "outgoing schedule" is a plan that includes the dates, times, and time periods for the system to make calls.

[0664] The "introduction talk" is the basic explanation and greeting that is given first when you call a customer.

[0665] "Switch Talk" is a response script that provides an appropriate reply based on the customer's response.

[0666] "Call content" refers to all conversation data regarding interactions with customers.

[0667] "Analysis data" is data generated based on the content of the call and is useful for improving the talk scenario for future calls.

[0668] A "generative AI model" is an artificial intelligence model that automatically generates sentences and talk scripts based on customer and product information.

[0669] A "prompt sentence" is an initial input sentence that is input into a generative AI model to generate a talk scenario.

[0670] "Product information" refers to information such as the name, characteristics, and price of the product being sold.

[0671] "Follow-up" refers to additional guidance or support provided if the customer expresses interest in purchasing.

[0672] This invention is a system that enables the complete automation of telemarketing operations at mail-order sites. The specific processing and operation of the program of this system will be described below.

[0673] System configuration

[0674] The system is broadly divided into three main parts: a terminal managed by the user, a server that hosts the entire system, and a terminal for making calls.Another feature is that it uses a generative AI model to automatically generate a conversation scenario optimized for each customer based on prompt text.

[0675] Program processing and behavior

[0676] 1. Data Entry

[0677] Users log in to the management screen and upload a conversation scenario containing customer and product information to the system. Customer information includes the customer's name, phone number, company name, etc., while product information includes the product's name, features, price, etc.

[0678] 2. Data analysis and talk scenario generation

[0679] The server analyzes the uploaded customer and product information and uses a generative AI model to automatically generate an individually optimized conversation scenario. For example, the prompt might look like this: "Hello {name}, I'd like to introduce you to {product_name}. This product..." A scenario is generated based on this prompt.

[0680] 3. Automatic call preparation

[0681] The server generates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[0682] 4. Auto-dial

[0683] The server automatically makes a call based on the calling list. When the customer answers, an introductory conversation immediately begins, explaining the product and providing guidance to the customer.

[0684] 5. Response analysis and switching talk

[0685] The server's AI system analyzes customer responses in real time and selects and executes the appropriate switching conversation. For example, if a customer asks, "What kind of product specifically are you talking about?", the AI ​​will select a switching conversation such as, "What we're recommending today is a new, multi-functional microwave oven that has 30% improved heating efficiency compared to conventional models."

[0686] 6. Making appointments and following up

[0687] The server and terminal continue to talk to the customer, aiming to gradually obtain an appointment or induce them to make a purchase. For example, if the customer says, "I'll consider purchasing," the server will follow up by saying, "We'll send you detailed information."

[0688] 7. Information Records

[0689] All call content and results are recorded on the server, and administrators can check this data on the management screen.In addition, the recorded data can be used to improve future talk scenarios.

[0690] Hardware and software used

[0691] Hardware: Customer support robots for smartphones and enterprises.

[0692] software:

[0693] Twilio API (for calls)

[0694] The transformers library (for generative AI models)

[0695] JSON library (for managing data)

[0696] Specific examples

[0697] For example, if an online shopping site runs a campaign selling a new microwave oven, it might do the following:

[0698] An administrator uploads a customer list and product information to the system, and uses a generative AI model to generate a talk scenario called "Introducing a new microwave oven."

[0699] The server analyzes the customer information on the list, initiates an automatic call based on the specified time period, and when the customer answers the phone, immediately makes a "proposal for a new microwave oven."

[0700] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[0701] Once an appointment is made, the results are recorded on the server and can be viewed by the administrator on the management screen.

[0702] This eliminates the need for tedious manual work and enables efficient new customer development and sales promotion.

[0703] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0704] Step 1:

[0705] The user logs in to the management screen and uploads a conversation scenario containing customer and product information in file format to the server. The input includes the customer's name, phone number, company name, and product name, features, price, etc. The output is saved on the server. Specifically, the user uploads files such as "Customer Information.csv" and "Product Information.json."

[0706] Step 2:

[0707] The server analyzes the uploaded customer and product information and uses a generative AI model to generate an optimal conversation scenario for each customer. The server receives the customer's individual information and product information as input, and processes the data using a prompt such as, "Hello, {name}, I'd like to introduce you to {product_name}." The server generates an optimized conversation scenario for each customer as output. Specifically, the server invokes a generative AI model (e.g., GPT-3) to automatically generate a conversation scenario.

[0708] Step 3:

[0709] The server generates a calling list and a calling schedule based on customer information. As input, the customer's phone number and the specified time period information are given. As output, a calling list and a calling schedule are generated. Specifically, the server performs an operation to create a list of customers to be called and a calling schedule based on that list.

[0710] Step 4:

[0711] The server automatically makes a call based on the call list. The call list and talk scenario are given as input. The output is a call to the customer and the execution of the talk scenario. Specifically, the call is made automatically using the Twilio API, and when the customer answers, the introduction talk of the talk scenario begins.

[0712] Step 5:

[0713] The server's AI system analyzes the customer's responses in real time and selects and executes the optimal switching conversation. As input, the customer's voice data and pre-generated switching conversation data are given. As output, the appropriate switching conversation is selected and executed. Specifically, speech analysis technology is used to convert the customer's words into text, and an appropriate response is generated based on that.

[0714] Step 6:

[0715] The server and terminal continue the conversation with the customer, gradually making appointments and offering purchase guidance. The inputs are the customer's response data and the conversation scenario. The output is making appointments and offering additional follow-up guidance. Specifically, the server detects the customer's desire to purchase, and the next step is to send detailed materials or set up another follow-up call.

[0716] Step 7:

[0717] All call content and results are recorded on the server, and users can view this data on the management screen. Input includes the content of calls with customers and their results. Output includes the content of calls and their results, which are saved in a database and provided in a viewable format on the management screen. Specifically, call recording data and its transcription are saved on the server, and users can review them and use them to improve their conversation scenarios.

[0718] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0719] This invention is a system that realizes the complete automation of telemarketing operations and further optimizes the content of conversations by recognizing emotions from customer responses. The specific processing and operation of the program for this system are explained below.

[0720] System configuration

[0721] This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[0722] Program processing

[0723] 1. Data Entry

[0724] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[0725] 2. Data analysis and talk scenario generation

[0726] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, for an IT company, it generates an introductory conversation for a "new cloud service."

[0727] 3. Automatic call preparation

[0728] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[0729] 4. Auto-dial

[0730] The server automatically makes a call based on the calling list, and once the device is connected to the customer, the introductory conversation begins immediately.

[0731] 5. Signal connection and introductory talk begins

[0732] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[0733] 6. Response analysis and switching talk

[0734] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if the customer shows interest, a detailed explanation will be provided, but if they show caution, a softer approach will be taken.

[0735] 7. Sentiment Analysis and Talk Modulation

[0736] When the emotion engine analyzes the customer's emotional state, it tags it as, for example, "favorable," "cold," or "anxious." Based on these tags, the server's AI adjusts the content of the conversation and switches appropriately to carry out the conversation. For example, when asked, "What kind of service is this specifically?", if the emotion is "favorable," the conversation will be, "What we are proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0737] 8. Make an appointment

[0738] The server and terminal continue to talk to the customer and aim to make an appointment based on the analysis results of the emotion engine. If the customer shows interest in a specific action, the server and terminal will suggest, for example, "Could you please give us some time to explain in more detail?"

[0739] 9. Information Records

[0740] All call content and the analysis results of the emotion engine are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve future conversations.

[0741] 10. Check the results

[0742] Users can log in to the dashboard and see all call results, a detailed list of appointments made, and even the emotional state of the customer, providing the basis for more effective planning of the next outbound campaign.

[0743] Specific examples

[0744] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[0745] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[0746] The server analyzes the customer information on the list and generates an introductory talk about the "new cloud service" and a switching talk using an emotion engine.

[0747] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[0748] When a customer asks a specific question, the server's AI analyzes it in real time, and the emotion engine recognizes the customer's emotions and adapts the conversation to suit their needs.

[0749] Once an appointment is made, the results and emotional state are recorded on the server and can be viewed by the user on the management screen.

[0750] This eliminates the hassle of manual work and enables efficient and effective new customer acquisition through optimal emotional responses.

[0751] The processing flow will be explained below.

[0752] Step 1:

[0753] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching conversation scenarios.

[0754] Step 2:

[0755] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[0756] Step 3:

[0757] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[0758] Step 4:

[0759] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[0760] Step 5:

[0761] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[0762] Step 6:

[0763] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service are you talking about specifically?", the AI ​​might select a conversation like, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0764] Step 7:

[0765] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please spare a moment to explain in more detail?") is executed. The emotion engine analyzes the customer's emotional state in real time and adjusts the conversation content accordingly.

[0766] Step 8:

[0767] The server records all call content and the analysis results of the emotion engine as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[0768] Step 9:

[0769] The user logs in to the management screen and checks all call results, a detailed list of appointments obtained, and the emotional state of the customer based on the emotion engine. Based on this, the user can adjust the scenario and plan the next outbound campaign.

[0770] This series of steps automates all processes in telemarketing, enabling efficient and effective new customer acquisition through optimal responses based on the customer's emotional state.

[0771] Example 2

[0772] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0773] Conventional telemarketing systems have difficulty automating a series of customer conversations, particularly when it comes to recognizing customer emotions and providing appropriate conversations. As a result, customer service efficiency is low and results are limited. Furthermore, manual data entry and confirmation of conversation status are required, resulting in poor operational efficiency. Furthermore, improving conversation scenarios requires a significant amount of effort.

[0774] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0775] In this invention, the server includes: means for receiving customer information and conversation scenarios from a user; means for generating a call list and a call schedule based on the customer information; means for automatically making a call and, when the customer answers, executing a pre-set introductory conversation using a generation AI model; means for analyzing the customer's response and selecting and executing an optimal switching conversation using an emotion engine; means for recording the call content and results and generating analytical data based on the analysis results using the emotion engine; and means for the user to review the recorded call content and analytical data. This automates customer service and utilizes emotion recognition to deliver optimal conversations, thereby improving the efficiency and results of customer service. It also reduces manual work and enables effective improvement of conversation scenarios.

[0776] A "user" is a person or organization that logs in to the management screen and uploads customer information and conversation scenarios in order to operate the system.

[0777] "Customer information" is data that includes information such as the customer's name, telephone number, and company name.

[0778] A "conversation scenario" is text data that describes a series of topics and questions to be used in a conversation with a customer.

[0779] A "server" is a computer system that manages customer information and conversation scenarios, and performs automatic calls, response analysis, and result recording.

[0780] A "terminal" is a device that is controlled by the server and is used to make calls to customers.

[0781] A "call list" is a server-generated order or list of calls to make based on customer information.

[0782] The "calling schedule" is a schedule generated by the server that indicates the time periods during which calls are made.

[0783] A "generative AI model" is an algorithm or system that uses AI technology to automatically generate optimal conversation scenarios.

[0784] An "emotion engine" is software or algorithms that analyze customer responses to recognize their emotional state.

[0785] An "introduction talk" is a set script or message that is delivered to the customer at the beginning of the call.

[0786] "Switch talk" refers to additional talk or response to a customer's response.

[0787] "Call Records" are records of all conversations you have with customers.

[0788] "Analysis data" is generated based on the recorded content of the call and the results of emotion analysis, and is data that will be useful for improving future talk scenarios.

[0789] This invention is a system that realizes the complete automation of telemarketing operations and optimizes the content of conversations by recognizing emotions from customer responses. This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[0790] System configuration

[0791] 1. User Device

[0792] Users access the management screen and upload customer information and conversation scenarios in CSV file format. The management screen is browser-based and is built using HTML, CSS, and JavaScript.

[0793] 2. Server

[0794] The server mainly performs the following tasks:

[0795] Data reception and analysis

[0796] The server receives and analyzes the CSV file containing customer information and conversation scenarios uploaded by the user. The server uses the Python language and the Pandas library to process the data.

[0797] Talk scenario generation

[0798] It uses a generative AI model to generate optimal introductory and transition conversations based on customer information, built using Transformers, an open-source natural language processing library.

[0799] Calling Lists and Scheduling

[0800] Create calling lists and schedules based on user-specified time periods.

[0801] Real-time analysis of customer responses

[0802] Customer responses are converted to text using speech recognition software and then analyzed for sentiment using an emotion engine, powered by the Google Cloud Speech-to-Text API and a sentiment analysis library.

[0803] Call recording and analytics data generation

[0804] The content of the call and the results of the analysis by the emotion engine are recorded in a database, generating analytical data that will be useful for future improvements. MySQL is used as the database.

[0805] 3. Call terminal

[0806] The call terminal receives instructions from the server and makes a call to the customer. After the call connection is established, it plays the introductory speech generated by the generative AI model. The Google Cloud Text-to-Speech API is used for voice synthesis.

[0807] Specific examples

[0808] For example, if a user wants to run a campaign proposing a new cloud service for an IT company, the following process occurs:

[0809] The user logs in to the management screen using a browser and uploads customer information and a conversation scenario (e.g., "Proposal for a new cloud service") in CSV file format.

[0810] The server receives the CSV file and uses Pandas to extract and analyze customer information.

[0811] The server uses a generative AI model using the Transformers library to generate optimal introductory and switching talks for IT companies.

[0812] The server creates a call schedule based on the specified time, and the call terminal automatically calls the customer.

[0813] When the customer answers the phone, the server uses the Google Cloud Text-to-Speech API to play an introductory conversation, such as, "Hello, this is a sales representative from a general company. We have a proposal for a new cloud service."

[0814] The customer's response is converted to text using the Google Cloud Speech-to-Text API, and the emotion engine analyzes the text to recognize the customer's emotional state. For example, if the customer shows interest, it generates an optimal transition phrase that provides detailed explanations.

[0815] The server records the call content and sentiment analysis results in a MySQL database and generates analytical data for future campaigns.

[0816] Prompt Sentence Examples

[0817] Here are some examples of prompts for generative AI models:

[0818] "Please create a talk scenario proposing a new cloud service to an IT company. The customer information is as follows: Name: Taro Yamada, Company: Tech Co., Ltd., Phone number: 012-345-6789."

[0819] This system eliminates the hassle of manual work and enables efficient and effective new customer acquisition by providing optimal emotional responses.

[0820] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0821] Step 1:

[0822] A user logs in to the admin page

[0823] A user accesses the administration screen using a browser and enters login information. Specifically, the user enters a username and password and clicks the "Login" button. The input is the user's authentication information, and the output is the success or failure status of the login authentication.

[0824] Step 2:

[0825] The user uploads customer information and conversation scenarios

[0826] The user uploads customer information and conversation scenarios in CSV file format to the server from the management screen. Specifically, the user clicks the "Select File" button, selects the CSV file, and presses the upload button. The input is the CSV file, and the output is the customer information and conversation scenario data received by the server.

[0827] Step 3:

[0828] The server analyzes customer information

[0829] The server reads the uploaded CSV file and parses the information for each customer. It processes the data using Python and the Pandas library. Specifically, it reads the data from the CSV file, converts it to a data frame, and extracts each field (such as name, phone number, or company name). The input is the contents of the CSV file, and the output is a data frame of parsed customer information.

[0830] Step 4:

[0831] The server generates a talk scenario

[0832] The server uses a generative AI model based on customer information to generate optimal introductory and switching talks. Specifically, the Transformers library is used to input prompts into the model and obtain the generated text. The input is customer information and prompts, and the output is the text of the generated talk scenario.

[0833] Step 5:

[0834] The server creates the delivery list and schedule

[0835] The server creates a call list based on the analyzed customer information and sets a call schedule based on the time period specified by the user. Specifically, it creates a list of customer information and distributes it among the designated call time periods. The input is the analyzed customer information and the designated time period, and the output is a call list and call schedule.

[0836] Step 6:

[0837] The server automatically makes the call

[0838] The server makes calls through the call terminal at the specified time based on the call list. Specifically, it automatically dials each customer in the list. The input is the call list and the call schedule, and the output is the information on successful call connections.

[0839] Step 7:

[0840] The device initiates a call to the customer

[0841] The terminal makes a call, and when the customer answers, it sends a connection signal to the server. Specifically, the terminal sends a "connection successful" signal to the server. The input is the call connection status, and the output is the connection signal.

[0842] Step 8:

[0843] The server executes the introductory token

[0844] The server receives the connection signal and synthesizes and plays the introductory speech generated by the generative AI model. Specifically, it uses the Google Cloud Text-to-Speech API to convert the text into speech and plays it to the customer. The input is the text of the generated introductory speech, and the output is audio data.

[0845] Step 9:

[0846] The server analyzes customer responses in real time.

[0847] The customer's response is converted into text using speech recognition software, and the text is then analyzed using an emotion engine. Specifically, the voice data is converted into text using the Google Cloud Speech-to-Text API, and the emotional state is recognized using an emotion analysis library. The input is the customer's voice data, and the output is text data and the emotion analysis results.

[0848] Step 10:

[0849] The server will switch optimally and execute the conversation.

[0850] The server generates the optimal switching conversation based on the analysis results of the emotion engine, synthesizes it into voice, and plays it back. Specifically, the conversation is generated using a generative AI model and converted into voice data using the Google Cloud Text-to-Speech API. The input is the emotion analysis result and the prompt, and the output is voice data.

[0851] Step 11:

[0852] The server records the call content and results

[0853] The server records the call content and the analysis results from the emotion engine in a database. Specifically, it stores voice data, text data, and emotion analysis results in a MySQL database. The input is the call content and analysis result data, and the output is a database record.

[0854] Step 12:

[0855] The user checks the results on the admin page

[0856] The user logs in to the management screen and checks the call results, the detailed list of appointments obtained, and the customer's emotional state. Specifically, the user clicks the "Check Results" button on the management screen to display a detailed report. The input is the recorded data in the database, and the output is the display of the results.

[0857] (Application example 2)

[0858] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0859] In traditional telemarketing, operators communicate one-on-one with customers, which increases labor costs and makes it difficult to provide consistent customer service. Furthermore, it is difficult to recognize customer emotions in real time and respond optimally accordingly. This results in lower customer satisfaction and a deterioration in sales efficiency. Especially for online shopping sites, the quality of customer support directly impacts customer satisfaction, so it was necessary to automate this process and achieve efficient, high-quality support.

[0860] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0861] In this invention, the server includes means for receiving customer information and conversation scenarios from users and generating a call list and call schedule, means for analyzing customer responses using an emotion engine and selecting and executing optimal switching conversations, and means for recording call content and results and generating data for analysis, thereby enabling the automation of optimal customer support based on customer emotions.

[0862] "User" refers to the person or organization that manages the system and provides customer information and conversation scenarios.

[0863] "Customer information" refers to all information about a customer, such as the customer's name, telephone number, and company name.

[0864] A "conversation scenario" is a pre-defined conversation content that the system sets up for customers.

[0865] A "call list" is a list of phone numbers or contacts that the system generates for automatic calling.

[0866] The "calling schedule" indicates the specific date, time, and time period set for the system to make automatic calls.

[0867] "Introductory talk" refers to the first conversation the system conducts when the customer answers the phone.

[0868] "Switch talk" refers to the content of the next conversation that the system selects and executes in response to the customer's response.

[0869] An "emotion engine" is an engine that analyzes the customer's emotional state in real time and adjusts the optimal content of the conversation based on the results.

[0870] "Call content" refers to all telephone conversations with customers.

[0871] "Analysis data" refers to data generated based on recorded call content and analysis of customer responses, and is used to help improve future calls.

[0872] "System" refers to the entire collection of devices and programs for performing automatic transmission, analysis, recording, and analysis based on this invention.

[0873] This invention is a system that fully automates customer support operations for online shopping sites and provides optimal responses based on customer emotions. The system is primarily composed of a server, a smartphone application, an AI emotion engine, and a data analysis engine.

[0874] System configuration and program processing

[0875] 1. Hardware and Software Configuration

[0876] Server platform: AWS (Amazon Web Services) is used for hosting the entire system and data processing.

[0877] AI Emotion Engine: Uses Google Cloud Natural Language API to analyze customer responses and emotions in real time.

[0878] Speech recognition engine: Using the Google Cloud Speech-to-Text API, the contents of voice calls are converted into text data and made analyzable.

[0879] Smartphone Framework: Build cross-platform smartphone applications using React Native.

[0880] 2. Data processing and calculation

[0881] Data Entry and Analysis:

[0882] Users use a smartphone application to upload customer information and conversation scenarios to the server, with this data provided in text format.

[0883] The server receives customer information and conversation scenarios and generates an optimal calling list and schedule for each customer.

[0884] Sentiment analysis and talk moderation:

[0885] When a chat or voice call is initiated with a customer, the Google Cloud Natural Language API analyzes the data in real time to recognize the customer's sentiment.

[0886] The emotion engine tags emotional states such as "satisfaction," "anxiety," and "anger" and dynamically adjusts the content of the conversation based on the results.

[0887] 3. Response generation and call recording

[0888] Chat Responses:

[0889] When customers use chat, AI automatically generates initial messages and responses to questions.

[0890] For example, if a customer types, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[0891] Voice call answering:

[0892] Even during voice calls, what the customer says is converted into text data using the Google Cloud Speech-to-Text API and analyzed by the emotion engine.

[0893] For example, if a customer says, "The item I ordered hasn't arrived yet," the AI ​​will respond flexibly by saying, "We apologize for the inconvenience. We will check the situation immediately and take appropriate action."

[0894] Information Record:

[0895] All chat and voice call content, as well as analysis results, are recorded on the server and used as analytical data to help improve the service in future.

[0896] Specific examples

[0897] Prompt Sentence Examples

[0898] For example, if a customer types in the chat, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[0899] Or, if a customer says in a voice call, "My order hasn't arrived yet," the AI ​​will respond, "We apologize for the inconvenience. We will immediately check the situation and take action."

[0900] In this way, this system can significantly improve the quality of customer support by analyzing customer sentiment in real time and automatically providing the most appropriate response based on that.

[0901] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0902] Step 1:

[0903] A user uses a smartphone application to upload customer information and conversation scenarios to a server.

[0904] Input: Customer information (name, phone number, company name, etc.), conversation scenario (text format)

[0905] Data processing and data calculation: The server stores the received customer information and conversation scenario in a database.

[0906] Output: Customer information and conversation scenarios stored in a database

[0907] Step 2:

[0908] The server generates a delivery list and a delivery schedule based on the received customer information.

[0909] Input: Customer information in the database

[0910] Data processing and data calculation: The server uses algorithms to create the optimal calling list and schedule for each customer.

[0911] Output: Call list and call schedule

[0912] Step 3:

[0913] The server automatically makes calls based on a calling list and plays a pre-defined introductory conversation when the customer answers.

[0914] Input: Call list, call schedule, conversation scenario

[0915] Data processing and data calculation: When making a call, a conversation scenario is retrieved from the database and an introductory talk is selected.

[0916] Output: Play an introductory talk to the customer

[0917] Step 4:

[0918] Analyze the customer's response and select and execute the optimal switching conversation.

[0919] Input: Customer response (voice or text)

[0920] Data processing and calculation: Convert speech to text using the Google Cloud Speech-to-Text API, and perform sentiment analysis using the Google Cloud Natural Language API. Select the optimal conversation based on sentiment tags.

[0921] Output: Translated responses, sentiment tags, and optimal conversation switching

[0922] Step 5:

[0923] It uses an emotion engine to recognize the customer's emotional state and adjust the content of the conversation based on that recognition.

[0924] Input: Customer response converted to text, sentiment tag

[0925] Data processing and data calculation: The emotion engine analyzes the customer's emotional state and dynamically adjusts the content of the conversation.

[0926] Output: Adjusted speech content

[0927] Step 6:

[0928] Calls and results are recorded to generate data for analysis.

[0929] Input: Call content converted to text, analysis results

[0930] Data processing and calculation: Call content and analysis results are stored in a database, and analytical data is generated for each call session.

[0931] Output: Recorded call content and analysis results, generated data for analysis

[0932] Step 7:

[0933] The user can check the recorded call content and analysis data on the management screen.

[0934] Input: Recorded call content and analysis results, generated analysis data

[0935] Data processing and data calculation: The server retrieves the call content and analysis data from the database and displays them on the management screen.

[0936] Output: Call details and analysis data displayed on the management screen

[0937] Through these steps, the system will achieve optimal customer support automation based on customer sentiment.

[0938] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0939] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0940] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0941] [Third embodiment]

[0942] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0943] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0944] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0945] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0946] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0947] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0948] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0949] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0950] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0951] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0952] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0953] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0954] This invention is a system that enables the complete automation of telemarketing operations. The specific processing and operation of the program of this system will be explained below.

[0955] System configuration

[0956] This system consists of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls to customers.

[0957] Program processing

[0958] 1. Data Entry

[0959] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[0960] 2. Data analysis and talk scenario generation

[0961] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, it generates an introductory conversation for a "new cloud service" for an IT company.

[0962] 3. Automatic call preparation

[0963] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[0964] 4. Auto-dial

[0965] The server automatically places a call based on the outgoing call list, and once the device is connected to the customer, an introductory conversation begins immediately.

[0966] 5. Signal connection and introductory talk begins

[0967] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[0968] 6. Response analysis and switching talk

[0969] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if the customer responds, "What kind of service are you talking about specifically?", the AI ​​will select the conversation, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0970] 7. Make an appointment

[0971] The server and terminal continue to talk to the customer, aiming to obtain an appointment step by step. For example, if the customer says, "Could I have some time to meet with you to explain in detail?", the server will respond, "Then, would 10:00 a.m. next Tuesday be convenient for you?"

[0972] 8. Information Records

[0973] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[0974] 9. Check the results

[0975] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[0976] Specific examples

[0977] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[0978] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[0979] The server analyzes the customer information on the list and generates an introduction talk for the "new cloud service" and the necessary switching talk.

[0980] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[0981] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[0982] Once an appointment is made, the results are recorded on the server and can be viewed by the user on the management screen.

[0983] This eliminates the need for manual work and enables efficient new customer acquisition.

[0984] The processing flow will be explained below.

[0985] Step 1:

[0986] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching chat scenarios.

[0987] Step 2:

[0988] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[0989] Step 3:

[0990] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[0991] Step 4:

[0992] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[0993] Step 5:

[0994] The device starts a call with the customer, and when the customer answers the phone, the device sends a connection signal to the server. The server's AI system executes the introductory conversation. For example, it outputs an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[0995] Step 6:

[0996] When the server receives a customer response, the AI ​​system analyzes it in real time. The AI ​​selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service do you want specifically?", the AI ​​will select a conversation such as, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[0997] Step 7:

[0998] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please give me a moment to explain in more detail?") is executed.

[0999] Step 8:

[1000] The server records all call content as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[1001] Step 9:

[1002] The user logs in to the management screen and checks the detailed list of all call results and appointments obtained. The user adjusts the scenario based on the analysis results and plans the next outbound campaign.

[1003] This series of steps automates all processes involved in telemarketing, enabling effective and efficient new customer acquisition.

[1004] Example 1

[1005] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1006] Traditional telemarketing operations required manual management of customer information, cumbersome outgoing calls, and a rapid response to customer responses. This made it difficult to efficiently acquire new customers and often resulted in wasted resources. Another problem was the significant time required to analyze customer responses and select appropriate conversation topics.

[1007] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1008] In this invention, the server includes means for receiving customer information and conversation scenarios from a user, means for generating a call list and a call schedule based on the customer information, means for automatically making a call and executing a pre-set introductory conversation when the customer answers, means for analyzing the customer's response in real time and using the generated AI model to select and execute an optimal switching conversation, means for recording the call content and results and generating analytical data, and means for the user to check the recorded call content and analytical data. This eliminates the complexity of manual work and enables efficient new customer development and rapid customer response.

[1009] "User" refers to the person who operates the system, logs in to the management screen, and uploads customer information and conversation scenarios.

[1010] "Customer information" refers to data including the customer's name, telephone number, company name, etc., and refers to important information used in telemarketing operations.

[1011] A "conversation scenario" refers to the text that constitutes the introductory talk and transition talk used when talking to a customer.

[1012] "Call List" means a list of customers created by the System to call.

[1013] "Call Schedule" refers to a schedule that sets the time periods during which the system will make calls.

[1014] "Introductory talk" refers to the initial sales pitch given to a customer when making a phone call.

[1015] "Switching talk" refers to an appropriate response talk selected based on the customer's response.

[1016] "Generated AI model" refers to a natural language processing model that is trained and generated using artificial intelligence technology.

[1017] "Real-time analysis" refers to the process of instantly analyzing customer responses and selecting an appropriate response.

[1018] "Call content" refers to all content discussed during a call with a customer.

[1019] "Analysis data" refers to data generated for analysis based on the content of a call and its results.

[1020] "Recorded call content" refers to the audio data recorded during a call and its textual equivalent.

[1021] "Administration screen" refers to the interface used by users to access and operate the system.

[1022] This invention is a system that enables the complete automation of telemarketing operations. This system is composed of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls with customers. A specific embodiment of this system will be described below.

[1023] Hardware and software used

[1024] This system uses the following hardware and software:

[1025] Devices: Used to communicate with users and customers. This includes PCs, tablets, and smartphones.

[1026] Server: Hosts the entire system, performs data analysis, and manages calls. The server implements a generative AI model (e.g., GPT-3).

[1027] Administration screen: The interface for users to operate the system. It is accessed using a web browser.

[1028] VoIP software: For example, Asterisk is used to make automated calls.

[1029] Database: We use MySQL or PostgreSQL to store customer information and call details.

[1030] Explanation of the processing flow

[1031] 1. Data Entry

[1032] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[1033] 2. Data analysis and talk scenario generation

[1034] The server analyzes the uploaded customer information and generates individually optimized introductory and transition conversation scenarios. The generative AI model used here is GPT-3, which generates optimal conversation scenarios based on customer information.

[1035] 3. Automatic call preparation

[1036] The server creates a call list and a call schedule based on the time slots specified by the user. The scheduling algorithm is implemented in Python, Java, or other languages.

[1037] 4. Auto-dial

[1038] The server automatically places a call based on a call list. When the terminal connects to the customer, an introductory conversation begins immediately. The call is made over the Internet using VoIP protocols.

[1039] 5. Signal connection and introductory talk begins

[1040] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from XX Company. We have a proposal for a new service."

[1041] 6. Response analysis and switching talk

[1042] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if a customer asks, "What kind of service is this specifically?", the AI ​​will respond with, "It's a new cloud service that will significantly improve business efficiency."

[1043] 7. Make an appointment

[1044] The server and terminal continue to talk to the customer, aiming to secure an appointment step by step. For example, if the customer requests, "Could I have some time to explain in detail?", the server will respond, "How about 10:00 a.m. next Tuesday?"

[1045] 8. Information Records

[1046] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[1047] 9. Check the results

[1048] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[1049] Specific examples

[1050] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[1051] 1. The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[1052] 2. The server analyzes the customer information on the list and generates an introduction token for the "new cloud service" and the necessary switching tokens.

[1053] 3. The server will automatically initiate a call at the specified time, and when the customer answers the call, an introductory conversation will be conducted through the terminal.

[1054] 4. When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[1055] 5. Once the appointment is made, the results are recorded on the server and the user can view them on the management screen.

[1056] Prompt Sentence Examples

[1057] Below are some example prompts to be input to the generative AI model:

[1058] Customer information:

[1059] Name: Yamada Taro

[1060] Company name: Sample Co., Ltd.

[1061] Phone number: 090-1234-5678

[1062] Conversation scenario:

[1063] Introduction talk: Proposing a new cloud service.

[1064] Switch Talk:

[1065] If you are asked about the content: What we are proposing this time is a new cloud service that will significantly improve your business efficiency.

[1066] These procedures enable the complete automation of telemarketing operations, improving the efficiency of new customer development and customer service.

[1067] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1068] Step 1: Data entry

[1069] Input: The user uploads customer information and conversation scenarios from the management screen. Customer information includes the customer's name, phone number, company name, etc., and the conversation scenarios are provided in text format.

[1070] Data processing: The server analyzes the uploaded file and stores the customer information and conversation scenario in a database.

[1071] Output: Customer information and conversation scenario stored in a database.

[1072] Specific operation: A user logs in to the management screen using a PC, selects and uploads customer information (CSV file format) and conversation scenarios (text files). The server receives this and stores it in the database.

[1073] Step 2: Data analysis and talk scenario generation

[1074] Input: Customer information and conversation scenario stored in the database.

[1075] Data processing: The server analyzes customer information and optimizes conversation scenarios. A generative AI model (GPT-3) is used to generate the optimal conversation scenario for each customer.

[1076] Output: Optimized talk scenario.

[1077] Specific operation: The server runs an analysis program, analyzes customer information, and extracts the name, company name, purpose, etc. Based on this, the generative AI model creates individually optimized introductory conversations and transition conversations, which are then stored in a database.

[1078] Step 3: Prepare for automatic calls

[1079] Input: Parsed customer information and optimized conversation scenario.

[1080] Data processing: The server creates a call list and call schedule. The server sets the call schedule based on the time period specified by the user.

[1081] Output: Call list and call schedule.

[1082] Specific operation: The server uses a scheduling algorithm to generate a calling list based on customer information, creates a calling schedule according to the time period specified by the user (e.g., 9:00 AM to 6:00 PM), and saves it in the database.

[1083] Step 4: Auto-call

[1084] Input: Call list and call schedule.

[1085] Data processing: The server automatically makes a call based on the calling list, and immediately starts the introductory conversation after connecting.

[1086] Output: Calls made and introductory talks performed.

[1087] Specific operation: The server uses VoIP software (e.g., Asterisk) to automatically make a call at a specified time based on a calling list. When the customer answers, the terminal plays an introductory message: "Hello, this is XX from XX Company. We have a new service proposal for you."

[1088] Step 5: Connect the signal and start the introduction talk

[1089] Input: Call connect signal when customer answers.

[1090] Data processing: The terminal notifies the server of the customer's response signal and starts the introductory conversation. The server sends the appropriate introductory conversation to the terminal based on the conversation scenario.

[1091] Output: The introductory talk played to the customer.

[1092] Specific operation: The terminal receives a response signal from the customer and notifies the server. The server refers to the conversation scenario and sends the introductory conversation audio data to the terminal, which then plays it back to the customer.

[1093] Step 6: Response analysis and switching talk

[1094] Input: Customer response data and real-time voice data.

[1095] Data processing: The server analyzes customer responses in real time and uses a generative AI model to select the optimal switching conversation.

[1096] Output: Selected switched talk.

[1097] How it works: The device records the customer's response and sends it to the server in real time. The server uses a generative AI model to analyze the response, selects the appropriate switching conversation, and sends it to the device. The device then plays back this conversation.

[1098] Step 7: Make an appointment

[1099] Input: Customer conversation data.

[1100] Data processing: The server and terminal continue the conversation with the customer, aiming to secure an appointment. If necessary, they select additional responses based on the generated conversation scenario.

[1101] Output: The retrieved appointment information.

[1102] Specific operation: The terminal continues the conversation with the customer, and if the customer requests an appointment, the server confirms it at the date and time suggested. For example, if the customer requests, "Could I have some time to explain in detail?", the server responds, "How about next Tuesday at 10:00 AM?"

[1103] Step 8: Record information

[1104] Input: Call content and outcome data.

[1105] Data processing: The server records the call content and results and generates data for analysis.

[1106] Output: Recorded call logs and data for analysis.

[1107] Specific operation: The device records the call and transmits it to the server in real time. The server stores the call data in a database and generates analytical data based on the call results.

[1108] Step 9: Check the results

[1109] Input: Recorded call content and analytical data.

[1110] Data processing: The server organizes the recorded call content and analytical data, and makes it available for users to view on the management screen.

[1111] Output: The confirmation result displayed on the management screen.

[1112] Specific operation: The user accesses the management screen and logs in with their ID and password. The management screen displays the results of each call and details of the appointments obtained. Based on this, the user can plan their next outgoing campaign.

[1113] (Application example 1)

[1114] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1115] In conventional telemarketing, the process of creating an optimal conversation scenario for each customer, manually making the call, and then switching to the appropriate conversation based on the response was extremely time-consuming and labor-intensive. Furthermore, recording and analyzing the content of the call also required manual work, making it difficult to improve the conversation scenario effectively. This meant that improvements in sales efficiency and customer satisfaction were not fully achieved.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1117] In this invention, the server includes a means for receiving customer information and conversation scenarios from a user, a means for generating a call list and a call schedule based on the customer information, and a means for automatically generating individually optimized conversation scenarios using a generative AI model. This enables automatic outgoing calls, execution of the optimal conversation scenario for each customer, and selection and execution of optimal conversation switching in real time based on the customer's response. Furthermore, the call content and results can be recorded and analyzed to help improve the conversation scenario for future calls, thereby improving sales efficiency and customer satisfaction.

[1118] A "user" is an entity that utilizes the system to provide customer information and conversation scenarios.

[1119] "Customer Information" refers to basic data about a customer, such as the customer's name, phone number, and company name.

[1120] A "conversation scenario" is a pre-set talk script used in a call with a customer.

[1121] A "call list" is a list of customers to whom the system will automatically call.

[1122] An "outgoing schedule" is a plan that includes the dates, times, and time periods for the system to make calls.

[1123] The "introduction talk" is the basic explanation and greeting that is given first when you call a customer.

[1124] "Switch Talk" is a response script that provides an appropriate reply based on the customer's response.

[1125] "Call content" refers to all conversation data regarding interactions with customers.

[1126] "Analysis data" is data generated based on the content of the call and is useful for improving the talk scenario for future calls.

[1127] A "generative AI model" is an artificial intelligence model that automatically generates sentences and talk scripts based on customer and product information.

[1128] A "prompt sentence" is an initial input sentence that is input into a generative AI model to generate a talk scenario.

[1129] "Product information" refers to information such as the name, characteristics, and price of the product being sold.

[1130] "Follow-up" refers to additional guidance or support provided if the customer expresses interest in purchasing.

[1131] This invention is a system that enables the complete automation of telemarketing operations at mail-order sites. The specific processing and operation of the program of this system will be described below.

[1132] System configuration

[1133] The system is broadly divided into three main parts: a terminal managed by the user, a server that hosts the entire system, and a terminal for making calls.Another feature is that it uses a generative AI model to automatically generate a conversation scenario optimized for each customer based on prompt text.

[1134] Program processing and behavior

[1135] 1. Data Entry

[1136] Users log in to the management screen and upload a conversation scenario containing customer and product information to the system. Customer information includes the customer's name, phone number, company name, etc., while product information includes the product's name, features, price, etc.

[1137] 2. Data analysis and talk scenario generation

[1138] The server analyzes the uploaded customer and product information and uses a generative AI model to automatically generate an individually optimized conversation scenario. For example, the prompt might look like this: "Hello {name}, I'd like to introduce you to {product_name}. This product..." A scenario is generated based on this prompt.

[1139] 3. Automatic call preparation

[1140] The server generates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[1141] 4. Auto-dial

[1142] The server automatically makes a call based on the calling list. When the customer answers, an introductory conversation immediately begins, explaining the product and providing guidance to the customer.

[1143] 5. Response analysis and switching talk

[1144] The server's AI system analyzes customer responses in real time and selects and executes the appropriate switching conversation. For example, if a customer asks, "What kind of product specifically are you talking about?", the AI ​​will select a switching conversation such as, "What we're recommending today is a new, multi-functional microwave oven that has 30% improved heating efficiency compared to conventional models."

[1145] 6. Making appointments and following up

[1146] The server and terminal continue to talk to the customer, aiming to gradually obtain an appointment or induce them to make a purchase. For example, if the customer says, "I'll consider purchasing," the server will follow up by saying, "We'll send you detailed information."

[1147] 7. Information Records

[1148] All call content and results are recorded on the server, and administrators can check this data on the management screen.In addition, the recorded data can be used to improve future talk scenarios.

[1149] Hardware and software used

[1150] Hardware: Customer support robots for smartphones and enterprises.

[1151] software:

[1152] Twilio API (for calls)

[1153] The transformers library (for generative AI models)

[1154] JSON library (for managing data)

[1155] Specific examples

[1156] For example, if an online shopping site runs a campaign selling a new microwave oven, it might do the following:

[1157] An administrator uploads a customer list and product information to the system, and uses a generative AI model to generate a talk scenario called "Introducing a new microwave oven."

[1158] The server analyzes the customer information on the list, initiates an automatic call based on the specified time period, and when the customer answers the phone, immediately makes a "proposal for a new microwave oven."

[1159] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[1160] Once an appointment is made, the results are recorded on the server and can be viewed by the administrator on the management screen.

[1161] This eliminates the need for tedious manual work and enables efficient new customer development and sales promotion.

[1162] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1163] Step 1:

[1164] The user logs in to the management screen and uploads a conversation scenario containing customer and product information in file format to the server. The input includes the customer's name, phone number, company name, and product name, features, price, etc. The output is saved on the server. Specifically, the user uploads files such as "Customer Information.csv" and "Product Information.json."

[1165] Step 2:

[1166] The server analyzes the uploaded customer and product information and uses a generative AI model to generate an optimal conversation scenario for each customer. The server receives the customer's individual information and product information as input, and processes the data using a prompt such as, "Hello, {name}, I'd like to introduce you to {product_name}." The server generates an optimized conversation scenario for each customer as output. Specifically, the server invokes a generative AI model (e.g., GPT-3) to automatically generate a conversation scenario.

[1167] Step 3:

[1168] The server generates a calling list and a calling schedule based on customer information. As input, the customer's phone number and the specified time period information are given. As output, a calling list and a calling schedule are generated. Specifically, the server performs an operation to create a list of customers to be called and a calling schedule based on that list.

[1169] Step 4:

[1170] The server automatically makes a call based on the call list. The call list and talk scenario are given as input. The output is a call to the customer and the execution of the talk scenario. Specifically, the call is made automatically using the Twilio API, and when the customer answers, the introduction talk of the talk scenario begins.

[1171] Step 5:

[1172] The server's AI system analyzes the customer's responses in real time and selects and executes the optimal switching conversation. As input, the customer's voice data and pre-generated switching conversation data are given. As output, the appropriate switching conversation is selected and executed. Specifically, speech analysis technology is used to convert the customer's words into text, and an appropriate response is generated based on that.

[1173] Step 6:

[1174] The server and terminal continue the conversation with the customer, gradually making appointments and offering purchase guidance. The inputs are the customer's response data and the conversation scenario. The output is making appointments and offering additional follow-up guidance. Specifically, the server detects the customer's desire to purchase, and the next step is to send detailed materials or set up another follow-up call.

[1175] Step 7:

[1176] All call content and results are recorded on the server, and users can view this data on the management screen. Input includes the content of calls with customers and their results. Output includes the content of calls and their results, which are saved in a database and provided in a viewable format on the management screen. Specifically, call recording data and its transcription are saved on the server, and users can review them and use them to improve their conversation scenarios.

[1177] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1178] This invention is a system that realizes the complete automation of telemarketing operations and further optimizes the content of conversations by recognizing emotions from customer responses. The specific processing and operation of the program for this system are explained below.

[1179] System configuration

[1180] This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[1181] Program processing

[1182] 1. Data Entry

[1183] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[1184] 2. Data analysis and talk scenario generation

[1185] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, for an IT company, it generates an introductory conversation for a "new cloud service."

[1186] 3. Automatic call preparation

[1187] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[1188] 4. Auto-dial

[1189] The server automatically makes a call based on the calling list, and once the device is connected to the customer, the introductory conversation begins immediately.

[1190] 5. Signal connection and introductory talk begins

[1191] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[1192] 6. Response analysis and switching talk

[1193] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if the customer shows interest, a detailed explanation will be provided, but if they show caution, a softer approach will be taken.

[1194] 7. Sentiment Analysis and Talk Modulation

[1195] When the emotion engine analyzes the customer's emotional state, it tags it as, for example, "favorable," "cold," or "anxious." Based on these tags, the server's AI adjusts the content of the conversation and switches appropriately to carry out the conversation. For example, when asked, "What kind of service is this specifically?", if the emotion is "favorable," the conversation will be, "What we are proposing today is a new cloud service that will significantly improve your company's business efficiency."

[1196] 8. Make an appointment

[1197] The server and terminal continue to talk to the customer and aim to make an appointment based on the analysis results of the emotion engine. If the customer shows interest in a specific action, the server and terminal will suggest, for example, "Could you please give us some time to explain in more detail?"

[1198] 9. Information Records

[1199] All call content and the analysis results of the emotion engine are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve future conversations.

[1200] 10. Check the results

[1201] Users can log in to the dashboard and see all call results, a detailed list of appointments made, and even the emotional state of the customer, providing the basis for more effective planning of the next outbound campaign.

[1202] Specific examples

[1203] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[1204] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[1205] The server analyzes the customer information on the list and generates an introductory talk about the "new cloud service" and a switching talk using an emotion engine.

[1206] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[1207] When a customer asks a specific question, the server's AI analyzes it in real time, and the emotion engine recognizes the customer's emotions and adapts the conversation to suit their needs.

[1208] Once an appointment is made, the results and emotional state are recorded on the server and can be viewed by the user on the management screen.

[1209] This eliminates the hassle of manual work and enables efficient and effective new customer acquisition through optimal emotional responses.

[1210] The processing flow will be explained below.

[1211] Step 1:

[1212] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching conversation scenarios.

[1213] Step 2:

[1214] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[1215] Step 3:

[1216] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[1217] Step 4:

[1218] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[1219] Step 5:

[1220] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[1221] Step 6:

[1222] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service are you talking about specifically?", the AI ​​might select a conversation like, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[1223] Step 7:

[1224] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please spare a moment to explain in more detail?") is executed. The emotion engine analyzes the customer's emotional state in real time and adjusts the conversation content accordingly.

[1225] Step 8:

[1226] The server records all call content and the analysis results of the emotion engine as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[1227] Step 9:

[1228] The user logs in to the management screen and checks all call results, a detailed list of appointments obtained, and the emotional state of the customer based on the emotion engine. Based on this, the user can adjust the scenario and plan the next outbound campaign.

[1229] This series of steps automates all processes in telemarketing, enabling efficient and effective new customer acquisition through optimal responses based on the customer's emotional state.

[1230] Example 2

[1231] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1232] Conventional telemarketing systems have difficulty automating a series of customer conversations, particularly when it comes to recognizing customer emotions and providing appropriate conversations. As a result, customer service efficiency is low and results are limited. Furthermore, manual data entry and confirmation of conversation status are required, resulting in poor operational efficiency. Furthermore, improving conversation scenarios requires a significant amount of effort.

[1233] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1234] In this invention, the server includes: means for receiving customer information and conversation scenarios from a user; means for generating a call list and a call schedule based on the customer information; means for automatically making a call and, when the customer answers, executing a pre-set introductory conversation using a generation AI model; means for analyzing the customer's response and selecting and executing an optimal switching conversation using an emotion engine; means for recording the call content and results and generating analytical data based on the analysis results using the emotion engine; and means for the user to review the recorded call content and analytical data. This automates customer service and utilizes emotion recognition to deliver optimal conversations, thereby improving the efficiency and results of customer service. It also reduces manual work and enables effective improvement of conversation scenarios.

[1235] A "user" is a person or organization that logs in to the management screen and uploads customer information and conversation scenarios in order to operate the system.

[1236] "Customer information" is data that includes information such as the customer's name, telephone number, and company name.

[1237] A "conversation scenario" is text data that describes a series of topics and questions to be used in a conversation with a customer.

[1238] A "server" is a computer system that manages customer information and conversation scenarios, and performs automatic calls, response analysis, and result recording.

[1239] A "terminal" is a device that is controlled by the server and is used to make calls to customers.

[1240] A "call list" is a server-generated order or list of calls to make based on customer information.

[1241] The "calling schedule" is a schedule generated by the server that indicates the time periods during which calls are made.

[1242] A "generative AI model" is an algorithm or system that uses AI technology to automatically generate optimal conversation scenarios.

[1243] An "emotion engine" is software or algorithms that analyze customer responses to recognize their emotional state.

[1244] An "introduction talk" is a set script or message that is delivered to the customer at the beginning of the call.

[1245] "Switch talk" refers to additional talk or response to a customer's response.

[1246] "Call Records" are records of all conversations you have with customers.

[1247] "Analysis data" is generated based on the recorded content of the call and the results of emotion analysis, and is data that will be useful for improving future talk scenarios.

[1248] This invention is a system that realizes the complete automation of telemarketing operations and optimizes the content of conversations by recognizing emotions from customer responses. This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[1249] System configuration

[1250] 1. User Device

[1251] Users access the management screen and upload customer information and conversation scenarios in CSV file format. The management screen is browser-based and is built using HTML, CSS, and JavaScript.

[1252] 2. Server

[1253] The server mainly performs the following tasks:

[1254] Data reception and analysis

[1255] The server receives and analyzes the CSV file containing customer information and conversation scenarios uploaded by the user. The server uses the Python language and the Pandas library to process the data.

[1256] Talk scenario generation

[1257] It uses a generative AI model to generate optimal introductory and transition conversations based on customer information, built using Transformers, an open-source natural language processing library.

[1258] Calling Lists and Scheduling

[1259] Create calling lists and schedules based on user-specified time periods.

[1260] Real-time analysis of customer responses

[1261] Customer responses are converted to text using speech recognition software and then analyzed for sentiment using an emotion engine, powered by the Google Cloud Speech-to-Text API and a sentiment analysis library.

[1262] Call recording and analytics data generation

[1263] The content of the call and the results of the analysis by the emotion engine are recorded in a database, generating analytical data that will be useful for future improvements. MySQL is used as the database.

[1264] 3. Call terminal

[1265] The call terminal receives instructions from the server and makes a call to the customer. After the call connection is established, it plays the introductory speech generated by the generative AI model. The Google Cloud Text-to-Speech API is used for voice synthesis.

[1266] Specific examples

[1267] For example, if a user wants to run a campaign proposing a new cloud service for an IT company, the following process occurs:

[1268] The user logs in to the management screen using a browser and uploads customer information and a conversation scenario (e.g., "Proposal for a new cloud service") in CSV file format.

[1269] The server receives the CSV file and uses Pandas to extract and analyze customer information.

[1270] The server uses a generative AI model using the Transformers library to generate optimal introductory and switching talks for IT companies.

[1271] The server creates a call schedule based on the specified time, and the call terminal automatically calls the customer.

[1272] When the customer answers the phone, the server uses the Google Cloud Text-to-Speech API to play an introductory conversation, such as, "Hello, this is a sales representative from a general company. We have a proposal for a new cloud service."

[1273] The customer's response is converted to text using the Google Cloud Speech-to-Text API, and the emotion engine analyzes the text to recognize the customer's emotional state. For example, if the customer shows interest, it generates an optimal transition phrase that provides detailed explanations.

[1274] The server records the call content and sentiment analysis results in a MySQL database and generates analytical data for future campaigns.

[1275] Prompt Sentence Examples

[1276] Here are some examples of prompts for generative AI models:

[1277] "Please create a talk scenario proposing a new cloud service to an IT company. The customer information is as follows: Name: Taro Yamada, Company: Tech Co., Ltd., Phone number: 012-345-6789."

[1278] This system eliminates the hassle of manual work and enables efficient and effective new customer acquisition by providing optimal emotional responses.

[1279] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1280] Step 1:

[1281] A user logs in to the admin page

[1282] A user accesses the administration screen using a browser and enters login information. Specifically, the user enters a username and password and clicks the "Login" button. The input is the user's authentication information, and the output is the success or failure status of the login authentication.

[1283] Step 2:

[1284] The user uploads customer information and conversation scenarios

[1285] The user uploads customer information and conversation scenarios in CSV file format to the server from the management screen. Specifically, the user clicks the "Select File" button, selects the CSV file, and presses the upload button. The input is the CSV file, and the output is the customer information and conversation scenario data received by the server.

[1286] Step 3:

[1287] The server analyzes customer information

[1288] The server reads the uploaded CSV file and parses the information for each customer. It processes the data using Python and the Pandas library. Specifically, it reads the data from the CSV file, converts it to a data frame, and extracts each field (such as name, phone number, or company name). The input is the contents of the CSV file, and the output is a data frame of parsed customer information.

[1289] Step 4:

[1290] The server generates a talk scenario

[1291] The server uses a generative AI model based on customer information to generate optimal introductory and switching talks. Specifically, the Transformers library is used to input prompts into the model and obtain the generated text. The input is customer information and prompts, and the output is the text of the generated talk scenario.

[1292] Step 5:

[1293] The server creates the delivery list and schedule

[1294] The server creates a call list based on the analyzed customer information and sets a call schedule based on the time period specified by the user. Specifically, it creates a list of customer information and distributes it among the designated call time periods. The input is the analyzed customer information and the designated time period, and the output is a call list and call schedule.

[1295] Step 6:

[1296] The server automatically makes the call

[1297] The server makes calls through the call terminal at the specified time based on the call list. Specifically, it automatically dials each customer in the list. The input is the call list and the call schedule, and the output is the information on successful call connections.

[1298] Step 7:

[1299] The device initiates a call to the customer

[1300] The terminal makes a call, and when the customer answers, it sends a connection signal to the server. Specifically, the terminal sends a "connection successful" signal to the server. The input is the call connection status, and the output is the connection signal.

[1301] Step 8:

[1302] The server executes the introductory token

[1303] The server receives the connection signal and synthesizes and plays the introductory speech generated by the generative AI model. Specifically, it uses the Google Cloud Text-to-Speech API to convert the text into speech and plays it to the customer. The input is the text of the generated introductory speech, and the output is audio data.

[1304] Step 9:

[1305] The server analyzes customer responses in real time.

[1306] The customer's response is converted into text using speech recognition software, and the text is then analyzed using an emotion engine. Specifically, the voice data is converted into text using the Google Cloud Speech-to-Text API, and the emotional state is recognized using an emotion analysis library. The input is the customer's voice data, and the output is text data and the emotion analysis results.

[1307] Step 10:

[1308] The server will switch optimally and execute the conversation.

[1309] The server generates the optimal switching conversation based on the analysis results of the emotion engine, synthesizes it into voice, and plays it back. Specifically, the conversation is generated using a generative AI model and converted into voice data using the Google Cloud Text-to-Speech API. The input is the emotion analysis result and the prompt, and the output is voice data.

[1310] Step 11:

[1311] The server records the call content and results

[1312] The server records the call content and the analysis results from the emotion engine in a database. Specifically, it stores voice data, text data, and emotion analysis results in a MySQL database. The input is the call content and analysis result data, and the output is a database record.

[1313] Step 12:

[1314] The user checks the results on the admin page

[1315] The user logs in to the management screen and checks the call results, the detailed list of appointments obtained, and the customer's emotional state. Specifically, the user clicks the "Check Results" button on the management screen to display a detailed report. The input is the recorded data in the database, and the output is the display of the results.

[1316] (Application example 2)

[1317] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1318] In traditional telemarketing, operators communicate one-on-one with customers, which increases labor costs and makes it difficult to provide consistent customer service. Furthermore, it is difficult to recognize customer emotions in real time and respond optimally accordingly. This results in lower customer satisfaction and a deterioration in sales efficiency. Especially for online shopping sites, the quality of customer support directly impacts customer satisfaction, so it was necessary to automate this process and achieve efficient, high-quality support.

[1319] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1320] In this invention, the server includes means for receiving customer information and conversation scenarios from users and generating a call list and call schedule, means for analyzing customer responses using an emotion engine and selecting and executing optimal switching conversations, and means for recording call content and results and generating data for analysis, thereby enabling the automation of optimal customer support based on customer emotions.

[1321] "User" refers to the person or organization that manages the system and provides customer information and conversation scenarios.

[1322] "Customer information" refers to all information about a customer, such as the customer's name, telephone number, and company name.

[1323] A "conversation scenario" is a pre-defined conversation content that the system sets up for customers.

[1324] A "call list" is a list of phone numbers or contacts that the system generates for automatic calling.

[1325] The "calling schedule" indicates the specific date, time, and time period set for the system to make automatic calls.

[1326] "Introductory talk" refers to the first conversation the system conducts when the customer answers the phone.

[1327] "Switch talk" refers to the content of the next conversation that the system selects and executes in response to the customer's response.

[1328] An "emotion engine" is an engine that analyzes the customer's emotional state in real time and adjusts the optimal content of the conversation based on the results.

[1329] "Call content" refers to all telephone conversations with customers.

[1330] "Analysis data" refers to data generated based on recorded call content and analysis of customer responses, and is used to help improve future calls.

[1331] "System" refers to the entire collection of devices and programs for performing automatic transmission, analysis, recording, and analysis based on this invention.

[1332] This invention is a system that fully automates customer support operations for online shopping sites and provides optimal responses based on customer emotions. The system is primarily composed of a server, a smartphone application, an AI emotion engine, and a data analysis engine.

[1333] System configuration and program processing

[1334] 1. Hardware and Software Configuration

[1335] Server platform: AWS (Amazon Web Services) is used for hosting the entire system and data processing.

[1336] AI Emotion Engine: Uses Google Cloud Natural Language API to analyze customer responses and emotions in real time.

[1337] Speech recognition engine: Using the Google Cloud Speech-to-Text API, the contents of voice calls are converted into text data and made analyzable.

[1338] Smartphone Framework: Build cross-platform smartphone applications using React Native.

[1339] 2. Data processing and calculation

[1340] Data Entry and Analysis:

[1341] Users use a smartphone application to upload customer information and conversation scenarios to the server, with this data provided in text format.

[1342] The server receives customer information and conversation scenarios and generates an optimal calling list and schedule for each customer.

[1343] Sentiment analysis and talk moderation:

[1344] When a chat or voice call is initiated with a customer, the Google Cloud Natural Language API analyzes the data in real time to recognize the customer's sentiment.

[1345] The emotion engine tags emotional states such as "satisfaction," "anxiety," and "anger" and dynamically adjusts the content of the conversation based on the results.

[1346] 3. Response generation and call recording

[1347] Chat Responses:

[1348] When customers use chat, AI automatically generates initial messages and responses to questions.

[1349] For example, if a customer types, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[1350] Voice call answering:

[1351] Even during voice calls, what the customer says is converted into text data using the Google Cloud Speech-to-Text API and analyzed by the emotion engine.

[1352] For example, if a customer says, "The item I ordered hasn't arrived yet," the AI ​​will respond flexibly by saying, "We apologize for the inconvenience. We will check the situation immediately and take appropriate action."

[1353] Information Record:

[1354] All chat and voice call content, as well as analysis results, are recorded on the server and used as analytical data to help improve the service in future.

[1355] Specific examples

[1356] Prompt Sentence Examples

[1357] For example, if a customer types in the chat, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[1358] Or, if a customer says in a voice call, "My order hasn't arrived yet," the AI ​​will respond, "We apologize for the inconvenience. We will immediately check the situation and take action."

[1359] In this way, this system can significantly improve the quality of customer support by analyzing customer sentiment in real time and automatically providing the most appropriate response based on that.

[1360] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1361] Step 1:

[1362] A user uses a smartphone application to upload customer information and conversation scenarios to a server.

[1363] Input: Customer information (name, phone number, company name, etc.), conversation scenario (text format)

[1364] Data processing and data calculation: The server stores the received customer information and conversation scenario in a database.

[1365] Output: Customer information and conversation scenarios stored in a database

[1366] Step 2:

[1367] The server generates a delivery list and a delivery schedule based on the received customer information.

[1368] Input: Customer information in the database

[1369] Data processing and data calculation: The server uses algorithms to create the optimal calling list and schedule for each customer.

[1370] Output: Call list and call schedule

[1371] Step 3:

[1372] The server automatically makes calls based on a calling list and plays a pre-defined introductory conversation when the customer answers.

[1373] Input: Call list, call schedule, conversation scenario

[1374] Data processing and data calculation: When making a call, a conversation scenario is retrieved from the database and an introductory talk is selected.

[1375] Output: Play an introductory talk to the customer

[1376] Step 4:

[1377] Analyze the customer's response and select and execute the optimal switching conversation.

[1378] Input: Customer response (voice or text)

[1379] Data processing and calculation: Convert speech to text using the Google Cloud Speech-to-Text API, and perform sentiment analysis using the Google Cloud Natural Language API. Select the optimal conversation based on sentiment tags.

[1380] Output: Translated responses, sentiment tags, and optimal conversation switching

[1381] Step 5:

[1382] It uses an emotion engine to recognize the customer's emotional state and adjust the content of the conversation based on that recognition.

[1383] Input: Customer response converted to text, sentiment tag

[1384] Data processing and data calculation: The emotion engine analyzes the customer's emotional state and dynamically adjusts the content of the conversation.

[1385] Output: Adjusted speech content

[1386] Step 6:

[1387] Calls and results are recorded to generate data for analysis.

[1388] Input: Call content converted to text, analysis results

[1389] Data processing and calculation: Call content and analysis results are stored in a database, and analytical data is generated for each call session.

[1390] Output: Recorded call content and analysis results, generated data for analysis

[1391] Step 7:

[1392] The user can check the recorded call content and analysis data on the management screen.

[1393] Input: Recorded call content and analysis results, generated analysis data

[1394] Data processing and data calculation: The server retrieves the call content and analysis data from the database and displays them on the management screen.

[1395] Output: Call details and analysis data displayed on the management screen

[1396] Through these steps, the system will achieve optimal customer support automation based on customer sentiment.

[1397] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1398] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1399] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1400] [Fourth embodiment]

[1401] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1402] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1403] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1404] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1405] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1406] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1407] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1408] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1409] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1410] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1411] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1412] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1413] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1414] This invention is a system that enables the complete automation of telemarketing operations. The specific processing and operation of the program of this system will be explained below.

[1415] System configuration

[1416] This system consists of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls to customers.

[1417] Program processing

[1418] 1. Data Entry

[1419] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[1420] 2. Data analysis and talk scenario generation

[1421] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, it generates an introductory conversation for a "new cloud service" for an IT company.

[1422] 3. Automatic call preparation

[1423] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[1424] 4. Auto-dial

[1425] The server automatically places a call based on the outgoing call list, and once the device is connected to the customer, an introductory conversation begins immediately.

[1426] 5. Signal connection and introductory talk begins

[1427] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[1428] 6. Response analysis and switching talk

[1429] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if the customer responds, "What kind of service are you talking about specifically?", the AI ​​will select the conversation, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[1430] 7. Make an appointment

[1431] The server and terminal continue to talk to the customer, aiming to obtain an appointment step by step. For example, if the customer says, "Could I have some time to meet with you to explain in detail?", the server will respond, "Then, would 10:00 a.m. next Tuesday be convenient for you?"

[1432] 8. Information Records

[1433] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[1434] 9. Check the results

[1435] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[1436] Specific examples

[1437] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[1438] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[1439] The server analyzes the customer information on the list and generates an introduction talk for the "new cloud service" and the necessary switching talk.

[1440] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[1441] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[1442] Once an appointment is made, the results are recorded on the server and can be viewed by the user on the management screen.

[1443] This eliminates the need for manual work and enables efficient new customer acquisition.

[1444] The processing flow will be explained below.

[1445] Step 1:

[1446] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching chat scenarios.

[1447] Step 2:

[1448] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[1449] Step 3:

[1450] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[1451] Step 4:

[1452] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[1453] Step 5:

[1454] The device starts a call with the customer, and when the customer answers the phone, the device sends a connection signal to the server. The server's AI system executes the introductory conversation. For example, it outputs an introductory conversation such as, "Hello, this is △△ from ○○ Company. We have a proposal for a new service."

[1455] Step 6:

[1456] When the server receives a customer response, the AI ​​system analyzes it in real time. The AI ​​selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service do you want specifically?", the AI ​​will select a conversation such as, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[1457] Step 7:

[1458] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please give me a moment to explain in more detail?") is executed.

[1459] Step 8:

[1460] The server records all call content as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[1461] Step 9:

[1462] The user logs in to the management screen and checks the detailed list of all call results and appointments obtained. The user adjusts the scenario based on the analysis results and plans the next outbound campaign.

[1463] This series of steps automates all processes involved in telemarketing, enabling effective and efficient new customer acquisition.

[1464] Example 1

[1465] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1466] Traditional telemarketing operations required manual management of customer information, cumbersome outgoing calls, and a rapid response to customer responses. This made it difficult to efficiently acquire new customers and often resulted in wasted resources. Another problem was the significant time required to analyze customer responses and select appropriate conversation topics.

[1467] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1468] In this invention, the server includes means for receiving customer information and conversation scenarios from a user, means for generating a call list and a call schedule based on the customer information, means for automatically making a call and executing a pre-set introductory conversation when the customer answers, means for analyzing the customer's response in real time and using the generated AI model to select and execute an optimal switching conversation, means for recording the call content and results and generating analytical data, and means for the user to check the recorded call content and analytical data. This eliminates the complexity of manual work and enables efficient new customer development and rapid customer response.

[1469] "User" refers to the person who operates the system, logs in to the management screen, and uploads customer information and conversation scenarios.

[1470] "Customer information" refers to data including the customer's name, telephone number, company name, etc., and refers to important information used in telemarketing operations.

[1471] A "conversation scenario" refers to the text that constitutes the introductory talk and transition talk used when talking to a customer.

[1472] "Call List" means a list of customers created by the System to call.

[1473] "Call Schedule" refers to a schedule that sets the time periods during which the system will make calls.

[1474] "Introductory talk" refers to the initial sales pitch given to a customer when making a phone call.

[1475] "Switching talk" refers to an appropriate response talk selected based on the customer's response.

[1476] "Generated AI model" refers to a natural language processing model that is trained and generated using artificial intelligence technology.

[1477] "Real-time analysis" refers to the process of instantly analyzing customer responses and selecting an appropriate response.

[1478] "Call content" refers to all content discussed during a call with a customer.

[1479] "Analysis data" refers to data generated for analysis based on the content of a call and its results.

[1480] "Recorded call content" refers to the audio data recorded during a call and its textual equivalent.

[1481] "Administration screen" refers to the interface used by users to access and operate the system.

[1482] This invention is a system that enables the complete automation of telemarketing operations. This system is composed of a terminal managed by the user, a server that hosts the entire system, and a terminal that makes calls with customers. A specific embodiment of this system will be described below.

[1483] Hardware and software used

[1484] This system uses the following hardware and software:

[1485] Devices: Used to communicate with users and customers. This includes PCs, tablets, and smartphones.

[1486] Server: Hosts the entire system, performs data analysis, and manages calls. The server implements a generative AI model (e.g., GPT-3).

[1487] Administration screen: The interface for users to operate the system. It is accessed using a web browser.

[1488] VoIP software: For example, Asterisk is used to make automated calls.

[1489] Database: We use MySQL or PostgreSQL to store customer information and call details.

[1490] Explanation of the processing flow

[1491] 1. Data Entry

[1492] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[1493] 2. Data analysis and talk scenario generation

[1494] The server analyzes the uploaded customer information and generates individually optimized introductory and transition conversation scenarios. The generative AI model used here is GPT-3, which generates optimal conversation scenarios based on customer information.

[1495] 3. Automatic call preparation

[1496] The server creates a call list and a call schedule based on the time slots specified by the user. The scheduling algorithm is implemented in Python, Java, or other languages.

[1497] 4. Auto-dial

[1498] The server automatically places a call based on a call list. When the terminal connects to the customer, an introductory conversation begins immediately. The call is made over the Internet using VoIP protocols.

[1499] 5. Signal connection and introductory talk begins

[1500] The terminal calls the customer, and when the customer answers, the server's AI system executes an introductory conversation such as, "Hello, this is △△ from XX Company. We have a proposal for a new service."

[1501] 6. Response analysis and switching talk

[1502] Based on the customer's response, the server's AI performs real-time analysis and selects the appropriate conversation. For example, if a customer asks, "What kind of service is this specifically?", the AI ​​will respond with, "It's a new cloud service that will significantly improve business efficiency."

[1503] 7. Make an appointment

[1504] The server and terminal continue to talk to the customer, aiming to secure an appointment step by step. For example, if the customer requests, "Could I have some time to explain in detail?", the server will respond, "How about 10:00 a.m. next Tuesday?"

[1505] 8. Information Records

[1506] All call content and results are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve it for future use.

[1507] 9. Check the results

[1508] Users can log in to the management screen and see a detailed list of all call results and appointments obtained, providing a basis for more effectively planning the next outbound campaign.

[1509] Specific examples

[1510] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[1511] 1. The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[1512] 2. The server analyzes the customer information on the list and generates an introduction token for the "new cloud service" and the necessary switching tokens.

[1513] 3. The server will automatically initiate a call at the specified time, and when the customer answers the call, an introductory conversation will be conducted through the terminal.

[1514] 4. When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[1515] 5. Once the appointment is made, the results are recorded on the server and the user can view them on the management screen.

[1516] Prompt Sentence Examples

[1517] Below are some example prompts to be input to the generative AI model:

[1518] Customer information:

[1519] Name: Yamada Taro

[1520] Company name: Sample Co., Ltd.

[1521] Phone number: 090-1234-5678

[1522] Conversation scenario:

[1523] Introduction talk: Proposing a new cloud service.

[1524] Switch Talk:

[1525] If you are asked about the content: What we are proposing this time is a new cloud service that will significantly improve your business efficiency.

[1526] These procedures enable the complete automation of telemarketing operations, improving the efficiency of new customer development and customer service.

[1527] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1528] Step 1: Data entry

[1529] Input: The user uploads customer information and conversation scenarios from the management screen. Customer information includes the customer's name, phone number, company name, etc., and the conversation scenarios are provided in text format.

[1530] Data processing: The server analyzes the uploaded file and stores the customer information and conversation scenario in a database.

[1531] Output: Customer information and conversation scenario stored in a database.

[1532] Specific operation: A user logs in to the management screen using a PC, selects and uploads customer information (CSV file format) and conversation scenarios (text files). The server receives this and stores it in the database.

[1533] Step 2: Data analysis and talk scenario generation

[1534] Input: Customer information and conversation scenario stored in the database.

[1535] Data processing: The server analyzes customer information and optimizes conversation scenarios. A generative AI model (GPT-3) is used to generate the optimal conversation scenario for each customer.

[1536] Output: Optimized talk scenario.

[1537] Specific operation: The server runs an analysis program, analyzes customer information, and extracts the name, company name, purpose, etc. Based on this, the generative AI model creates individually optimized introductory conversations and transition conversations, which are then stored in a database.

[1538] Step 3: Prepare for automatic calls

[1539] Input: Parsed customer information and optimized conversation scenario.

[1540] Data processing: The server creates a call list and call schedule. The server sets the call schedule based on the time period specified by the user.

[1541] Output: Call list and call schedule.

[1542] Specific operation: The server uses a scheduling algorithm to generate a calling list based on customer information, creates a calling schedule according to the time period specified by the user (e.g., 9:00 AM to 6:00 PM), and saves it in the database.

[1543] Step 4: Auto-call

[1544] Input: Call list and call schedule.

[1545] Data processing: The server automatically makes a call based on the calling list, and immediately starts the introductory conversation after connecting.

[1546] Output: Calls made and introductory talks performed.

[1547] Specific operation: The server uses VoIP software (e.g., Asterisk) to automatically make a call at a specified time based on a calling list. When the customer answers, the terminal plays an introductory message: "Hello, this is XX from XX Company. We have a new service proposal for you."

[1548] Step 5: Connect the signal and start the introduction talk

[1549] Input: Call connect signal when customer answers.

[1550] Data processing: The terminal notifies the server of the customer's response signal and starts the introductory conversation. The server sends the appropriate introductory conversation to the terminal based on the conversation scenario.

[1551] Output: The introductory talk played to the customer.

[1552] Specific operation: The terminal receives a response signal from the customer and notifies the server. The server refers to the conversation scenario and sends the introductory conversation audio data to the terminal, which then plays it back to the customer.

[1553] Step 6: Response analysis and switching talk

[1554] Input: Customer response data and real-time voice data.

[1555] Data processing: The server analyzes customer responses in real time and uses a generative AI model to select the optimal switching conversation.

[1556] Output: Selected switched talk.

[1557] How it works: The device records the customer's response and sends it to the server in real time. The server uses a generative AI model to analyze the response, selects the appropriate switching conversation, and sends it to the device. The device then plays back this conversation.

[1558] Step 7: Make an appointment

[1559] Input: Customer conversation data.

[1560] Data processing: The server and terminal continue the conversation with the customer, aiming to secure an appointment. If necessary, they select additional responses based on the generated conversation scenario.

[1561] Output: The retrieved appointment information.

[1562] Specific operation: The terminal continues the conversation with the customer, and if the customer requests an appointment, the server confirms it at the date and time suggested. For example, if the customer requests, "Could I have some time to explain in detail?", the server responds, "How about next Tuesday at 10:00 AM?"

[1563] Step 8: Record information

[1564] Input: Call content and outcome data.

[1565] Data processing: The server records the call content and results and generates data for analysis.

[1566] Output: Recorded call logs and data for analysis.

[1567] Specific operation: The device records the call and transmits it to the server in real time. The server stores the call data in a database and generates analytical data based on the call results.

[1568] Step 9: Check the results

[1569] Input: Recorded call content and analytical data.

[1570] Data processing: The server organizes the recorded call content and analytical data, and makes it available for users to view on the management screen.

[1571] Output: The confirmation result displayed on the management screen.

[1572] Specific operation: The user accesses the management screen and logs in with their ID and password. The management screen displays the results of each call and details of the appointments obtained. Based on this, the user can plan their next outgoing campaign.

[1573] (Application example 1)

[1574] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1575] In conventional telemarketing, the process of creating an optimal conversation scenario for each customer, manually making the call, and then switching to the appropriate conversation based on the response was extremely time-consuming and labor-intensive. Furthermore, recording and analyzing the content of the call also required manual work, making it difficult to improve the conversation scenario effectively. This meant that improvements in sales efficiency and customer satisfaction were not fully achieved.

[1576] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1577] In this invention, the server includes a means for receiving customer information and conversation scenarios from a user, a means for generating a call list and a call schedule based on the customer information, and a means for automatically generating individually optimized conversation scenarios using a generative AI model. This enables automatic outgoing calls, execution of the optimal conversation scenario for each customer, and selection and execution of optimal conversation switching in real time based on the customer's response. Furthermore, the call content and results can be recorded and analyzed to help improve the conversation scenario for future calls, thereby improving sales efficiency and customer satisfaction.

[1578] A "user" is an entity that utilizes the system to provide customer information and conversation scenarios.

[1579] "Customer Information" refers to basic data about a customer, such as the customer's name, phone number, and company name.

[1580] A "conversation scenario" is a pre-set talk script used in a call with a customer.

[1581] A "call list" is a list of customers to whom the system will automatically call.

[1582] An "outgoing schedule" is a plan that includes the dates, times, and time periods for the system to make calls.

[1583] The "introduction talk" is the basic explanation and greeting that is given first when you call a customer.

[1584] "Switch Talk" is a response script that provides an appropriate reply based on the customer's response.

[1585] "Call content" refers to all conversation data regarding interactions with customers.

[1586] "Analysis data" is data generated based on the content of the call and is useful for improving the talk scenario for future calls.

[1587] A "generative AI model" is an artificial intelligence model that automatically generates sentences and talk scripts based on customer and product information.

[1588] A "prompt sentence" is an initial input sentence that is input into a generative AI model to generate a talk scenario.

[1589] "Product information" refers to information such as the name, characteristics, and price of the product being sold.

[1590] "Follow-up" refers to additional guidance or support provided if the customer expresses interest in purchasing.

[1591] This invention is a system that enables the complete automation of telemarketing operations at mail-order sites. The specific processing and operation of the program of this system will be described below.

[1592] System configuration

[1593] The system is broadly divided into three main parts: a terminal managed by the user, a server that hosts the entire system, and a terminal for making calls.Another feature is that it uses a generative AI model to automatically generate a conversation scenario optimized for each customer based on prompt text.

[1594] Program processing and behavior

[1595] 1. Data Entry

[1596] Users log in to the management screen and upload a conversation scenario containing customer and product information to the system. Customer information includes the customer's name, phone number, company name, etc., while product information includes the product's name, features, price, etc.

[1597] 2. Data analysis and talk scenario generation

[1598] The server analyzes the uploaded customer and product information and uses a generative AI model to automatically generate an individually optimized conversation scenario. For example, the prompt might look like this: "Hello {name}, I'd like to introduce you to {product_name}. This product..." A scenario is generated based on this prompt.

[1599] 3. Automatic call preparation

[1600] The server generates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[1601] 4. Auto-dial

[1602] The server automatically makes a call based on the calling list. When the customer answers, an introductory conversation immediately begins, explaining the product and providing guidance to the customer.

[1603] 5. Response analysis and switching talk

[1604] The server's AI system analyzes customer responses in real time and selects and executes the appropriate switching conversation. For example, if a customer asks, "What kind of product specifically are you talking about?", the AI ​​will select a switching conversation such as, "What we're recommending today is a new, multi-functional microwave oven that has 30% improved heating efficiency compared to conventional models."

[1605] 6. Making appointments and following up

[1606] The server and terminal continue to talk to the customer, aiming to gradually obtain an appointment or induce them to make a purchase. For example, if the customer says, "I'll consider purchasing," the server will follow up by saying, "We'll send you detailed information."

[1607] 7. Information Records

[1608] All call content and results are recorded on the server, and administrators can check this data on the management screen.In addition, the recorded data can be used to improve future talk scenarios.

[1609] Hardware and software used

[1610] Hardware: Customer support robots for smartphones and enterprises.

[1611] software:

[1612] Twilio API (for calls)

[1613] The transformers library (for generative AI models)

[1614] JSON library (for managing data)

[1615] Specific examples

[1616] For example, if an online shopping site runs a campaign selling a new microwave oven, it might do the following:

[1617] An administrator uploads a customer list and product information to the system, and uses a generative AI model to generate a talk scenario called "Introducing a new microwave oven."

[1618] The server analyzes the customer information on the list, initiates an automatic call based on the specified time period, and when the customer answers the phone, immediately makes a "proposal for a new microwave oven."

[1619] When a customer asks a specific question, the server's AI analyzes it in real time and switches to the most appropriate conversation.

[1620] Once an appointment is made, the results are recorded on the server and can be viewed by the administrator on the management screen.

[1621] This eliminates the need for tedious manual work and enables efficient new customer development and sales promotion.

[1622] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1623] Step 1:

[1624] The user logs in to the management screen and uploads a conversation scenario containing customer and product information in file format to the server. The input includes the customer's name, phone number, company name, and product name, features, price, etc. The output is saved on the server. Specifically, the user uploads files such as "Customer Information.csv" and "Product Information.json."

[1625] Step 2:

[1626] The server analyzes the uploaded customer and product information and uses a generative AI model to generate an optimal conversation scenario for each customer. The server receives the customer's individual information and product information as input, and processes the data using a prompt such as, "Hello, {name}, I'd like to introduce you to {product_name}." The server generates an optimized conversation scenario for each customer as output. Specifically, the server invokes a generative AI model (e.g., GPT-3) to automatically generate a conversation scenario.

[1627] Step 3:

[1628] The server generates a calling list and a calling schedule based on customer information. As input, the customer's phone number and the specified time period information are given. As output, a calling list and a calling schedule are generated. Specifically, the server performs an operation to create a list of customers to be called and a calling schedule based on that list.

[1629] Step 4:

[1630] The server automatically makes a call based on the call list. The call list and talk scenario are given as input. The output is a call to the customer and the execution of the talk scenario. Specifically, the call is made automatically using the Twilio API, and when the customer answers, the introduction talk of the talk scenario begins.

[1631] Step 5:

[1632] The server's AI system analyzes the customer's responses in real time and selects and executes the optimal switching conversation. As input, the customer's voice data and pre-generated switching conversation data are given. As output, the appropriate switching conversation is selected and executed. Specifically, speech analysis technology is used to convert the customer's words into text, and an appropriate response is generated based on that.

[1633] Step 6:

[1634] The server and terminal continue the conversation with the customer, gradually making appointments and offering purchase guidance. The inputs are the customer's response data and the conversation scenario. The output is making appointments and offering additional follow-up guidance. Specifically, the server detects the customer's desire to purchase, and the next step is to send detailed materials or set up another follow-up call.

[1635] Step 7:

[1636] All call content and results are recorded on the server, and users can view this data on the management screen. Input includes the content of calls with customers and their results. Output includes the content of calls and their results, which are saved in a database and provided in a viewable format on the management screen. Specifically, call recording data and its transcription are saved on the server, and users can review them and use them to improve their conversation scenarios.

[1637] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1638] This invention is a system that realizes the complete automation of telemarketing operations and further optimizes the content of conversations by recognizing emotions from customer responses. The specific processing and operation of the program for this system are explained below.

[1639] System configuration

[1640] This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[1641] Program processing

[1642] 1. Data Entry

[1643] Users log in to the management screen and upload customer information and conversation scenarios to the server in file format. The customer information includes the customer's name, phone number, company name, etc.

[1644] 2. Data analysis and talk scenario generation

[1645] The server analyzes the uploaded customer information and generates individually optimized introductory and switching conversation scenarios. For example, for an IT company, it generates an introductory conversation for a "new cloud service."

[1646] 3. Automatic call preparation

[1647] The server creates a delivery list and delivery schedule, which is based on a user-specified time period (e.g., 9:00 AM to 6:00 PM).

[1648] 4. Auto-dial

[1649] The server automatically makes a call based on the calling list, and once the device is connected to the customer, the introductory conversation begins immediately.

[1650] 5. Signal connection and introductory talk begins

[1651] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[1652] 6. Response analysis and switching talk

[1653] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if the customer shows interest, a detailed explanation will be provided, but if they show caution, a softer approach will be taken.

[1654] 7. Sentiment Analysis and Talk Modulation

[1655] When the emotion engine analyzes the customer's emotional state, it tags it as, for example, "favorable," "cold," or "anxious." Based on these tags, the server's AI adjusts the content of the conversation and switches appropriately to carry out the conversation. For example, when asked, "What kind of service is this specifically?", if the emotion is "favorable," the conversation will be, "What we are proposing today is a new cloud service that will significantly improve your company's business efficiency."

[1656] 8. Make an appointment

[1657] The server and terminal continue to talk to the customer and aim to make an appointment based on the analysis results of the emotion engine. If the customer shows interest in a specific action, the server and terminal will suggest, for example, "Could you please give us some time to explain in more detail?"

[1658] 9. Information Records

[1659] All call content and the analysis results of the emotion engine are recorded on the server. The recorded data is used to analyze the effectiveness of the talk scenario and to help improve future conversations.

[1660] 10. Check the results

[1661] Users can log in to the dashboard and see all call results, a detailed list of appointments made, and even the emotional state of the customer, providing the basis for more effective planning of the next outbound campaign.

[1662] Specific examples

[1663] For example, when a user attempts to run a campaign proposing a new cloud service for IT companies, the following process is carried out.

[1664] The user uploads a customer list and a conversation scenario (e.g., "Proposal for a new cloud service") to the server.

[1665] The server analyzes the customer information on the list and generates an introductory talk about the "new cloud service" and a switching talk using an emotion engine.

[1666] The server will automatically initiate a call at the specified time, and when the customer answers the call, a "proposal for a new cloud service" will be made through the terminal.

[1667] When a customer asks a specific question, the server's AI analyzes it in real time, and the emotion engine recognizes the customer's emotions and adapts the conversation to suit their needs.

[1668] Once an appointment is made, the results and emotional state are recorded on the server and can be viewed by the user on the management screen.

[1669] This eliminates the hassle of manual work and enables efficient and effective new customer acquisition through optimal emotional responses.

[1670] The processing flow will be explained below.

[1671] Step 1:

[1672] The user logs in to the administration screen. The user uploads a new customer list (customer name, phone number, company name, etc.) to the server in CSV or Excel format. The user can also set up introductory and switching conversation scenarios.

[1673] Step 2:

[1674] The server receives the uploaded customer list and analyzes each record (customer information). The server accesses the scenario data for the introduction talk and switching talk, and generates the optimal talk scenario based on the customer information.

[1675] Step 3:

[1676] The server creates a calling list and calling schedule, and sets the time periods for calling based on the user-specified times (e.g., 9:00 AM to 6:00 PM).

[1677] Step 4:

[1678] The server automatically calls the first customer on the call list, the server processes the call, and the terminal connects to the customer through the telephone line.

[1679] Step 5:

[1680] The device initiates a call to the customer, and when the customer answers, the device sends a connection signal to the server. The server's AI system then executes the introductory conversation. For example, it might output an introductory conversation such as, "Hello, this is a sales representative from a general company. We have a proposal for a new service."

[1681] Step 6:

[1682] When the server receives a customer's response, the AI ​​system analyzes it in real time. Furthermore, the emotion engine recognizes the customer's emotional state and selects the optimal conversation based on the analysis results. For example, if a customer asks, "What kind of service are you talking about specifically?", the AI ​​might select a conversation like, "What we're proposing today is a new cloud service that will significantly improve your company's business efficiency."

[1683] Step 7:

[1684] The device then executes the AI-selected conversation. If the conversation continues and the customer continues to show interest, the next action (e.g., "Could you please spare a moment to explain in more detail?") is executed. The emotion engine analyzes the customer's emotional state in real time and adjusts the conversation content accordingly.

[1685] Step 8:

[1686] The server records all call content and the analysis results of the emotion engine as a log. The server analyzes the recorded call content and generates data to evaluate the effectiveness of the talk scenario. This data can be used to improve future outgoing campaigns.

[1687] Step 9:

[1688] The user logs in to the management screen and checks all call results, a detailed list of appointments obtained, and the emotional state of the customer based on the emotion engine. Based on this, the user can adjust the scenario and plan the next outbound campaign.

[1689] This series of steps automates all processes in telemarketing, enabling efficient and effective new customer acquisition through optimal responses based on the customer's emotional state.

[1690] Example 2

[1691] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1692] Conventional telemarketing systems have difficulty automating a series of customer conversations, particularly when it comes to recognizing customer emotions and providing appropriate conversations. As a result, customer service efficiency is low and results are limited. Furthermore, manual data entry and confirmation of conversation status are required, resulting in poor operational efficiency. Furthermore, improving conversation scenarios requires a significant amount of effort.

[1693] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1694] In this invention, the server includes: means for receiving customer information and conversation scenarios from a user; means for generating a call list and a call schedule based on the customer information; means for automatically making a call and, when the customer answers, executing a pre-set introductory conversation using a generation AI model; means for analyzing the customer's response and selecting and executing an optimal switching conversation using an emotion engine; means for recording the call content and results and generating analytical data based on the analysis results using the emotion engine; and means for the user to review the recorded call content and analytical data. This automates customer service and utilizes emotion recognition to deliver optimal conversations, thereby improving the efficiency and results of customer service. It also reduces manual work and enables effective improvement of conversation scenarios.

[1695] A "user" is a person or organization that logs in to the management screen and uploads customer information and conversation scenarios in order to operate the system.

[1696] "Customer information" is data that includes information such as the customer's name, telephone number, and company name.

[1697] A "conversation scenario" is text data that describes a series of topics and questions to be used in a conversation with a customer.

[1698] A "server" is a computer system that manages customer information and conversation scenarios, and performs automatic calls, response analysis, and result recording.

[1699] A "terminal" is a device that is controlled by the server and is used to make calls to customers.

[1700] A "call list" is a server-generated order or list of calls to make based on customer information.

[1701] The "calling schedule" is a schedule generated by the server that indicates the time periods during which calls are made.

[1702] A "generative AI model" is an algorithm or system that uses AI technology to automatically generate optimal conversation scenarios.

[1703] An "emotion engine" is software or algorithms that analyze customer responses to recognize their emotional state.

[1704] An "introduction talk" is a set script or message that is delivered to the customer at the beginning of the call.

[1705] "Switch talk" refers to additional talk or response to a customer's response.

[1706] "Call Records" are records of all conversations you have with customers.

[1707] "Analysis data" is generated based on the recorded content of the call and the results of emotion analysis, and is data that will be useful for improving future talk scenarios.

[1708] This invention is a system that realizes the complete automation of telemarketing operations and optimizes the content of conversations by recognizing emotions from customer responses. This system consists of a terminal managed by the user, a server that hosts the entire system, a terminal that communicates with customers, and an emotion engine that recognizes customer emotions.

[1709] System configuration

[1710] 1. User Device

[1711] Users access the management screen and upload customer information and conversation scenarios in CSV file format. The management screen is browser-based and is built using HTML, CSS, and JavaScript.

[1712] 2. Server

[1713] The server mainly performs the following tasks:

[1714] Data reception and analysis

[1715] The server receives and analyzes the CSV file containing customer information and conversation scenarios uploaded by the user. The server uses the Python language and the Pandas library to process the data.

[1716] Talk scenario generation

[1717] It uses a generative AI model to generate optimal introductory and transition conversations based on customer information, built using Transformers, an open-source natural language processing library.

[1718] Calling Lists and Scheduling

[1719] Create calling lists and schedules based on user-specified time periods.

[1720] Real-time analysis of customer responses

[1721] Customer responses are converted to text using speech recognition software and then analyzed for sentiment using an emotion engine, powered by the Google Cloud Speech-to-Text API and a sentiment analysis library.

[1722] Call recording and analytics data generation

[1723] The content of the call and the results of the analysis by the emotion engine are recorded in a database, generating analytical data that will be useful for future improvements. MySQL is used as the database.

[1724] 3. Call terminal

[1725] The call terminal receives instructions from the server and makes a call to the customer. After the call connection is established, it plays the introductory speech generated by the generative AI model. The Google Cloud Text-to-Speech API is used for voice synthesis.

[1726] Specific examples

[1727] For example, if a user wants to run a campaign proposing a new cloud service for an IT company, the following process occurs:

[1728] The user logs in to the management screen using a browser and uploads customer information and a conversation scenario (e.g., "Proposal for a new cloud service") in CSV file format.

[1729] The server receives the CSV file and uses Pandas to extract and analyze customer information.

[1730] The server uses a generative AI model using the Transformers library to generate optimal introductory and switching talks for IT companies.

[1731] The server creates a call schedule based on the specified time, and the call terminal automatically calls the customer.

[1732] When the customer answers the phone, the server uses the Google Cloud Text-to-Speech API to play an introductory conversation, such as, "Hello, this is a sales representative from a general company. We have a proposal for a new cloud service."

[1733] The customer's response is converted to text using the Google Cloud Speech-to-Text API, and the emotion engine analyzes the text to recognize the customer's emotional state. For example, if the customer shows interest, it generates an optimal transition phrase that provides detailed explanations.

[1734] The server records the call content and sentiment analysis results in a MySQL database and generates analytical data for future campaigns.

[1735] Prompt Sentence Examples

[1736] Here are some examples of prompts for generative AI models:

[1737] "Please create a talk scenario proposing a new cloud service to an IT company. The customer information is as follows: Name: Taro Yamada, Company: Tech Co., Ltd., Phone number: 012-345-6789."

[1738] This system eliminates the hassle of manual work and enables efficient and effective new customer acquisition by providing optimal emotional responses.

[1739] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1740] Step 1:

[1741] A user logs in to the admin page

[1742] A user accesses the administration screen using a browser and enters login information. Specifically, the user enters a username and password and clicks the "Login" button. The input is the user's authentication information, and the output is the success or failure status of the login authentication.

[1743] Step 2:

[1744] The user uploads customer information and conversation scenarios

[1745] The user uploads customer information and conversation scenarios in CSV file format to the server from the management screen. Specifically, the user clicks the "Select File" button, selects the CSV file, and presses the upload button. The input is the CSV file, and the output is the customer information and conversation scenario data received by the server.

[1746] Step 3:

[1747] The server analyzes customer information

[1748] The server reads the uploaded CSV file and parses the information for each customer. It processes the data using Python and the Pandas library. Specifically, it reads the data from the CSV file, converts it to a data frame, and extracts each field (such as name, phone number, or company name). The input is the contents of the CSV file, and the output is a data frame of parsed customer information.

[1749] Step 4:

[1750] The server generates a talk scenario

[1751] The server uses a generative AI model based on customer information to generate optimal introductory and switching talks. Specifically, the Transformers library is used to input prompts into the model and obtain the generated text. The input is customer information and prompts, and the output is the text of the generated talk scenario.

[1752] Step 5:

[1753] The server creates the delivery list and schedule

[1754] The server creates a call list based on the analyzed customer information and sets a call schedule based on the time period specified by the user. Specifically, it creates a list of customer information and distributes it among the designated call time periods. The input is the analyzed customer information and the designated time period, and the output is a call list and call schedule.

[1755] Step 6:

[1756] The server automatically makes the call

[1757] The server makes calls through the call terminal at the specified time based on the call list. Specifically, it automatically dials each customer in the list. The input is the call list and the call schedule, and the output is the information on successful call connections.

[1758] Step 7:

[1759] The device initiates a call to the customer

[1760] The terminal makes a call, and when the customer answers, it sends a connection signal to the server. Specifically, the terminal sends a "connection successful" signal to the server. The input is the call connection status, and the output is the connection signal.

[1761] Step 8:

[1762] The server executes the introductory token

[1763] The server receives the connection signal and synthesizes and plays the introductory speech generated by the generative AI model. Specifically, it uses the Google Cloud Text-to-Speech API to convert the text into speech and plays it to the customer. The input is the text of the generated introductory speech, and the output is audio data.

[1764] Step 9:

[1765] The server analyzes customer responses in real time.

[1766] The customer's response is converted into text using speech recognition software, and the text is then analyzed using an emotion engine. Specifically, the voice data is converted into text using the Google Cloud Speech-to-Text API, and the emotional state is recognized using an emotion analysis library. The input is the customer's voice data, and the output is text data and the emotion analysis results.

[1767] Step 10:

[1768] The server will switch optimally and execute the conversation.

[1769] The server generates the optimal switching conversation based on the analysis results of the emotion engine, synthesizes it into voice, and plays it back. Specifically, the conversation is generated using a generative AI model and converted into voice data using the Google Cloud Text-to-Speech API. The input is the emotion analysis result and the prompt, and the output is voice data.

[1770] Step 11:

[1771] The server records the call content and results

[1772] The server records the call content and the analysis results from the emotion engine in a database. Specifically, it stores voice data, text data, and emotion analysis results in a MySQL database. The input is the call content and analysis result data, and the output is a database record.

[1773] Step 12:

[1774] The user checks the results on the admin page

[1775] The user logs in to the management screen and checks the call results, the detailed list of appointments obtained, and the customer's emotional state. Specifically, the user clicks the "Check Results" button on the management screen to display a detailed report. The input is the recorded data in the database, and the output is the display of the results.

[1776] (Application example 2)

[1777] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1778] In traditional telemarketing, operators communicate one-on-one with customers, which increases labor costs and makes it difficult to provide consistent customer service. Furthermore, it is difficult to recognize customer emotions in real time and respond optimally accordingly. This results in lower customer satisfaction and a deterioration in sales efficiency. Especially for online shopping sites, the quality of customer support directly impacts customer satisfaction, so it was necessary to automate this process and achieve efficient, high-quality support.

[1779] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1780] In this invention, the server includes means for receiving customer information and conversation scenarios from users and generating a call list and call schedule, means for analyzing customer responses using an emotion engine and selecting and executing optimal switching conversations, and means for recording call content and results and generating data for analysis, thereby enabling the automation of optimal customer support based on customer emotions.

[1781] "User" refers to the person or organization that manages the system and provides customer information and conversation scenarios.

[1782] "Customer information" refers to all information about a customer, such as the customer's name, telephone number, and company name.

[1783] A "conversation scenario" is a pre-defined conversation content that the system sets up for customers.

[1784] A "call list" is a list of phone numbers or contacts that the system generates for automatic calling.

[1785] The "calling schedule" indicates the specific date, time, and time period set for the system to make automatic calls.

[1786] "Introductory talk" refers to the first conversation the system conducts when the customer answers the phone.

[1787] "Switch talk" refers to the content of the next conversation that the system selects and executes in response to the customer's response.

[1788] An "emotion engine" is an engine that analyzes the customer's emotional state in real time and adjusts the optimal content of the conversation based on the results.

[1789] "Call content" refers to all telephone conversations with customers.

[1790] "Analysis data" refers to data generated based on recorded call content and analysis of customer responses, and is used to help improve future calls.

[1791] "System" refers to the entire collection of devices and programs for performing automatic transmission, analysis, recording, and analysis based on this invention.

[1792] This invention is a system that fully automates customer support operations for online shopping sites and provides optimal responses based on customer emotions. The system is primarily composed of a server, a smartphone application, an AI emotion engine, and a data analysis engine.

[1793] System configuration and program processing

[1794] 1. Hardware and Software Configuration

[1795] Server platform: AWS (Amazon Web Services) is used for hosting the entire system and data processing.

[1796] AI Emotion Engine: Uses Google Cloud Natural Language API to analyze customer responses and emotions in real time.

[1797] Speech recognition engine: Using the Google Cloud Speech-to-Text API, the contents of voice calls are converted into text data and made analyzable.

[1798] Smartphone Framework: Build cross-platform smartphone applications using React Native.

[1799] 2. Data processing and calculation

[1800] Data Entry and Analysis:

[1801] Users use a smartphone application to upload customer information and conversation scenarios to the server, with this data provided in text format.

[1802] The server receives customer information and conversation scenarios and generates an optimal calling list and schedule for each customer.

[1803] Sentiment analysis and talk moderation:

[1804] When a chat or voice call is initiated with a customer, the Google Cloud Natural Language API analyzes the data in real time to recognize the customer's sentiment.

[1805] The emotion engine tags emotional states such as "satisfaction," "anxiety," and "anger" and dynamically adjusts the content of the conversation based on the results.

[1806] 3. Response generation and call recording

[1807] Chat Responses:

[1808] When customers use chat, AI automatically generates initial messages and responses to questions.

[1809] For example, if a customer types, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[1810] Voice call answering:

[1811] Even during voice calls, what the customer says is converted into text data using the Google Cloud Speech-to-Text API and analyzed by the emotion engine.

[1812] For example, if a customer says, "The item I ordered hasn't arrived yet," the AI ​​will respond flexibly by saying, "We apologize for the inconvenience. We will check the situation immediately and take appropriate action."

[1813] Information Record:

[1814] All chat and voice call content, as well as analysis results, are recorded on the server and used as analytical data to help improve the service in future.

[1815] Specific examples

[1816] Prompt Sentence Examples

[1817] For example, if a customer types in the chat, "Please tell me about returning a product," the AI ​​will respond, "We will assist you with the return process. Could you please tell us the reason for the return?"

[1818] Or, if a customer says in a voice call, "My order hasn't arrived yet," the AI ​​will respond, "We apologize for the inconvenience. We will immediately check the situation and take action."

[1819] In this way, this system can significantly improve the quality of customer support by analyzing customer sentiment in real time and automatically providing the most appropriate response based on that.

[1820] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1821] Step 1:

[1822] A user uses a smartphone application to upload customer information and conversation scenarios to a server.

[1823] Input: Customer information (name, phone number, company name, etc.), conversation scenario (text format)

[1824] Data processing and data calculation: The server stores the received customer information and conversation scenario in a database.

[1825] Output: Customer information and conversation scenarios stored in a database

[1826] Step 2:

[1827] The server generates a delivery list and a delivery schedule based on the received customer information.

[1828] Input: Customer information in the database

[1829] Data processing and data calculation: The server uses algorithms to create the optimal calling list and schedule for each customer.

[1830] Output: Call list and call schedule

[1831] Step 3:

[1832] The server automatically makes calls based on a calling list and plays a pre-defined introductory conversation when the customer answers.

[1833] Input: Call list, call schedule, conversation scenario

[1834] Data processing and data calculation: When making a call, a conversation scenario is retrieved from the database and an introductory talk is selected.

[1835] Output: Play an introductory talk to the customer

[1836] Step 4:

[1837] Analyze the customer's response and select and execute the optimal switching conversation.

[1838] Input: Customer response (voice or text)

[1839] Data processing and calculation: Convert speech to text using the Google Cloud Speech-to-Text API, and perform sentiment analysis using the Google Cloud Natural Language API. Select the optimal conversation based on sentiment tags.

[1840] Output: Translated responses, sentiment tags, and optimal conversation switching

[1841] Step 5:

[1842] It uses an emotion engine to recognize the customer's emotional state and adjust the content of the conversation based on that recognition.

[1843] Input: Customer response converted to text, sentiment tag

[1844] Data processing and data calculation: The emotion engine analyzes the customer's emotional state and dynamically adjusts the content of the conversation.

[1845] Output: Adjusted speech content

[1846] Step 6:

[1847] Calls and results are recorded to generate data for analysis.

[1848] Input: Call content converted to text, analysis results

[1849] Data processing and calculation: Call content and analysis results are stored in a database, and analytical data is generated for each call session.

[1850] Output: Recorded call content and analysis results, generated data for analysis

[1851] Step 7:

[1852] The user can check the recorded call content and analysis data on the management screen.

[1853] Input: Recorded call content and analysis results, generated analysis data

[1854] Data processing and data calculation: The server retrieves the call content and analysis data from the database and displays them on the management screen.

[1855] Output: Call details and analysis data displayed on the management screen

[1856] Through these steps, the system will achieve optimal customer support automation based on customer sentiment.

[1857] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1858] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1859] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1860] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1861] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1862] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1863] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1864] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1865] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1866] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1867] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1868] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1869] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1870] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1871] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1872] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1873] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1874] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1875] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1876] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1877] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1878] The following is further disclosed regarding the above embodiment.

[1879] (Claim 1)

[1880] means for receiving customer information and a conversation scenario from a ...

Claims

1. means for receiving customer information and a conversation scenario from a user; means for generating a calling list and a calling schedule based on the customer information; A way to automatically make calls and play pre-defined introductory conversations when customers answer; A means to analyze customer responses, select and execute the optimal switching conversation, and means for recording call content and results and generating data for analysis; means for a user to review the recorded call content and analysis data; A system including:

2. The system according to claim 1, wherein customer information is analyzed and an optimal talk scenario is generated for each customer.

3. The system of claim 1, wherein a log of call content is stored and data is generated for analyzing the effectiveness of talk scenarios.

Images