System

Abstract

A system is provided.SOLUTION: A system comprising: means for importing a target list into a database; means for automatically placing a call; means for detecting the start of a call and speaking an introductory talk; means for analyzing the reaction of the other party and generating an appropriate response; and means for storing the call content in the database.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional telemarketing work relies on manual labor, resulting in inefficient time-use and a heavy burden on training new recruits. Furthermore, the large number of phone calls placed on salespeople places a mental and physical burden, resulting in lower work efficiency. Furthermore, even if a lot of time and resources are spent, it is often difficult to secure enough appointments. There is a need for a system that can solve these issues and streamline sales activities. [Means for solving the problem]

[0005] The present invention solves these problems with a system that includes a means for importing a target list into a database, a means for automatically making a call, a means for detecting the start of a call and delivering an introductory speech, a means for analyzing the other party's reaction and generating an appropriate response, and a means for saving the contents of the call in a database. If the other party shows no interest or expresses an objection, the system further includes a means for switching to another conversation and generating and delivering a new speech, enabling flexible responses. After the call ends, the system also includes a means for saving appointment information in a database, allowing salespeople to focus on high-value-added tasks.

[0006] A "target list" is a database or file containing contact information for customers and potential customers.

[0007] A "database" is a system for systematically storing and managing information.

[0008] An "importing means" is a mechanism for importing information from an external data file into an internal database.

[0009] "Means for automatically making phone calls" refers to a function that allows the system to automatically make phone calls without human intervention.

[0010] "Means for detecting the start of a call" refers to the mechanism by which the system recognizes that the call has been connected and the other party has answered.

[0011] An "introduction talk" is a script that is used at the beginning of a phone call to introduce yourself and explain the purpose of the call.

[0012] "Means for speaking" refers to the function of converting text into speech and outputting it as actual speech.

[0013] "Means for analyzing the other party's reaction" refers to an analysis function that allows the system to understand the other party's comments and emotions during a call.

[0014] "Means for generating an appropriate response" refers to the function of the system automatically creating the optimal reply based on the analyzed reaction of the other party.

[0015] "Means for storing call content in a database" refers to a mechanism for recording the information exchanged during a call and the results in a database.

[0016] "Switching talk" is a response that keeps a conversation going when the other person is not interested or objects.

[0017] "Appointment information" refers to detailed information such as the date, time, and location of an arranged meeting or business meeting. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention is a fully automated telesales system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0040] Program processing

[0041] 1. Importing a Target List

[0042] The server uploads the CSV file specified by the user to the system and performs analysis. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically eliminated.

[0043] Examples:

[0044] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[0045] 2. Auto-dial

[0046] The device will automatically retrieve phone numbers from the target list and make calls according to a set schedule. It can also retry calls until it is successful.

[0047] Examples:

[0048] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[0049] 3. Introductory Talk

[0050] When the server detects that the call has been connected, it automatically speaks an introductory phrase. This is done using text-to-speech (TTS) technology. For example, you might say, "Hello, this is XX from XX Company. Do you have time?"

[0051] Examples:

[0052] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[0053] 4. Hearing and responding

[0054] The server analyzes the other party's responses in real time during the call and generates appropriate responses based on the other party's needs and reactions, using natural language processing (NLP) technology to understand what the other party is saying and derive the most appropriate response.

[0055] Examples:

[0056] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0057] 5. Switch Talk

[0058] If the other party is not interested or expresses an objection, the server generates and speaks a transitional conversation appropriate to the situation. The transitional conversation is intended to keep the other party engaged.

[0059] Examples:

[0060] If the other person says, "I'm busy right now," the server suggests, "Even if you don't have time, would you like me to send you the details via email?"

[0061] 6. Appointment Setting

[0062] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. If the other party accepts the date and time, the information is saved in the database.

[0063] Examples:

[0064] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[0065] 7. Storage of call and appointment information

[0066] After the call is over, the server saves the contents of the call, the interview results, and the appointment information in a database, allowing the salesperson to refer to this information later and use it for the next follow-up.

[0067] Examples:

[0068] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[0069] Overall flow

[0070] In this way, the system automates everything from importing target lists to automatically making calls, analyzing and answering calls, switching between conversations, and setting appointments, which results in a significant improvement in sales efficiency and a reduction in the burden on users.

[0071] The processing flow will be explained below.

[0072] Program processing

[0073] 1. Importing a Target List

[0074] server

[0075] Step 1.1: Upload the CSV file specified by the administrator to the system.

[0076] The administrator selects the target list containing the customer information and clicks the upload button.

[0077] Step 1.2: The server parses the CSV file and stores it in the database.

[0078] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[0079] 2. Automated Call Process

[0080] Terminal

[0081] Step 2.1: Get contacts from your target list on a schedule.

[0082] The terminal checks the current time and the set outgoing call schedule, and obtains the customer's phone number to be called next.

[0083] Step 2.2: Make a call automatically.

[0084] The device will automatically call the acquired phone number and retry until the call is connected.

[0085] 3. Introductory Talk

[0086] server

[0087] Step 3.1: Detect the start of a call.

[0088] The server detects that the call connection was successful and proceeds to the next step.

[0089] Step 3.2: Generate and speak the introductory speech.

[0090] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[0091] 4. Hearing and response

[0092] server

[0093] Step 4.1: Analyze the other person's reaction.

[0094] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[0095] Step 4.2: Generate an appropriate response talk.

[0096] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[0097] 5. Switch Talk

[0098] server

[0099] Step 5.1: Analyze the other party's objections and interest.

[0100] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[0101] Step 5.2: Switch and generate a talk.

[0102] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[0103] 6. Appointment Setting

[0104] Terminal

[0105] Step 6.1: Present your available schedule.

[0106] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[0107] Step 6.2: Set an appointment.

[0108] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[0109] 7. Storage of call and appointment information

[0110] server

[0111] Step 7.1: Save the call details, interview results, and appointment information in the database.

[0112] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[0113] Specific examples

[0114] Importing a Target List

[0115] 1. Step 1.1: The user uploads "CustomerList_2023.csv".

[0116] The user selects the CSV file from the management screen and clicks the upload button.

[0117] 2. Step 1.2: The server parses the CSV file and stores the customer information in the database.

[0118] The server analyzes each line of data in the CSV and inserts "Name," "Phone Number," "Company Name," etc. into the database.

[0119] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] In conventional telemarketing systems, operators had to manually make calls, analyze responses, and think of appropriate responses, which required a lot of effort and reduced productivity. Furthermore, the lack of centralized management of customer information and storage of call content made follow-up difficult. Another issue was the lack of scripts for responding appropriately when a customer showed no interest or raised an objection.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes means for importing a target list into a database, means for automatically making a call, means for detecting the start of a call and delivering an introductory speech, means for analyzing the other party's reaction and generating an appropriate response, and means for saving the call content and appointment information in the database. This automates the entire telemarketing process, significantly reducing the operator's workload and improving productivity.

[0125] A "target list" is data that lists targets to call, including information such as customer names, telephone numbers, and company names.

[0126] A "database" is an information management system that stores information in an organized manner and makes it easy to search and update.

[0127] "Means for automatically making phone calls" refers to technology in which a system automatically makes calls to pre-set phone numbers without human intervention.

[0128] "Means for detecting the start of a call and speaking an introductory speech" refers to a technology that detects that a call has been connected and automatically speaks a pre-set message.

[0129] "Means for analyzing the other party's reaction and generating an appropriate response" refers to technology that analyzes the other party's response in real time during a call and automatically generates the optimal response accordingly.

[0130] "Means for storing call content and appointment information in a database" refers to technology that organizes information about call content and appointments and stores it in a database, allowing it to be viewed and analyzed later.

[0131] "Switching talk" is a script or response message that allows you to quickly respond appropriately to situations when the other person is not interested or objects.

[0132] A "generative AI model" is a model that uses artificial intelligence technology to make appropriate responses and inferences from accumulated data.

[0133] This invention is a telemarketing system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0134] First, the user prepares a target list in the form of a CSV file. The server accepts the process of uploading this CSV file to the system. The server then analyzes the uploaded CSV file and stores target list data such as customer names, phone numbers, and company names in a database. The system is designed to automatically eliminate duplicate and incomplete data during analysis.

[0135] Next, the terminal automatically makes a call according to a set schedule based on the target list data acquired from the server, and attempts to make the call based on a certain number of retries until the call is successful.

[0136] Once the call is successfully placed and connected, the server detects the start of the call and automatically uses text-to-speech (TTS) technology to deliver a pre-defined introductory message, such as "Hello, this is XX from XX Company. Is it your time?"

[0137] During the call, the server analyzes the other party's responses in real time and uses a generative AI model to generate an appropriate response based on the other party's needs and reactions. If the other party asks, "What can I do for you?", the server might respond, "I'm calling to provide information about ____. May I speak to you briefly?"

[0138] If the other party is not interested or objects, the server switches over and generates the conversation, using TTS technology to speak it. For example, if the other party says, "I'm busy right now," the server might suggest, "Even if you don't have time, would you like me to send you the details by email?"

[0139] If the other party suggests a date and time for an appointment during a call, the device will refer to the user's schedule database and confirm the available date and time. For example, if the other party suggests, "How about 3:00 p.m. next Tuesday?", the device will reserve this date and time and record it in the database.

[0140] Finally, once the call ends, the server analyzes the call content in real time and uses a generative AI model to store the interview results and appointment information in a database. For example, if the analysis shows that the person is interested in something, this information is recorded in the database.

[0141] This system automates the entire telemarketing process, significantly reducing user effort and improving the efficiency of sales activities.

[0142] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0143] Step 1: Importing your target list

[0144] The server receives a CSV file uploaded by the user. As input, the CSV file contains customer names, phone numbers, company names, etc. The server parses this file and stores the data for each row in a database. Duplicate and incomplete data is automatically eliminated. As output, the organized target list data is stored in the database.

[0145] Specific operation: When a user uploads "Customer List.csv", the server reads this file and records information such as "Taro Tanaka" and "09012345678" in the database.

[0146] Step 2: Auto-dial

[0147] The terminal automatically makes calls according to a set schedule based on the target list data obtained from the server. It uses the target list data and schedule information as input. It attempts to make calls based on a set number of retries until the call is successful. The output is when the call is connected or the number of retries has expired.

[0148] Specific operation: The device will automatically call "09012345678". If the connection fails, it will try to dial again after 10 seconds. This operation will be repeated until the number of retries (e.g. 3 times) is reached.

[0149] Step 3: Introductory Talk

[0150] The server detects the start of a call and uses text-to-speech (TTS) technology to speak a pre-defined introductory phrase. As input, it uses the call connection detection and the introductory phrase script. As output, a voice message is spoken.

[0151] Specific operation: The moment the call is connected, the server starts the speech synthesis engine and speaks, "Hello, this is △△ from ○○ Company. Is it okay if I speak with you?"

[0152] Step 4: Listen and respond

[0153] The server analyzes the other party's responses in real time during the call and generates an appropriate response using a generative AI model. The server uses the other party's voice data and the generative AI model as input. The generated response is then spoken using TTS technology as output.

[0154] Specific operation: When the caller asks, "What can I do for you?", the server analyzes the language data in real time and responds, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0155] Step 5: Switch Talk

[0156] The server generates an appropriate switching conversation if the other party is not interested or objects and speaks it through TTS. As input, it uses the other party's response data and the switching conversation script. As output, the switching conversation is spoken.

[0157] Specific behavior: For example, if the other party says, "I'm busy right now," the server will say, "Even if you don't have time, would you like me to send you the details by email?"

[0158] Step 6: Set up an appointment

[0159] If the other party expresses interest, the terminal consults the user's schedule database and presents available dates and times. The terminal uses the other party's schedule request and the user's schedule data as input. The confirmed appointment information is recorded as output.

[0160] Specific operation: If the other party says during a call, "How about next Tuesday at 3 p.m.?", the device will reserve that date and time and record it in the database.

[0161] Step 7: Save the call

[0162] The server analyzes the call content in real time after the call ends and uses a generative AI model to store the interview results and appointment information in a database. The server uses the call content data and the generative AI model as input. The server outputs the analyzed call content and appointment information, storing them in a database.

[0163] Specific operation: For example, if the content of a call is analyzed as "interested in XX," this information is recorded in the database, along with information such as "set up an appointment for 3 p.m. next Tuesday."

[0164] (Application example 1)

[0165] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0166] Until now, factory quality control and process management have often relied on manual work by many people, and automation has not progressed. In such situations, early detection and response to quality problems can be delayed, which can have a negative impact on production efficiency and product quality. Furthermore, when an abnormality occurs, it is not possible to contact the responsible engineer quickly, which can result in a long time until the problem is resolved. To solve these problems, a system was needed that could automatically detect quality problems, quickly report them, generate appropriate responses, and resolve the problem quickly.

[0167] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0168] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for reporting a quality problem to an engineer, means for performing process management using an automatic call system, and means for automatically notifying the engineer of the generated response. This enables rapid automatic reporting and response generation when a quality problem occurs, significantly improving the efficiency of factory quality control.

[0169] A "target list" is a list of data that contains information necessary for quality control and process management.

[0170] A "database" is a system for storing and managing information in an organized manner.

[0171] "Means for automatically making phone calls" refers to a system that automatically makes calls to designated phone numbers.

[0172] The "means for detecting the start of a call and issuing an introductory message" is a system that confirms that the call has been connected and automatically issues a pre-set introductory message.

[0173] "Means for analyzing the other party's reaction and generating an appropriate response" refers to a system that analyzes the other party's remarks during a call in real time and generates an appropriate reply.

[0174] "Means for storing call content in a database" refers to a system that records information exchanged during a call and stores it in a database.

[0175] The "means of reporting quality issues to engineers" is a system that automatically reports quality issues to the relevant engineers when they occur.

[0176] "Means for process management using an automatic telephone calling system" refers to a system that automatically makes telephone calls to report and handle quality issues as part of the management process.

[0177] The "means for automatically communicating the generated response to the engineer" is a system that automatically transmits the response generated based on the analysis to the engineer in charge.

[0178] The system for carrying out the present invention includes an automatic telephone call system used for quality control in a factory. Specifically, it is composed of the following steps:

[0179] Hardware and Software Use

[0180] The system uses smartphones and dedicated factory terminals and utilizes the following key software components:

[0181] Twilio API (automatic call)

[0182] Google Cloud Speech-to-Text API

[0183] Google Cloud Text-to-Speech API

[0184] Python (programming language)

[0185] Program processing explanation

[0186] First, the user prepares a target list (a list of products with quality issues) in CSV file format and uploads it to the server. The server parses this CSV file and imports it into a database. Each row contains, for example, the product ID, details of the quality issue, and the contact information of the engineer.

[0187] When a call is placed, the device automatically dials the number to the technician's contact information stored in the database. Once the call is connected, the server uses text-to-speech (TTS) technology to deliver a pre-defined introductory message. For example, "There is a quality issue with product ID A1234. Please check for details."

[0188] When the technician responds, the server analyzes the real-time response using speech-to-text technology (Google Cloud Speech-to-Text). For example, if the technician asks, "What is the specific problem?", the server analyzes the response and generates an appropriate answer, such as, "There is a crack. It needs immediate attention."

[0189] All call content is stored in a database, along with details of any quality issues that occur and the technician's responses, ensuring that data is always available for future analysis and improvement.

[0190] Specific examples and prompts for the generative AI model

[0191] As a concrete example, the system will run with the following prompt:

[0192] "There is a quality problem with Product A. Please call the person in charge and report the abnormality."

[0193] "If the agent gives you instructions during the call, please record them and display them here."

[0194] Based on these prompts, the hardware and software work together to automatically report quality issues and generate appropriate responses, greatly streamlining factory quality control.

[0195] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0196] Step 1:

[0197] The user prepares a target list and uploads it to the server in the form of a CSV file, which contains product IDs, details of quality issues, and contact information of engineers. The CSV file is then input to the server, which triggers the next step of the process.

[0198] Step 2:

[0199] The server parses the uploaded CSV file. The server reads this file and imports each row of data into the database. Duplicate and incomplete data is automatically eliminated. The input is a CSV file, and the output is organized database entries.

[0200] Step 3:

[0201] The terminal retrieves the technician's contact information from a database and automatically initiates the call. A phone number is retrieved and a call command is entered into the terminal. This results in an output in which the terminal calls the specified technician's phone number. Retries can be made until the call is established.

[0202] Step 4:

[0203] When the server detects the start of a call, it generates an introductory message and speaks it using text-to-speech (TTS) technology. For example, a message such as "There is a quality problem with product ID A1234. Please check the details." is generated and conveyed to the technician. The input is the call start signal, and the output is the voice message.

[0204] Step 5:

[0205] When the technician responds, the audio is recorded by the server and analyzed in real time. The technician's speech is analyzed using speech-to-text conversion technology (Google Cloud Speech-to-Text). The input is the technician's voice, and the output is text data.

[0206] Step 6:

[0207] The server generates an appropriate response based on the analyzed text data. This response is then converted into audio using text-to-speech (TTS) technology. For example, a message such as "There is a crack. Please check it immediately" is generated. The input is the analyzed text data, and the output is the response message.

[0208] Step 7:

[0209] The server stores all call content in a database, including information exchanged during the call, details of any quality issues that occurred, and the technician's responses. The input is the call content, and the output is the stored database entries.

[0210] Step 8:

[0211] Users can refer to the call content and response information from the database as needed to improve quality control and for subsequent analysis. The input is a database query, and the output is the acquired call records and analysis data.

[0212] This automates the quality control process within the factory and enables rapid response to any problems that arise.

[0213] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0214] This invention is a fully automated telesales system that imports target lists, automatically makes calls, analyzes and responds to calls, switches between conversations appropriately, recognizes emotions using an emotion engine, sets appointments, and saves information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0215] Program processing

[0216] 1. Importing a Target List

[0217] server

[0218] Processing: The administrator uploads a CSV file to the system. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically removed.

[0219] Examples:

[0220] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[0221] 2. Auto-dial

[0222] Terminal

[0223] Action: According to the set schedule, the system retrieves phone numbers from the target list and automatically makes calls. It can also retry calls until it is successful.

[0224] Examples:

[0225] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[0226] 3. Introductory Talk

[0227] server

[0228] Processing: When the system detects that the call is connected, it automatically speaks an introductory phrase. The introductory phrase is delivered using text-to-speech (TTS) technology. For example, "Hello, this is XX from XX Company. Do you have time?"

[0229] Examples:

[0230] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[0231] 4. Hearing and response

[0232] server

[0233] Processing: Analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions. It uses natural language processing (NLP) technology to understand what the other person is saying and derive the best response.

[0234] Examples:

[0235] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0236] 5. Switch Talk

[0237] server

[0238] Processing: If the other person is not interested or objects, the system will switch to a situation-appropriate conversation and generate and speak. For example, if the other person says, "I'm busy right now," the system will suggest, "Even if you don't have time, would you like me to send you the details by email?"

[0239] Examples:

[0240] If the other party says "I'm busy right now" during a call, the server will switch to the appropriate situation, generate a conversation, and speak it.

[0241] 6. Emotion Recognition by Emotion Engine

[0242] server

[0243] Processing: During a call, the emotion engine analyzes the other person's emotions, collecting emotional data from the way the other person speaks, their tone of voice, and their speed to assess their interest and stress levels.

[0244] Examples:

[0245] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[0246] 7. Appointment Setting

[0247] Terminal

[0248] Process: If the other party is interested, the system checks the user's schedule, suggests a free date and time, and sets up an appointment. If the other party approves the date and time, the information is saved in the database.

[0249] Examples:

[0250] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[0251] 8. Storage of call and appointment information

[0252] server

[0253] Processing: After the call is completed, the contents of the call, the interview results, and the appointment information are saved in the database. This allows the salesperson to refer to this information later and use it for the next follow-up.

[0254] Examples:

[0255] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[0256] Overall flow

[0257] In this way, this system automates everything from importing target lists to automatically making phone calls, analyzing and answering call content, switching between conversations, recognizing emotions with an emotion engine, setting appointments, and saving information. As a result, it can significantly improve sales efficiency and reduce the burden on users.

[0258] The processing flow will be explained below.

[0259] Program processing

[0260] 1. Importing a Target List

[0261] server

[0262] Step 1: Upload the CSV file specified by the administrator to the system.

[0263] The administrator selects the target list file containing the customer information and clicks the upload button.

[0264] Step 2: The server parses the CSV file and stores it in the database.

[0265] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[0266] 2. Automated Call Process

[0267] Terminal

[0268] Step 3: Get contacts from your target list on a schedule.

[0269] The terminal checks the current time and the set outgoing call schedule, and obtains the phone number of the next customer to call.

[0270] Step 4: Make the call automatically.

[0271] The device will automatically call the acquired phone number and retry until the call is connected.

[0272] 3. Introductory Talk

[0273] server

[0274] Step 5: Detect the start of a call.

[0275] The server detects that the call connection was successful and proceeds to the next step.

[0276] Step 6: Generate and deliver your introductory speech.

[0277] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[0278] 4. Hearing and response

[0279] server

[0280] Step 7: Analyze the other person's reaction.

[0281] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[0282] Step 8: Generate an appropriate response talk.

[0283] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[0284] 5. Switch Talk

[0285] server

[0286] Step 9: Analyze the other person's objections and interest.

[0287] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[0288] Step 10: Switch and generate a conversation.

[0289] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[0290] 6. Emotion Recognition by Emotion Engine

[0291] server

[0292] Step 11: During the call, the emotion engine analyzes the other person's emotions.

[0293] The emotion engine collects emotional data from the other person's speaking style, tone, and speed to assess their level of interest and stress.

[0294] Step 12: Adjust the response based on the emotion engine analysis.

[0295] The emotion engine analyzes the other person's psychological state, and the AI ​​engine uses that data to adjust and generate response talk.

[0296] 7. Appointment Setting

[0297] Terminal

[0298] Step 13: Present your available schedule.

[0299] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[0300] Step 14: Set an appointment.

[0301] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[0302] 8. Storage of call and appointment information

[0303] server

[0304] Step 15: Save the call details, interview results, and appointment information in the database.

[0305] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[0306] Specific examples

[0307] Importing a Target List

[0308] server

[0309] Step 1: User uploads "CustomerList_2023.csv".

[0310] The user selects the CSV file from the management screen and clicks the upload button.

[0311] Step 2: The server parses the CSV file and stores the customer information in a database.

[0312] The server analyzes each line of data in the CSV and inserts "Name," "Telephone Number," "Company Name," etc. into the database.

[0313] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[0314] Example 2

[0315] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0316] Conventional telemarketing systems required a lot of manual work, resulting in inefficient customer service. Furthermore, call recording and analysis were insufficient, making it difficult to properly understand customer emotions and reactions, making it difficult to improve sales activities. Furthermore, it was difficult to provide appropriate real-time responses and set appointments, leading to declining sales results.

[0317] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for importing a target list into a data record, a means for automatically making a call, a means for detecting the start of a call and uttering an introductory speech using voice synthesis, a means for analyzing the other party's reaction using natural language processing technology and generating an appropriate response, a means for analyzing the other party's emotion during the call using emotion recognition technology, and a means for saving the call content and appointment information in a data record. This enables automated calls and efficient customer service, significantly improving the efficiency of sales activities.

[0318] A "target list" is a data record that contains information about customers who will be targeted for sales and marketing activities.

[0319] A "data record" is a structure within a database that records and stores specific information.

[0320] "Making an automatic call" means that the system automatically makes a call based on a set schedule or conditions.

[0321] "Detecting the start of a call" means that the system confirms that a call has been connected.

[0322] "Speech synthesis" is a technology that artificially generates speech based on text data.

[0323] "Natural language processing technology" refers to technology that allows computers to analyze, understand, and generate human language.

[0324] "Emotion recognition technology" is a technology that analyzes and evaluates the emotions of others from voice and text.

[0325] "Call content" refers to the dialogue and information exchanged during a call.

[0326] "Appointment information" refers to information regarding scheduled meetings or phone calls arranged with customers.

[0327] This invention is a fully automated telesales system that handles everything from importing target lists to automatically making calls, analyzing and answering call content, switching between conversations, recognizing emotions using an emotion engine, setting appointments, and saving information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0328] To implement this system, the following hardware and software are used.

[0329] 1. Importing a Target List

[0330] server

[0331] The server uploads a CSV file specified by the user to the system. The target list contains information such as customer names, phone numbers, and company names. The server uses Python's pandas library to parse the CSV file and insert it into the database. The system is designed to automatically eliminate duplicate and incomplete data.

[0332] Examples:

[0333] When a user uploads "CustomerList.csv", the server parses the file using pandas' read_csv function and saves it in the database.

[0334] 2. Auto-dial

[0335] Terminal

[0336] The device retrieves phone numbers from a database and automatically makes calls based on a set schedule. The device uses VoIP services such as the Twilio API to make calls. It includes a function to retry calls until a successful call is made.

[0337] Examples:

[0338] The device will automatically call the phone number in the first line of "Customer List.csv" using the Twilio API, and will repeat the process until the call is connected or the set number of retries is reached.

[0339] 3. Introductory Talk

[0340] server

[0341] When the server detects that the call is connected, it automatically uses text-to-speech (TTS) technology to deliver an introductory speech. This process uses the Google TTS API. The introductory speech is pre-defined.

[0342] Examples:

[0343] Once the server recognizes that the call has been connected, it uses the Google TTS API to speak, "Hello, this is △△ from XX Company. Is it free for you?"

[0344] 4. Hearing and response

[0345] server

[0346] The server analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions, using Google Cloud's natural language processing (NLP) API to understand what the other person is saying.

[0347] Examples:

[0348] If the caller asks, "What can I do for you?", the server converts the speech into text and uses NLP to respond, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0349] 5. Switch Talk

[0350] server

[0351] If the other party shows no interest or a negative reaction, the server switches to a pre-defined talk pattern and generates and speaks a conversation.

[0352] Examples:

[0353] If the other party says, "I'm busy right now," the server generates and speaks a suggestion: "Even if you don't have time, would you like me to send you the details by email?"

[0354] 6. Emotion Recognition by Emotion Engine

[0355] server

[0356] The server's emotion engine analyzes the other party's emotions during the call. The emotion recognition technology uses IBM Watson's emotion analysis API and other technologies. The emotion is assessed based on the tone, speed, and content of the other party's voice.

[0357] Examples:

[0358] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[0359] 7. Appointment Setting

[0360] Terminal

[0361] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. The datetime module is used to manage the schedule. Once the appointment is set, the information is saved in the database.

[0362] Examples:

[0363] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device records that date and time information in a database.

[0364] 8. Storage of call and appointment information

[0365] server

[0366] After the call is completed, the server stores the call content, interview results, and appointment information in a database, allowing the salesperson to plan the next follow-up.

[0367] Examples:

[0368] If the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[0369] Prompt Sentence Examples

[0370] 1. Please provide detailed steps to import the target list.

[0371] 2. "Please tell me the specific process flow of the auto-dial function."

[0372] 3. "Please explain in detail the steps to deliver an introductory talk using TTS technology."

[0373] 4. "Please tell me how Real-Time Hearing and NLP technology work in real time during a call."

[0374] 5. "Please describe in detail how the call and appointment information will be stored in the database."

[0375] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0376] Step 1:

[0377] Importing a Target List

[0378] server

[0379] Input: CSV file uploaded by the user (e.g. "CustomerList.csv")

[0380] What happens: The server receives the CSV file provided by the user. It reads the CSV file using Python's pandas library. It parses the data in each row and converts it to the correct format.

[0381] Data processing: The server loops through each row of the CSV file and extracts information such as name, phone number, company name, etc. It filters out duplicate and incomplete data.

[0382] Output: Customer information inserted into the database

[0383] Step 2:

[0384] Auto-dial

[0385] Terminal

[0386] Input: Customer information stored in the database (e.g., phone number)

[0387] Specific operation: The device retrieves the next phone number to call from the database based on the configured schedule, and automatically makes the call using a VoIP service (e.g., Twilio API).

[0388] Data calculation: Records the call result and retries as necessary until the call is successful.

[0389] Output: Call connection result

[0390] Step 3:

[0391] Introductory Talk

[0392] server

[0393] Input: Call connection confirmation signal

[0394] What happens: The server detects that the call is connected and uses the Google TTS API to voice a pre-defined introductory phrase.

[0395] Data calculation: The server converts the text of the introductory speech into audio data.

[0396] Output: Sending and playing audio data

[0397] Step 4:

[0398] Hearing and response

[0399] server

[0400] Input: Voice data from the customer during the call

[0401] What it does: The server uses Google Cloud's NLP API to convert the voice data to text, analyzes the customer's response, and applies algorithms to generate the optimal response.

[0402] Data arithmetic: Analyzes customer responses and generates appropriate response text.

[0403] Output: Response audio data

[0404] Step 5:

[0405] Switch Talk

[0406] server

[0407] Input: Negative customer responses (text data and sentiment analysis data)

[0408] Specific operation: The server uses the emotion engine to analyze the customer's emotions, switches and generates conversations according to the situation, and converts text to speech using Google TTS API.

[0409] Data Calculation: Evaluate customer sentiment and generate optimal conversations.

[0410] Output: Switched talk audio data

[0411] Step 6:

[0412] Emotion recognition by emotion engine

[0413] server

[0414] Input: Voice data during a call

[0415] Specific operations: Using emotion recognition technology (e.g., IBM Watson's emotion analysis API), collect and analyze customer emotion data, quantifying the level of emotion based on tone of voice, speed, and choice of words.

[0416] Data calculation: Emotional data is quantified and used as feedback to adjust talk.

[0417] Output: Quantified emotion data and emotion-based responses

[0418] Step 7:

[0419] Appointment setting

[0420] Terminal

[0421] Input: Appointment approvals from customers and schedules from users

[0422] Specific operation: The terminal refers to the user's schedule stored in the database, calculates and presents available time slots using the Python datetime module, and saves the date and time information in the database after receiving approval from the customer.

[0423] Data calculation: Calculates available time in the schedule and sets appointments.

[0424] Output: Appointment information stored in the database

[0425] Step 8:

[0426] Save call and appointment information

[0427] server

[0428] Input: Recorded data and analysis results after the call ends

[0429] Specific operation: The server saves the call contents (recorded data and analysis results) and the set appointment information in the database.

[0430] Data calculation: Converts recorded data into text and saves the analysis results. Also, updates the appointment information to the database.

[0431] Output: Updated database record

[0432] (Application example 2)

[0433] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0434] Conventional automated telesales systems offer features such as importing target lists, automatically making calls, analyzing and responding to call content, setting appointments, and saving information. However, they lack the ability to respond appropriately to user emotions and context. Furthermore, to support a comfortable and efficient travel experience for passengers in autonomous vehicles, more advanced responses and suggestions are required in addition to simply automating calls. This invention aims to solve these issues and significantly improve the user experience in autonomous vehicles.

[0435] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0436] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for analyzing emotions and generating an appropriate response based on the emotions, and means for setting a destination and proposing a route to support the user's travel experience. This enables appropriate responses that take the user's emotions into consideration and makes it possible to provide passengers of autonomous vehicles with a comfortable and efficient travel experience.

[0437] A "target list" is a data list containing specific customer information (such as names, phone numbers, and company names).

[0438] A "database" is a system for efficiently storing, retrieving, and managing information.

[0439] "Automatically dialing" means that the system makes calls programmatically without human intervention.

[0440] "Detecting the start of a call" means that the system automatically recognizes that a call has been connected.

[0441] "Introductory talk" refers to basic conversations such as greetings and self-introductions that take place at the beginning of a call.

[0442] "Analyzing the other person's reactions" means understanding the words and attitudes of the other person during a call in real time.

[0443] "Generating an appropriate response" means automatically creating the optimal reply based on the other person's reaction.

[0444] "Storing call content" means recording the conversation during a call and storing it in a database.

[0445] "Analyzing emotions" means reading the emotional state of the person on the other end of the phone call from their voice and words.

[0446] "Generating an appropriate response based on emotions" means providing a reply or response that is most appropriate to the other person's emotions based on the analysis results.

[0447] "Supporting the user's mobility experience" means providing convenient and efficient services to users while they are on the move.

[0448] "Set destination" is a function that allows the user to specify a place they would like to visit.

[0449] "Route suggestion" refers to showing the best route to a specified destination.

[0450] This invention is applied to a smart glasses application aimed at supporting passengers in autonomous vehicles. The entire system consists of a server and a terminal.

[0451] Program Overview

[0452] The server includes the following means:

[0453] How to import a target list into a database: Read customer data from a CSV file and store it in a database.

[0454] A way to automatically make phone calls: Get phone numbers from a target list and make calls automatically. It also has a retry function.

[0455] Means for detecting the start of a call and uttering introductory speech: Detecting the call connection and uttering introductory speech using a text-to-speech (TTS) engine.

[0456] A means of analyzing the other person's reaction and generating an appropriate response: A natural language processing (NLP) engine is used to analyze the other person's reaction and generate an appropriate response.

[0457] A means of storing call content: Call conversations are recorded and stored in a database.

[0458] A means of analyzing emotions and generating appropriate responses based on emotions: An emotion engine is used to analyze the emotions of the other party and generate a response based on that.

[0459] A means of setting destinations and suggesting routes to support the user's travel experience: Set destinations according to passenger requests and suggest optimal routes.

[0460] Hardware and Software

[0461] Hardware used

[0462] Smart glasses: A device with display and voice input capabilities.

[0463] Server: A high-performance server that performs recognition, analysis, and data storage.

[0464] Software used

[0465] Text-to-speech (TTS) engine: Converts text to speech using Google TTS API etc.

[0466] Natural language processing (NLP) engine: Analyzes the other person's speech using SpaCy and Google Cloud Natural Language API.

[0467] Emotion engine: Analyzes the emotions of the other party during a call using IBM Watson Tone Analyzer.

[0468] Database: PostgreSQL is used to store call details and appointment information.

[0469] Appointment Scheduler: Uses the Google Calendar API to suggest available schedules and manage reservations.

[0470] Specific use cases

[0471] The scenario assumes that a user will wear smart glasses while on the move. The server to which the glasses are connected imports the target list specified by the user (e.g., "Customer List.csv") into a database and automatically initiates a call. Once the call is connected, a TTS engine is used to speak an introductory phrase such as, "Hello, this is a sales representative. May I speak to you?" The NLP engine analyzes the caller's response, and the emotion engine reads the caller's emotions and generates an appropriate response based on this. For example, if the caller responds with "interested," the appointment scheduler suggests available dates and times.

[0472] Prompt Sentence Examples

[0473] "Hello, this is your sales representative. May I speak to you?"

[0474] Example response:

[0475] I'm interested, tell me a bit more about that product.

[0476] This system will enable appropriate responses to be generated taking into account the user's emotions, providing a comfortable and efficient travel experience for passengers in autonomous vehicles.

[0477] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0478] Step 1:

[0479] The server imports the target list into the database. By uploading a CSV file specified by the administrator, the server analyzes each line of the CSV file and stores information such as customer names, phone numbers, and company names in the database. Duplicate and incomplete data is automatically eliminated.

[0480] Input: Target list (CSV file)

[0481] Data processing: Parsing CSV files, eliminating duplicate and incomplete data

[0482] Output: Target list stored in database

[0483] Step 2:

[0484] The server retrieves phone numbers from the target list according to a set schedule and automatically makes calls, and can also retry calls until successful.

[0485] Input: Customer phone number retrieved from database

[0486] Data calculation: Automatic call retry processing

[0487] Output: Phone connection

[0488] Step 3:

[0489] The device detects the start of a call and uses a text-to-speech (TTS) engine to speak an introductory phrase (e.g., "Hello, this is sales. May I speak to you?").

[0490] Input: Call connection detection signal

[0491] Data processing: Text-to-speech (TTS) speech generation

[0492] Output: Introductory speech to be spoken

[0493] Step 4:

[0494] The server analyzes the other party's responses in real time during the call and generates appropriate responses, using a natural language processing (NLP) engine to understand what the other party is saying and derive the most appropriate response.

[0495] Input: The other person's response (audio data)

[0496] Data Computing: Parsing Responses with Natural Language Processing (NLP)

[0497] Output: Appropriate response

[0498] Step 5:

[0499] The server collects emotional data from the other person's tone and speed of voice, analyzes it with an emotion engine, and generates an appropriate response based on the analysis results.

[0500] Input: Voice characteristics of the other person (tone, speed)

[0501] Data Computation: Emotion Analysis with Emotion Engine

[0502] Output: Emotion-based response

[0503] Step 6:

[0504] If the other party shows interest, the device uses the appointment setting system to reference the user's schedule, suggest an available date and time, and set up an appointment. If the other party approves the date and time, the information is saved in a database.

[0505] Input: Appointment date and time accepted by the other party

[0506] Data calculation: setting appointments and saving them to the database

[0507] Output: Saved appointment information

[0508] Step 7:

[0509] After the call is completed, the server stores the contents of the call, the results of the interview, and the appointment information in a database, allowing the user to refer to this information later and use it for the next follow-up.

[0510] Input: Call details, interview results, appointment information

[0511] Data processing: saving to database

[0512] Output: Saved call and appointment information

[0513] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0514] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0515] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0516] [Second embodiment]

[0517] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0518] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0519] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0520] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0521] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0522] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0523] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0524] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0525] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0526] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0527] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0528] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0529] This invention is a fully automated telesales system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0530] Program processing

[0531] 1. Importing a Target List

[0532] The server uploads the CSV file specified by the user to the system and performs analysis. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically eliminated.

[0533] Examples:

[0534] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[0535] 2. Auto-dial

[0536] The device will automatically retrieve phone numbers from the target list and make calls according to a set schedule. It can also retry calls until it is successful.

[0537] Examples:

[0538] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[0539] 3. Introductory Talk

[0540] When the server detects that the call has been connected, it automatically speaks an introductory phrase. This is done using text-to-speech (TTS) technology. For example, you might say, "Hello, this is XX from XX Company. Do you have time?"

[0541] Examples:

[0542] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[0543] 4. Hearing and responding

[0544] The server analyzes the other party's responses in real time during the call and generates appropriate responses based on the other party's needs and reactions, using natural language processing (NLP) technology to understand what the other party is saying and derive the most appropriate response.

[0545] Examples:

[0546] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0547] 5. Switch Talk

[0548] If the other party is not interested or expresses an objection, the server generates and speaks a transitional conversation appropriate to the situation. The transitional conversation is intended to keep the other party engaged.

[0549] Examples:

[0550] If the other person says, "I'm busy right now," the server suggests, "Even if you don't have time, would you like me to send you the details via email?"

[0551] 6. Appointment Setting

[0552] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. If the other party accepts the date and time, the information is saved in the database.

[0553] Examples:

[0554] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[0555] 7. Storage of call and appointment information

[0556] After the call is over, the server saves the contents of the call, the interview results, and the appointment information in a database, allowing the salesperson to refer to this information later and use it for the next follow-up.

[0557] Examples:

[0558] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[0559] Overall flow

[0560] In this way, the system automates everything from importing target lists to automatically making calls, analyzing and answering calls, switching between conversations, and setting appointments, which results in a significant improvement in sales efficiency and a reduction in the burden on users.

[0561] The processing flow will be explained below.

[0562] Program processing

[0563] 1. Importing a Target List

[0564] server

[0565] Step 1.1: Upload the CSV file specified by the administrator to the system.

[0566] The administrator selects the target list containing the customer information and clicks the upload button.

[0567] Step 1.2: The server parses the CSV file and stores it in the database.

[0568] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[0569] 2. Automated Call Process

[0570] Terminal

[0571] Step 2.1: Get contacts from your target list on a schedule.

[0572] The terminal checks the current time and the set outgoing call schedule, and obtains the customer's phone number to be called next.

[0573] Step 2.2: Make a call automatically.

[0574] The device will automatically call the acquired phone number and retry until the call is connected.

[0575] 3. Introductory Talk

[0576] server

[0577] Step 3.1: Detect the start of a call.

[0578] The server detects that the call connection was successful and proceeds to the next step.

[0579] Step 3.2: Generate and speak the introductory speech.

[0580] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[0581] 4. Hearing and response

[0582] server

[0583] Step 4.1: Analyze the other person's reaction.

[0584] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[0585] Step 4.2: Generate an appropriate response talk.

[0586] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[0587] 5. Switch Talk

[0588] server

[0589] Step 5.1: Analyze the other party's objections and interest.

[0590] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[0591] Step 5.2: Switch and generate a talk.

[0592] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[0593] 6. Appointment Setting

[0594] Terminal

[0595] Step 6.1: Present your available schedule.

[0596] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[0597] Step 6.2: Set an appointment.

[0598] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[0599] 7. Storage of call and appointment information

[0600] server

[0601] Step 7.1: Save the call details, interview results, and appointment information in the database.

[0602] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[0603] Specific examples

[0604] Importing a Target List

[0605] 1. Step 1.1: The user uploads "CustomerList_2023.csv".

[0606] The user selects the CSV file from the management screen and clicks the upload button.

[0607] 2. Step 1.2: The server parses the CSV file and stores the customer information in the database.

[0608] The server analyzes each line of data in the CSV and inserts "Name," "Phone Number," "Company Name," etc. into the database.

[0609] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[0610] Example 1

[0611] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] In conventional telemarketing systems, operators had to manually make calls, analyze responses, and think of appropriate responses, which required a lot of effort and reduced productivity. Furthermore, the lack of centralized management of customer information and storage of call content made follow-up difficult. Another issue was the lack of scripts for responding appropriately when a customer showed no interest or raised an objection.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0614] In this invention, the server includes means for importing a target list into a database, means for automatically making a call, means for detecting the start of a call and delivering an introductory speech, means for analyzing the other party's reaction and generating an appropriate response, and means for saving the call content and appointment information in the database. This automates the entire telemarketing process, significantly reducing the operator's workload and improving productivity.

[0615] A "target list" is data that lists targets to call, including information such as customer names, telephone numbers, and company names.

[0616] A "database" is an information management system that stores information in an organized manner and makes it easy to search and update.

[0617] "Means for automatically making phone calls" refers to technology in which a system automatically makes calls to pre-set phone numbers without human intervention.

[0618] "Means for detecting the start of a call and speaking an introductory speech" refers to a technology that detects that a call has been connected and automatically speaks a pre-set message.

[0619] "Means for analyzing the other party's reaction and generating an appropriate response" refers to technology that analyzes the other party's response in real time during a call and automatically generates the optimal response accordingly.

[0620] "Means for storing call content and appointment information in a database" refers to technology that organizes information about call content and appointments and stores it in a database, allowing it to be viewed and analyzed later.

[0621] "Switching talk" is a script or response message that allows you to quickly respond appropriately to situations when the other person is not interested or objects.

[0622] A "generative AI model" is a model that uses artificial intelligence technology to make appropriate responses and inferences from accumulated data.

[0623] This invention is a telemarketing system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0624] First, the user prepares a target list in the form of a CSV file. The server accepts the process of uploading this CSV file to the system. The server then analyzes the uploaded CSV file and stores target list data such as customer names, phone numbers, and company names in a database. The system is designed to automatically eliminate duplicate and incomplete data during analysis.

[0625] Next, the terminal automatically makes a call according to a set schedule based on the target list data acquired from the server, and attempts to make the call based on a certain number of retries until the call is successful.

[0626] Once the call is successfully placed and connected, the server detects the start of the call and automatically uses text-to-speech (TTS) technology to deliver a pre-defined introductory message, such as "Hello, this is XX from XX Company. Is it your time?"

[0627] During the call, the server analyzes the other party's responses in real time and uses a generative AI model to generate an appropriate response based on the other party's needs and reactions. If the other party asks, "What can I do for you?", the server might respond, "I'm calling to provide information about ____. May I speak to you briefly?"

[0628] If the other party is not interested or objects, the server switches over and generates the conversation, using TTS technology to speak it. For example, if the other party says, "I'm busy right now," the server might suggest, "Even if you don't have time, would you like me to send you the details by email?"

[0629] If the other party suggests a date and time for an appointment during a call, the device will refer to the user's schedule database and confirm the available date and time. For example, if the other party suggests, "How about 3:00 p.m. next Tuesday?", the device will reserve this date and time and record it in the database.

[0630] Finally, once the call ends, the server analyzes the call content in real time and uses a generative AI model to store the interview results and appointment information in a database. For example, if the analysis shows that the person is interested in something, this information is recorded in the database.

[0631] This system automates the entire telemarketing process, significantly reducing user effort and improving the efficiency of sales activities.

[0632] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0633] Step 1: Importing your target list

[0634] The server receives a CSV file uploaded by the user. As input, the CSV file contains customer names, phone numbers, company names, etc. The server parses this file and stores the data for each row in a database. Duplicate and incomplete data is automatically eliminated. As output, the organized target list data is stored in the database.

[0635] Specific operation: When a user uploads "Customer List.csv", the server reads this file and records information such as "Taro Tanaka" and "09012345678" in the database.

[0636] Step 2: Auto-dial

[0637] The terminal automatically makes calls according to a set schedule based on the target list data obtained from the server. It uses the target list data and schedule information as input. It attempts to make calls based on a set number of retries until the call is successful. The output is when the call is connected or the number of retries has expired.

[0638] Specific operation: The device will automatically call "09012345678". If the connection fails, it will try to dial again after 10 seconds. This operation will be repeated until the number of retries (e.g. 3 times) is reached.

[0639] Step 3: Introductory Talk

[0640] The server detects the start of a call and uses text-to-speech (TTS) technology to speak a pre-defined introductory phrase. As input, it uses the call connection detection and the introductory phrase script. As output, a voice message is spoken.

[0641] Specific operation: The moment the call is connected, the server starts the speech synthesis engine and speaks, "Hello, this is △△ from ○○ Company. Is it okay if I speak with you?"

[0642] Step 4: Listen and respond

[0643] The server analyzes the other party's responses in real time during the call and generates an appropriate response using a generative AI model. The server uses the other party's voice data and the generative AI model as input. The generated response is then spoken using TTS technology as output.

[0644] Specific operation: When the caller asks, "What can I do for you?", the server analyzes the language data in real time and responds, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0645] Step 5: Switch Talk

[0646] The server generates an appropriate switching conversation if the other party is not interested or objects and speaks it through TTS. As input, it uses the other party's response data and the switching conversation script. As output, the switching conversation is spoken.

[0647] Specific behavior: For example, if the other party says, "I'm busy right now," the server will say, "Even if you don't have time, would you like me to send you the details by email?"

[0648] Step 6: Set up an appointment

[0649] If the other party expresses interest, the terminal consults the user's schedule database and presents available dates and times. The terminal uses the other party's schedule request and the user's schedule data as input. The confirmed appointment information is recorded as output.

[0650] Specific operation: If the other party says during a call, "How about next Tuesday at 3 p.m.?", the device will reserve that date and time and record it in the database.

[0651] Step 7: Save the call

[0652] The server analyzes the call content in real time after the call ends and uses a generative AI model to store the interview results and appointment information in a database. The server uses the call content data and the generative AI model as input. The server outputs the analyzed call content and appointment information, storing them in a database.

[0653] Specific operation: For example, if the content of a call is analyzed as "interested in XX," this information is recorded in the database, along with information such as "set up an appointment for 3 p.m. next Tuesday."

[0654] (Application example 1)

[0655] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] Until now, factory quality control and process management have often relied on manual work by many people, and automation has not progressed. In such situations, early detection and response to quality problems can be delayed, which can have a negative impact on production efficiency and product quality. Furthermore, when an abnormality occurs, it is not possible to contact the responsible engineer quickly, which can result in a long time until the problem is resolved. To solve these problems, a system was needed that could automatically detect quality problems, quickly report them, generate appropriate responses, and resolve the problem quickly.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0658] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for reporting a quality problem to an engineer, means for performing process management using an automatic call system, and means for automatically notifying the engineer of the generated response. This enables rapid automatic reporting and response generation when a quality problem occurs, significantly improving the efficiency of factory quality control.

[0659] A "target list" is a list of data that contains information necessary for quality control and process management.

[0660] A "database" is a system for storing and managing information in an organized manner.

[0661] "Means for automatically making phone calls" refers to a system that automatically makes calls to designated phone numbers.

[0662] The "means for detecting the start of a call and issuing an introductory message" is a system that confirms that the call has been connected and automatically issues a pre-set introductory message.

[0663] "Means for analyzing the other party's reaction and generating an appropriate response" refers to a system that analyzes the other party's remarks during a call in real time and generates an appropriate reply.

[0664] "Means for storing call content in a database" refers to a system that records information exchanged during a call and stores it in a database.

[0665] The "means of reporting quality issues to engineers" is a system that automatically reports quality issues to the relevant engineers when they occur.

[0666] "Means for process management using an automatic telephone calling system" refers to a system that automatically makes telephone calls to report and handle quality issues as part of the management process.

[0667] The "means for automatically communicating the generated response to the engineer" is a system that automatically transmits the response generated based on the analysis to the engineer in charge.

[0668] The system for carrying out the present invention includes an automatic telephone call system used for quality control in a factory. Specifically, it is composed of the following steps:

[0669] Hardware and Software Use

[0670] The system uses smartphones and dedicated factory terminals and utilizes the following key software components:

[0671] Twilio API (automatic call)

[0672] Google Cloud Speech-to-Text API

[0673] Google Cloud Text-to-Speech API

[0674] Python (programming language)

[0675] Program processing explanation

[0676] First, the user prepares a target list (a list of products with quality issues) in CSV file format and uploads it to the server. The server parses this CSV file and imports it into a database. Each row contains, for example, the product ID, details of the quality issue, and the contact information of the engineer.

[0677] When a call is placed, the device automatically dials the number to the technician's contact information stored in the database. Once the call is connected, the server uses text-to-speech (TTS) technology to deliver a pre-defined introductory message. For example, "There is a quality issue with product ID A1234. Please check for details."

[0678] When the technician responds, the server analyzes the real-time response using speech-to-text technology (Google Cloud Speech-to-Text). For example, if the technician asks, "What is the specific problem?", the server analyzes the response and generates an appropriate answer, such as, "There is a crack. It needs immediate attention."

[0679] All call content is stored in a database, along with details of any quality issues that occur and the technician's responses, ensuring that data is always available for future analysis and improvement.

[0680] Specific examples and prompts for the generative AI model

[0681] As a concrete example, the system will run with the following prompt:

[0682] "There is a quality problem with Product A. Please call the person in charge and report the abnormality."

[0683] "If the agent gives you instructions during the call, please record them and display them here."

[0684] Based on these prompts, the hardware and software work together to automatically report quality issues and generate appropriate responses, greatly streamlining factory quality control.

[0685] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0686] Step 1:

[0687] The user prepares a target list and uploads it to the server in the form of a CSV file, which contains product IDs, details of quality issues, and contact information of engineers. The CSV file is then input to the server, which triggers the next step of the process.

[0688] Step 2:

[0689] The server parses the uploaded CSV file. The server reads this file and imports each row of data into the database. Duplicate and incomplete data is automatically eliminated. The input is a CSV file, and the output is organized database entries.

[0690] Step 3:

[0691] The terminal retrieves the technician's contact information from a database and automatically initiates the call. A phone number is retrieved and a call command is entered into the terminal. This results in an output in which the terminal calls the specified technician's phone number. Retries can be made until the call is established.

[0692] Step 4:

[0693] When the server detects the start of a call, it generates an introductory message and speaks it using text-to-speech (TTS) technology. For example, a message such as "There is a quality problem with product ID A1234. Please check the details." is generated and conveyed to the technician. The input is the call start signal, and the output is the voice message.

[0694] Step 5:

[0695] When the technician responds, the audio is recorded by the server and analyzed in real time. The technician's speech is analyzed using speech-to-text conversion technology (Google Cloud Speech-to-Text). The input is the technician's voice, and the output is text data.

[0696] Step 6:

[0697] The server generates an appropriate response based on the analyzed text data. This response is then converted into audio using text-to-speech (TTS) technology. For example, a message such as "There is a crack. Please check it immediately" is generated. The input is the analyzed text data, and the output is the response message.

[0698] Step 7:

[0699] The server stores all call content in a database, including information exchanged during the call, details of any quality issues that occurred, and the technician's responses. The input is the call content, and the output is the stored database entries.

[0700] Step 8:

[0701] Users can refer to the call content and response information from the database as needed to improve quality control and for subsequent analysis. The input is a database query, and the output is the acquired call records and analysis data.

[0702] This automates the quality control process within the factory and enables rapid response to any problems that arise.

[0703] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0704] This invention is a fully automated telesales system that imports target lists, automatically makes calls, analyzes and responds to calls, switches between conversations appropriately, recognizes emotions using an emotion engine, sets appointments, and saves information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0705] Program processing

[0706] 1. Importing a Target List

[0707] server

[0708] Processing: The administrator uploads a CSV file to the system. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically removed.

[0709] Examples:

[0710] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[0711] 2. Auto-dial

[0712] Terminal

[0713] Action: According to the set schedule, the system retrieves phone numbers from the target list and automatically makes calls. It can also retry calls until it is successful.

[0714] Examples:

[0715] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[0716] 3. Introductory Talk

[0717] server

[0718] Processing: When the system detects that the call is connected, it automatically speaks an introductory phrase. The introductory phrase is delivered using text-to-speech (TTS) technology. For example, "Hello, this is XX from XX Company. Do you have time?"

[0719] Examples:

[0720] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[0721] 4. Hearing and response

[0722] server

[0723] Processing: Analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions. It uses natural language processing (NLP) technology to understand what the other person is saying and derive the best response.

[0724] Examples:

[0725] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0726] 5. Switch Talk

[0727] server

[0728] Processing: If the other person is not interested or objects, the system will switch to a situation-appropriate conversation and generate and speak. For example, if the other person says, "I'm busy right now," the system will suggest, "Even if you don't have time, would you like me to send you the details by email?"

[0729] Examples:

[0730] If the other party says "I'm busy right now" during a call, the server will switch to the appropriate situation, generate a conversation, and speak it.

[0731] 6. Emotion Recognition by Emotion Engine

[0732] server

[0733] Processing: During a call, the emotion engine analyzes the other person's emotions, collecting emotional data from the way the other person speaks, their tone of voice, and their speed to assess their interest and stress levels.

[0734] Examples:

[0735] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[0736] 7. Appointment Setting

[0737] Terminal

[0738] Process: If the other party is interested, the system checks the user's schedule, suggests a free date and time, and sets up an appointment. If the other party approves the date and time, the information is saved in the database.

[0739] Examples:

[0740] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[0741] 8. Storage of call and appointment information

[0742] server

[0743] Processing: After the call is completed, the contents of the call, the interview results, and the appointment information are saved in the database. This allows the salesperson to refer to this information later and use it for the next follow-up.

[0744] Examples:

[0745] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[0746] Overall flow

[0747] In this way, this system automates everything from importing target lists to automatically making phone calls, analyzing and answering call content, switching between conversations, recognizing emotions with an emotion engine, setting appointments, and saving information. As a result, it can significantly improve sales efficiency and reduce the burden on users.

[0748] The processing flow will be explained below.

[0749] Program processing

[0750] 1. Importing a Target List

[0751] server

[0752] Step 1: Upload the CSV file specified by the administrator to the system.

[0753] The administrator selects the target list file containing the customer information and clicks the upload button.

[0754] Step 2: The server parses the CSV file and stores it in the database.

[0755] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[0756] 2. Automated Call Process

[0757] Terminal

[0758] Step 3: Get contacts from your target list on a schedule.

[0759] The terminal checks the current time and the set outgoing call schedule, and obtains the phone number of the next customer to call.

[0760] Step 4: Make the call automatically.

[0761] The device will automatically call the acquired phone number and retry until the call is connected.

[0762] 3. Introductory Talk

[0763] server

[0764] Step 5: Detect the start of a call.

[0765] The server detects that the call connection was successful and proceeds to the next step.

[0766] Step 6: Generate and deliver your introductory speech.

[0767] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[0768] 4. Hearing and response

[0769] server

[0770] Step 7: Analyze the other person's reaction.

[0771] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[0772] Step 8: Generate an appropriate response talk.

[0773] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[0774] 5. Switch Talk

[0775] server

[0776] Step 9: Analyze the other person's objections and interest.

[0777] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[0778] Step 10: Switch and generate a conversation.

[0779] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[0780] 6. Emotion Recognition by Emotion Engine

[0781] server

[0782] Step 11: During the call, the emotion engine analyzes the other person's emotions.

[0783] The emotion engine collects emotional data from the other person's speaking style, tone, and speed to assess their level of interest and stress.

[0784] Step 12: Adjust the response based on the emotion engine analysis.

[0785] The emotion engine analyzes the other person's psychological state, and the AI ​​engine uses that data to adjust and generate response talk.

[0786] 7. Appointment Setting

[0787] Terminal

[0788] Step 13: Present your available schedule.

[0789] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[0790] Step 14: Set an appointment.

[0791] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[0792] 8. Storage of call and appointment information

[0793] server

[0794] Step 15: Save the call details, interview results, and appointment information in the database.

[0795] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[0796] Specific examples

[0797] Importing a Target List

[0798] server

[0799] Step 1: User uploads "CustomerList_2023.csv".

[0800] The user selects the CSV file from the management screen and clicks the upload button.

[0801] Step 2: The server parses the CSV file and stores the customer information in a database.

[0802] The server analyzes each line of data in the CSV and inserts "Name," "Telephone Number," "Company Name," etc. into the database.

[0803] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[0804] Example 2

[0805] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0806] Conventional telemarketing systems required a lot of manual work, resulting in inefficient customer service. Furthermore, call recording and analysis were insufficient, making it difficult to properly understand customer emotions and reactions, making it difficult to improve sales activities. Furthermore, it was difficult to provide appropriate real-time responses and set appointments, leading to declining sales results.

[0807] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for importing a target list into a data record, a means for automatically making a call, a means for detecting the start of a call and uttering an introductory speech using voice synthesis, a means for analyzing the other party's reaction using natural language processing technology and generating an appropriate response, a means for analyzing the other party's emotion during the call using emotion recognition technology, and a means for saving the call content and appointment information in a data record. This enables automated calls and efficient customer service, significantly improving the efficiency of sales activities.

[0808] A "target list" is a data record that contains information about customers who will be targeted for sales and marketing activities.

[0809] A "data record" is a structure within a database that records and stores specific information.

[0810] "Making an automatic call" means that the system automatically makes a call based on a set schedule or conditions.

[0811] "Detecting the start of a call" means that the system confirms that a call has been connected.

[0812] "Speech synthesis" is a technology that artificially generates speech based on text data.

[0813] "Natural language processing technology" refers to technology that allows computers to analyze, understand, and generate human language.

[0814] "Emotion recognition technology" is a technology that analyzes and evaluates the emotions of others from voice and text.

[0815] "Call content" refers to the dialogue and information exchanged during a call.

[0816] "Appointment information" refers to information regarding scheduled meetings or phone calls arranged with customers.

[0817] This invention is a fully automated telesales system that handles everything from importing target lists to automatically making calls, analyzing and answering call content, switching between conversations, recognizing emotions using an emotion engine, setting appointments, and saving information. This system is designed to reduce the burden on users and realize efficient sales activities.

[0818] To implement this system, the following hardware and software are used.

[0819] 1. Importing a Target List

[0820] server

[0821] The server uploads a CSV file specified by the user to the system. The target list contains information such as customer names, phone numbers, and company names. The server uses Python's pandas library to parse the CSV file and insert it into the database. The system is designed to automatically eliminate duplicate and incomplete data.

[0822] Examples:

[0823] When a user uploads "CustomerList.csv", the server parses the file using pandas' read_csv function and saves it in the database.

[0824] 2. Auto-dial

[0825] Terminal

[0826] The device retrieves phone numbers from a database and automatically makes calls based on a set schedule. The device uses VoIP services such as the Twilio API to make calls. It includes a function to retry calls until a successful call is made.

[0827] Examples:

[0828] The device will automatically call the phone number in the first line of "Customer List.csv" using the Twilio API, and will repeat the process until the call is connected or the set number of retries is reached.

[0829] 3. Introductory Talk

[0830] server

[0831] When the server detects that the call is connected, it automatically uses text-to-speech (TTS) technology to deliver an introductory speech. This process uses the Google TTS API. The introductory speech is pre-defined.

[0832] Examples:

[0833] Once the server recognizes that the call has been connected, it uses the Google TTS API to speak, "Hello, this is △△ from XX Company. Is it free for you?"

[0834] 4. Hearing and response

[0835] server

[0836] The server analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions, using Google Cloud's natural language processing (NLP) API to understand what the other person is saying.

[0837] Examples:

[0838] If the caller asks, "What can I do for you?", the server converts the speech into text and uses NLP to respond, "I'm calling to provide information about XX. May I speak to you for a moment?"

[0839] 5. Switch Talk

[0840] server

[0841] If the other party shows no interest or a negative reaction, the server switches to a pre-defined talk pattern and generates and speaks a conversation.

[0842] Examples:

[0843] If the other party says, "I'm busy right now," the server generates and speaks a suggestion: "Even if you don't have time, would you like me to send you the details by email?"

[0844] 6. Emotion Recognition by Emotion Engine

[0845] server

[0846] The server's emotion engine analyzes the other party's emotions during the call. The emotion recognition technology uses IBM Watson's emotion analysis API and other technologies. The emotion is assessed based on the tone, speed, and content of the other party's voice.

[0847] Examples:

[0848] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[0849] 7. Appointment Setting

[0850] Terminal

[0851] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. The datetime module is used to manage the schedule. Once the appointment is set, the information is saved in the database.

[0852] Examples:

[0853] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device records that date and time information in a database.

[0854] 8. Storage of call and appointment information

[0855] server

[0856] After the call is completed, the server stores the call content, interview results, and appointment information in a database, allowing the salesperson to plan the next follow-up.

[0857] Examples:

[0858] If the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[0859] Prompt Sentence Examples

[0860] 1. Please provide detailed steps to import the target list.

[0861] 2. "Please tell me the specific process flow of the auto-dial function."

[0862] 3. "Please explain in detail the steps to deliver an introductory talk using TTS technology."

[0863] 4. "Please tell me how Real-Time Hearing and NLP technology work in real time during a call."

[0864] 5. "Please describe in detail how the call and appointment information will be stored in the database."

[0865] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0866] Step 1:

[0867] Importing a Target List

[0868] server

[0869] Input: CSV file uploaded by the user (e.g. "CustomerList.csv")

[0870] What happens: The server receives the CSV file provided by the user. It reads the CSV file using Python's pandas library. It parses the data in each row and converts it to the correct format.

[0871] Data processing: The server loops through each row of the CSV file and extracts information such as name, phone number, company name, etc. It filters out duplicate and incomplete data.

[0872] Output: Customer information inserted into the database

[0873] Step 2:

[0874] Auto-dial

[0875] Terminal

[0876] Input: Customer information stored in the database (e.g., phone number)

[0877] Specific operation: The device retrieves the next phone number to call from the database based on the configured schedule, and automatically makes the call using a VoIP service (e.g., Twilio API).

[0878] Data calculation: Records the call result and retries as necessary until the call is successful.

[0879] Output: Call connection result

[0880] Step 3:

[0881] Introductory Talk

[0882] server

[0883] Input: Call connection confirmation signal

[0884] What happens: The server detects that the call is connected and uses the Google TTS API to voice a pre-defined introductory phrase.

[0885] Data calculation: The server converts the text of the introductory speech into audio data.

[0886] Output: Sending and playing audio data

[0887] Step 4:

[0888] Hearing and response

[0889] server

[0890] Input: Voice data from the customer during the call

[0891] What it does: The server uses Google Cloud's NLP API to convert the voice data to text, analyzes the customer's response, and applies algorithms to generate the optimal response.

[0892] Data arithmetic: Analyzes customer responses and generates appropriate response text.

[0893] Output: Response audio data

[0894] Step 5:

[0895] Switch Talk

[0896] server

[0897] Input: Negative customer responses (text data and sentiment analysis data)

[0898] Specific operation: The server uses the emotion engine to analyze the customer's emotions, switches and generates conversations according to the situation, and converts text to speech using Google TTS API.

[0899] Data Calculation: Evaluate customer sentiment and generate optimal conversations.

[0900] Output: Switched talk audio data

[0901] Step 6:

[0902] Emotion recognition by emotion engine

[0903] server

[0904] Input: Voice data during a call

[0905] Specific operations: Using emotion recognition technology (e.g., IBM Watson's emotion analysis API), collect and analyze customer emotion data, quantifying the level of emotion based on tone of voice, speed, and choice of words.

[0906] Data calculation: Emotional data is quantified and used as feedback to adjust talk.

[0907] Output: Quantified emotion data and emotion-based responses

[0908] Step 7:

[0909] Appointment setting

[0910] Terminal

[0911] Input: Appointment approvals from customers and schedules from users

[0912] Specific operation: The terminal refers to the user's schedule stored in the database, calculates and presents available time slots using the Python datetime module, and saves the date and time information in the database after receiving approval from the customer.

[0913] Data calculation: Calculates available time in the schedule and sets appointments.

[0914] Output: Appointment information stored in the database

[0915] Step 8:

[0916] Save call and appointment information

[0917] server

[0918] Input: Recorded data and analysis results after the call ends

[0919] Specific operation: The server saves the call contents (recorded data and analysis results) and the set appointment information in the database.

[0920] Data calculation: Converts recorded data into text and saves the analysis results. Also, updates the appointment information to the database.

[0921] Output: Updated database record

[0922] (Application example 2)

[0923] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0924] Conventional automated telesales systems offer features such as importing target lists, automatically making calls, analyzing and responding to call content, setting appointments, and saving information. However, they lack the ability to respond appropriately to user emotions and context. Furthermore, to support a comfortable and efficient travel experience for passengers in autonomous vehicles, more advanced responses and suggestions are required in addition to simply automating calls. This invention aims to solve these issues and significantly improve the user experience in autonomous vehicles.

[0925] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0926] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for analyzing emotions and generating an appropriate response based on the emotions, and means for setting a destination and proposing a route to support the user's travel experience. This enables appropriate responses that take the user's emotions into consideration and makes it possible to provide passengers of autonomous vehicles with a comfortable and efficient travel experience.

[0927] A "target list" is a data list containing specific customer information (such as names, phone numbers, and company names).

[0928] A "database" is a system for efficiently storing, retrieving, and managing information.

[0929] "Automatically dialing" means that the system makes calls programmatically without human intervention.

[0930] "Detecting the start of a call" means that the system automatically recognizes that a call has been connected.

[0931] "Introductory talk" refers to basic conversations such as greetings and self-introductions that take place at the beginning of a call.

[0932] "Analyzing the other person's reactions" means understanding the words and attitudes of the other person during a call in real time.

[0933] "Generating an appropriate response" means automatically creating the optimal reply based on the other person's reaction.

[0934] "Storing call content" means recording the conversation during a call and storing it in a database.

[0935] "Analyzing emotions" means reading the emotional state of the person on the other end of the phone call from their voice and words.

[0936] "Generating an appropriate response based on emotions" means providing a reply or response that is most appropriate to the other person's emotions based on the analysis results.

[0937] "Supporting the user's mobility experience" means providing convenient and efficient services to users while they are on the move.

[0938] "Set destination" is a function that allows the user to specify a place they would like to visit.

[0939] "Route suggestion" refers to showing the best route to a specified destination.

[0940] This invention is applied to a smart glasses application aimed at supporting passengers in autonomous vehicles. The entire system consists of a server and a terminal.

[0941] Program Overview

[0942] The server includes the following means:

[0943] How to import a target list into a database: Read customer data from a CSV file and store it in a database.

[0944] A way to automatically make phone calls: Get phone numbers from a target list and make calls automatically. It also has a retry function.

[0945] Means for detecting the start of a call and uttering introductory speech: Detecting the call connection and uttering introductory speech using a text-to-speech (TTS) engine.

[0946] A means of analyzing the other person's reaction and generating an appropriate response: A natural language processing (NLP) engine is used to analyze the other person's reaction and generate an appropriate response.

[0947] A means of storing call content: Call conversations are recorded and stored in a database.

[0948] A means of analyzing emotions and generating appropriate responses based on emotions: An emotion engine is used to analyze the emotions of the other party and generate a response based on that.

[0949] A means of setting destinations and suggesting routes to support the user's travel experience: Set destinations according to passenger requests and suggest optimal routes.

[0950] Hardware and Software

[0951] Hardware used

[0952] Smart glasses: A device with display and voice input capabilities.

[0953] Server: A high-performance server that performs recognition, analysis, and data storage.

[0954] Software used

[0955] Text-to-speech (TTS) engine: Converts text to speech using Google TTS API etc.

[0956] Natural language processing (NLP) engine: Analyzes the other person's speech using SpaCy and Google Cloud Natural Language API.

[0957] Emotion engine: Analyzes the emotions of the other party during a call using IBM Watson Tone Analyzer.

[0958] Database: PostgreSQL is used to store call details and appointment information.

[0959] Appointment Scheduler: Uses the Google Calendar API to suggest available schedules and manage reservations.

[0960] Specific use cases

[0961] The scenario assumes that a user will wear smart glasses while on the move. The server to which the glasses are connected imports the target list specified by the user (e.g., "Customer List.csv") into a database and automatically initiates a call. Once the call is connected, a TTS engine is used to speak an introductory phrase such as, "Hello, this is a sales representative. May I speak to you?" The NLP engine analyzes the caller's response, and the emotion engine reads the caller's emotions and generates an appropriate response based on this. For example, if the caller responds with "interested," the appointment scheduler suggests available dates and times.

[0962] Prompt Sentence Examples

[0963] "Hello, this is your sales representative. May I speak to you?"

[0964] Example response:

[0965] I'm interested, tell me a bit more about that product.

[0966] This system will enable appropriate responses to be generated taking into account the user's emotions, providing a comfortable and efficient travel experience for passengers in autonomous vehicles.

[0967] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0968] Step 1:

[0969] The server imports the target list into the database. By uploading a CSV file specified by the administrator, the server analyzes each line of the CSV file and stores information such as customer names, phone numbers, and company names in the database. Duplicate and incomplete data is automatically eliminated.

[0970] Input: Target list (CSV file)

[0971] Data processing: Parsing CSV files, eliminating duplicate and incomplete data

[0972] Output: Target list stored in database

[0973] Step 2:

[0974] The server retrieves phone numbers from the target list according to a set schedule and automatically makes calls, and can also retry calls until successful.

[0975] Input: Customer phone number retrieved from database

[0976] Data calculation: Automatic call retry processing

[0977] Output: Phone connection

[0978] Step 3:

[0979] The device detects the start of a call and uses a text-to-speech (TTS) engine to speak an introductory phrase (e.g., "Hello, this is sales. May I speak to you?").

[0980] Input: Call connection detection signal

[0981] Data processing: Text-to-speech (TTS) speech generation

[0982] Output: Introductory speech to be spoken

[0983] Step 4:

[0984] The server analyzes the other party's responses in real time during the call and generates appropriate responses, using a natural language processing (NLP) engine to understand what the other party is saying and derive the most appropriate response.

[0985] Input: The other person's response (audio data)

[0986] Data Computing: Parsing Responses with Natural Language Processing (NLP)

[0987] Output: Appropriate response

[0988] Step 5:

[0989] The server collects emotional data from the other person's tone and speed of voice, analyzes it with an emotion engine, and generates an appropriate response based on the analysis results.

[0990] Input: Voice characteristics of the other person (tone, speed)

[0991] Data Computation: Emotion Analysis with Emotion Engine

[0992] Output: Emotion-based response

[0993] Step 6:

[0994] If the other party shows interest, the device uses the appointment setting system to reference the user's schedule, suggest an available date and time, and set up an appointment. If the other party approves the date and time, the information is saved in a database.

[0995] Input: Appointment date and time accepted by the other party

[0996] Data calculation: setting appointments and saving them to the database

[0997] Output: Saved appointment information

[0998] Step 7:

[0999] After the call is completed, the server stores the contents of the call, the results of the interview, and the appointment information in a database, allowing the user to refer to this information later and use it for the next follow-up.

[1000] Input: Call details, interview results, appointment information

[1001] Data processing: saving to database

[1002] Output: Saved call and appointment information

[1003] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1004] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1005] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1006] [Third embodiment]

[1007] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1008] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1009] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1010] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1011] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1012] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1013] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1014] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1015] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1016] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1017] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1018] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1019] This invention is a fully automated telesales system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1020] Program processing

[1021] 1. Importing a Target List

[1022] The server uploads the CSV file specified by the user to the system and performs analysis. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically eliminated.

[1023] Examples:

[1024] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[1025] 2. Auto-dial

[1026] The device will automatically retrieve phone numbers from the target list and make calls according to a set schedule. It can also retry calls until it is successful.

[1027] Examples:

[1028] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[1029] 3. Introductory Talk

[1030] When the server detects that the call has been connected, it automatically speaks an introductory phrase. This is done using text-to-speech (TTS) technology. For example, you might say, "Hello, this is XX from XX Company. Do you have time?"

[1031] Examples:

[1032] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[1033] 4. Hearing and responding

[1034] The server analyzes the other party's responses in real time during the call and generates appropriate responses based on the other party's needs and reactions, using natural language processing (NLP) technology to understand what the other party is saying and derive the most appropriate response.

[1035] Examples:

[1036] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1037] 5. Switch Talk

[1038] If the other party is not interested or expresses an objection, the server generates and speaks a transitional conversation appropriate to the situation. The transitional conversation is intended to keep the other party engaged.

[1039] Examples:

[1040] If the other person says, "I'm busy right now," the server suggests, "Even if you don't have time, would you like me to send you the details via email?"

[1041] 6. Appointment Setting

[1042] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. If the other party accepts the date and time, the information is saved in the database.

[1043] Examples:

[1044] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[1045] 7. Storage of call and appointment information

[1046] After the call is over, the server saves the contents of the call, the interview results, and the appointment information in a database, allowing the salesperson to refer to this information later and use it for the next follow-up.

[1047] Examples:

[1048] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[1049] Overall flow

[1050] In this way, the system automates everything from importing target lists to automatically making calls, analyzing and answering calls, switching between conversations, and setting appointments, which results in a significant improvement in sales efficiency and a reduction in the burden on users.

[1051] The processing flow will be explained below.

[1052] Program processing

[1053] 1. Importing a Target List

[1054] server

[1055] Step 1.1: Upload the CSV file specified by the administrator to the system.

[1056] The administrator selects the target list containing the customer information and clicks the upload button.

[1057] Step 1.2: The server parses the CSV file and stores it in the database.

[1058] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[1059] 2. Automated Call Process

[1060] Terminal

[1061] Step 2.1: Get contacts from your target list on a schedule.

[1062] The terminal checks the current time and the set outgoing call schedule, and obtains the customer's phone number to be called next.

[1063] Step 2.2: Make a call automatically.

[1064] The device will automatically call the acquired phone number and retry until the call is connected.

[1065] 3. Introductory Talk

[1066] server

[1067] Step 3.1: Detect the start of a call.

[1068] The server detects that the call connection was successful and proceeds to the next step.

[1069] Step 3.2: Generate and speak the introductory speech.

[1070] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[1071] 4. Hearing and response

[1072] server

[1073] Step 4.1: Analyze the other person's reaction.

[1074] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[1075] Step 4.2: Generate an appropriate response talk.

[1076] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[1077] 5. Switch Talk

[1078] server

[1079] Step 5.1: Analyze the other party's objections and interests.

[1080] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[1081] Step 5.2: Switch and generate a talk.

[1082] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[1083] 6. Appointment Setting

[1084] Terminal

[1085] Step 6.1: Present your available schedule.

[1086] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[1087] Step 6.2: Set an appointment.

[1088] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[1089] 7. Storage of call and appointment information

[1090] server

[1091] Step 7.1: Save the call details, interview results, and appointment information in the database.

[1092] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[1093] Specific examples

[1094] Importing a Target List

[1095] 1. Step 1.1: The user uploads "CustomerList_2023.csv".

[1096] The user selects the CSV file from the management screen and clicks the upload button.

[1097] 2. Step 1.2: The server parses the CSV file and stores the customer information in the database.

[1098] The server analyzes each line of data in the CSV and inserts "Name," "Phone Number," "Company Name," etc. into the database.

[1099] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[1100] Example 1

[1101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1102] In conventional telemarketing systems, operators had to manually make calls, analyze responses, and think of appropriate responses, which required a lot of effort and reduced productivity. Furthermore, the lack of centralized management of customer information and storage of call content made follow-up difficult. Another issue was the lack of scripts for responding appropriately when a customer showed no interest or raised an objection.

[1103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1104] In this invention, the server includes means for importing a target list into a database, means for automatically making a call, means for detecting the start of a call and delivering an introductory speech, means for analyzing the other party's reaction and generating an appropriate response, and means for saving the call content and appointment information in the database. This automates the entire telemarketing process, significantly reducing the operator's workload and improving productivity.

[1105] A "target list" is data that lists targets to call, including information such as customer names, telephone numbers, and company names.

[1106] A "database" is an information management system that stores information in an organized manner and makes it easy to search and update.

[1107] "Means for automatically making phone calls" refers to technology in which a system automatically makes calls to pre-set phone numbers without human intervention.

[1108] "Means for detecting the start of a call and speaking an introductory speech" refers to a technology that detects that a call has been connected and automatically speaks a pre-set message.

[1109] "Means for analyzing the other party's reaction and generating an appropriate response" refers to technology that analyzes the other party's response in real time during a call and automatically generates the optimal response accordingly.

[1110] "Means for storing call content and appointment information in a database" refers to technology that organizes information about call content and appointments and stores it in a database, allowing it to be viewed and analyzed later.

[1111] "Switching talk" is a script or response message that allows you to quickly respond appropriately to situations when the other person is not interested or objects.

[1112] A "generative AI model" is a model that uses artificial intelligence technology to make appropriate responses and inferences from accumulated data.

[1113] This invention is a telemarketing system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1114] First, the user prepares a target list in the form of a CSV file. The server accepts the process of uploading this CSV file to the system. The server then analyzes the uploaded CSV file and stores target list data such as customer names, phone numbers, and company names in a database. The system is designed to automatically eliminate duplicate and incomplete data during analysis.

[1115] Next, the terminal automatically makes a call according to a set schedule based on the target list data acquired from the server, and attempts to make the call based on a certain number of retries until the call is successful.

[1116] Once the call is successfully placed and connected, the server detects the start of the call and automatically uses text-to-speech (TTS) technology to deliver a pre-defined introductory message, such as "Hello, this is XX from XX Company. Is it your time?"

[1117] During the call, the server analyzes the other party's responses in real time and uses a generative AI model to generate an appropriate response based on the other party's needs and reactions. If the other party asks, "What can I do for you?", the server might respond, "I'm calling to provide information about ____. May I speak to you briefly?"

[1118] If the other party is not interested or objects, the server switches over and generates the conversation, using TTS technology to speak it. For example, if the other party says, "I'm busy right now," the server might suggest, "Even if you don't have time, would you like me to send you the details by email?"

[1119] If the other party suggests a date and time for an appointment during a call, the device will refer to the user's schedule database and confirm the available date and time. For example, if the other party suggests, "How about 3:00 p.m. next Tuesday?", the device will reserve this date and time and record it in the database.

[1120] Finally, once the call ends, the server analyzes the call content in real time and uses a generative AI model to store the interview results and appointment information in a database. For example, if the analysis shows that the person is interested in something, this information is recorded in the database.

[1121] This system automates the entire telemarketing process, significantly reducing user effort and improving the efficiency of sales activities.

[1122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1123] Step 1: Importing your target list

[1124] The server receives a CSV file uploaded by the user. As input, the CSV file contains customer names, phone numbers, company names, etc. The server parses this file and stores the data for each row in a database. Duplicate and incomplete data is automatically eliminated. As output, the organized target list data is stored in the database.

[1125] Specific operation: When a user uploads "Customer List.csv", the server reads this file and records information such as "Taro Tanaka" and "09012345678" in the database.

[1126] Step 2: Auto-dial

[1127] The terminal automatically makes calls according to a set schedule based on the target list data obtained from the server. It uses the target list data and schedule information as input. It attempts to make calls based on a set number of retries until the call is successful. The output is when the call is connected or the number of retries has expired.

[1128] Specific operation: The device will automatically call "09012345678". If the connection fails, it will try to dial again after 10 seconds. This operation will be repeated until the number of retries (e.g. 3 times) is reached.

[1129] Step 3: Introductory Talk

[1130] The server detects the start of a call and uses text-to-speech (TTS) technology to speak a pre-defined introductory phrase. As input, it uses the call connection detection and the introductory phrase script. As output, a voice message is spoken.

[1131] Specific operation: The moment the call is connected, the server starts the speech synthesis engine and speaks, "Hello, this is △△ from ○○ Company. Is it okay if I speak with you?"

[1132] Step 4: Listen and respond

[1133] The server analyzes the other party's responses in real time during the call and generates an appropriate response using a generative AI model. The server uses the other party's voice data and the generative AI model as input. The generated response is then spoken using TTS technology as output.

[1134] Specific operation: When the caller asks, "What can I do for you?", the server analyzes the language data in real time and responds, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1135] Step 5: Switch Talk

[1136] The server generates an appropriate switching conversation if the other party is not interested or objects and speaks it through TTS. As input, it uses the other party's response data and the switching conversation script. As output, the switching conversation is spoken.

[1137] Specific behavior: For example, if the other party says, "I'm busy right now," the server will say, "Even if you don't have time, would you like me to send you the details by email?"

[1138] Step 6: Set up an appointment

[1139] If the other party expresses interest, the terminal consults the user's schedule database and presents available dates and times. The terminal uses the other party's schedule request and the user's schedule data as input. The confirmed appointment information is recorded as output.

[1140] Specific operation: If the other party says during a call, "How about next Tuesday at 3 p.m.?", the device will reserve that date and time and record it in the database.

[1141] Step 7: Save the call

[1142] The server analyzes the call content in real time after the call ends and uses a generative AI model to store the interview results and appointment information in a database. The server uses the call content data and the generative AI model as input. The server outputs the analyzed call content and appointment information, storing them in a database.

[1143] Specific operation: For example, if the content of a call is analyzed as "interested in XX," this information is recorded in the database, along with information such as "set up an appointment for 3 p.m. next Tuesday."

[1144] (Application example 1)

[1145] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1146] Until now, factory quality control and process management have often relied on manual work by many people, and automation has not progressed. In such situations, early detection and response to quality problems can be delayed, which can have a negative impact on production efficiency and product quality. Furthermore, when an abnormality occurs, it is not possible to contact the responsible engineer quickly, which can result in a long time until the problem is resolved. To solve these problems, a system was needed that could automatically detect quality problems, quickly report them, generate appropriate responses, and resolve the problem quickly.

[1147] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1148] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for reporting a quality problem to an engineer, means for performing process management using an automatic call system, and means for automatically notifying the engineer of the generated response. This enables rapid automatic reporting and response generation when a quality problem occurs, significantly improving the efficiency of factory quality control.

[1149] A "target list" is a list of data that contains information necessary for quality control and process management.

[1150] A "database" is a system for storing and managing information in an organized manner.

[1151] "Means for automatically making phone calls" refers to a system that automatically makes calls to designated phone numbers.

[1152] The "means for detecting the start of a call and issuing an introductory message" is a system that confirms that the call has been connected and automatically issues a pre-set introductory message.

[1153] "Means for analyzing the other party's reaction and generating an appropriate response" refers to a system that analyzes the other party's remarks during a call in real time and generates an appropriate reply.

[1154] "Means for storing call content in a database" refers to a system that records information exchanged during a call and stores it in a database.

[1155] The "means of reporting quality issues to engineers" is a system that automatically reports quality issues to the relevant engineers when they occur.

[1156] "Means for process management using an automatic telephone calling system" refers to a system that automatically makes telephone calls to report and handle quality issues as part of the management process.

[1157] The "means for automatically communicating the generated response to the engineer" is a system that automatically transmits the response generated based on the analysis to the engineer in charge.

[1158] The system for carrying out the present invention includes an automatic telephone call system used for quality control in a factory. Specifically, it is composed of the following steps:

[1159] Hardware and Software Use

[1160] The system uses smartphones and dedicated factory terminals and utilizes the following key software components:

[1161] Twilio API (automatic call)

[1162] Google Cloud Speech-to-Text API

[1163] Google Cloud Text-to-Speech API

[1164] Python (programming language)

[1165] Program processing explanation

[1166] First, the user prepares a target list (a list of products with quality issues) in CSV file format and uploads it to the server. The server parses this CSV file and imports it into a database. Each row contains, for example, the product ID, details of the quality issue, and the contact information of the engineer.

[1167] When a call is placed, the device automatically dials the number to the technician's contact information stored in the database. Once the call is connected, the server uses text-to-speech (TTS) technology to deliver a pre-defined introductory message. For example, "There is a quality issue with product ID A1234. Please check for details."

[1168] When the technician responds, the server analyzes the real-time response using speech-to-text technology (Google Cloud Speech-to-Text). For example, if the technician asks, "What is the specific problem?", the server analyzes the response and generates an appropriate answer, such as, "There is a crack. It needs immediate attention."

[1169] All call content is stored in a database, along with details of any quality issues that occur and the technician's responses, ensuring that data is always available for future analysis and improvement.

[1170] Specific examples and prompts for the generative AI model

[1171] As a concrete example, the system will run with the following prompt:

[1172] "There is a quality problem with Product A. Please call the person in charge and report the abnormality."

[1173] "If the agent gives you instructions during the call, please record them and display them here."

[1174] Based on these prompts, the hardware and software work together to automatically report quality issues and generate appropriate responses, greatly streamlining factory quality control.

[1175] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1176] Step 1:

[1177] The user prepares a target list and uploads it to the server in the form of a CSV file, which contains product IDs, details of quality issues, and contact information of engineers. The CSV file is then input to the server, which triggers the next step of the process.

[1178] Step 2:

[1179] The server parses the uploaded CSV file. The server reads this file and imports each row of data into the database. Duplicate and incomplete data is automatically eliminated. The input is a CSV file, and the output is organized database entries.

[1180] Step 3:

[1181] The terminal retrieves the technician's contact information from a database and automatically initiates the call. A phone number is retrieved and a call command is entered into the terminal. This results in an output in which the terminal calls the specified technician's phone number. Retries can be made until the call is established.

[1182] Step 4:

[1183] When the server detects the start of a call, it generates an introductory message and speaks it using text-to-speech (TTS) technology. For example, a message such as "There is a quality problem with product ID A1234. Please check the details." is generated and conveyed to the technician. The input is the call start signal, and the output is the voice message.

[1184] Step 5:

[1185] When the technician responds, the audio is recorded by the server and analyzed in real time. The technician's speech is analyzed using speech-to-text conversion technology (Google Cloud Speech-to-Text). The input is the technician's voice, and the output is text data.

[1186] Step 6:

[1187] The server generates an appropriate response based on the analyzed text data. This response is then converted into audio using text-to-speech (TTS) technology. For example, a message such as "There is a crack. Please check it immediately" is generated. The input is the analyzed text data, and the output is the response message.

[1188] Step 7:

[1189] The server stores all call content in a database, including information exchanged during the call, details of any quality issues that occurred, and the technician's responses. The input is the call content, and the output is the stored database entries.

[1190] Step 8:

[1191] Users can refer to the call content and response information from the database as needed to improve quality control and for subsequent analysis. The input is a database query, and the output is the acquired call records and analysis data.

[1192] This automates the quality control process within the factory and enables rapid response to any problems that arise.

[1193] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1194] This invention is a fully automated telesales system that imports target lists, automatically makes calls, analyzes and responds to calls, switches between conversations appropriately, recognizes emotions using an emotion engine, sets appointments, and saves information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1195] Program processing

[1196] 1. Importing a Target List

[1197] server

[1198] Processing: The administrator uploads a CSV file to the system. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically removed.

[1199] Examples:

[1200] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[1201] 2. Auto-dial

[1202] Terminal

[1203] Action: According to the set schedule, the system retrieves phone numbers from the target list and automatically makes calls. It can also retry calls until it is successful.

[1204] Examples:

[1205] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[1206] 3. Introductory Talk

[1207] server

[1208] Processing: When the system detects that the call is connected, it automatically speaks an introductory phrase. The introductory phrase is delivered using text-to-speech (TTS) technology. For example, "Hello, this is XX from XX Company. Do you have time?"

[1209] Examples:

[1210] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[1211] 4. Hearing and response

[1212] server

[1213] Processing: Analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions. It uses natural language processing (NLP) technology to understand what the other person is saying and derive the best response.

[1214] Examples:

[1215] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1216] 5. Switch Talk

[1217] server

[1218] Processing: If the other person is not interested or objects, the system will switch to a situation-appropriate conversation and generate and speak. For example, if the other person says, "I'm busy right now," the system will suggest, "Even if you don't have time, would you like me to send you the details by email?"

[1219] Examples:

[1220] If the other party says "I'm busy right now" during a call, the server will switch to the appropriate situation, generate a conversation, and speak it.

[1221] 6. Emotion Recognition by Emotion Engine

[1222] server

[1223] Processing: During a call, the emotion engine analyzes the other person's emotions, collecting emotional data from the way the other person speaks, their tone of voice, and their speed to assess their interest and stress levels.

[1224] Examples:

[1225] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[1226] 7. Appointment Setting

[1227] Terminal

[1228] Process: If the other party is interested, the system checks the user's schedule, suggests a free date and time, and sets up an appointment. If the other party approves the date and time, the information is saved in the database.

[1229] Examples:

[1230] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[1231] 8. Storage of call and appointment information

[1232] server

[1233] Processing: After the call is completed, the contents of the call, the interview results, and the appointment information are saved in the database. This allows the salesperson to refer to this information later and use it for the next follow-up.

[1234] Examples:

[1235] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[1236] Overall flow

[1237] In this way, this system automates everything from importing target lists to automatically making phone calls, analyzing and answering call content, switching between conversations, recognizing emotions with an emotion engine, setting appointments, and saving information. As a result, it can significantly improve sales efficiency and reduce the burden on users.

[1238] The processing flow will be explained below.

[1239] Program processing

[1240] 1. Importing a Target List

[1241] server

[1242] Step 1: Upload the CSV file specified by the administrator to the system.

[1243] The administrator selects the target list file containing the customer information and clicks the upload button.

[1244] Step 2: The server parses the CSV file and stores it in the database.

[1245] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[1246] 2. Automated Call Process

[1247] Terminal

[1248] Step 3: Get contacts from your target list on a schedule.

[1249] The terminal checks the current time and the set outgoing call schedule, and obtains the phone number of the next customer to call.

[1250] Step 4: Make the call automatically.

[1251] The device will automatically call the acquired phone number and retry until the call is connected.

[1252] 3. Introductory Talk

[1253] server

[1254] Step 5: Detect the start of a call.

[1255] The server detects that the call connection was successful and proceeds to the next step.

[1256] Step 6: Generate and deliver your introductory speech.

[1257] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[1258] 4. Hearing and response

[1259] server

[1260] Step 7: Analyze the other person's reaction.

[1261] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[1262] Step 8: Generate an appropriate response talk.

[1263] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[1264] 5. Switch Talk

[1265] server

[1266] Step 9: Analyze the other person's objections and interest.

[1267] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[1268] Step 10: Switch and generate a conversation.

[1269] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[1270] 6. Emotion Recognition by Emotion Engine

[1271] server

[1272] Step 11: During the call, the emotion engine analyzes the other person's emotions.

[1273] The emotion engine collects emotional data from the other person's speaking style, tone, and speed to assess their level of interest and stress.

[1274] Step 12: Adjust the response based on the emotion engine analysis.

[1275] The emotion engine analyzes the other person's psychological state, and the AI ​​engine uses that data to adjust and generate response talk.

[1276] 7. Appointment Setting

[1277] Terminal

[1278] Step 13: Present your available schedule.

[1279] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[1280] Step 14: Set an appointment.

[1281] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[1282] 8. Storage of call and appointment information

[1283] server

[1284] Step 15: Save the call details, interview results, and appointment information in the database.

[1285] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[1286] Specific examples

[1287] Importing a Target List

[1288] server

[1289] Step 1: User uploads "CustomerList_2023.csv".

[1290] The user selects the CSV file from the management screen and clicks the upload button.

[1291] Step 2: The server parses the CSV file and stores the customer information in a database.

[1292] The server analyzes each line of data in the CSV and inserts "Name," "Telephone Number," "Company Name," etc. into the database.

[1293] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[1294] Example 2

[1295] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1296] Conventional telemarketing systems required a lot of manual work, resulting in inefficient customer service. Furthermore, call recording and analysis were insufficient, making it difficult to properly understand customer emotions and reactions, making it difficult to improve sales activities. Furthermore, it was difficult to provide appropriate real-time responses and set appointments, leading to declining sales results.

[1297] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for importing a target list into a data record, a means for automatically making a call, a means for detecting the start of a call and uttering an introductory speech using voice synthesis, a means for analyzing the other party's reaction using natural language processing technology and generating an appropriate response, a means for analyzing the other party's emotion during the call using emotion recognition technology, and a means for saving the call content and appointment information in a data record. This enables automated calls and efficient customer service, significantly improving the efficiency of sales activities.

[1298] A "target list" is a data record that contains information about customers who will be targeted for sales and marketing activities.

[1299] A "data record" is a structure within a database that records and stores specific information.

[1300] "Making an automatic call" means that the system automatically makes a call based on a set schedule or conditions.

[1301] "Detecting the start of a call" means that the system confirms that a call has been connected.

[1302] "Speech synthesis" is a technology that artificially generates speech based on text data.

[1303] "Natural language processing technology" refers to technology that allows computers to analyze, understand, and generate human language.

[1304] "Emotion recognition technology" is a technology that analyzes and evaluates the emotions of others from voice and text.

[1305] "Call content" refers to the dialogue and information exchanged during a call.

[1306] "Appointment information" refers to information regarding scheduled meetings or phone calls arranged with customers.

[1307] This invention is a fully automated telesales system that handles everything from importing target lists to automatically making calls, analyzing and answering call content, switching between conversations, recognizing emotions using an emotion engine, setting appointments, and saving information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1308] To implement this system, the following hardware and software are used.

[1309] 1. Importing a Target List

[1310] server

[1311] The server uploads a CSV file specified by the user to the system. The target list contains information such as customer names, phone numbers, and company names. The server uses Python's pandas library to parse the CSV file and insert it into the database. The system is designed to automatically eliminate duplicate and incomplete data.

[1312] Examples:

[1313] When a user uploads "CustomerList.csv", the server parses the file using pandas' read_csv function and saves it in the database.

[1314] 2. Auto-dial

[1315] Terminal

[1316] The device retrieves phone numbers from a database and automatically makes calls based on a set schedule. The device uses VoIP services such as the Twilio API to make calls. It includes a function to retry calls until a successful call is made.

[1317] Examples:

[1318] The device will automatically call the phone number in the first line of "Customer List.csv" using the Twilio API, and will repeat the process until the call is connected or the set number of retries is reached.

[1319] 3. Introductory Talk

[1320] server

[1321] When the server detects that the call is connected, it automatically uses text-to-speech (TTS) technology to deliver an introductory speech. This process uses the Google TTS API. The introductory speech is pre-defined.

[1322] Examples:

[1323] Once the server recognizes that the call has been connected, it uses the Google TTS API to speak, "Hello, this is △△ from XX Company. Is it free for you?"

[1324] 4. Hearing and response

[1325] server

[1326] The server analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions, using Google Cloud's natural language processing (NLP) API to understand what the other person is saying.

[1327] Examples:

[1328] If the caller asks, "What can I do for you?", the server converts the speech into text and uses NLP to respond, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1329] 5. Switch Talk

[1330] server

[1331] If the other party shows no interest or a negative reaction, the server switches to a pre-defined talk pattern and generates and speaks a conversation.

[1332] Examples:

[1333] If the other party says, "I'm busy right now," the server generates and speaks a suggestion: "Even if you don't have time, would you like me to send you the details by email?"

[1334] 6. Emotion Recognition by Emotion Engine

[1335] server

[1336] The server's emotion engine analyzes the other party's emotions during the call. The emotion recognition technology uses IBM Watson's emotion analysis API and other technologies. The emotion is assessed based on the tone, speed, and content of the other party's voice.

[1337] Examples:

[1338] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[1339] 7. Appointment Setting

[1340] Terminal

[1341] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. The datetime module is used to manage the schedule. Once the appointment is set, the information is saved in the database.

[1342] Examples:

[1343] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device records that date and time information in a database.

[1344] 8. Storage of call and appointment information

[1345] server

[1346] After the call is completed, the server stores the call content, interview results, and appointment information in a database, allowing the salesperson to plan the next follow-up.

[1347] Examples:

[1348] If the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[1349] Prompt Sentence Examples

[1350] 1. Please provide detailed steps to import the target list.

[1351] 2. "Please tell me the specific process flow of the auto-dial function."

[1352] 3. "Please explain in detail the steps to deliver an introductory talk using TTS technology."

[1353] 4. "Please tell me how Real-Time Hearing and NLP technology work in real time during a call."

[1354] 5. "Please describe in detail how the call and appointment information will be stored in the database."

[1355] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1356] Step 1:

[1357] Importing a Target List

[1358] server

[1359] Input: CSV file uploaded by the user (e.g. "CustomerList.csv")

[1360] What happens: The server receives the CSV file provided by the user. It reads the CSV file using Python's pandas library. It parses the data in each row and converts it to the correct format.

[1361] Data processing: The server loops through each row of the CSV file and extracts information such as name, phone number, company name, etc. It filters out duplicate and incomplete data.

[1362] Output: Customer information inserted into the database

[1363] Step 2:

[1364] Auto-dial

[1365] Terminal

[1366] Input: Customer information stored in the database (e.g., phone number)

[1367] Specific operation: The device retrieves the next phone number to call from the database based on the configured schedule, and automatically makes the call using a VoIP service (e.g., Twilio API).

[1368] Data calculation: Records the call result and retries as necessary until the call is successful.

[1369] Output: Call connection result

[1370] Step 3:

[1371] Introductory Talk

[1372] server

[1373] Input: Call connection confirmation signal

[1374] What happens: The server detects that the call is connected and uses the Google TTS API to voice a pre-defined introductory phrase.

[1375] Data calculation: The server converts the text of the introductory speech into audio data.

[1376] Output: Sending and playing audio data

[1377] Step 4:

[1378] Hearing and response

[1379] server

[1380] Input: Voice data from the customer during the call

[1381] What it does: The server uses Google Cloud's NLP API to convert the voice data to text, analyzes the customer's response, and applies algorithms to generate the optimal response.

[1382] Data arithmetic: Analyzes customer responses and generates appropriate response text.

[1383] Output: Response audio data

[1384] Step 5:

[1385] Switch Talk

[1386] server

[1387] Input: Negative customer responses (text data and sentiment analysis data)

[1388] Specific operation: The server uses the emotion engine to analyze the customer's emotions, switches and generates conversations according to the situation, and converts text to speech using Google TTS API.

[1389] Data Calculation: Evaluate customer sentiment and generate optimal conversations.

[1390] Output: Switched talk audio data

[1391] Step 6:

[1392] Emotion recognition by emotion engine

[1393] server

[1394] Input: Voice data during a call

[1395] Specific operations: Using emotion recognition technology (e.g., IBM Watson's emotion analysis API), collect and analyze customer emotion data, quantifying the level of emotion based on tone of voice, speed, and choice of words.

[1396] Data calculation: Emotional data is quantified and used as feedback to adjust talk.

[1397] Output: Quantified emotion data and emotion-based responses

[1398] Step 7:

[1399] Appointment setting

[1400] Terminal

[1401] Input: Appointment approvals from customers and schedules from users

[1402] Specific operation: The terminal refers to the user's schedule stored in the database, calculates and presents available time slots using the Python datetime module, and saves the date and time information in the database after receiving approval from the customer.

[1403] Data calculation: Calculates available time in the schedule and sets appointments.

[1404] Output: Appointment information stored in the database

[1405] Step 8:

[1406] Save call and appointment information

[1407] server

[1408] Input: Recorded data and analysis results after the call ends

[1409] Specific operation: The server saves the call contents (recorded data and analysis results) and the set appointment information in the database.

[1410] Data calculation: Converts recorded data into text and saves the analysis results. Also, updates the appointment information to the database.

[1411] Output: Updated database record

[1412] (Application example 2)

[1413] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] Conventional automated telesales systems offer features such as importing target lists, automatically making calls, analyzing and responding to call content, setting appointments, and saving information. However, they lack the ability to respond appropriately to user emotions and context. Furthermore, to support a comfortable and efficient travel experience for passengers in autonomous vehicles, more advanced responses and suggestions are required in addition to simply automating calls. This invention aims to solve these issues and significantly improve the user experience in autonomous vehicles.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1416] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for analyzing emotions and generating an appropriate response based on the emotions, and means for setting a destination and proposing a route to support the user's travel experience. This enables appropriate responses that take the user's emotions into consideration and makes it possible to provide passengers of autonomous vehicles with a comfortable and efficient travel experience.

[1417] A "target list" is a data list containing specific customer information (such as names, phone numbers, and company names).

[1418] A "database" is a system for efficiently storing, retrieving, and managing information.

[1419] "Automatically dialing" means that the system makes calls programmatically without human intervention.

[1420] "Detecting the start of a call" means that the system automatically recognizes that a call has been connected.

[1421] "Introductory talk" refers to basic conversations such as greetings and self-introductions that take place at the beginning of a call.

[1422] "Analyzing the other person's reactions" means understanding the words and attitudes of the other person during a call in real time.

[1423] "Generating an appropriate response" means automatically creating the optimal reply based on the other person's reaction.

[1424] "Storing call content" means recording the conversation during a call and storing it in a database.

[1425] "Analyzing emotions" means reading the emotional state of the person on the other end of the phone call from their voice and words.

[1426] "Generating an appropriate response based on emotions" means providing a reply or response that is most appropriate to the other person's emotions based on the analysis results.

[1427] "Supporting the user's mobility experience" means providing convenient and efficient services to users while they are on the move.

[1428] "Set destination" is a function that allows the user to specify a place they would like to visit.

[1429] "Route suggestion" refers to showing the best route to a specified destination.

[1430] This invention is applied to a smart glasses application aimed at supporting passengers in autonomous vehicles. The entire system consists of a server and a terminal.

[1431] Program Overview

[1432] The server includes the following means:

[1433] How to import a target list into a database: Read customer data from a CSV file and store it in a database.

[1434] A way to automatically make phone calls: Get phone numbers from a target list and make calls automatically. It also has a retry function.

[1435] Means for detecting the start of a call and uttering introductory speech: Detecting the call connection and uttering introductory speech using a text-to-speech (TTS) engine.

[1436] A means of analyzing the other person's reaction and generating an appropriate response: A natural language processing (NLP) engine is used to analyze the other person's reaction and generate an appropriate response.

[1437] A means of storing call content: Call conversations are recorded and stored in a database.

[1438] A means of analyzing emotions and generating appropriate responses based on emotions: An emotion engine is used to analyze the emotions of the other party and generate a response based on that.

[1439] A means of setting destinations and suggesting routes to support the user's travel experience: Set destinations according to passenger requests and suggest optimal routes.

[1440] Hardware and Software

[1441] Hardware used

[1442] Smart glasses: A device with display and voice input capabilities.

[1443] Server: A high-performance server that performs recognition, analysis, and data storage.

[1444] Software used

[1445] Text-to-speech (TTS) engine: Converts text to speech using Google TTS API etc.

[1446] Natural language processing (NLP) engine: Analyzes the other person's speech using SpaCy and Google Cloud Natural Language API.

[1447] Emotion engine: Analyzes the emotions of the other party during a call using IBM Watson Tone Analyzer.

[1448] Database: PostgreSQL is used to store call details and appointment information.

[1449] Appointment Scheduler: Uses the Google Calendar API to suggest available schedules and manage reservations.

[1450] Specific use cases

[1451] The scenario assumes that a user will wear smart glasses while on the move. The server to which the glasses are connected imports the target list specified by the user (e.g., "Customer List.csv") into a database and automatically initiates a call. Once the call is connected, a TTS engine is used to speak an introductory phrase such as, "Hello, this is a sales representative. May I speak to you?" The NLP engine analyzes the caller's response, and the emotion engine reads the caller's emotions and generates an appropriate response based on this. For example, if the caller responds with "interested," the appointment scheduler suggests available dates and times.

[1452] Prompt Sentence Examples

[1453] "Hello, this is your sales representative. May I speak to you?"

[1454] Example response:

[1455] I'm interested, tell me a bit more about that product.

[1456] This system will enable appropriate responses to be generated taking into account the user's emotions, providing a comfortable and efficient travel experience for passengers in autonomous vehicles.

[1457] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1458] Step 1:

[1459] The server imports the target list into the database. By uploading a CSV file specified by the administrator, the server analyzes each line of the CSV file and stores information such as customer names, phone numbers, and company names in the database. Duplicate and incomplete data is automatically eliminated.

[1460] Input: Target list (CSV file)

[1461] Data processing: Parsing CSV files, eliminating duplicate and incomplete data

[1462] Output: Target list stored in database

[1463] Step 2:

[1464] The server retrieves phone numbers from the target list according to a set schedule and automatically makes calls, and can also retry calls until successful.

[1465] Input: Customer phone number retrieved from database

[1466] Data calculation: Automatic call retry processing

[1467] Output: Phone connection

[1468] Step 3:

[1469] The device detects the start of a call and uses a text-to-speech (TTS) engine to speak an introductory phrase (e.g., "Hello, this is sales. May I speak to you?").

[1470] Input: Call connection detection signal

[1471] Data processing: Text-to-speech (TTS) speech generation

[1472] Output: Introductory speech to be spoken

[1473] Step 4:

[1474] The server analyzes the other party's responses in real time during the call and generates appropriate responses, using a natural language processing (NLP) engine to understand what the other party is saying and derive the most appropriate response.

[1475] Input: The other person's response (audio data)

[1476] Data Computing: Parsing Responses with Natural Language Processing (NLP)

[1477] Output: Appropriate response

[1478] Step 5:

[1479] The server collects emotional data from the other person's tone and speed of voice, analyzes it with an emotion engine, and generates an appropriate response based on the analysis results.

[1480] Input: Voice characteristics of the other person (tone, speed)

[1481] Data Computation: Emotion Analysis with Emotion Engine

[1482] Output: Emotion-based response

[1483] Step 6:

[1484] If the other party shows interest, the device uses the appointment setting system to reference the user's schedule, suggest an available date and time, and set up an appointment. If the other party approves the date and time, the information is saved in a database.

[1485] Input: Appointment date and time accepted by the other party

[1486] Data calculation: setting appointments and saving them to the database

[1487] Output: Saved appointment information

[1488] Step 7:

[1489] After the call is completed, the server stores the contents of the call, the results of the interview, and the appointment information in a database, allowing the user to refer to this information later and use it for the next follow-up.

[1490] Input: Call details, interview results, appointment information

[1491] Data processing: saving to database

[1492] Output: Saved call and appointment information

[1493] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1494] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1495] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1496] [Fourth embodiment]

[1497] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1498] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1499] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1500] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1501] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1502] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1503] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1504] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1505] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1506] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1507] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1508] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1509] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1510] This invention is a fully automated telesales system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1511] Program processing

[1512] 1. Importing a Target List

[1513] The server uploads the CSV file specified by the user to the system and performs analysis. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically eliminated.

[1514] Examples:

[1515] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[1516] 2. Auto-dial

[1517] The device will automatically retrieve phone numbers from the target list and make calls according to a set schedule. It can also retry calls until it is successful.

[1518] Examples:

[1519] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[1520] 3. Introductory Talk

[1521] When the server detects that the call has been connected, it automatically speaks an introductory phrase. This is done using text-to-speech (TTS) technology. For example, you might say, "Hello, this is XX from XX Company. Do you have time?"

[1522] Examples:

[1523] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[1524] 4. Hearing and responding

[1525] The server analyzes the other party's responses in real time during the call and generates appropriate responses based on the other party's needs and reactions, using natural language processing (NLP) technology to understand what the other party is saying and derive the most appropriate response.

[1526] Examples:

[1527] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1528] 5. Switch Talk

[1529] If the other party is not interested or expresses an objection, the server generates and speaks a transitional conversation appropriate to the situation. The transitional conversation is intended to keep the other party engaged.

[1530] Examples:

[1531] If the other person says, "I'm busy right now," the server suggests, "Even if you don't have time, would you like me to send you the details via email?"

[1532] 6. Appointment Setting

[1533] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. If the other party accepts the date and time, the information is saved in the database.

[1534] Examples:

[1535] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[1536] 7. Storage of call and appointment information

[1537] After the call is over, the server saves the contents of the call, the interview results, and the appointment information in a database, allowing the salesperson to refer to this information later and use it for the next follow-up.

[1538] Examples:

[1539] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[1540] Overall flow

[1541] In this way, the system automates everything from importing target lists to automatically making calls, analyzing and answering calls, switching between conversations, and setting appointments, which results in a significant improvement in sales efficiency and a reduction in the burden on users.

[1542] The processing flow will be explained below.

[1543] Program processing

[1544] 1. Importing a Target List

[1545] server

[1546] Step 1.1: Upload the CSV file specified by the administrator to the system.

[1547] The administrator selects the target list containing the customer information and clicks the upload button.

[1548] Step 1.2: The server parses the CSV file and stores it in the database.

[1549] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[1550] 2. Automated Call Process

[1551] Terminal

[1552] Step 2.1: Get contacts from your target list on a schedule.

[1553] The terminal checks the current time and the set outgoing call schedule, and obtains the customer's phone number to be called next.

[1554] Step 2.2: Make a call automatically.

[1555] The device will automatically call the acquired phone number and retry until the call is connected.

[1556] 3. Introductory Talk

[1557] server

[1558] Step 3.1: Detect the start of a call.

[1559] The server detects that the call connection was successful and proceeds to the next step.

[1560] Step 3.2: Generate and speak the introductory speech.

[1561] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[1562] 4. Hearing and response

[1563] server

[1564] Step 4.1: Analyze the other person's reaction.

[1565] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[1566] Step 4.2: Generate an appropriate response talk.

[1567] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[1568] 5. Switch Talk

[1569] server

[1570] Step 5.1: Analyze the other party's objections and interest.

[1571] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[1572] Step 5.2: Switch and generate a talk.

[1573] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[1574] 6. Appointment Setting

[1575] Terminal

[1576] Step 6.1: Present your available schedule.

[1577] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[1578] Step 6.2: Set an appointment.

[1579] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[1580] 7. Storage of call and appointment information

[1581] server

[1582] Step 7.1: Save the call details, interview results, and appointment information in the database.

[1583] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[1584] Specific examples

[1585] Importing a Target List

[1586] 1. Step 1.1: The user uploads "CustomerList_2023.csv".

[1587] The user selects the CSV file from the management screen and clicks the upload button.

[1588] 2. Step 1.2: The server parses the CSV file and stores the customer information in the database.

[1589] The server analyzes each line of data in the CSV and inserts "Name," "Phone Number," "Company Name," etc. into the database.

[1590] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[1591] Example 1

[1592] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1593] In conventional telemarketing systems, operators had to manually make calls, analyze responses, and think of appropriate responses, which required a lot of effort and reduced productivity. Furthermore, the lack of centralized management of customer information and storage of call content made follow-up difficult. Another issue was the lack of scripts for responding appropriately when a customer showed no interest or raised an objection.

[1594] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1595] In this invention, the server includes means for importing a target list into a database, means for automatically making a call, means for detecting the start of a call and delivering an introductory speech, means for analyzing the other party's reaction and generating an appropriate response, and means for saving the call content and appointment information in the database. This automates the entire telemarketing process, significantly reducing the operator's workload and improving productivity.

[1596] A "target list" is data that lists targets to call, including information such as customer names, telephone numbers, and company names.

[1597] A "database" is an information management system that stores information in an organized manner and makes it easy to search and update.

[1598] "Means for automatically making phone calls" refers to technology in which a system automatically makes calls to pre-set phone numbers without human intervention.

[1599] "Means for detecting the start of a call and speaking an introductory speech" refers to a technology that detects that a call has been connected and automatically speaks a pre-set message.

[1600] "Means for analyzing the other party's reaction and generating an appropriate response" refers to technology that analyzes the other party's response in real time during a call and automatically generates the optimal response accordingly.

[1601] "Means for storing call content and appointment information in a database" refers to technology that organizes information about call content and appointments and stores it in a database, allowing it to be viewed and analyzed later.

[1602] "Switching talk" is a script or response message that allows you to quickly respond appropriately to situations when the other person is not interested or objects.

[1603] A "generative AI model" is a model that uses artificial intelligence technology to make appropriate responses and inferences from accumulated data.

[1604] This invention is a telemarketing system that imports target lists, automatically places calls, analyzes call content, generates appropriate responses, saves call content, and manages appointment information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1605] First, the user prepares a target list in the form of a CSV file. The server accepts the process of uploading this CSV file to the system. The server then analyzes the uploaded CSV file and stores target list data such as customer names, phone numbers, and company names in a database. The system is designed to automatically eliminate duplicate and incomplete data during analysis.

[1606] Next, the terminal automatically makes a call according to a set schedule based on the target list data acquired from the server, and attempts to make the call based on a certain number of retries until the call is successful.

[1607] Once the call is successfully placed and connected, the server detects the start of the call and automatically uses text-to-speech (TTS) technology to deliver a pre-defined introductory message, such as "Hello, this is XX from XX Company. Is it your time?"

[1608] During the call, the server analyzes the other party's responses in real time and uses a generative AI model to generate an appropriate response based on the other party's needs and reactions. If the other party asks, "What can I do for you?", the server might respond, "I'm calling to provide information about ____. May I speak to you briefly?"

[1609] If the other party is not interested or objects, the server switches over and generates the conversation, using TTS technology to speak it. For example, if the other party says, "I'm busy right now," the server might suggest, "Even if you don't have time, would you like me to send you the details by email?"

[1610] If the other party suggests a date and time for an appointment during a call, the device will refer to the user's schedule database and confirm the available date and time. For example, if the other party suggests, "How about 3:00 p.m. next Tuesday?", the device will reserve this date and time and record it in the database.

[1611] Finally, once the call ends, the server analyzes the call content in real time and uses a generative AI model to store the interview results and appointment information in a database. For example, if the analysis shows that the person is interested in something, this information is recorded in the database.

[1612] This system automates the entire telemarketing process, significantly reducing user effort and improving the efficiency of sales activities.

[1613] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1614] Step 1: Importing your target list

[1615] The server receives a CSV file uploaded by the user. As input, the CSV file contains customer names, phone numbers, company names, etc. The server parses this file and stores the data for each row in a database. Duplicate and incomplete data is automatically eliminated. As output, the organized target list data is stored in the database.

[1616] Specific operation: When a user uploads "Customer List.csv", the server reads this file and records information such as "Taro Tanaka" and "09012345678" in the database.

[1617] Step 2: Auto-dial

[1618] The terminal automatically makes calls according to a set schedule based on the target list data obtained from the server. It uses the target list data and schedule information as input. It attempts to make calls based on a set number of retries until the call is successful. The output is when the call is connected or the number of retries has expired.

[1619] Specific operation: The device will automatically call "09012345678". If the connection fails, it will try to dial again after 10 seconds. This operation will be repeated until the number of retries (e.g. 3 times) is reached.

[1620] Step 3: Introductory Talk

[1621] The server detects the start of a call and uses text-to-speech (TTS) technology to speak a pre-defined introductory phrase. As input, it uses the call connection detection and the introductory phrase script. As output, a voice message is spoken.

[1622] Specific operation: The moment the call is connected, the server starts the speech synthesis engine and speaks, "Hello, this is △△ from ○○ Company. Is it okay if I speak with you?"

[1623] Step 4: Listen and respond

[1624] The server analyzes the other party's responses in real time during the call and generates an appropriate response using a generative AI model. The server uses the other party's voice data and the generative AI model as input. The generated response is then spoken using TTS technology as output.

[1625] Specific operation: When the caller asks, "What can I do for you?", the server analyzes the language data in real time and responds, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1626] Step 5: Switch Talk

[1627] The server generates an appropriate switching conversation if the other party is not interested or objects and speaks it through TTS. As input, it uses the other party's response data and the switching conversation script. As output, the switching conversation is spoken.

[1628] Specific behavior: For example, if the other party says, "I'm busy right now," the server will say, "Even if you don't have time, would you like me to send you the details by email?"

[1629] Step 6: Set up an appointment

[1630] If the other party expresses interest, the terminal consults the user's schedule database and presents available dates and times. The terminal uses the other party's schedule request and the user's schedule data as input. The confirmed appointment information is recorded as output.

[1631] Specific operation: If the other party says during a call, "How about next Tuesday at 3 p.m.?", the device will reserve that date and time and record it in the database.

[1632] Step 7: Save the call

[1633] The server analyzes the call content in real time after the call ends and uses a generative AI model to store the interview results and appointment information in a database. The server uses the call content data and the generative AI model as input. The server outputs the analyzed call content and appointment information, storing them in a database.

[1634] Specific operation: For example, if the content of a call is analyzed as "interested in XX," this information is recorded in the database, along with information such as "set up an appointment for 3 p.m. next Tuesday."

[1635] (Application example 1)

[1636] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1637] Until now, factory quality control and process management have often relied on manual work by many people, and automation has not progressed. In such situations, early detection and response to quality problems can be delayed, which can have a negative impact on production efficiency and product quality. Furthermore, when an abnormality occurs, it is not possible to contact the responsible engineer quickly, which can result in a long time until the problem is resolved. To solve these problems, a system was needed that could automatically detect quality problems, quickly report them, generate appropriate responses, and resolve the problem quickly.

[1638] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1639] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for reporting a quality problem to an engineer, means for performing process management using an automatic call system, and means for automatically notifying the engineer of the generated response. This enables rapid automatic reporting and response generation when a quality problem occurs, significantly improving the efficiency of factory quality control.

[1640] A "target list" is a list of data that contains information necessary for quality control and process management.

[1641] A "database" is a system for storing and managing information in an organized manner.

[1642] "Means for automatically making phone calls" refers to a system that automatically makes calls to designated phone numbers.

[1643] The "means for detecting the start of a call and issuing an introductory message" is a system that confirms that the call has been connected and automatically issues a pre-set introductory message.

[1644] "Means for analyzing the other party's reaction and generating an appropriate response" refers to a system that analyzes the other party's remarks during a call in real time and generates an appropriate reply.

[1645] "Means for storing call content in a database" refers to a system that records information exchanged during a call and stores it in a database.

[1646] The "means of reporting quality issues to engineers" is a system that automatically reports quality issues to the relevant engineers when they occur.

[1647] "Means for process management using an automatic telephone calling system" refers to a system that automatically makes telephone calls to report and handle quality issues as part of the management process.

[1648] The "means for automatically communicating the generated response to the engineer" is a system that automatically transmits the response generated based on the analysis to the engineer in charge.

[1649] The system for carrying out the present invention includes an automatic telephone call system used for quality control in a factory. Specifically, it is composed of the following steps:

[1650] Hardware and Software Use

[1651] The system uses smartphones and dedicated factory terminals and utilizes the following key software components:

[1652] Twilio API (automatic call)

[1653] Google Cloud Speech-to-Text API

[1654] Google Cloud Text-to-Speech API

[1655] Python (programming language)

[1656] Program processing explanation

[1657] First, the user prepares a target list (a list of products with quality issues) in CSV file format and uploads it to the server. The server parses this CSV file and imports it into a database. Each row contains, for example, the product ID, details of the quality issue, and the contact information of the engineer.

[1658] When a call is placed, the device automatically dials the number to the technician's contact information stored in the database. Once the call is connected, the server uses text-to-speech (TTS) technology to deliver a pre-defined introductory message. For example, "There is a quality issue with product ID A1234. Please check for details."

[1659] When the technician responds, the server analyzes the real-time response using speech-to-text technology (Google Cloud Speech-to-Text). For example, if the technician asks, "What is the specific problem?", the server analyzes the response and generates an appropriate answer, such as, "There is a crack. It needs immediate attention."

[1660] All call content is stored in a database, along with details of any quality issues that occur and the technician's responses, ensuring that data is always available for future analysis and improvement.

[1661] Specific examples and prompts for the generative AI model

[1662] As a concrete example, the system will run with the following prompt:

[1663] "There is a quality problem with Product A. Please call the person in charge and report the abnormality."

[1664] "If the agent gives you instructions during the call, please record them and display them here."

[1665] Based on these prompts, the hardware and software work together to automatically report quality issues and generate appropriate responses, greatly streamlining factory quality control.

[1666] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1667] Step 1:

[1668] The user prepares a target list and uploads it to the server in the form of a CSV file, which contains product IDs, details of quality issues, and contact information of engineers. The CSV file is then input to the server, which triggers the next step of the process.

[1669] Step 2:

[1670] The server parses the uploaded CSV file. The server reads this file and imports each row of data into the database. Duplicate and incomplete data is automatically eliminated. The input is a CSV file, and the output is organized database entries.

[1671] Step 3:

[1672] The terminal retrieves the technician's contact information from a database and automatically initiates the call. A phone number is retrieved and a call command is entered into the terminal. This results in an output in which the terminal calls the specified technician's phone number. Retries can be made until the call is established.

[1673] Step 4:

[1674] When the server detects the start of a call, it generates an introductory message and speaks it using text-to-speech (TTS) technology. For example, a message such as "There is a quality problem with product ID A1234. Please check the details." is generated and conveyed to the technician. The input is the call start signal, and the output is the voice message.

[1675] Step 5:

[1676] When the technician responds, the audio is recorded by the server and analyzed in real time. The technician's speech is analyzed using speech-to-text conversion technology (Google Cloud Speech-to-Text). The input is the technician's voice, and the output is text data.

[1677] Step 6:

[1678] The server generates an appropriate response based on the analyzed text data. This response is then converted into audio using text-to-speech (TTS) technology. For example, a message such as "There is a crack. Please check it immediately" is generated. The input is the analyzed text data, and the output is the response message.

[1679] Step 7:

[1680] The server stores all call content in a database, including information exchanged during the call, details of any quality issues that occurred, and the technician's responses. The input is the call content, and the output is the stored database entries.

[1681] Step 8:

[1682] Users can refer to the call content and response information from the database as needed to improve quality control and for subsequent analysis. The input is a database query, and the output is the acquired call records and analysis data.

[1683] This automates the quality control process within the factory and enables rapid response to any problems that arise.

[1684] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1685] This invention is a fully automated telesales system that imports target lists, automatically makes calls, analyzes and responds to calls, switches between conversations appropriately, recognizes emotions using an emotion engine, sets appointments, and saves information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1686] Program processing

[1687] 1. Importing a Target List

[1688] server

[1689] Processing: The administrator uploads a CSV file to the system. The target list contains information such as customer names, phone numbers, and company names. The server parses each line of the CSV file and inserts it into the database. Duplicate and incomplete data is automatically removed.

[1690] Examples:

[1691] When a user uploads "CustomerList.csv", the server parses the file and stores each customer's information in a database.

[1692] 2. Auto-dial

[1693] Terminal

[1694] Action: According to the set schedule, the system retrieves phone numbers from the target list and automatically makes calls. It can also retry calls until it is successful.

[1695] Examples:

[1696] The device will automatically call the phone number in the first line of "Customer List.csv" and repeat the process until the call is connected or the set number of retries is reached.

[1697] 3. Introductory Talk

[1698] server

[1699] Processing: When the system detects that the call is connected, it automatically speaks an introductory phrase. The introductory phrase is delivered using text-to-speech (TTS) technology. For example, "Hello, this is XX from XX Company. Do you have time?"

[1700] Examples:

[1701] When the server detects that the call has been connected, the AI ​​generates an introductory conversation and speaks it using TTS technology.

[1702] 4. Hearing and response

[1703] server

[1704] Processing: Analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions. It uses natural language processing (NLP) technology to understand what the other person is saying and derive the best response.

[1705] Examples:

[1706] If the caller asks, "What can I do for you?" the server will respond with, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1707] 5. Switch Talk

[1708] server

[1709] Processing: If the other person is not interested or objects, the system will switch to a situation-appropriate conversation and generate and speak. For example, if the other person says, "I'm busy right now," the system will suggest, "Even if you don't have time, would you like me to send you the details by email?"

[1710] Examples:

[1711] If the other party says "I'm busy right now" during a call, the server will switch to the appropriate situation, generate a conversation, and speak it.

[1712] 6. Emotion Recognition by Emotion Engine

[1713] server

[1714] Processing: During a call, the emotion engine analyzes the other person's emotions, collecting emotional data from the way the other person speaks, their tone of voice, and their speed to assess their interest and stress levels.

[1715] Examples:

[1716] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[1717] 7. Appointment Setting

[1718] Terminal

[1719] Process: If the other party is interested, the system checks the user's schedule, suggests a free date and time, and sets up an appointment. If the other party approves the date and time, the information is saved in the database.

[1720] Examples:

[1721] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device will add that date and time and record it in the database.

[1722] 8. Storage of call and appointment information

[1723] server

[1724] Processing: After the call is completed, the contents of the call, the interview results, and the appointment information are saved in the database. This allows the salesperson to refer to this information later and use it for the next follow-up.

[1725] Examples:

[1726] If the content of the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[1727] Overall flow

[1728] In this way, this system automates everything from importing target lists to automatically making phone calls, analyzing and answering call content, switching between conversations, recognizing emotions with an emotion engine, setting appointments, and saving information. As a result, it can significantly improve sales efficiency and reduce the burden on users.

[1729] The processing flow will be explained below.

[1730] Program processing

[1731] 1. Importing a Target List

[1732] server

[1733] Step 1: Upload the CSV file specified by the administrator to the system.

[1734] The administrator selects the target list file containing the customer information and clicks the upload button.

[1735] Step 2: The server parses the CSV file and stores it in the database.

[1736] The server parses the CSV data and inserts each record as a separate entry into the database, automatically detecting and eliminating duplicates and incomplete data.

[1737] 2. Automated Call Process

[1738] Terminal

[1739] Step 3: Get contacts from your target list on a schedule.

[1740] The terminal checks the current time and the set outgoing call schedule, and obtains the phone number of the next customer to call.

[1741] Step 4: Make the call automatically.

[1742] The device will automatically call the acquired phone number and retry until the call is connected.

[1743] 3. Introductory Talk

[1744] server

[1745] Step 5: Detect the start of a call.

[1746] The server detects that the call connection was successful and proceeds to the next step.

[1747] Step 6: Generate and deliver your introductory speech.

[1748] The AI ​​engine generates an introductory speech based on a pre-set script and uses text-to-speech (TTS) to deliver it. Example: "Hello, this is XX from XX Company. Do you have time?"

[1749] 4. Hearing and response

[1750] server

[1751] Step 7: Analyze the other person's reaction.

[1752] The server analyzes the voice data during the call in real time and uses natural language processing (NLP) to understand what the other person is saying.

[1753] Step 8: Generate an appropriate response talk.

[1754] The AI ​​engine generates an appropriate response based on the caller's response. For example, in response to the question, "What can I do for you?", the engine responds with, "I'm calling to inform you about ____. May I speak to you for a moment?"

[1755] 5. Switch Talk

[1756] server

[1757] Step 9: Analyze the other person's objections and interest.

[1758] The server analyzes the other person's speech and tone to determine their level of interest and whether they have any objections.

[1759] Step 10: Switch and generate a conversation.

[1760] The AI ​​engine generates and speaks conversations that adapt to the other person's situation. For example, if someone says, "I'm busy right now," the AI ​​engine suggests, "Even if you don't have time, would you like me to send you the details by email?"

[1761] 6. Emotion Recognition by Emotion Engine

[1762] server

[1763] Step 11: During the call, the emotion engine analyzes the other person's emotions.

[1764] The emotion engine collects emotional data from the other person's speaking style, tone, and speed to assess their level of interest and stress.

[1765] Step 12: Adjust the response based on the emotion engine analysis.

[1766] The emotion engine analyzes the other person's psychological state, and the AI ​​engine uses that data to adjust and generate response talk.

[1767] 7. Appointment Setting

[1768] Terminal

[1769] Step 13: Present your available schedule.

[1770] The terminal refers to the schedule data in the system and checks available dates and times that can be presented to the other party.

[1771] Step 14: Set an appointment.

[1772] If the other party approves the proposed date and time, the information is saved in the database and the appointment is set.

[1773] 8. Storage of call and appointment information

[1774] server

[1775] Step 15: Save the call details, interview results, and appointment information in the database.

[1776] The server records the entire call, the results of the interview, and the appointment information in a database for easy access later.

[1777] Specific examples

[1778] Importing a Target List

[1779] server

[1780] Step 1: User uploads "CustomerList_2023.csv".

[1781] The user selects the CSV file from the management screen and clicks the upload button.

[1782] Step 2: The server parses the CSV file and stores the customer information in a database.

[1783] The server analyzes each line of data in the CSV and inserts "Name," "Telephone Number," "Company Name," etc. into the database.

[1784] In this way, by explaining in detail the specific actions that the subject performs at each step, the processing flow of the system becomes clear.

[1785] Example 2

[1786] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1787] Conventional telemarketing systems required a lot of manual work, resulting in inefficient customer service. Furthermore, call recording and analysis were insufficient, making it difficult to properly understand customer emotions and reactions, making it difficult to improve sales activities. Furthermore, it was difficult to provide appropriate real-time responses and set appointments, leading to declining sales results.

[1788] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for importing a target list into a data record, a means for automatically making a call, a means for detecting the start of a call and uttering an introductory speech using voice synthesis, a means for analyzing the other party's reaction using natural language processing technology and generating an appropriate response, a means for analyzing the other party's emotion during the call using emotion recognition technology, and a means for saving the call content and appointment information in a data record. This enables automated calls and efficient customer service, significantly improving the efficiency of sales activities.

[1789] A "target list" is a data record that contains information about customers who will be targeted for sales and marketing activities.

[1790] A "data record" is a structure within a database that records and stores specific information.

[1791] "Making an automatic call" means that the system automatically makes a call based on a set schedule or conditions.

[1792] "Detecting the start of a call" means that the system confirms that a call has been connected.

[1793] "Speech synthesis" is a technology that artificially generates speech based on text data.

[1794] "Natural language processing technology" refers to technology that allows computers to analyze, understand, and generate human language.

[1795] "Emotion recognition technology" is a technology that analyzes and evaluates the emotions of others from voice and text.

[1796] "Call content" refers to the dialogue and information exchanged during a call.

[1797] "Appointment information" refers to information regarding scheduled meetings or phone calls arranged with customers.

[1798] This invention is a fully automated telesales system that handles everything from importing target lists to automatically making calls, analyzing and answering call content, switching between conversations, recognizing emotions using an emotion engine, setting appointments, and saving information. This system is designed to reduce the burden on users and realize efficient sales activities.

[1799] To implement this system, the following hardware and software are used.

[1800] 1. Importing a Target List

[1801] server

[1802] The server uploads a CSV file specified by the user to the system. The target list contains information such as customer names, phone numbers, and company names. The server uses Python's pandas library to parse the CSV file and insert it into the database. The system is designed to automatically eliminate duplicate and incomplete data.

[1803] Examples:

[1804] When a user uploads "CustomerList.csv", the server parses the file using pandas' read_csv function and saves it in the database.

[1805] 2. Auto-dial

[1806] Terminal

[1807] The device retrieves phone numbers from a database and automatically makes calls based on a set schedule. The device uses VoIP services such as the Twilio API to make calls. It includes a function to retry calls until a successful call is made.

[1808] Examples:

[1809] The device will automatically call the phone number in the first line of "Customer List.csv" using the Twilio API, and will repeat the process until the call is connected or the set number of retries is reached.

[1810] 3. Introductory Talk

[1811] server

[1812] When the server detects that the call is connected, it automatically uses text-to-speech (TTS) technology to deliver an introductory speech. This process uses the Google TTS API. The introductory speech is pre-defined.

[1813] Examples:

[1814] Once the server recognizes that the call has been connected, it uses the Google TTS API to speak, "Hello, this is △△ from XX Company. Is it free for you?"

[1815] 4. Hearing and response

[1816] server

[1817] The server analyzes the other person's responses in real time during the call and generates appropriate responses based on the other person's needs and reactions, using Google Cloud's natural language processing (NLP) API to understand what the other person is saying.

[1818] Examples:

[1819] If the caller asks, "What can I do for you?", the server converts the speech into text and uses NLP to respond, "I'm calling to provide information about XX. May I speak to you for a moment?"

[1820] 5. Switch Talk

[1821] server

[1822] If the other party shows no interest or a negative reaction, the server switches to a pre-defined talk pattern and generates and speaks a conversation.

[1823] Examples:

[1824] If the other party says, "I'm busy right now," the server generates and speaks a suggestion: "Even if you don't have time, would you like me to send you the details by email?"

[1825] 6. Emotion Recognition by Emotion Engine

[1826] server

[1827] The server's emotion engine analyzes the other party's emotions during the call. The emotion recognition technology uses IBM Watson's emotion analysis API and other technologies. The emotion is assessed based on the tone, speed, and content of the other party's voice.

[1828] Examples:

[1829] If the other person is hesitant, the emotion engine analyzes their emotions and the system generates softer, more reassuring talk.

[1830] 7. Appointment Setting

[1831] Terminal

[1832] If the other party shows interest, the device will check the user's schedule, suggest a free date and time, and set up an appointment. The datetime module is used to manage the schedule. Once the appointment is set, the information is saved in the database.

[1833] Examples:

[1834] If the call progresses and the other party says, "Next Tuesday at 3 p.m. would be fine," the device records that date and time information in a database.

[1835] 8. Storage of call and appointment information

[1836] server

[1837] After the call is completed, the server stores the call content, interview results, and appointment information in a database, allowing the salesperson to plan the next follow-up.

[1838] Examples:

[1839] If the call is analyzed as "interested in XX" and an appointment is set for "next Tuesday at 3pm," this information is updated in the database.

[1840] Prompt Sentence Examples

[1841] 1. Please provide detailed steps to import the target list.

[1842] 2. "Please tell me the specific process flow of the auto-dial function."

[1843] 3. "Please explain in detail the steps to deliver an introductory talk using TTS technology."

[1844] 4. "Please tell me how Real-Time Hearing and NLP technology work in real time during a call."

[1845] 5. "Please describe in detail how the call and appointment information will be stored in the database."

[1846] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1847] Step 1:

[1848] Importing a Target List

[1849] server

[1850] Input: CSV file uploaded by the user (e.g. "CustomerList.csv")

[1851] What happens: The server receives the CSV file provided by the user. It reads the CSV file using Python's pandas library. It parses the data in each row and converts it to the correct format.

[1852] Data processing: The server loops through each row of the CSV file and extracts information such as name, phone number, company name, etc. It filters out duplicate and incomplete data.

[1853] Output: Customer information inserted into the database

[1854] Step 2:

[1855] Auto-dial

[1856] Terminal

[1857] Input: Customer information stored in the database (e.g., phone number)

[1858] Specific operation: The device retrieves the next phone number to call from the database based on the configured schedule, and automatically makes the call using a VoIP service (e.g., Twilio API).

[1859] Data calculation: Records the call result and retries as necessary until the call is successful.

[1860] Output: Call connection result

[1861] Step 3:

[1862] Introductory Talk

[1863] server

[1864] Input: Call connection confirmation signal

[1865] What happens: The server detects that the call is connected and uses the Google TTS API to voice a pre-defined introductory phrase.

[1866] Data calculation: The server converts the text of the introductory speech into audio data.

[1867] Output: Sending and playing audio data

[1868] Step 4:

[1869] Hearing and response

[1870] server

[1871] Input: Voice data from the customer during the call

[1872] What it does: The server uses Google Cloud's NLP API to convert the voice data to text, analyzes the customer's response, and applies algorithms to generate the optimal response.

[1873] Data arithmetic: Analyzes customer responses and generates appropriate response text.

[1874] Output: Response audio data

[1875] Step 5:

[1876] Switch Talk

[1877] server

[1878] Input: Negative customer responses (text data and sentiment analysis data)

[1879] Specific operation: The server uses the emotion engine to analyze the customer's emotions, switches and generates conversations according to the situation, and converts text to speech using Google TTS API.

[1880] Data Calculation: Evaluate customer sentiment and generate optimal conversations.

[1881] Output: Switched talk audio data

[1882] Step 6:

[1883] Emotion recognition by emotion engine

[1884] server

[1885] Input: Voice data during a call

[1886] Specific operations: Using emotion recognition technology (e.g., IBM Watson's emotion analysis API), collect and analyze customer emotion data, quantifying the level of emotion based on tone of voice, speed, and choice of words.

[1887] Data calculation: Emotional data is quantified and used as feedback to adjust talk.

[1888] Output: Quantified emotion data and emotion-based responses

[1889] Step 7:

[1890] Appointment setting

[1891] Terminal

[1892] Input: Appointment approvals from customers and schedules from users

[1893] Specific operation: The terminal refers to the user's schedule stored in the database, calculates and presents available time slots using the Python datetime module, and saves the date and time information in the database after receiving approval from the customer.

[1894] Data calculation: Calculates available time in the schedule and sets appointments.

[1895] Output: Appointment information stored in the database

[1896] Step 8:

[1897] Save call and appointment information

[1898] server

[1899] Input: Recorded data and analysis results after the call ends

[1900] Specific operation: The server saves the call contents (recorded data and analysis results) and the set appointment information in the database.

[1901] Data calculation: Converts recorded data into text and saves the analysis results. Also, updates the appointment information to the database.

[1902] Output: Updated database record

[1903] (Application example 2)

[1904] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1905] Conventional automated telesales systems offer features such as importing target lists, automatically making calls, analyzing and responding to call content, setting appointments, and saving information. However, they lack the ability to respond appropriately to user emotions and context. Furthermore, to support a comfortable and efficient travel experience for passengers in autonomous vehicles, more advanced responses and suggestions are required in addition to simply automating calls. This invention aims to solve these issues and significantly improve the user experience in autonomous vehicles.

[1906] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1907] In this invention, the server includes means for importing the target list into a database, means for automatically making a call, means for detecting the start of a call and speaking an introductory phrase, means for analyzing the other party's reaction and generating an appropriate response, means for saving the call content in a database, means for analyzing emotions and generating an appropriate response based on the emotions, and means for setting a destination and proposing a route to support the user's travel experience. This enables appropriate responses that take the user's emotions into consideration and makes it possible to provide passengers of autonomous vehicles with a comfortable and efficient travel experience.

[1908] A "target list" is a data list containing specific customer information (such as names, phone numbers, and company names).

[1909] A "database" is a system for efficiently storing, retrieving, and managing information.

[1910] "Automatically dialing" means that the system makes calls programmatically without human intervention.

[1911] "Detecting the start of a call" means that the system automatically recognizes that a call has been connected.

[1912] "Introductory talk" refers to basic conversations such as greetings and self-introductions that take place at the beginning of a call.

[1913] "Analyzing the other person's reactions" means understanding the words and attitudes of the other person during a call in real time.

[1914] "Generating an appropriate response" means automatically creating the optimal reply based on the other person's reaction.

[1915] "Storing call content" means recording the conversation during a call and storing it in a database.

[1916] "Analyzing emotions" means reading the emotional state of the person on the other end of the phone call from their voice and words.

[1917] "Generating an appropriate response based on emotions" means providing a reply or response that is most appropriate to the other person's emotions based on the analysis results.

[1918] "Supporting the user's mobility experience" means providing convenient and efficient services to users while they are on the move.

[1919] "Set destination" is a function that allows the user to specify a place they would like to visit.

[1920] "Route suggestion" refers to showing the best route to a specified destination.

[1921] This invention is applied to a smart glasses application aimed at supporting passengers in autonomous vehicles. The entire system consists of a server and a terminal.

[1922] Program Overview

[1923] The server includes the following means:

[1924] How to import a target list into a database: Read customer data from a CSV file and store it in a database.

[1925] A way to automatically make phone calls: Get phone numbers from a target list and make calls automatically. It also has a retry function.

[1926] Means for detecting the start of a call and uttering introductory speech: Detecting the call connection and uttering introductory speech using a text-to-speech (TTS) engine.

[1927] A means of analyzing the other person's reaction and generating an appropriate response: A natural language processing (NLP) engine is used to analyze the other person's reaction and generate an appropriate response.

[1928] A means of storing call content: Call conversations are recorded and stored in a database.

[1929] A means of analyzing emotions and generating appropriate responses based on emotions: An emotion engine is used to analyze the emotions of the other party and generate a response based on that.

[1930] A means of setting destinations and suggesting routes to support the user's travel experience: Set destinations according to passenger requests and suggest optimal routes.

[1931] Hardware and Software

[1932] Hardware used

[1933] Smart glasses: A device with display and voice input capabilities.

[1934] Server: A high-performance server that performs recognition, analysis, and data storage.

[1935] Software used

[1936] Text-to-speech (TTS) engine: Converts text to speech using Google TTS API etc.

[1937] Natural language processing (NLP) engine: Analyzes the other person's speech using SpaCy and Google Cloud Natural Language API.

[1938] Emotion engine: Analyzes the emotions of the other party during a call using IBM Watson Tone Analyzer.

[1939] Database: PostgreSQL is used to store call details and appointment information.

[1940] Appointment Scheduler: Uses the Google Calendar API to suggest available schedules and manage reservations.

[1941] Specific use cases

[1942] The scenario assumes that a user will wear smart glasses while on the move. The server to which the glasses are connected imports the target list specified by the user (e.g., "Customer List.csv") into a database and automatically initiates a call. Once the call is connected, a TTS engine is used to speak an introductory phrase such as, "Hello, this is a sales representative. May I speak to you?" The NLP engine analyzes the caller's response, and the emotion engine reads the caller's emotions and generates an appropriate response based on this. For example, if the caller responds with "interested," the appointment scheduler suggests available dates and times.

[1943] Prompt Sentence Examples

[1944] "Hello, this is your sales representative. May I speak to you?"

[1945] Example response:

[1946] I'm interested, tell me a bit more about that product.

[1947] This system will enable appropriate responses to be generated taking into account the user's emotions, providing a comfortable and efficient travel experience for passengers in autonomous vehicles.

[1948] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1949] Step 1:

[1950] The server imports the target list into the database. By uploading a CSV file specified by the administrator, the server analyzes each line of the CSV file and stores information such as customer names, phone numbers, and company names in the database. Duplicate and incomplete data is automatically eliminated.

[1951] Input: Target list (CSV file)

[1952] Data processing: Parsing CSV files, eliminating duplicate and incomplete data

[1953] Output: Target list stored in database

[1954] Step 2:

[1955] The server retrieves phone numbers from the target list according to a set schedule and automatically makes calls, and can also retry calls until successful.

[1956] Input: Customer phone number retrieved from database

[1957] Data calculation: Automatic call retry processing

[1958] Output: Phone connection

[1959] Step 3:

[1960] The device detects the start of a call and uses a text-to-speech (TTS) engine to speak an introductory phrase (e.g., "Hello, this is sales. May I speak to you?").

[1961] Input: Call connection detection signal

[1962] Data processing: Text-to-speech (TTS) speech generation

[1963] Output: Introductory speech to be spoken

[1964] Step 4:

[1965] The server analyzes the other party's responses in real time during the call and generates appropriate responses, using a natural language processing (NLP) engine to understand what the other party is saying and derive the most appropriate response.

[1966] Input: The other person's response (audio data)

[1967] Data Computing: Parsing Responses with Natural Language Processing (NLP)

[1968] Output: Appropriate response

[1969] Step 5:

[1970] The server collects emotional data from the other person's tone and speed of voice, analyzes it with an emotion engine, and generates an appropriate response based on the analysis results.

[1971] Input: Voice characteristics of the other person (tone, speed)

[1972] Data Computation: Emotion Analysis with Emotion Engine

[1973] Output: Emotion-based response

[1974] Step 6:

[1975] If the other party shows interest, the device uses the appointment setting system to reference the user's schedule, suggest an available date and time, and set up an appointment. If the other party approves the date and time, the information is saved in a database.

[1976] Input: Appointment date and time accepted by the other party

[1977] Data calculation: setting appointments and saving them to the database

[1978] Output: Saved appointment information

[1979] Step 7:

[1980] After the call is completed, the server stores the contents of the call, the results of the interview, and the appointment information in a database, allowing the user to refer to this information later and use it for the next follow-up.

[1981] Input: Call details, interview results, appointment information

[1982] Data processing: saving to database

[1983] Output: Saved call and appointment information

[1984] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1985] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1986] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1987] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1988] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1989] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1990] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1991] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1992] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1993] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1994] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1995] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1996] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1997] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1998] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1999] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2000] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2001] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2002] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2003] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2004] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2005] The following is further disclosed regarding the above embodiment.

[2006] (Claim 1)

[2007] A means of importing the target list into the database;

[2008] A means of automatically making phone calls;

[2009] means for detecting the start of a call and uttering an introductory speech;

[2010] A means of analyzing the other person's reaction and generating an appropriate response;

[2011] a means for storing the call content in a database;

[2012] A system including:

[2013] (Claim 2)

[2014] 10. The system of claim 1, further comprising means for switching over and generating and uttering talk when the other party is not interested or objects.

[2015] (Claim 3)

[2016] 10. The system of claim 1, further comprising means for storing the appointment information in a database after the call is completed.

[2017] "Example 1"

[2018] (Claim 1)

[2019] A means of importing the target list into the database;

[2020] A means of automatically making phone calls;

[2021] means for detecting the start of a call and uttering an introductory speech;

[2022] A means of analyzing the other person's reaction and generating an appropriate response;

[2023] means for storing call content and appointment information in a database;

[2024] A system including:

[2025] (Claim 2)

[2026] 10. The system of claim 1, further comprising means for switching over and generating and uttering talk when the other party is not interested or objects.

[2027] (Claim 3)

[2028] 10. The system of claim 1, further comprising means for analyzing the call content in real time after the call ends and storing the hearing results in a database using a generative AI model.

[2029] "Application Example 1"

[2030] (Claim 1)

[2031] A means of importing the target list into the database;

[2032] A means of automatically making phone calls;

[2033] means for detecting the start of a call and uttering an introductory speech;

[2034] A means of analyzing the other person's reaction and generating an appropriate response;

[2035] a means for storing the call content in a database;

[2036] a means of reporting quality issues to a technician;

[2037] a means for managing the process using an automated telephone calling system;

[2038] a means for automatically communicating the generated response to a technician;

[2039] A system including:

[2040] (Claim 2)

[2041] 10. The system of claim 1, further comprising means for switching over and generating and uttering talk when the other party is not interested or objects.

[2042] (Claim 3)

[2043] 10. The system of claim 1, further comprising means for storing the appointment information in a database after the call is completed.

[2044] "Example 2: Combining Emotion Engines"

[2045] (Claim 1)

[2046] a means for importing the target list into the data record;

[2047] A means of automatically making phone calls;

[2048] a means for detecting the start of a call and uttering an introductory speech using speech synthesis;

[2049] A means of analyzing the other person's reaction using natural language processing technology and generating an appropriate response,

[2050] A means for analyzing the emotions of the other party during a call using emotion recognition technology;

[2051] means for storing call content and appointment information in a data record;

[2052] A system including:

[2053] (Claim 2)

[2054] 10. The system of claim 1, further comprising means for switching over and generating and uttering a conversation when the other party is not interested or has a negative reaction.

[2055] (Claim 3)

[2056] 2. The system of claim 1, further comprising means for storing the call content and hearing results in a data record after the call is completed.

[2057] "Application example 2 when combining emotion engines"

[2058] (Claim 1)

[2059] A means of importing the target list into the database;

[2060] A means of automatically making phone calls;

[2061] means for detecting the start of a call and uttering an introductory speech;

[2062] A means of analyzing the other person's reaction and generating an appropriate response;

[2063] a means for storing the call content in a database;

[2064] means for analyzing emotions and generating appropriate responses based on the emotions;

[2065] A means for setting destinations and proposing routes to support the user's travel experience;

[2066] A system including:

[2067] (Claim 2)

[2068] 10. The system of claim 1, further comprising means for switching over and generating and uttering talk when the other party is not interested or objects.

[2069] (Claim 3)

[2070] 10. The system of claim 1, further comprising means for storing the appointment information in a database after the call is completed. [Explanation of symbols]

[2071] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of importing the target list into the database; A means of automatically making phone calls; means for detecting the start of a call and uttering an introductory speech; A means of analyzing the other person's reaction and generating an appropriate response; a means for storing the call content in a database; A system including:

2. 2. The system of claim 1, further comprising means for switching over and generating and uttering talk when the other party is not interested or expresses an objection.

3. 2. The system of claim 1, further comprising means for storing the appointment information in a database after the call is completed.

Images