System

Abstract

To provide a system allowing a customer to use a service without anxiety even outside business hours.SOLUTION: Means for recording a voice of a customer, means for transmitting the recorded voice data to a server, means for registering the voice data received by the server in a voice recognition system, means for starting an automatic voice guidance in a call center, means for checking a customer based on the received voice data, means for requesting the customer to change information or change a plan in an interactive manner with an AI after the checking is completed, means for temporarily storing changed information in the server, means for providing the temporarily stored data to a call center staff, means for double-checking the data by the call center staff, and means for permanently storing the checked data; Means for communicating the result of the change to the customer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, it is difficult to respond to customer problems and concerns outside of call center and store business hours. Elderly people in particular often forget their PIN numbers and have difficulty using the internet, resulting in a lack of support. As a result, conventional systems are unable to provide adequate service, resulting in a loss of customer satisfaction. There is a need to solve these issues and provide services that can be used safely by customers of all ages. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides the following measures.

[0006] When customers sign a new contract, the system provides a means for them to record their voice and send that voice data to a server. The server then registers the received voice data in a voice recognition system. When a customer calls the call center outside of business hours, an automated voice guidance system is activated and their identity is verified based on the recorded voice data. After identity verification is complete, the customer interacts with the AI ​​to request changes to their information or plan. This information about the changes is then temporarily stored on the server. Furthermore, during business hours, the server provides the temporarily stored data to call center staff, who then double-check the data. The verified data is permanently stored, and the customer is notified of the changes. This system allows customers to use the service with peace of mind, even outside of business hours.

[0007] "Customer" refers to the user of the call center or service.

[0008] "Voice" refers to data that records the words and vocalizations spoken by customers as sound.

[0009] "Recording means" refers to a device or application for recording the customer's voice as digital data.

[0010] "Voice Data" refers to information stored in digital format that is a recorded Customer voice.

[0011] "Server" means a computer system for receiving, storing, and processing audio data.

[0012] "Transmission Means" means the technical means for transmitting Customer's voice data in digital form to the Server.

[0013] A "voice recognition system" refers to a system that analyzes input voice data and recognizes the content of the speech as text data.

[0014] "Automatic voice guidance" refers to the process in which a system automatically plays voice messages to guide customers.

[0015] "Identity verification" refers to the process by which a voice recognition system uses a customer's voice data to verify their identity.

[0016] "AI" refers to systems and programs that use artificial intelligence technology, and refers to technology that can respond to customer requests in an interactive format.

[0017] "Dialogue" refers to a method in which the AI ​​and the customer exchange information alternately through voice.

[0018] "Information Change" refers to the action of updating a customer's registered information.

[0019] "Plan change" refers to the act of a customer changing their service plan to another plan.

[0020] "Temporary storage" refers to temporarily storing change information in a database before final confirmation.

[0021] "Database" refers to a system that stores information in an organized manner and makes it accessible when needed.

[0022] "Call center staff" refers to the person in charge of customer service and data verification.

[0023] "Double checking" refers to the process by which call center staff double-check the data provided by the customer.

[0024] "Durable storage" refers to the long-term storage of verified data.

[0025] "Calling" refers to the act of making a phone call, and in this case refers to a call made to confirm or notify a customer.

[0026] A "smartphone" refers to a mobile information terminal that can use various applications in addition to telephone functions.

[0027] "Specialized Device" refers to an electronic device designed and used for a specific purpose. [Brief explanation of the drawings]

[0028] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0029] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0032] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0033] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0034] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0035] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0036] [First embodiment]

[0037] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0038] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0039] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0040] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0041] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0042] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0043] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0044] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0045] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0046] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0047] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0048] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] This invention utilizes voice recognition technology and AI to provide a system that automates identity verification and various information changes at call centers, aiming to enable customers of all ages, including the elderly, to use the service with peace of mind.

[0050] System Overview

[0051] The system consists of a customer terminal, a server, a voice recognition system, an AI system, and a call center. The customer terminal is a smartphone or a dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[0052] Program processing and specific examples

[0053] Customer voice registration

[0054] User device:

[0055] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[0056] server:

[0057] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[0058] Identity verification and information changes outside business hours

[0059] User device:

[0060] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[0061] server:

[0062] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[0063] User device:

[0064] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the request, then temporarily stores the changes on the server.

[0065] server:

[0066] The temporarily saved change information will be provided to call center staff during the next business hours.

[0067] Data confirmation and double-checking during business hours

[0068] server:

[0069] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[0070] Call Center Staff:

[0071] Our staff will review the data provided and double-check it for accuracy and consistency. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[0072] Final confirmation to the customer:

[0073] A staff member will call the customer to let them know the change has been made.

[0074] Specific examples

[0075] Example 1:

[0076] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[0077] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0078] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[0079] 4. On the next business day, call center staff will double-check the data and complete the verification.

[0080] 5. Call the customer to notify them that the address change is complete.

[0081] Example 2:

[0082] 1. The customer registers the voice call when signing the contract.

[0083] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[0084] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[0085] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[0086] 5. Make a confirmation call and notify the customer that the plan change is complete.

[0087] This allows customers to use the service with peace of mind even outside of business hours. The entire system is automated, ensuring efficient and accurate support.

[0088] The processing flow will be explained below.

[0089] Step 1:

[0090] User device: The customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[0091] Step 2:

[0092] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[0093] Step 3:

[0094] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[0095] Step 4:

[0096] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[0097] Step 5:

[0098] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[0099] Step 6:

[0100] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[0101] Step 7:

[0102] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[0103] Step 8:

[0104] User: A customer requests an AI system to change information or plans.

[0105] Step 9:

[0106] Server: Temporarily stores changes received from the AI ​​system.

[0107] Step 10:

[0108] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[0109] Step 11:

[0110] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[0111] Step 12:

[0112] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[0113] Step 13:

[0114] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[0115] In this way, the entire system works in cooperation, providing an environment in which customers can use services with peace of mind.

[0116] Example 1

[0117] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] Traditionally, identity verification and information change procedures at call centers have been done manually, which not only required a great deal of effort and time, but also posed the risk of human error. Furthermore, the process of identity verification and information change was cumbersome and difficult to use, especially for elderly people and non-technical users. Furthermore, the inability to respond outside of business hours limited the ability for customers to complete the procedures at their desired times.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0120] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the received voice data in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request information changes or plan changes interactively with AI after identity verification is complete; means for temporarily storing the change information on the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; a voice recognition system for processing the customer's voice; means for analyzing and understanding the customer's request using an AI system; means for storing the voice data and change information in storage; means for the server to add the stored information to a queue; and means for call center staff to verify and correct the data. This automates identity verification and information change procedures, improving efficiency and convenience. In particular, it provides an environment where customers of all ages, including seniors and non-technical personnel, can use the service with peace of mind, and also realizes a system that can be used outside of business hours.

[0121] "Means for recording the customer's voice" means any device or software that the customer uses to record his or her own voice.

[0122] The "means for transmitting recorded voice data to a server" is a technology for transmitting recorded voice data from a customer's terminal to a server via a network.

[0123] The "means for registering the voice data received by the server in the voice recognition system" refers to the process of sending the voice data received by the server to the voice recognition system and registering the data.

[0124] The "means for starting automated voice guidance at a call center" refers to a system or program for automatically starting voice guidance at a call center.

[0125] "Means for verifying identity based on received voice data" refers to a technology that uses a voice recognition system to verify the identity of a customer by comparing the received voice data with registered voice data.

[0126] "A means for customers to interactively request changes to their information or plans through AI after identity verification has been completed" is a function that uses AI linked to a voice recognition system to enable customers to interactively request changes to their desired information or plans.

[0127] The "means for temporarily storing change information on a server" is a method for temporarily storing change information generated in response to a request from a customer on a server.

[0128] The "means for providing temporarily stored data to call center staff" is a mechanism for providing data temporarily stored on a server to call center staff.

[0129] "Means for call center staff to double-check data" refers to a process by which call center staff manually review the data provided and check for accuracy.

[0130] "Means for permanently storing confirmed data and notifying customers of any changes" refers to procedures for storing data confirmed by call center staff on a long-term basis and notifying customers of any changes.

[0131] A "voice recognition system for processing customer voice" is a system for analyzing and recognizing customer voice data.

[0132] "Means for analyzing and understanding customer requests using an AI system" refers to a method for using AI technology to analyze customer requests and understand their meaning.

[0133] The "means for saving voice data and change information in storage" refers to a technology for saving recorded voice data and change request information from a customer in digital storage.

[0134] The term "means for a server to add information stored in storage to a queue" refers to a technique for adding information stored in storage to a queue in order to process it in a fixed order.

[0135] "Means for call center staff to check and correct data" refers to a method by which call center staff can check the contents of the data and correct it as necessary.

[0136] This invention relates to a system that utilizes voice recognition technology and artificial intelligence (AI) to automate identity verification and various information changes at call centers. This system consists of customer terminals, servers, a voice recognition system, an AI system, and the call center departments.

[0137] System configuration

[0138] Customer terminal

[0139] The customer terminal is a smartphone or other dedicated device that the customer uses to record voice and send it to the server. For example, a dedicated application is installed on the smartphone, and the customer starts the application to record their own voice. The recorded voice data is automatically sent to the server.

[0140] server

[0141] The server receives and stores the voice data sent from the customer's device and links it with the voice recognition system and AI system. The server is responsible for the following processes:

[0142] 1. Receive the voice data sent from the customer terminal.

[0143] 2. The received voice data is temporarily stored and transferred to the voice recognition system.

[0144] 3. The data analyzed by the voice recognition system is stored in a database along with the customer ID.

[0145] 4. Process customer identity verification and information change requests outside of business hours.

[0146] 5. The analysis results and changes from the AI ​​system are saved and provided to call center staff during the next business hours.

[0147] Voice Recognition System

[0148] The voice recognition system analyzes the voice data received from the customer and registers the customer ID and voice data in a database. When verifying the customer's identity, it also plays a role in comparing the registered voice data with newly recorded voice data.

[0149] AI systems

[0150] The AI ​​system understands and analyzes requests for information changes and plan changes through dialogue with customers. The AI ​​system analyzes customer utterances and temporarily stores the request content on the server.

[0151] Call Center

[0152] At the call center, staff double-check the data provided by the server during business hours and permanently store the confirmed data. Staff manually check the accuracy of the data and contact the customer to request corrections if necessary. After confirming the data, they notify the customer that the changes have been made.

[0153] Specific examples

[0154] Example 1: Address change

[0155] 1. When a user signs up for a new contract, they record a voice message and send it to the server.

[0156] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0157] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[0158] 4. On the next business day, call center staff will double-check the data and complete the verification.

[0159] 5. Call the user to let them know the address change is complete.

[0160] Example 2: Plan change

[0161] 1. The user registers the voice call when signing up.

[0162] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[0163] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[0164] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[0165] 5. Make a confirmation call and notify the user that the plan change is complete.

[0166] Prompt Sentence Examples

[0167] "Please tell me about identity verification systems that use voice recognition technology at call centers."

[0168] This provides customers with an environment where they can use the service with peace of mind even outside of business hours, and because the entire system is automated, efficient and accurate support is achieved.

[0169] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0170] Step 1: User records audio

[0171] input:

[0172] A smartphone or dedicated device used by users when signing up for a new contract or verifying their identity

[0173] process:

[0174] 1. The user launches the dedicated application and follows the application's instructions.

[0175] 2. The user records audio. The application starts recording audio data when the user presses the record button, and ends recording when the user presses the stop button.

[0176] output:

[0177] Recorded audio data

[0178] ---

[0179] Step 2: Send the audio data to the server

[0180] input:

[0181] Recorded audio data

[0182] process:

[0183] 1. The user device sends the recorded audio data to the server via a dedicated application.

[0184] 2. The application packets the audio data and sends it using the HTTPS protocol.

[0185] output:

[0186] Audio data sent to the server

[0187] ---

[0188] Step 3: Temporarily store the voice data and transfer it to the voice recognition system

[0189] input:

[0190] Audio data sent to the server

[0191] process:

[0192] 1. The server temporarily stores the received voice data.

[0193] 2. The server transfers the voice data to the voice recognition system. The voice recognition system analyzes the voice data, extracts customer information, and registers it in the database. The server updates the database based on the analysis results.

[0194] output:

[0195] Customer voice data and customer ID stored in a database

[0196] ---

[0197] Step 4: The customer contacts the call center and receives an automated message

[0198] input:

[0199] A database containing customer voice data and customer IDs

[0200] process:

[0201] 1. A user calls the call center outside business hours.

[0202] 2. The server launches an automated voice guidance system, prompting the user to re-record their voice for identity verification.

[0203] output:

[0204] Recorded voice data (for identity verification)

[0205] ---

[0206] Step 5: Verify your identity

[0207] input:

[0208] Recorded voice data for identity verification, customer voice data stored on the server

[0209] process:

[0210] 1. The server sends the recorded voice data back to the voice recognition system.

[0211] 2. The voice recognition system compares the newly recorded voice data with the voice data already registered.

[0212] 3. If the match is successful, the server notifies the user terminal that identity verification is complete.

[0213] output:

[0214] Identity verification result (success / failure)

[0215] ---

[0216] Step 6: The AI ​​system receives requests for information changes

[0217] input:

[0218] Identity verification result (success)

[0219] process:

[0220] 1. After successful identity verification, the server initiates a dialogue with the AI ​​system for the customer.

[0221] 2. The user speaks a request to change information or plan.

[0222] 3. The AI ​​system analyzes the user's speech and understands the request.

[0223] output:

[0224] Parsed request content

[0225] ---

[0226] Step 7: Save the changes

[0227] input:

[0228] Parsed request content

[0229] process:

[0230] 1. The server temporarily stores the change information received from the AI ​​system.

[0231] 2. The saved changes are stored in storage and added to a queue for the next business hours.

[0232] output:

[0233] Temporarily saved changes

[0234] ---

[0235] Step 8: Data verification and double-checking by call center staff

[0236] input:

[0237] Temporarily saved changes

[0238] process:

[0239] 1. During business hours, the server provides the saved changes to the call center staff.

[0240] 2. Call center staff will review the data provided and manually double-check the accuracy of the data.

[0241] output:

[0242] Confirmed Changes

[0243] ---

[0244] Step 9: Persistent data storage and customer notification

[0245] input:

[0246] Confirmed Changes

[0247] process:

[0248] 1. After the staff confirms the accuracy of the data, the server will permanently store the confirmed data.

[0249] 2. The staff member calls the customer to inform them that the change has been completed.

[0250] output:

[0251] Permanently stored data, customer notification

[0252] (Application example 1)

[0253] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0254] Traditional call center identity verification and information change procedures are often dependent on business hours, making them inconvenient for customers. Traditional authentication methods are often manual, requiring time and resources, and risking errors and unauthorized access. Authentication and transaction processing at financial institutions, in particular, require high security, requiring fast and accurate identity verification.

[0255] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0256] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at a call center; means for verifying the customer's identity based on the received voice data; means for the customer to request changes to their information or plan in an interactive format with AI after identity verification is complete; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; means for authenticating the financial institution and for users to access accounts using voice; means for the voice recognition system to verify the customer's identity by comparing it with pre-registered voice data; and means for the AI ​​system to process the transaction details and action details if identity verification is successful. This enables customers to safely and quickly verify their identity and process transactions 24 hours a day.

[0257] "Means for recording the customer's voice" refers to a device or software that has the function of recording the customer's voice as digital data.

[0258] The "means for transmitting recorded voice data to a server" is a function or device for transmitting recorded voice data to a server via a network.

[0259] The "means for registering voice data received by the server in the voice recognition system" is a function for registering voice data received by the server in the voice recognition system, making it available for use in the subsequent authentication process.

[0260] "Means for activating automated voice guidance at a call center" refers to a system or software for automatically starting voice guidance in response to a call from a customer.

[0261] "Means for verifying identity based on received voice data" refers to technical means for verifying identity using voice data sent to the server.

[0262] "A means for customers to interact with AI to request changes to their information or plans after identity verification is complete" refers to a function that allows customers to interact with the AI ​​system and request changes to their information or plans after successful identity verification.

[0263] The "means for temporarily storing change information on a server" refers to a system or function for temporarily storing change information requested by a customer on a server.

[0264] The "means for providing temporarily stored data to call center staff" refers to a system or function that provides temporarily stored modified data to call center staff.

[0265] "Means for call center staff to double-check data" refers to the process by which call center staff verify the accuracy and consistency of temporarily stored data.

[0266] "Means for permanently storing confirmed data and notifying customers of the results of changes" refers to a system or function that permanently stores data after double-checking and notifies customers that changes have been completed.

[0267] "Financial institution authentication method" means a system or technology used to verify the identity of a user in order to ensure the security of financial transactions.

[0268] "A means for users to access their accounts using their voice" refers to the ability for users to access their bank accounts or online accounts using their voice.

[0269] "Means for verifying identity by having a voice recognition system compare the user's speech with pre-registered voice data" refers to a technology in which a voice recognition system compares the user's speech with pre-registered data to verify the user's identity.

[0270] "Means for the AI ​​system to process transaction details and action details if identity verification is successful" refers to technology that enables the AI ​​system to analyze the user's request and process the transaction or action after identity verification is successful.

[0271] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. This system not only improves customer convenience but also provides high security for financial institutions and other organizations. An embodiment of the present invention is described below.

[0272] Overall system configuration

[0273] The system mainly consists of the following parts:

[0274] Customer's device

[0275] server

[0276] Voice Recognition System

[0277] AI systems

[0278] Call center departments

[0279] Hardware and software used

[0280] Hardware:

[0281] A microphone built into a smartphone or dedicated device

[0282] software:

[0283] Voice Recording Application

[0284] Python

[0285] speech_recognition library (speech recognition)

[0286] requests library (HTTP communication)

[0287] AI system (imported as ai_system)

[0288] Processing flow

[0289] Customer voice registration and identity verification

[0290] 1. Customer device: The customer launches the voice recording app and speaks a specific phrase to record it. Once recording is complete, the voice data is sent to the server.

[0291] 2. Server: The received voice data is forwarded to the voice recognition system and registered with the customer ID, which is used for the subsequent identity verification process.

[0292] 3. Call center response: When a customer contacts the center outside of business hours, an automated voice guidance system is activated, prompting the customer to record a voice message for identity verification, which is then sent to the server.

[0293] 4. Voice recognition system: The system analyzes the recorded voice of the customer and verifies their identity by comparing it with registered voice data. If the comparison is successful, identity verification is complete.

[0294] AI-based information change processing

[0295] 5. Customer terminal: After completing identity verification, the customer interacts with the AI ​​system to request changes to their information or plans. The AI ​​system analyzes the content of their speech and understands their request.

[0296] 6. Server: The AI ​​system will temporarily store the analyzed changes on the server, and this data will be provided to the call center staff during the next business hours.

[0297] Call center confirmation and notification

[0298] 7. Call center staff: During business hours, they will review the temporarily stored data, double-check the accuracy and consistency of the data, and if there are no problems, store the data permanently.

[0299] 8. Customer Notification: Once verification is complete, a call center staff member will contact the customer to inform them that the change has been completed.

[0300] Specific examples

[0301] Example 1: Address change

[0302] 1. The customer records the audio and sends it to the server.

[0303] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0304] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[0305] 4. On the next business day, call center staff will double-check the data and complete the confirmation.

[0306] 5. Call the customer to notify them that the address change is complete.

[0307] Example 2: Plan change

[0308] 1. The customer registers the voice call when signing the contract.

[0309] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[0310] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[0311] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[0312] 5. Make a confirmation call and notify the customer that the plan change is complete.

[0313] Prompt Sentence Examples

[0314] The user launches the application and speaks the following phrase:

[0315] "Log in, please use face recognition"

[0316] "Will you pay this month's electricity bill?"

[0317] In this way, customers can rest assured that their services are safe and the entire system can provide efficient and accurate support.

[0318] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0319] Step 1:

[0320] explanation:

[0321] The user starts the voice recording app, speaks a specific phrase, and records it. Once the recording is complete, the voice data is stored on the device.

[0322] input:

[0323] What the user said.

[0324] output:

[0325] Recorded audio data file.

[0326] Specific behavior:

[0327] The user launches the application and speaks a specific phrase following the app's guidance. The speech is recorded through the device's microphone and saved as an audio data file in the device's internal storage or memory.

[0328] Step 2:

[0329] explanation:

[0330] The device sends the recorded voice data to the server via the network, and the server stores the received voice data.

[0331] input:

[0332] Recorded audio data file.

[0333] output:

[0334] Audio data transferred to the server.

[0335] Specific behavior:

[0336] The device creates an HTTP request and sends it to the server with the recorded audio file attached. The server receives the request and stores the audio data in a database.

[0337] Step 3:

[0338] explanation:

[0339] The server transfers the received voice data to the voice recognition system and registers it along with the customer ID.

[0340] input:

[0341] Voice data and customer ID stored on the server.

[0342] output:

[0343] A pair of voice data and customer ID registered in the voice recognition system.

[0344] Specific behavior:

[0345] The server sends the voice data and customer ID to the voice recognition system's API, and the voice recognition system registers the data in its internal database.

[0346] Step 4:

[0347] explanation:

[0348] When a customer contacts the call center outside of business hours, an automated voice guidance system is activated, prompting them to record a voice message to verify their identity.

[0349] input:

[0350] Customer calls and voice guidance scripts.

[0351] output:

[0352] Recorded voice data for identity verification.

[0353] Specific behavior:

[0354] The call center system activates an automated voice guidance system, prompting the customer to speak phrases, which are then recorded and saved as audio data.

[0355] Step 5:

[0356] explanation:

[0357] The voice recognition system analyzes the recorded customer voice and compares it with pre-registered voice data to verify the customer's identity.

[0358] input:

[0359] Recorded voice data for identity verification, pre-registered voice data.

[0360] output:

[0361] The result of identity verification (success / failure).

[0362] Specific behavior:

[0363] The voice recognition system analyzes the recorded voice data and extracts features. These are then compared with pre-registered voice features. If they match, the identity verification is successful; if they do not, it is deemed unsuccessful.

[0364] Step 6:

[0365] explanation:

[0366] Once identity verification is complete, customers can interact with the AI ​​system and request changes to their information or plans.

[0367] input:

[0368] Identity verification result (success), customer's statement.

[0369] output:

[0370] Text data for information change or plan change requests.

[0371] Specific behavior:

[0372] Once identity verification is successful, the AI ​​system will be activated and will receive voice requests from customers to change plans or information, which will be converted into text by a voice recognition system.

[0373] Step 7:

[0374] explanation:

[0375] The server temporarily stores the change information analyzed by the AI ​​system.

[0376] input:

[0377] Text data for information change or plan change requests.

[0378] output:

[0379] Changed data temporarily saved on the server.

[0380] Specific behavior:

[0381] The text data generated by the AI ​​system is sent to a server, which temporarily stores the data.

[0382] Step 8:

[0383] explanation:

[0384] Call center staff will check and double-check the data that has been temporarily saved during business hours.

[0385] input:

[0386] Changed data temporarily saved on the server.

[0387] output:

[0388] Results of double-checked data (confirmation / correction).

[0389] Specific behavior:

[0390] Call center staff retrieve the stored data from the server and verify the accuracy and consistency of the input. Once verified, the data is permanently stored in the final database.

[0391] Step 9:

[0392] explanation:

[0393] A call center staff member will notify the customer that the verification has been completed.

[0394] input:

[0395] Results of double-checked data (confirmation).

[0396] output:

[0397] Notifying customers (calls, emails, etc.).

[0398] Specific behavior:

[0399] Call center staff will contact customers by phone or email to inform them that their information or plan change has been completed.

[0400] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0401] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves more flexible and human-like responses. It also aims to enable customers of all ages, including the elderly, to use the service with peace of mind.

[0402] System Overview

[0403] The system consists of a customer terminal, server, voice recognition system, AI system, emotion engine, and call center department. The customer terminal is a smartphone or dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. The emotion engine recognizes emotions from the customer's voice and the content of the dialogue, and responds accordingly. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[0404] Program processing and specific examples

[0405] Customer voice registration

[0406] User device:

[0407] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[0408] server:

[0409] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[0410] Identity verification and information changes outside business hours

[0411] User device:

[0412] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[0413] server:

[0414] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[0415] User device:

[0416] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the content of the request. At this time, the emotion engine also recognizes the customer's emotions and adjusts the response as necessary. For example, if the customer is feeling stressed, the system will respond in a calmer tone. The changes are then temporarily saved on the server.

[0417] server:

[0418] The temporarily saved change information will be provided to call center staff during the next business hours.

[0419] Data confirmation and double-checking during business hours

[0420] server:

[0421] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[0422] Call Center Staff:

[0423] Our staff will review the data provided and double-check the accuracy and consistency of the information. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[0424] Final confirmation to the customer:

[0425] A staff member will call the customer to let them know the change has been made.

[0426] Specific examples

[0427] Example 1:

[0428] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[0429] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0430] 3. The customer requests a change of address through a dialogue with the AI ​​system, and the emotion engine recognizes the customer's emotions and adjusts the response accordingly. The change information is temporarily stored on the server.

[0431] 4. On the next business day, call center staff will double-check the data and complete the verification.

[0432] 5. Call the customer to notify them that the address change is complete.

[0433] Example 2:

[0434] 1. The customer registers the voice call when signing the contract.

[0435] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[0436] 3. After a second attempt, the customer is successfully identified and requests a plan change through a dialogue with the AI. The emotion engine recognizes the customer's stress and responds calmly.

[0437] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[0438] 5. Make a confirmation call and notify the customer that the plan change is complete.

[0439] In this way, the entire system works together to provide an environment where customers can use the service with peace of mind. The introduction of an emotion engine makes it possible to provide customer service with a more human touch.

[0440] The processing flow will be explained below.

[0441] Step 1:

[0442] User device: When signing up for a new contract, the customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[0443] Step 2:

[0444] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[0445] Step 3:

[0446] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[0447] Step 4:

[0448] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[0449] Step 5:

[0450] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[0451] Step 6:

[0452] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[0453] Step 7:

[0454] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[0455] Step 8:

[0456] Emotion engine: Recognizes emotions from customer voices and conversations and provides that emotional information to the AI ​​system. For example, if the emotion engine determines that a customer is stressed, the AI ​​system will adjust its responses to use a calmer tone.

[0457] Step 9:

[0458] Server: Temporarily stores data on information changes and plan changes received from the AI ​​system.

[0459] Step 10:

[0460] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[0461] Step 11:

[0462] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[0463] Step 12:

[0464] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[0465] Step 13:

[0466] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[0467] Specific examples

[0468] Example 1:

[0469] 1. Step 1: When a customer signs up for a new service, their voice is recorded and sent to the server.

[0470] 2. Step 2: The server registers the received audio data.

[0471] 3. Step 3-6: The customer calls the call center outside business hours and uses automated voice guidance to verify their identity.

[0472] 4. Step 7: Following the automated voice guidance, the AI ​​system is activated and the customer requests a change of address.

[0473] 5. Step 8: The emotion engine recognizes customer emotions and provides adjustment information to the AI ​​system.

[0474] 6. Step 9: The changes are saved to the server.

[0475] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[0476] 8. Step 12: Call center staff will verify and double-check the data.

[0477] 9. Step 13: A staff member will call the customer to inform them that the address change has been completed.

[0478] Example 2:

[0479] 1. Step 1: The customer registers by voice upon signing up.

[0480] 2. Step 2: The server registers the audio data.

[0481] 3. Step 3-6: You try to change your plan outside of business hours, but the initial identity verification fails.

[0482] 4. Step 7: After successfully verifying your identity, you can request a plan change through a dialogue with the AI.

[0483] 5. Step 8: The emotion engine recognizes the customer's stress level and instructs the AI ​​system to adjust.

[0484] 6. Step 9: The changes are saved to the server.

[0485] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[0486] 8. Step 12: Call center staff will verify and double-check the data.

[0487] 9. Step 13: A staff member will call the customer to inform them that the plan change has been completed.

[0488] This trend allows customers to receive fast and accurate service through automated systems, and the introduction of emotion engines makes responses more flexible and human, improving customer satisfaction.

[0489] Example 2

[0490] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0491] Call center operations require fast and accurate customer identity verification and information changes. However, conventional systems require a lot of manual work, which can lead to errors and delays. Furthermore, conventional automated systems lack emotion recognition and provide mechanical customer service, making it difficult to improve customer satisfaction. Furthermore, few systems are available outside of business hours, resulting in poor customer convenience.

[0492] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request an information change or a plan change in an interactive format with AI after the identity verification is completed; means including an emotion engine for recognizing the customer's emotions and adjusting the response; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results. This makes it possible to realize a system that quickly and accurately changes customer information and verifies the customer's identity, enhances customer service with a more human touch, and is available even outside business hours.

[0493] A "customer voice recording device" is any device or software that allows a customer to digitally record their own voice.

[0494] The "means for transmitting recorded audio data to a server" refers to the functions and protocols for transferring recorded audio files to a server via a network.

[0495] The "means for the server to register the received voice data in the voice recognition system" refers to the process in which the server passes the received voice file to the voice recognition algorithm for analysis and registration.

[0496] "Means for activating automated voice guidance at a call center" refers to a technology that enables a call center system to automatically start voice guidance and play a voice message to a customer.

[0497] "Means for verifying identity based on received voice data" refers to the process of analyzing the recorded voice file and comparing it with pre-registered voice data to verify the identity of the customer.

[0498] "A means for customers to interactively request changes to information or plans through AI after identity verification is complete" refers to a function that, once identity verification is successful, activates the AI ​​system and interactively accepts requests from customers to change information or plans.

[0499] The "means including an emotion engine that recognizes customer emotions and adjusts responses" refers to an algorithm or system that detects customer emotions through voice analysis and adjusts response content according to those emotions.

[0500] The "means for temporarily saving change information on the server" is a function for temporarily saving information change requests from customers on the server.

[0501] The "means for providing temporarily stored data to call center staff" is a function that allows call center staff to access temporarily stored changed data during business hours.

[0502] "Means for call center staff to double-check data" refers to the process by which staff review the data provided and verify its accuracy and consistency.

[0503] "Means for permanently storing confirmed data and informing customers of changes" refers to a database that stores double-checked data for a long period of time and a means of communication for notifying customers of changes.

[0504] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves a more flexible and human-like response. The purpose of this invention is to provide an environment where customers of all generations can use the service with peace of mind.

[0505] System Overview

[0506] The system consists of the following elements:

[0507] 1. Customer terminal (smartphone or dedicated device): Customers record their own voice and send it to the server. Specifically, this provides a means to capture voice using a recording application and send it to the server.

[0508] 2. Server: Responsible for receiving, registering, temporarily storing, and providing voice data. The server is integrated with the voice recognition system and AI system.

[0509] 3. Voice recognition system: Analyzes customer voice data and verifies their identity. It uses algorithms such as deep learning models.

[0510] 4. AI system: Handles information changes and plan changes through customer interactions.

[0511] 5. Emotion engine: Recognizes emotions from customer voice and dialogue and responds accordingly.

[0512] Hardware and Software

[0513] Customer terminal: Smartphone, dedicated device

[0514] Servers: Database servers, application servers

[0515] Speech recognition system: speech analysis software (e.g., Google® Cloud Speech-to-Text)

[0516] AI system: Conversational AI platform (e.g., Dialogflow)

[0517] Emotion engine: Emotion analysis software (e.g., IBM Watson® Tone Analyzer)

[0518] Example of system operation

[0519] 1. Registering customer feedback

[0520] User device: When a customer signs up for a new service, they launch the application and record their own voice. Once the recording is complete, they press the "Send" button to send the voice data to the server.

[0521] Server: Receives the transmitted voice data and forwards it to the voice recognition system. It registers it along with the customer ID, which is used in the subsequent identity verification process.

[0522] 2. Identity Verification Process (Outside Business Hours)

[0523] User device: When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[0524] Server: Receives the customer's recorded voice, analyzes it with a voice recognition system, and compares it with registered voice data. If the comparison is successful, identity verification is complete.

[0525] User device: Once identity verification is complete, a dialogue interface with the AI ​​system is launched, allowing the user to request changes to information or plans. The emotion engine recognizes the user's emotions and adjusts the response as necessary.

[0526] 3. Data verification and double-checking (during business hours)

[0527] Server: Provides the temporarily stored data to the terminals of call center staff during business hours.

[0528] Call center staff: verify the data provided, double-check the information for accuracy and consistency, permanently store it if correct, and notify the customer of any changes.

[0529] Specific examples

[0530] Example 1:

[0531] 1. When a user signs up for a new contract, they record a voice message and send it to the server.

[0532] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0533] 3. The user requests a change of address through an interactive dialogue with the AI ​​system, and the emotion engine recognizes the user's emotions and adjusts the response accordingly. The change information is temporarily stored on the server.

[0534] 4. On the next business day, call center staff will double-check the data and complete the verification.

[0535] 5. Call the user to let them know the address change is complete.

[0536] Example prompt sentence:

[0537] Please write a statement:

[0538] 1. Overview of the call center system using voice recognition technology and AI

[0539] 2. The role of an emotion engine that recognizes customer emotions

[0540] 3. Specific examples of each processing step

[0541] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0542] Step 1: Record and send audio data

[0543] User device: When signing up for a new contract, the user launches the recording application. The user presses the "Start Recording" button to record their own voice. When recording is complete, the user presses the "Send" button to send the voice data to the server.

[0544] Input: User's voice

[0545] Output: Audio data file (e.g., wav format)

[0546] What happens: A user uses the application's recording feature to record audio and send the data to the server.

[0547] Step 2: Receiving and registering voice data

[0548] Server: The server receives the voice data, transfers it to the voice recognition system, and registers it in the server's database along with the customer ID.

[0549] Input: Voice data file, customer ID

[0550] Output: Registration information in the database

[0551] How it works: The server receives the voice data, transfers it to a voice recognition system, and registers it in a database along with the customer ID.

[0552] Step 3: Launch the automated voice guidance

[0553] User device: A user calls the call center outside business hours. An automated voice prompt is activated and provides instructions for recording.

[0554] Input: User's call

[0555] Output: Voice guidance to start recording

[0556] How it works: When a user calls, an automated answering system is activated and provides voice prompts.

[0557] Step 4: Record and send customer voice (identity verification)

[0558] User device: Following the automated voice guidance, the user records a voice message for identity verification and sends it to the server.

[0559] Input: User's recorded voice

[0560] Output: Audio data file (for confirmation)

[0561] How it works: The user follows the audio prompts to record audio, then presses the send button to send it to the server.

[0562] Step 5: Verify your identity with voice data

[0563] Server: The server sends the received voice data to the voice recognition system in real time and compares it with the registered data. If the comparison is successful, identity verification is complete.

[0564] Input: Audio data file

[0565] Output: Matching result

[0566] How it works: The server sends the voice data to a voice recognition system, which checks it against the enrollment data and returns a confirmation result.

[0567] Step 6: Interact with the AI ​​system

[0568] User device: Once identity verification is successful, the user begins a dialogue with the AI ​​system, requesting changes to information or plans via voice.

[0569] Input: What the user says

[0570] Output: Dialogue

[0571] How it works: The user gives voice instructions to the AI ​​system, which then analyzes them and proceeds with the dialogue.

[0572] Step 7: Emotion recognition and response adjustment

[0573] Emotion Engine: The AI ​​system analyzes the content and tone of the user's speech, and the emotion engine recognizes the user's emotions and adjusts the tone of the response as needed.

[0574] Input: User's voice data

[0575] Output: Recognized emotions and adjusted responses

[0576] How it works: The emotion engine analyzes the emotion in the user's voice and adjusts the system's response tone.

[0577] Step 8: Temporarily save the changed information data

[0578] Server: The AI ​​system temporarily stores the information changes data obtained as a result of the dialogue on the server.

[0579] Input: Information change data

[0580] Output: Temporarily saved data

[0581] How it works: The server records the information changes in a temporary database.

[0582] Step 9: Provide data

[0583] Server: Provides temporarily stored data to call center staff devices during business hours.

[0584] Input: Temporarily saved data

[0585] Output: Data displayed on the call center staff's terminal

[0586] Operation: The server provides the temporarily stored data to the staff terminal so that the staff can check it.

[0587] Step 10: Double-check your data

[0588] Call center staff: Staff will review the data provided and double-check the information for accuracy and consistency.

[0589] Input: Temporarily saved data

[0590] Output: Double-check result (confirmed data or correction request)

[0591] Action: Staff will check the data and save it if it is correct, or ask for corrections if it is incorrect.

[0592] Step 11: Final confirmation and notification

[0593] Call center staff: Staff will contact you to let you know that the change has been made.

[0594] Input: Confirmed data

[0595] Output: User notification

[0596] What happens: An agent calls or messages the customer to let them know about the changes.

[0597] (Application example 2)

[0598] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0599] Conventional food delivery services have faced problems such as the hassle it takes for customers to change or cancel their orders, and delayed response outside of business hours. Furthermore, it can be difficult to respond appropriately to customers' emotions, leading to lower customer satisfaction. The present invention aims to solve these problems by providing a system that allows customers to change their orders quickly and smoothly without feeling stressed.

[0600] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0601] In this invention, the server includes means for recording the customer's voice, means for transmitting the recorded voice data to the server, means for registering the voice data received by the server in a voice recognition system, means for activating automated voice guidance at the call center, means for verifying the customer's identity based on the received voice data, means for the customer to interactively request changes to information or order details through AI after identity verification is complete, means for recognizing the customer's emotions and adjusting the response, means for temporarily storing the changed information on the server, means for providing the temporarily stored data to call center staff, means for the call center staff to double-check the data, and means for permanently storing the verified data and notifying the customer of the change results. This allows customers to change or cancel their orders at any time, and also enables appropriate responses based on their emotions.

[0602] A "means for recording the customer's voice" is any device or software that electronically records the voice uttered by the customer.

[0603] The "means for transmitting recorded voice data to a server" is a device or software for transferring recorded voice data to a remote server via a communication network.

[0604] "Means for registering voice data received by the server in the voice recognition system" refers to a device or software that stores the voice data received by the server in the voice recognition system for analysis and makes it reusable as needed.

[0605] A "means for activating automated voice guidance in a call center" is a device or software that initiates a system that automatically plays pre-set voice messages in response to calls from customers.

[0606] "Means for verifying identity based on received voice data" refers to a device or software that analyzes the recorded voice data and verifies whether it matches the voice data of the registered customer.

[0607] "A means for customers to request changes to information or order details through an interactive AI dialogue after identity verification has been completed" refers to a device or software that allows verified customers to converse with AI and request changes to information or order details.

[0608] "Means for recognizing customer emotions and adjusting responses" refers to devices or software that analyze emotions from a customer's voice or text and adjust the content and tone of responses based on the results.

[0609] The "means for temporarily storing change information on a server" refers to a device or software that temporarily stores change requests from customers so that they can be verified and processed later.

[0610] "Means for providing temporarily stored data to call center staff" refers to a device or software that presents temporarily stored data to the terminal of a call center staff member for processing or confirmation.

[0611] "Means for call center staff to double-check data" means devices or software that allow call center staff to verify the accuracy of the data provided and correct or re-verify it as necessary.

[0612] "Means for permanently storing confirmed data and notifying customers of the results of changes" refers to a device or software for storing confirmed data for a long period of time and notifying customers of the results.

[0613] This invention provides a system for efficiently changing or canceling orders in food delivery services. The system aims to increase customer satisfaction by combining customer identity verification using voice, conversational order changes using AI, and emotion recognition. The system consists of the following components:

[0614] System configuration

[0615] The system is broadly composed as follows:

[0616] Customer device: A smartphone or dedicated device that records the customer's voice and sends it to the server.

[0617] Server: Responsible for receiving voice data, recognizing the voice, temporarily storing the data, and finally permanently storing the data. The following software components are installed on the server:

[0618] Voice Recognition System

[0619] Emotion Recognition Engine

[0620] AI order processing engine

[0621] Call center system: Double-checks data and notifies customers of changes.

[0622] Program processing

[0623] Audio recording and transmission

[0624] The user starts a dedicated application on their smartphone and records their voice request for an order change. The recorded voice data is sent to the server in real time.

[0625] Voice recognition and identity verification

[0626] The server passes the received voice data to a voice recognition system for analysis, which compares the voice data with registered voice data to verify the user's identity.

[0627] Order changes through AI dialogue

[0628] After completing identity verification, the AI ​​system receives a conversational request from the customer to change their order. An emotion recognition engine analyzes the customer's emotions, recognizing stress, joy, etc., and adjusts the response accordingly.

[0629] Emotion Recognition and Data Storage

[0630] If the customer is feeling stressed, the AI ​​system will respond in a calmer tone. The modified order details are temporarily saved on the server.

[0631] Response and data check

[0632] Call center staff will review the temporarily stored data during business hours, double-check for accuracy, and if there are no issues, store the data permanently and notify the customer of the changes.

[0633] Specific examples

[0634] 1. The customer changes their order by voice, saying, "I'd like one more pizza and one more Coke," and the emotion engine recognizes the customer's joy.

[0635] 2. The AI ​​order processing engine calmly replies, "We've added a pizza and a Coke. Thank you for your order!"

[0636] Prompt Sentence Examples

[0637] "A customer says, 'I'd like one more pizza and one more Coke.' You process this order, and your emotion recognition engine recognizes the pleasure. Respond in the most appropriate order to make the customer happy."

[0638] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0639] Step 1:

[0640] Audio recording and transmission

[0641] Customers use their smartphones to launch a dedicated application and record voice requests for changes to their order, such as "I'd like one more pizza and one more Coke." The recorded voice data is sent to the server in real time.

[0642] Input: Customer voice

[0643] Data processing: Recorded as audio data

[0644] Output: Audio data sent to the server

[0645] Step 2:

[0646] Voice recognition and identity verification

[0647] The server passes the received voice data to a voice recognition system for analysis, which compares the voice data with registered voice data to verify the user's identity.

[0648] Input: Received audio data

[0649] Data calculation: Analysis of voice data and comparison with registered data

[0650] Output: Identity verification result

[0651] Step 3:

[0652] Order changes through AI dialogue

[0653] After identity verification is complete, the AI ​​system receives order change requests from customers in an interactive format.

[0654] Input: Voice data after identity verification

[0655] Data calculation: Analyzes voice and understands order details

[0656] Output: Text format data of the order change request

[0657] Step 4:

[0658] Emotion recognition and response adjustment

[0659] An emotion recognition engine analyzes the customer's voice and text to determine whether they are feeling happy or stressed, and the AI ​​system adjusts the response accordingly.

[0660] Input: Customer voice and text data

[0661] Data Computing: Emotion Recognition Analysis

[0662] Output: Emotion recognition results and corresponding policy

[0663] Step 5:

[0664] Temporarily save order details

[0665] The analyzed order details and emotion recognition results are temporarily stored on the server, allowing call center staff to check them later.

[0666] Input: Order details and emotion recognition results

[0667] Data processing: Temporary storage of data

[0668] Output: Data temporarily saved on the server

[0669] Step 6:

[0670] Data check and storage during business hours

[0671] Call center staff will review the temporarily stored data during business hours, double-check for accuracy, and if there are no issues, store the data permanently.

[0672] Input:Temporarily saved data

[0673] Data calculations: data verification and double-checking

[0674] Output: Verified and persistently stored data

[0675] Step 7:

[0676] Notify customers of the changes

[0677] Finally, the changes are notified to the customer, who can confirm that the order details have been changed.

[0678] Input: Verified permanently stored data

[0679] Data processing: Creating a notification of the changes

[0680] Output: Customer notification

[0681] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0682] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0683] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0684] [Second embodiment]

[0685] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0686] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0687] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0688] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0689] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0690] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0691] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0692] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0693] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0694] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0695] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0696] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0697] This invention utilizes voice recognition technology and AI to provide a system that automates identity verification and various information changes at call centers, aiming to enable customers of all ages, including the elderly, to use the service with peace of mind.

[0698] System Overview

[0699] The system consists of a customer terminal, a server, a voice recognition system, an AI system, and a call center. The customer terminal is a smartphone or a dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[0700] Program processing and specific examples

[0701] Customer voice registration

[0702] User device:

[0703] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[0704] server:

[0705] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[0706] Identity verification and information changes outside business hours

[0707] User device:

[0708] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[0709] server:

[0710] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[0711] User device:

[0712] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the request, then temporarily stores the changes on the server.

[0713] server:

[0714] The temporarily saved change information will be provided to call center staff during the next business hours.

[0715] Data confirmation and double-checking during business hours

[0716] server:

[0717] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[0718] Call Center Staff:

[0719] Our staff will review the data provided and double-check it for accuracy and consistency. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[0720] Final confirmation to the customer:

[0721] A staff member will call the customer to let them know the change has been made.

[0722] Specific examples

[0723] Example 1:

[0724] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[0725] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0726] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[0727] 4. On the next business day, call center staff will double-check the data and complete the verification.

[0728] 5. Call the customer to notify them that the address change is complete.

[0729] Example 2:

[0730] 1. The customer registers the voice call when signing the contract.

[0731] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[0732] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[0733] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[0734] 5. Make a confirmation call and notify the customer that the plan change is complete.

[0735] This allows customers to use the service with peace of mind even outside of business hours. The entire system is automated, ensuring efficient and accurate support.

[0736] The processing flow will be explained below.

[0737] Step 1:

[0738] User device: The customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[0739] Step 2:

[0740] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[0741] Step 3:

[0742] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[0743] Step 4:

[0744] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[0745] Step 5:

[0746] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[0747] Step 6:

[0748] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[0749] Step 7:

[0750] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[0751] Step 8:

[0752] User: A customer requests an AI system to change information or plans.

[0753] Step 9:

[0754] Server: Temporarily stores changes received from the AI ​​system.

[0755] Step 10:

[0756] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[0757] Step 11:

[0758] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[0759] Step 12:

[0760] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[0761] Step 13:

[0762] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[0763] In this way, the entire system works in cooperation, providing an environment in which customers can use services with peace of mind.

[0764] Example 1

[0765] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0766] Traditionally, identity verification and information change procedures at call centers have been done manually, which not only required a great deal of effort and time, but also posed the risk of human error. Furthermore, the process of identity verification and information change was cumbersome and difficult to use, especially for elderly people and non-technical users. Furthermore, the inability to respond outside of business hours limited the ability for customers to complete the procedures at their desired times.

[0767] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0768] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the received voice data in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request information changes or plan changes interactively with AI after identity verification is complete; means for temporarily storing the change information on the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; a voice recognition system for processing the customer's voice; means for analyzing and understanding the customer's request using an AI system; means for storing the voice data and change information in storage; means for the server to add the stored information to a queue; and means for call center staff to verify and correct the data. This automates identity verification and information change procedures, improving efficiency and convenience. In particular, it provides an environment where customers of all ages, including seniors and non-technical personnel, can use the service with peace of mind, and also realizes a system that can be used outside of business hours.

[0769] "Means for recording the customer's voice" means any device or software that the customer uses to record his or her own voice.

[0770] The "means for transmitting recorded voice data to a server" is a technology for transmitting recorded voice data from a customer's terminal to a server via a network.

[0771] The "means for registering the voice data received by the server in the voice recognition system" refers to the process of sending the voice data received by the server to the voice recognition system and registering the data.

[0772] The "means for starting automated voice guidance at a call center" refers to a system or program for automatically starting voice guidance at a call center.

[0773] "Means for verifying identity based on received voice data" refers to a technology that uses a voice recognition system to verify the identity of a customer by comparing the received voice data with registered voice data.

[0774] "A means for customers to interactively request changes to their information or plans through AI after identity verification has been completed" is a function that uses AI linked to a voice recognition system to enable customers to interactively request changes to their desired information or plans.

[0775] The "means for temporarily storing change information on a server" is a method for temporarily storing change information generated in response to a request from a customer on a server.

[0776] The "means for providing temporarily stored data to call center staff" is a mechanism for providing data temporarily stored on a server to call center staff.

[0777] "Means for call center staff to double-check data" refers to a process by which call center staff manually review the data provided and check for accuracy.

[0778] "Means for permanently storing confirmed data and notifying customers of any changes" refers to procedures for storing data confirmed by call center staff on a long-term basis and notifying customers of any changes.

[0779] A "voice recognition system for processing customer voice" is a system for analyzing and recognizing customer voice data.

[0780] "Means for analyzing and understanding customer requests using an AI system" refers to a method for using AI technology to analyze customer requests and understand their meaning.

[0781] The "means for saving voice data and change information in storage" refers to a technology for saving recorded voice data and change request information from a customer in digital storage.

[0782] The term "means for a server to add information stored in storage to a queue" refers to a technique for adding information stored in storage to a queue in order to process it in a fixed order.

[0783] "Means for call center staff to check and correct data" refers to a method by which call center staff can check the contents of the data and correct it as necessary.

[0784] This invention relates to a system that utilizes voice recognition technology and artificial intelligence (AI) to automate identity verification and various information changes at call centers. This system consists of customer terminals, servers, a voice recognition system, an AI system, and the call center departments.

[0785] System configuration

[0786] Customer terminal

[0787] The customer terminal is a smartphone or other dedicated device that the customer uses to record voice and send it to the server. For example, a dedicated application is installed on the smartphone, and the customer starts the application to record their own voice. The recorded voice data is automatically sent to the server.

[0788] server

[0789] The server receives and stores the voice data sent from the customer's device and links it with the voice recognition system and AI system. The server is responsible for the following processes:

[0790] 1. Receive the voice data sent from the customer terminal.

[0791] 2. The received voice data is temporarily stored and transferred to the voice recognition system.

[0792] 3. The data analyzed by the voice recognition system is stored in a database along with the customer ID.

[0793] 4. Process customer identity verification and information change requests outside of business hours.

[0794] 5. The analysis results and changes from the AI ​​system are saved and provided to call center staff during the next business hours.

[0795] Voice Recognition System

[0796] The voice recognition system analyzes the voice data received from the customer and registers the customer ID and voice data in a database. When verifying the customer's identity, it also plays a role in comparing the registered voice data with newly recorded voice data.

[0797] AI systems

[0798] The AI ​​system understands and analyzes requests for information changes and plan changes through dialogue with customers. The AI ​​system analyzes customer utterances and temporarily stores the request content on the server.

[0799] Call Center

[0800] At the call center, staff double-check the data provided by the server during business hours and permanently store the confirmed data. Staff manually check the accuracy of the data and contact the customer to request corrections if necessary. After confirming the data, they notify the customer that the changes have been made.

[0801] Specific examples

[0802] Example 1: Address change

[0803] 1. When a user signs up for a new contract, they record a voice message and send it to the server.

[0804] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0805] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[0806] 4. On the next business day, call center staff will double-check the data and complete the verification.

[0807] 5. Call the user to let them know the address change is complete.

[0808] Example 2: Plan change

[0809] 1. The user registers the voice call when signing up.

[0810] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[0811] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[0812] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[0813] 5. Make a confirmation call and notify the user that the plan change is complete.

[0814] Prompt Sentence Examples

[0815] "Please tell me about identity verification systems that use voice recognition technology at call centers."

[0816] This provides customers with an environment where they can use the service with peace of mind even outside of business hours, and because the entire system is automated, efficient and accurate support is achieved.

[0817] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0818] Step 1: User records audio

[0819] input:

[0820] A smartphone or dedicated device used by users when signing up for a new contract or verifying their identity

[0821] process:

[0822] 1. The user launches the dedicated application and follows the application's instructions.

[0823] 2. The user records audio. The application starts recording audio data when the user presses the record button, and ends recording when the user presses the stop button.

[0824] output:

[0825] Recorded audio data

[0826] ---

[0827] Step 2: Send the audio data to the server

[0828] input:

[0829] Recorded audio data

[0830] process:

[0831] 1. The user device sends the recorded audio data to the server via a dedicated application.

[0832] 2. The application packets the audio data and sends it using the HTTPS protocol.

[0833] output:

[0834] Audio data sent to the server

[0835] ---

[0836] Step 3: Temporarily store the voice data and transfer it to the voice recognition system

[0837] input:

[0838] Audio data sent to the server

[0839] process:

[0840] 1. The server temporarily stores the received voice data.

[0841] 2. The server transfers the voice data to the voice recognition system. The voice recognition system analyzes the voice data, extracts customer information, and registers it in the database. The server updates the database based on the analysis results.

[0842] output:

[0843] Customer voice data and customer ID stored in a database

[0844] ---

[0845] Step 4: The customer contacts the call center and receives an automated message

[0846] input:

[0847] A database containing customer voice data and customer IDs

[0848] process:

[0849] 1. A user calls the call center outside business hours.

[0850] 2. The server launches an automated voice guidance system, prompting the user to re-record their voice for identity verification.

[0851] output:

[0852] Recorded voice data (for identity verification)

[0853] ---

[0854] Step 5: Verify your identity

[0855] input:

[0856] Recorded voice data for identity verification, customer voice data stored on the server

[0857] process:

[0858] 1. The server sends the recorded voice data back to the voice recognition system.

[0859] 2. The voice recognition system compares the newly recorded voice data with the voice data already registered.

[0860] 3. If the match is successful, the server notifies the user terminal that identity verification is complete.

[0861] output:

[0862] Identity verification result (success / failure)

[0863] ---

[0864] Step 6: The AI ​​system receives requests for information changes

[0865] input:

[0866] Identity verification result (success)

[0867] process:

[0868] 1. After successful identity verification, the server initiates a dialogue with the AI ​​system for the customer.

[0869] 2. The user speaks a request to change information or plan.

[0870] 3. The AI ​​system analyzes the user's speech and understands the request.

[0871] output:

[0872] Parsed request content

[0873] ---

[0874] Step 7: Save the changes

[0875] input:

[0876] Parsed request content

[0877] process:

[0878] 1. The server temporarily stores the change information received from the AI ​​system.

[0879] 2. The saved changes are stored in storage and added to a queue for the next business hours.

[0880] output:

[0881] Temporarily saved changes

[0882] ---

[0883] Step 8: Data verification and double-checking by call center staff

[0884] input:

[0885] Temporarily saved changes

[0886] process:

[0887] 1. During business hours, the server provides the saved changes to the call center staff.

[0888] 2. Call center staff will review the data provided and manually double-check the accuracy of the data.

[0889] output:

[0890] Confirmed Changes

[0891] ---

[0892] Step 9: Persistent data storage and customer notification

[0893] input:

[0894] Confirmed Changes

[0895] process:

[0896] 1. After the staff confirms the accuracy of the data, the server will permanently store the confirmed data.

[0897] 2. The staff member calls the customer to inform them that the change has been completed.

[0898] output:

[0899] Permanently stored data, customer notification

[0900] (Application example 1)

[0901] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] Traditional call center identity verification and information change procedures are often dependent on business hours, making them inconvenient for customers. Traditional authentication methods are often manual, requiring time and resources, and risking errors and unauthorized access. Authentication and transaction processing at financial institutions, in particular, require high security, requiring fast and accurate identity verification.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0904] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at a call center; means for verifying the customer's identity based on the received voice data; means for the customer to request changes to their information or plan in an interactive format with AI after identity verification is complete; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; means for authenticating the financial institution and for users to access accounts using voice; means for the voice recognition system to verify the customer's identity by comparing it with pre-registered voice data; and means for the AI ​​system to process the transaction details and action details if identity verification is successful. This enables customers to safely and quickly verify their identity and process transactions 24 hours a day.

[0905] "Means for recording the customer's voice" refers to a device or software that has the function of recording the customer's voice as digital data.

[0906] The "means for transmitting recorded voice data to a server" is a function or device for transmitting recorded voice data to a server via a network.

[0907] The "means for registering voice data received by the server in the voice recognition system" is a function for registering voice data received by the server in the voice recognition system, making it available for use in the subsequent authentication process.

[0908] "Means for activating automated voice guidance at a call center" refers to a system or software for automatically starting voice guidance in response to a call from a customer.

[0909] "Means for verifying identity based on received voice data" refers to technical means for verifying identity using voice data sent to the server.

[0910] "A means for customers to interact with AI to request changes to their information or plans after identity verification is complete" refers to a function that allows customers to interact with the AI ​​system and request changes to their information or plans after successful identity verification.

[0911] The "means for temporarily storing change information on a server" refers to a system or function for temporarily storing change information requested by a customer on a server.

[0912] The "means for providing temporarily stored data to call center staff" refers to a system or function that provides temporarily stored modified data to call center staff.

[0913] "Means for call center staff to double-check data" refers to the process by which call center staff verify the accuracy and consistency of temporarily stored data.

[0914] "Means for permanently storing confirmed data and notifying customers of the results of changes" refers to a system or function that permanently stores data after double-checking and notifies customers that changes have been completed.

[0915] "Financial institution authentication method" means a system or technology used to verify the identity of a user in order to ensure the security of financial transactions.

[0916] "A means for users to access their accounts using their voice" refers to the ability for users to access their bank accounts or online accounts using their voice.

[0917] "Means for verifying identity by having a voice recognition system compare the user's speech with pre-registered voice data" refers to a technology in which a voice recognition system compares the user's speech with pre-registered data to verify the user's identity.

[0918] "Means for the AI ​​system to process transaction details and action details if identity verification is successful" refers to technology that enables the AI ​​system to analyze the user's request and process the transaction or action after identity verification is successful.

[0919] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. This system not only improves customer convenience but also provides high security for financial institutions and other organizations. An embodiment of the present invention is described below.

[0920] Overall system configuration

[0921] The system mainly consists of the following parts:

[0922] Customer's device

[0923] server

[0924] Voice Recognition System

[0925] AI systems

[0926] Call center departments

[0927] Hardware and software used

[0928] Hardware:

[0929] A microphone built into a smartphone or dedicated device

[0930] software:

[0931] Voice Recording Application

[0932] Python

[0933] speech_recognition library (speech recognition)

[0934] requests library (HTTP communication)

[0935] AI system (imported as ai_system)

[0936] Processing flow

[0937] Customer voice registration and identity verification

[0938] 1. Customer device: The customer launches the voice recording app and speaks a specific phrase to record it. Once recording is complete, the voice data is sent to the server.

[0939] 2. Server: The received voice data is forwarded to the voice recognition system and registered with the customer ID, which is used for the subsequent identity verification process.

[0940] 3. Call center response: When a customer contacts the center outside of business hours, an automated voice guidance system is activated, prompting the customer to record a voice message for identity verification, which is then sent to the server.

[0941] 4. Voice recognition system: The system analyzes the recorded voice of the customer and verifies their identity by comparing it with registered voice data. If the comparison is successful, identity verification is complete.

[0942] AI-based information change processing

[0943] 5. Customer terminal: After completing identity verification, the customer interacts with the AI ​​system to request changes to their information or plans. The AI ​​system analyzes the content of their speech and understands their request.

[0944] 6. Server: The AI ​​system will temporarily store the analyzed changes on the server, and this data will be provided to the call center staff during the next business hours.

[0945] Call center confirmation and notification

[0946] 7. Call center staff: During business hours, they will review the temporarily stored data, double-check the accuracy and consistency of the data, and if there are no problems, store the data permanently.

[0947] 8. Customer Notification: Once verification is complete, a call center staff member will contact the customer to inform them that the change has been completed.

[0948] Specific examples

[0949] Example 1: Address change

[0950] 1. The customer records the audio and sends it to the server.

[0951] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[0952] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[0953] 4. On the next business day, call center staff will double-check the data and complete the confirmation.

[0954] 5. Call the customer to notify them that the address change is complete.

[0955] Example 2: Plan change

[0956] 1. The customer registers the voice call when signing the contract.

[0957] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[0958] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[0959] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[0960] 5. Make a confirmation call and notify the customer that the plan change is complete.

[0961] Prompt Sentence Examples

[0962] The user launches the application and speaks the following phrase:

[0963] "Log in, please use face recognition"

[0964] "Will you pay this month's electricity bill?"

[0965] In this way, customers can rest assured that their services are safe and the entire system can provide efficient and accurate support.

[0966] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0967] Step 1:

[0968] explanation:

[0969] The user starts the voice recording app, speaks a specific phrase, and records it. Once the recording is complete, the voice data is stored on the device.

[0970] input:

[0971] What the user said.

[0972] output:

[0973] Recorded audio data file.

[0974] Specific behavior:

[0975] The user launches the application and speaks a specific phrase following the app's guidance. The speech is recorded through the device's microphone and saved as an audio data file in the device's internal storage or memory.

[0976] Step 2:

[0977] explanation:

[0978] The device sends the recorded voice data to the server via the network, and the server stores the received voice data.

[0979] input:

[0980] Recorded audio data file.

[0981] output:

[0982] Audio data transferred to the server.

[0983] Specific behavior:

[0984] The device creates an HTTP request and sends it to the server with the recorded audio file attached. The server receives the request and stores the audio data in a database.

[0985] Step 3:

[0986] explanation:

[0987] The server transfers the received voice data to the voice recognition system and registers it along with the customer ID.

[0988] input:

[0989] Voice data and customer ID stored on the server.

[0990] output:

[0991] A pair of voice data and customer ID registered in the voice recognition system.

[0992] Specific behavior:

[0993] The server sends the voice data and customer ID to the voice recognition system's API, and the voice recognition system registers the data in its internal database.

[0994] Step 4:

[0995] explanation:

[0996] When a customer contacts the call center outside of business hours, an automated voice guidance system is activated, prompting them to record a voice message to verify their identity.

[0997] input:

[0998] Customer calls and voice guidance scripts.

[0999] output:

[1000] Recorded voice data for identity verification.

[1001] Specific behavior:

[1002] The call center system activates an automated voice guidance system, prompting the customer to speak phrases, which are then recorded and saved as audio data.

[1003] Step 5:

[1004] explanation:

[1005] The voice recognition system analyzes the recorded customer voice and compares it with pre-registered voice data to verify the customer's identity.

[1006] input:

[1007] Recorded voice data for identity verification, pre-registered voice data.

[1008] output:

[1009] The result of identity verification (success / failure).

[1010] Specific behavior:

[1011] The voice recognition system analyzes the recorded voice data and extracts features. These are then compared with pre-registered voice features. If they match, the identity verification is successful; if they do not, it is deemed unsuccessful.

[1012] Step 6:

[1013] explanation:

[1014] Once identity verification is complete, customers can interact with the AI ​​system and request changes to their information or plans.

[1015] input:

[1016] Identity verification result (success), customer's statement.

[1017] output:

[1018] Text data for information change or plan change requests.

[1019] Specific behavior:

[1020] Once identity verification is successful, the AI ​​system will be activated and will receive voice requests from customers to change plans or information, which will be converted into text by a voice recognition system.

[1021] Step 7:

[1022] explanation:

[1023] The server temporarily stores the change information analyzed by the AI ​​system.

[1024] input:

[1025] Text data for information change or plan change requests.

[1026] output:

[1027] Changed data temporarily saved on the server.

[1028] Specific behavior:

[1029] The text data generated by the AI ​​system is sent to a server, which temporarily stores the data.

[1030] Step 8:

[1031] explanation:

[1032] Call center staff will check and double-check the data that has been temporarily saved during business hours.

[1033] input:

[1034] Changed data temporarily saved on the server.

[1035] output:

[1036] Results of double-checked data (confirmation / correction).

[1037] Specific behavior:

[1038] Call center staff retrieve the stored data from the server and verify the accuracy and consistency of the input. Once verified, the data is permanently stored in the final database.

[1039] Step 9:

[1040] explanation:

[1041] A call center staff member will notify the customer that the verification has been completed.

[1042] input:

[1043] Results of double-checked data (confirmation).

[1044] output:

[1045] Notifying customers (calls, emails, etc.).

[1046] Specific behavior:

[1047] Call center staff will contact customers by phone or email to inform them that their information or plan change has been completed.

[1048] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1049] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves more flexible and human-like responses. It also aims to enable customers of all ages, including the elderly, to use the service with peace of mind.

[1050] System Overview

[1051] The system consists of a customer terminal, server, voice recognition system, AI system, emotion engine, and call center department. The customer terminal is a smartphone or dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. The emotion engine recognizes emotions from the customer's voice and the content of the dialogue, and responds accordingly. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[1052] Program processing and specific examples

[1053] Customer voice registration

[1054] User device:

[1055] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[1056] server:

[1057] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[1058] Identity verification and information changes outside business hours

[1059] User device:

[1060] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[1061] server:

[1062] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[1063] User device:

[1064] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the content of the request. At this time, the emotion engine also recognizes the customer's emotions and adjusts the response as necessary. For example, if the customer is feeling stressed, the system will respond in a calmer tone. The changes are then temporarily saved on the server.

[1065] server:

[1066] The temporarily saved change information will be provided to call center staff during the next business hours.

[1067] Data confirmation and double-checking during business hours

[1068] server:

[1069] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[1070] Call Center Staff:

[1071] Our staff will review the data provided and double-check the accuracy and consistency of the information. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[1072] Final confirmation to the customer:

[1073] A staff member will call the customer to let them know the change has been made.

[1074] Specific examples

[1075] Example 1:

[1076] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[1077] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[1078] 3. The customer requests a change of address through a dialogue with the AI ​​system, and the emotion engine recognizes the customer's emotions and adjusts the response accordingly. The change information is temporarily stored on the server.

[1079] 4. On the next business day, call center staff will double-check the data and complete the verification.

[1080] 5. Call the customer to notify them that the address change is complete.

[1081] Example 2:

[1082] 1. The customer registers the voice call when signing the contract.

[1083] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[1084] 3. After a second attempt, the customer is successfully identified and requests a plan change through a dialogue with the AI. The emotion engine recognizes the customer's stress and responds calmly.

[1085] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[1086] 5. Make a confirmation call and notify the customer that the plan change is complete.

[1087] In this way, the entire system works together to provide an environment where customers can use the service with peace of mind. The introduction of an emotion engine makes it possible to provide customer service with a more human touch.

[1088] The processing flow will be explained below.

[1089] Step 1:

[1090] User device: When signing up for a new contract, the customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[1091] Step 2:

[1092] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[1093] Step 3:

[1094] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[1095] Step 4:

[1096] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[1097] Step 5:

[1098] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[1099] Step 6:

[1100] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[1101] Step 7:

[1102] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[1103] Step 8:

[1104] Emotion engine: Recognizes emotions from customer voices and conversations and provides that emotional information to the AI ​​system. For example, if the emotion engine determines that a customer is stressed, the AI ​​system will adjust its responses to use a calmer tone.

[1105] Step 9:

[1106] Server: Temporarily stores data on information changes and plan changes received from the AI ​​system.

[1107] Step 10:

[1108] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[1109] Step 11:

[1110] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[1111] Step 12:

[1112] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[1113] Step 13:

[1114] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[1115] Specific examples

[1116] Example 1:

[1117] 1. Step 1: When a customer signs up for a new service, their voice is recorded and sent to the server.

[1118] 2. Step 2: The server registers the received audio data.

[1119] 3. Step 3-6: The customer calls the call center outside business hours and uses automated voice guidance to verify their identity.

[1120] 4. Step 7: Following the automated voice guidance, the AI ​​system is activated and the customer requests a change of address.

[1121] 5. Step 8: The emotion engine recognizes customer emotions and provides adjustment information to the AI ​​system.

[1122] 6. Step 9: The changes are saved to the server.

[1123] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[1124] 8. Step 12: Call center staff will verify and double-check the data.

[1125] 9. Step 13: A staff member will call the customer to inform them that the address change has been completed.

[1126] Example 2:

[1127] 1. Step 1: The customer registers by voice upon signing up.

[1128] 2. Step 2: The server registers the audio data.

[1129] 3. Step 3-6: You try to change your plan outside of business hours, but the initial identity verification fails.

[1130] 4. Step 7: After successfully verifying your identity, you can request a plan change through a dialogue with the AI.

[1131] 5. Step 8: The emotion engine recognizes the customer's stress level and instructs the AI ​​system to adjust.

[1132] 6. Step 9: The changes are saved to the server.

[1133] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[1134] 8. Step 12: Call center staff will verify and double-check the data.

[1135] 9. Step 13: A staff member will call the customer to inform them that the plan change has been completed.

[1136] This trend allows customers to receive fast and accurate service through automated systems, and the introduction of emotion engines makes responses more flexible and human, improving customer satisfaction.

[1137] Example 2

[1138] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1139] Call center operations require fast and accurate customer identity verification and information changes. However, conventional systems require a lot of manual work, which can lead to errors and delays. Furthermore, conventional automated systems lack emotion recognition and provide mechanical customer service, making it difficult to improve customer satisfaction. Furthermore, few systems are available outside of business hours, resulting in poor customer convenience.

[1140] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request an information change or a plan change in an interactive format with AI after the identity verification is completed; means including an emotion engine for recognizing the customer's emotions and adjusting the response; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results. This makes it possible to realize a system that quickly and accurately changes customer information and verifies the customer's identity, enhances customer service with a more human touch, and is available even outside business hours.

[1141] A "customer voice recording device" is any device or software that allows a customer to digitally record their own voice.

[1142] The "means for transmitting recorded audio data to a server" refers to the functions and protocols for transferring recorded audio files to a server via a network.

[1143] The "means for the server to register the received voice data in the voice recognition system" refers to the process in which the server passes the received voice file to the voice recognition algorithm for analysis and registration.

[1144] "Means for activating automated voice guidance at a call center" refers to a technology that enables a call center system to automatically start voice guidance and play a voice message to a customer.

[1145] "Means for verifying identity based on received voice data" refers to the process of analyzing the recorded voice file and comparing it with pre-registered voice data to verify the identity of the customer.

[1146] "A means for customers to interactively request changes to information or plans through AI after identity verification is complete" refers to a function that, once identity verification is successful, activates the AI ​​system and interactively accepts requests from customers to change information or plans.

[1147] The "means including an emotion engine that recognizes customer emotions and adjusts responses" refers to an algorithm or system that detects customer emotions through voice analysis and adjusts response content according to those emotions.

[1148] The "means for temporarily saving change information on the server" is a function for temporarily saving information change requests from customers on the server.

[1149] The "means for providing temporarily stored data to call center staff" is a function that allows call center staff to access temporarily stored changed data during business hours.

[1150] "Means for call center staff to double-check data" refers to the process by which staff review the data provided and verify its accuracy and consistency.

[1151] "Means for permanently storing confirmed data and informing customers of changes" refers to a database that stores double-checked data for a long period of time and a means of communication for notifying customers of changes.

[1152] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves a more flexible and human-like response. The purpose of this invention is to provide an environment where customers of all generations can use the service with peace of mind.

[1153] System Overview

[1154] The system consists of the following elements:

[1155] 1. Customer terminal (smartphone or dedicated device): Customers record their own voice and send it to the server. Specifically, this provides a means to capture voice using a recording application and send it to the server.

[1156] 2. Server: Responsible for receiving, registering, temporarily storing, and providing voice data. The server is integrated with the voice recognition system and AI system.

[1157] 3. Voice recognition system: Analyzes customer voice data and verifies their identity. It uses algorithms such as deep learning models.

[1158] 4. AI system: Handles information changes and plan changes through customer interactions.

[1159] 5. Emotion engine: Recognizes emotions from customer voice and dialogue and responds accordingly.

[1160] Hardware and Software

[1161] Customer terminal: Smartphone, dedicated device

[1162] Servers: Database servers, application servers

[1163] Speech recognition system: speech analysis software (e.g., Google Cloud Speech-to-Text)

[1164] AI system: Conversational AI platform (e.g., Dialogflow)

[1165] Emotion engine: Emotion analysis software (e.g. IBM Watson Tone Analyzer)

[1166] Example of system operation

[1167] 1. Registering customer feedback

[1168] User device: When a customer signs up for a new service, they launch the application and record their own voice. Once the recording is complete, they press the "Send" button to send the voice data to the server.

[1169] Server: Receives the transmitted voice data and forwards it to the voice recognition system. It registers it along with the customer ID, which is used in the subsequent identity verification process.

[1170] 2. Identity Verification Process (Outside Business Hours)

[1171] User device: When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[1172] Server: Receives the customer's recorded voice, analyzes it with a voice recognition system, and compares it with registered voice data. If the comparison is successful, identity verification is complete.

[1173] User device: Once identity verification is complete, a dialogue interface with the AI ​​system is launched, allowing the user to request changes to information or plans. The emotion engine recognizes the user's emotions and adjusts the response as necessary.

[1174] 3. Data verification and double-checking (during business hours)

[1175] Server: Provides the temporarily stored data to the terminals of call center staff during business hours.

[1176] Call center staff: verify the data provided, double-check the information for accuracy and consistency, permanently store it if correct, and notify the customer of any changes.

[1177] Specific examples

[1178] Example 1:

[1179] 1. When a user signs up for a new contract, they record a voice message and send it to the server.

[1180] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[1181] 3. The user requests a change of address through an interactive dialogue with the AI ​​system, and the emotion engine recognizes the user's emotions and adjusts the response accordingly. The change information is temporarily stored on the server.

[1182] 4. On the next business day, call center staff will double-check the data and complete the verification.

[1183] 5. Call the user to let them know the address change is complete.

[1184] Example prompt sentence:

[1185] Please write a statement:

[1186] 1. Overview of the call center system using voice recognition technology and AI

[1187] 2. The role of an emotion engine that recognizes customer emotions

[1188] 3. Specific examples of each processing step

[1189] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1190] Step 1: Record and send audio data

[1191] User device: When signing up for a new contract, the user launches the recording application. The user presses the "Start Recording" button to record their own voice. When recording is complete, the user presses the "Send" button to send the voice data to the server.

[1192] Input: User's voice

[1193] Output: Audio data file (e.g., wav format)

[1194] What happens: A user uses the application's recording feature to record audio and send the data to the server.

[1195] Step 2: Receiving and registering voice data

[1196] Server: The server receives the voice data, transfers it to the voice recognition system, and registers it in the server's database along with the customer ID.

[1197] Input: Voice data file, customer ID

[1198] Output: Registration information in the database

[1199] How it works: The server receives the voice data, transfers it to a voice recognition system, and registers it in a database along with the customer ID.

[1200] Step 3: Launch the automated voice guidance

[1201] User device: A user calls the call center outside business hours. An automated voice prompt is activated and provides instructions for recording.

[1202] Input: User's call

[1203] Output: Voice guidance to start recording

[1204] How it works: When a user calls, an automated answering system is activated and provides voice prompts.

[1205] Step 4: Record and send customer voice (identity verification)

[1206] User device: Following the automated voice guidance, the user records a voice message for identity verification and sends it to the server.

[1207] Input: User's recorded voice

[1208] Output: Audio data file (for confirmation)

[1209] How it works: The user follows the audio prompts to record audio, then presses the send button to send it to the server.

[1210] Step 5: Verify your identity with voice data

[1211] Server: The server sends the received voice data to the voice recognition system in real time and compares it with the registered data. If the comparison is successful, identity verification is complete.

[1212] Input: Audio data file

[1213] Output: Matching result

[1214] How it works: The server sends the voice data to a voice recognition system, which checks it against the enrollment data and returns a confirmation result.

[1215] Step 6: Interact with the AI ​​system

[1216] User device: Once identity verification is successful, the user begins a dialogue with the AI ​​system, requesting changes to information or plans via voice.

[1217] Input: What the user says

[1218] Output: Dialogue

[1219] How it works: The user gives voice instructions to the AI ​​system, which then analyzes them and proceeds with the dialogue.

[1220] Step 7: Emotion recognition and response adjustment

[1221] Emotion Engine: The AI ​​system analyzes the content and tone of the user's speech, and the emotion engine recognizes the user's emotions and adjusts the tone of the response as needed.

[1222] Input: User's voice data

[1223] Output: Recognized emotions and adjusted responses

[1224] How it works: The emotion engine analyzes the emotion in the user's voice and adjusts the system's response tone.

[1225] Step 8: Temporarily save the changed information data

[1226] Server: The AI ​​system temporarily stores the information changes data obtained as a result of the dialogue on the server.

[1227] Input: Information change data

[1228] Output: Temporarily saved data

[1229] How it works: The server records the information changes in a temporary database.

[1230] Step 9: Provide data

[1231] Server: Provides temporarily stored data to call center staff devices during business hours.

[1232] Input: Temporarily saved data

[1233] Output: Data displayed on the call center staff's terminal

[1234] Operation: The server provides the temporarily stored data to the staff terminal so that the staff can check it.

[1235] Step 10: Double-check your data

[1236] Call center staff: Staff will review the data provided and double-check the information for accuracy and consistency.

[1237] Input: Temporarily saved data

[1238] Output: Double-check result (confirmed data or correction request)

[1239] Action: Staff will check the data and save it if it is correct, or ask for corrections if it is incorrect.

[1240] Step 11: Final confirmation and notification

[1241] Call center staff: Staff will contact you to let you know that the change has been made.

[1242] Input: Confirmed data

[1243] Output: User notification

[1244] What happens: An agent calls or messages the customer to let them know about the changes.

[1245] (Application example 2)

[1246] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1247] Conventional food delivery services have faced problems such as the hassle it takes for customers to change or cancel their orders, and delayed response outside of business hours. Furthermore, it can be difficult to respond appropriately to customers' emotions, leading to lower customer satisfaction. The present invention aims to solve these problems by providing a system that allows customers to change their orders quickly and smoothly without feeling stressed.

[1248] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1249] In this invention, the server includes means for recording the customer's voice, means for transmitting the recorded voice data to the server, means for registering the voice data received by the server in a voice recognition system, means for activating automated voice guidance at the call center, means for verifying the customer's identity based on the received voice data, means for the customer to interactively request changes to information or order details through AI after identity verification is complete, means for recognizing the customer's emotions and adjusting the response, means for temporarily storing the changed information on the server, means for providing the temporarily stored data to call center staff, means for the call center staff to double-check the data, and means for permanently storing the verified data and notifying the customer of the change results. This allows customers to change or cancel their orders at any time, and also enables appropriate responses based on their emotions.

[1250] A "means for recording the customer's voice" is any device or software that electronically records the voice uttered by the customer.

[1251] The "means for transmitting recorded voice data to a server" is a device or software for transferring recorded voice data to a remote server via a communication network.

[1252] "Means for registering voice data received by the server in the voice recognition system" refers to a device or software that stores the voice data received by the server in the voice recognition system for analysis and makes it reusable as needed.

[1253] A "means for activating automated voice guidance in a call center" is a device or software that initiates a system that automatically plays pre-set voice messages in response to calls from customers.

[1254] "Means for verifying identity based on received voice data" refers to a device or software that analyzes the recorded voice data and verifies whether it matches the voice data of the registered customer.

[1255] "A means for customers to request changes to information or order details through an interactive AI dialogue after identity verification has been completed" refers to a device or software that allows verified customers to converse with AI and request changes to information or order details.

[1256] "Means for recognizing customer emotions and adjusting responses" refers to devices or software that analyze emotions from a customer's voice or text and adjust the content and tone of responses based on the results.

[1257] The "means for temporarily storing change information on a server" refers to a device or software that temporarily stores change requests from customers so that they can be verified and processed later.

[1258] "Means for providing temporarily stored data to call center staff" refers to a device or software that presents temporarily stored data to the terminal of a call center staff member for processing or confirmation.

[1259] "Means for call center staff to double-check data" means devices or software that allow call center staff to verify the accuracy of the data provided and correct or re-verify it as necessary.

[1260] "Means for permanently storing confirmed data and notifying customers of the results of changes" refers to a device or software for storing confirmed data for a long period of time and notifying customers of the results.

[1261] This invention provides a system for efficiently changing or canceling orders in food delivery services. The system aims to increase customer satisfaction by combining customer identity verification using voice, conversational order changes using AI, and emotion recognition. The system consists of the following components:

[1262] System configuration

[1263] The system is broadly composed as follows:

[1264] Customer device: A smartphone or dedicated device that records the customer's voice and sends it to the server.

[1265] Server: Responsible for receiving voice data, recognizing the voice, temporarily storing the data, and finally permanently storing the data. The following software components are installed on the server:

[1266] Voice Recognition System

[1267] Emotion Recognition Engine

[1268] AI order processing engine

[1269] Call center system: Double-checks data and notifies customers of changes.

[1270] Program processing

[1271] Audio recording and transmission

[1272] The user starts a dedicated application on their smartphone and records their voice request for an order change. The recorded voice data is sent to the server in real time.

[1273] Voice recognition and identity verification

[1274] The server passes the received voice data to a voice recognition system for analysis, which compares the voice data with registered voice data to verify the user's identity.

[1275] Order changes through AI dialogue

[1276] After completing identity verification, the AI ​​system receives a conversational request from the customer to change their order. An emotion recognition engine analyzes the customer's emotions, recognizing stress, joy, etc., and adjusts the response accordingly.

[1277] Emotion Recognition and Data Storage

[1278] If the customer is feeling stressed, the AI ​​system will respond in a calmer tone. The modified order details are temporarily saved on the server.

[1279] Response and data check

[1280] Call center staff will review the temporarily stored data during business hours, double-check for accuracy, and if there are no issues, store the data permanently and notify the customer of the changes.

[1281] Specific examples

[1282] 1. The customer changes their order by voice, saying, "I'd like one more pizza and one more Coke," and the emotion engine recognizes the customer's joy.

[1283] 2. The AI ​​order processing engine calmly replies, "We've added a pizza and a Coke. Thank you for your order!"

[1284] Prompt Sentence Examples

[1285] "A customer says, 'I'd like one more pizza and one more Coke.' You process this order, and your emotion recognition engine recognizes the pleasure. Respond in the most appropriate order to make the customer happy."

[1286] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1287] Step 1:

[1288] Audio recording and transmission

[1289] Customers use their smartphones to launch a dedicated application and record voice requests for changes to their order, such as "I'd like one more pizza and one more Coke." The recorded voice data is sent to the server in real time.

[1290] Input: Customer voice

[1291] Data processing: Recorded as audio data

[1292] Output: Audio data sent to the server

[1293] Step 2:

[1294] Voice recognition and identity verification

[1295] The server passes the received voice data to a voice recognition system for analysis, which compares the voice data with registered voice data to verify the user's identity.

[1296] Input: Received audio data

[1297] Data calculation: Analysis of voice data and comparison with registered data

[1298] Output: Identity verification result

[1299] Step 3:

[1300] Order changes through AI dialogue

[1301] After identity verification is complete, the AI ​​system receives order change requests from customers in an interactive format.

[1302] Input: Voice data after identity verification

[1303] Data calculation: Analyzes voice and understands order details

[1304] Output: Text format data of the order change request

[1305] Step 4:

[1306] Emotion recognition and response adjustment

[1307] An emotion recognition engine analyzes the customer's voice and text to determine whether they are feeling happy or stressed, and the AI ​​system adjusts the response accordingly.

[1308] Input: Customer voice and text data

[1309] Data Computing: Emotion Recognition Analysis

[1310] Output: Emotion recognition results and corresponding policy

[1311] Step 5:

[1312] Temporarily save order details

[1313] The analyzed order details and emotion recognition results are temporarily stored on the server, allowing call center staff to check them later.

[1314] Input: Order details and emotion recognition results

[1315] Data processing: Temporary storage of data

[1316] Output: Data temporarily saved on the server

[1317] Step 6:

[1318] Data check and storage during business hours

[1319] Call center staff will review the temporarily stored data during business hours, double-check for accuracy, and if there are no issues, store the data permanently.

[1320] Input:Temporarily saved data

[1321] Data calculations: data verification and double-checking

[1322] Output: Verified and persistently stored data

[1323] Step 7:

[1324] Notify customers of the changes

[1325] Finally, the changes are notified to the customer, who can confirm that the order details have been changed.

[1326] Input: Verified permanently stored data

[1327] Data processing: Creating a notification of the changes

[1328] Output: Customer notification

[1329] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1330] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1331] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1332] [Third embodiment]

[1333] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1334] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1335] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1336] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1337] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1338] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1339] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1340] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1341] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1342] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1343] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1344] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1345] This invention utilizes voice recognition technology and AI to provide a system that automates identity verification and various information changes at call centers, aiming to enable customers of all ages, including the elderly, to use the service with peace of mind.

[1346] System Overview

[1347] The system consists of a customer terminal, a server, a voice recognition system, an AI system, and a call center. The customer terminal is a smartphone or a dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[1348] Program processing and specific examples

[1349] Customer voice registration

[1350] User device:

[1351] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[1352] server:

[1353] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[1354] Identity verification and information changes outside business hours

[1355] User device:

[1356] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[1357] server:

[1358] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[1359] User device:

[1360] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the request, then temporarily stores the changes on the server.

[1361] server:

[1362] The temporarily saved change information will be provided to call center staff during the next business hours.

[1363] Data confirmation and double-checking during business hours

[1364] server:

[1365] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[1366] Call Center Staff:

[1367] Our staff will review the data provided and double-check it for accuracy and consistency. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[1368] Final confirmation to the customer:

[1369] A staff member will call the customer to let them know the change has been made.

[1370] Specific examples

[1371] Example 1:

[1372] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[1373] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[1374] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[1375] 4. On the next business day, call center staff will double-check the data and complete the verification.

[1376] 5. Call the customer to notify them that the address change is complete.

[1377] Example 2:

[1378] 1. The customer registers the voice call when signing the contract.

[1379] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[1380] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[1381] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[1382] 5. Make a confirmation call and notify the customer that the plan change is complete.

[1383] This allows customers to use the service with peace of mind even outside of business hours. The entire system is automated, ensuring efficient and accurate support.

[1384] The processing flow will be explained below.

[1385] Step 1:

[1386] User device: The customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[1387] Step 2:

[1388] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[1389] Step 3:

[1390] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[1391] Step 4:

[1392] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[1393] Step 5:

[1394] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[1395] Step 6:

[1396] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[1397] Step 7:

[1398] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[1399] Step 8:

[1400] User: A customer requests an AI system to change information or plans.

[1401] Step 9:

[1402] Server: Temporarily stores changes received from the AI ​​system.

[1403] Step 10:

[1404] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[1405] Step 11:

[1406] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[1407] Step 12:

[1408] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[1409] Step 13:

[1410] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[1411] In this way, the entire system works in cooperation, providing an environment in which customers can use services with peace of mind.

[1412] Example 1

[1413] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] Traditionally, identity verification and information change procedures at call centers have been done manually, which not only required a great deal of effort and time, but also posed the risk of human error. Furthermore, the process of identity verification and information change was cumbersome and difficult to use, especially for elderly people and non-technical users. Furthermore, the inability to respond outside of business hours limited the ability for customers to complete the procedures at their desired times.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1416] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the received voice data in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request information changes or plan changes interactively with AI after identity verification is complete; means for temporarily storing the change information on the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; a voice recognition system for processing the customer's voice; means for analyzing and understanding the customer's request using an AI system; means for storing the voice data and change information in storage; means for the server to add the stored information to a queue; and means for call center staff to verify and correct the data. This automates identity verification and information change procedures, improving efficiency and convenience. In particular, it provides an environment where customers of all ages, including seniors and non-technical personnel, can use the service with peace of mind, and also realizes a system that can be used outside of business hours.

[1417] "Means for recording the customer's voice" means any device or software that the customer uses to record his or her own voice.

[1418] The "means for transmitting recorded voice data to a server" is a technology for transmitting recorded voice data from a customer's terminal to a server via a network.

[1419] The "means for registering the voice data received by the server in the voice recognition system" refers to the process of sending the voice data received by the server to the voice recognition system and registering the data.

[1420] The "means for starting automated voice guidance at a call center" refers to a system or program for automatically starting voice guidance at a call center.

[1421] "Means for verifying identity based on received voice data" refers to a technology that uses a voice recognition system to verify the identity of a customer by comparing the received voice data with registered voice data.

[1422] "A means for customers to interactively request changes to their information or plans through AI after identity verification has been completed" is a function that uses AI linked to a voice recognition system to enable customers to interactively request changes to their desired information or plans.

[1423] The "means for temporarily storing change information on a server" is a method for temporarily storing change information generated in response to a request from a customer on a server.

[1424] The "means for providing temporarily stored data to call center staff" is a mechanism for providing data temporarily stored on a server to call center staff.

[1425] "Means for call center staff to double-check data" refers to a process by which call center staff manually review the data provided and check for accuracy.

[1426] "Means for permanently storing confirmed data and notifying customers of any changes" refers to procedures for storing data confirmed by call center staff on a long-term basis and notifying customers of any changes.

[1427] A "voice recognition system for processing customer voice" is a system for analyzing and recognizing customer voice data.

[1428] "Means for analyzing and understanding customer requests using an AI system" refers to a method for using AI technology to analyze customer requests and understand their meaning.

[1429] The "means for saving voice data and change information in storage" refers to a technology for saving recorded voice data and change request information from a customer in digital storage.

[1430] The term "means for a server to add information stored in storage to a queue" refers to a technique for adding information stored in storage to a queue in order to process it in a fixed order.

[1431] "Means for call center staff to check and correct data" refers to a method by which call center staff can check the contents of the data and correct it as necessary.

[1432] This invention relates to a system that utilizes voice recognition technology and artificial intelligence (AI) to automate identity verification and various information changes at call centers. This system consists of customer terminals, servers, a voice recognition system, an AI system, and the call center departments.

[1433] System configuration

[1434] Customer terminal

[1435] The customer terminal is a smartphone or other dedicated device that the customer uses to record voice and send it to the server. For example, a dedicated application is installed on the smartphone, and the customer starts the application to record their own voice. The recorded voice data is automatically sent to the server.

[1436] server

[1437] The server receives and stores the voice data sent from the customer's device and links it with the voice recognition system and AI system. The server is responsible for the following processes:

[1438] 1. Receive the voice data sent from the customer terminal.

[1439] 2. The received voice data is temporarily stored and transferred to the voice recognition system.

[1440] 3. The data analyzed by the voice recognition system is stored in a database along with the customer ID.

[1441] 4. Process customer identity verification and information change requests outside of business hours.

[1442] 5. The analysis results and changes from the AI ​​system are saved and provided to call center staff during the next business hours.

[1443] Voice Recognition System

[1444] The voice recognition system analyzes the voice data received from the customer and registers the customer ID and voice data in a database. When verifying the customer's identity, it also plays a role in comparing the registered voice data with newly recorded voice data.

[1445] AI systems

[1446] The AI ​​system understands and analyzes requests for information changes and plan changes through dialogue with customers. The AI ​​system analyzes customer utterances and temporarily stores the request content on the server.

[1447] Call Center

[1448] At the call center, staff double-check the data provided by the server during business hours and permanently store the confirmed data. Staff manually check the accuracy of the data and contact the customer to request corrections if necessary. After confirming the data, they notify the customer that the changes have been made.

[1449] Specific examples

[1450] Example 1: Address change

[1451] 1. When a user signs up for a new contract, they record a voice message and send it to the server.

[1452] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[1453] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[1454] 4. On the next business day, call center staff will double-check the data and complete the verification.

[1455] 5. Call the user to let them know the address change is complete.

[1456] Example 2: Plan change

[1457] 1. The user registers the voice call when signing up.

[1458] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[1459] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[1460] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[1461] 5. Make a confirmation call and notify the user that the plan change is complete.

[1462] Prompt Sentence Examples

[1463] "Please tell me about identity verification systems that use voice recognition technology at call centers."

[1464] This provides customers with an environment where they can use the service with peace of mind even outside of business hours, and because the entire system is automated, efficient and accurate support is achieved.

[1465] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1466] Step 1: User records audio

[1467] input:

[1468] A smartphone or dedicated device used by users when signing up for a new contract or verifying their identity

[1469] process:

[1470] 1. The user launches the dedicated application and follows the application's instructions.

[1471] 2. The user records audio. The application starts recording audio data when the user presses the record button, and ends recording when the user presses the stop button.

[1472] output:

[1473] Recorded audio data

[1474] ---

[1475] Step 2: Send the audio data to the server

[1476] input:

[1477] Recorded audio data

[1478] process:

[1479] 1. The user device sends the recorded audio data to the server via a dedicated application.

[1480] 2. The application packets the audio data and sends it using the HTTPS protocol.

[1481] output:

[1482] Audio data sent to the server

[1483] ---

[1484] Step 3: Temporarily store the voice data and transfer it to the voice recognition system

[1485] input:

[1486] Audio data sent to the server

[1487] process:

[1488] 1. The server temporarily stores the received voice data.

[1489] 2. The server transfers the voice data to the voice recognition system. The voice recognition system analyzes the voice data, extracts customer information, and registers it in the database. The server updates the database based on the analysis results.

[1490] output:

[1491] Customer voice data and customer ID stored in a database

[1492] ---

[1493] Step 4: The customer contacts the call center and receives an automated message

[1494] input:

[1495] A database containing customer voice data and customer IDs

[1496] process:

[1497] 1. A user calls the call center outside business hours.

[1498] 2. The server launches an automated voice guidance system, prompting the user to re-record their voice for identity verification.

[1499] output:

[1500] Recorded voice data (for identity verification)

[1501] ---

[1502] Step 5: Verify your identity

[1503] input:

[1504] Recorded voice data for identity verification, customer voice data stored on the server

[1505] process:

[1506] 1. The server sends the recorded voice data back to the voice recognition system.

[1507] 2. The voice recognition system compares the newly recorded voice data with the voice data already registered.

[1508] 3. If the match is successful, the server notifies the user terminal that identity verification is complete.

[1509] output:

[1510] Identity verification result (success / failure)

[1511] ---

[1512] Step 6: The AI ​​system receives requests for information changes

[1513] input:

[1514] Identity verification result (success)

[1515] process:

[1516] 1. After successful identity verification, the server initiates a dialogue with the AI ​​system for the customer.

[1517] 2. The user speaks a request to change information or plan.

[1518] 3. The AI ​​system analyzes the user's speech and understands the request.

[1519] output:

[1520] Parsed request content

[1521] ---

[1522] Step 7: Save the changes

[1523] input:

[1524] Parsed request content

[1525] process:

[1526] 1. The server temporarily stores the change information received from the AI ​​system.

[1527] 2. The saved changes are stored in storage and added to a queue for the next business hours.

[1528] output:

[1529] Temporarily saved changes

[1530] ---

[1531] Step 8: Data verification and double-checking by call center staff

[1532] input:

[1533] Temporarily saved changes

[1534] process:

[1535] 1. During business hours, the server provides the saved changes to the call center staff.

[1536] 2. Call center staff will review the data provided and manually double-check the accuracy of the data.

[1537] output:

[1538] Confirmed Changes

[1539] ---

[1540] Step 9: Persistent data storage and customer notification

[1541] input:

[1542] Confirmed Changes

[1543] process:

[1544] 1. After the staff confirms the accuracy of the data, the server will permanently store the confirmed data.

[1545] 2. The staff member calls the customer to inform them that the change has been completed.

[1546] output:

[1547] Permanently stored data, customer notification

[1548] (Application example 1)

[1549] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1550] Traditional call center identity verification and information change procedures are often dependent on business hours, making them inconvenient for customers. Traditional authentication methods are often manual, requiring time and resources, and risking errors and unauthorized access. Authentication and transaction processing at financial institutions, in particular, require high security, requiring fast and accurate identity verification.

[1551] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1552] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at a call center; means for verifying the customer's identity based on the received voice data; means for the customer to request changes to their information or plan in an interactive format with AI after identity verification is complete; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; means for authenticating the financial institution and for users to access accounts using voice; means for the voice recognition system to verify the customer's identity by comparing it with pre-registered voice data; and means for the AI ​​system to process the transaction details and action details if identity verification is successful. This enables customers to safely and quickly verify their identity and process transactions 24 hours a day.

[1553] "Means for recording the customer's voice" refers to a device or software that has the function of recording the customer's voice as digital data.

[1554] The "means for transmitting recorded voice data to a server" is a function or device for transmitting recorded voice data to a server via a network.

[1555] The "means for registering voice data received by the server in the voice recognition system" is a function for registering voice data received by the server in the voice recognition system, making it available for use in the subsequent authentication process.

[1556] "Means for activating automated voice guidance at a call center" refers to a system or software for automatically starting voice guidance in response to a call from a customer.

[1557] "Means for verifying identity based on received voice data" refers to technical means for verifying identity using voice data sent to the server.

[1558] "A means for customers to interact with AI to request changes to their information or plans after identity verification is complete" refers to a function that allows customers to interact with the AI ​​system and request changes to their information or plans after successful identity verification.

[1559] The "means for temporarily storing change information on a server" refers to a system or function for temporarily storing change information requested by a customer on a server.

[1560] The "means for providing temporarily stored data to call center staff" refers to a system or function that provides temporarily stored modified data to call center staff.

[1561] "Means for call center staff to double-check data" refers to the process by which call center staff verify the accuracy and consistency of temporarily stored data.

[1562] "Means for permanently storing confirmed data and notifying customers of the results of changes" refers to a system or function that permanently stores data after double-checking and notifies customers that changes have been completed.

[1563] "Financial institution authentication method" means a system or technology used to verify the identity of a user in order to ensure the security of financial transactions.

[1564] "A means for users to access their accounts using their voice" refers to the ability for users to access their bank accounts or online accounts using their voice.

[1565] "Means for verifying identity by having a voice recognition system compare the user's speech with pre-registered voice data" refers to a technology in which a voice recognition system compares the user's speech with pre-registered data to verify the user's identity.

[1566] "Means for the AI ​​system to process transaction details and action details if identity verification is successful" refers to technology that enables the AI ​​system to analyze the user's request and process the transaction or action after identity verification is successful.

[1567] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. This system not only improves customer convenience but also provides high security for financial institutions and other organizations. An embodiment of the present invention is described below.

[1568] Overall system configuration

[1569] The system mainly consists of the following parts:

[1570] Customer's device

[1571] server

[1572] Voice Recognition System

[1573] AI systems

[1574] Call center departments

[1575] Hardware and software used

[1576] Hardware:

[1577] A microphone built into a smartphone or dedicated device

[1578] software:

[1579] Voice Recording Application

[1580] Python

[1581] speech_recognition library (speech recognition)

[1582] requests library (HTTP communication)

[1583] AI system (imported as ai_system)

[1584] Processing flow

[1585] Customer voice registration and identity verification

[1586] 1. Customer device: The customer launches the voice recording app and speaks a specific phrase to record it. Once recording is complete, the voice data is sent to the server.

[1587] 2. Server: The received voice data is forwarded to the voice recognition system and registered with the customer ID, which is used for the subsequent identity verification process.

[1588] 3. Call center response: When a customer contacts the center outside of business hours, an automated voice guidance system is activated, prompting the customer to record a voice message for identity verification, which is then sent to the server.

[1589] 4. Voice recognition system: The system analyzes the recorded voice of the customer and verifies their identity by comparing it with registered voice data. If the comparison is successful, identity verification is complete.

[1590] AI-based information change processing

[1591] 5. Customer terminal: After completing identity verification, the customer interacts with the AI ​​system to request changes to their information or plans. The AI ​​system analyzes the content of their speech and understands their request.

[1592] 6. Server: The AI ​​system will temporarily store the analyzed changes on the server, and this data will be provided to the call center staff during the next business hours.

[1593] Call center confirmation and notification

[1594] 7. Call center staff: During business hours, they will review the temporarily stored data, double-check the accuracy and consistency of the data, and if there are no problems, store the data permanently.

[1595] 8. Customer Notification: Once verification is complete, a call center staff member will contact the customer to inform them that the change has been completed.

[1596] Specific examples

[1597] Example 1: Address change

[1598] 1. The customer records the audio and sends it to the server.

[1599] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[1600] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[1601] 4. On the next business day, call center staff will double-check the data and complete the confirmation.

[1602] 5. Call the customer to notify them that the address change is complete.

[1603] Example 2: Plan change

[1604] 1. The customer registers the voice call when signing the contract.

[1605] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[1606] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[1607] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[1608] 5. Make a confirmation call and notify the customer that the plan change is complete.

[1609] Prompt Sentence Examples

[1610] The user launches the application and speaks the following phrase:

[1611] "Log in, please use face recognition"

[1612] "Will you pay this month's electricity bill?"

[1613] In this way, customers can rest assured that their services are safe and the entire system can provide efficient and accurate support.

[1614] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1615] Step 1:

[1616] explanation:

[1617] The user starts the voice recording app, speaks a specific phrase, and records it. Once the recording is complete, the voice data is stored on the device.

[1618] input:

[1619] What the user said.

[1620] output:

[1621] Recorded audio data file.

[1622] Specific behavior:

[1623] The user launches the application and speaks a specific phrase following the app's guidance. The speech is recorded through the device's microphone and saved as an audio data file in the device's internal storage or memory.

[1624] Step 2:

[1625] explanation:

[1626] The device sends the recorded voice data to the server via the network, and the server stores the received voice data.

[1627] input:

[1628] Recorded audio data file.

[1629] output:

[1630] Audio data transferred to the server.

[1631] Specific behavior:

[1632] The device creates an HTTP request and sends it to the server with the recorded audio file attached. The server receives the request and stores the audio data in a database.

[1633] Step 3:

[1634] explanation:

[1635] The server transfers the received voice data to the voice recognition system and registers it along with the customer ID.

[1636] input:

[1637] Voice data and customer ID stored on the server.

[1638] output:

[1639] A pair of voice data and customer ID registered in the voice recognition system.

[1640] Specific behavior:

[1641] The server sends the voice data and customer ID to the voice recognition system's API, and the voice recognition system registers the data in its internal database.

[1642] Step 4:

[1643] explanation:

[1644] When a customer contacts the call center outside of business hours, an automated voice guidance system is activated, prompting them to record a voice message to verify their identity.

[1645] input:

[1646] Customer calls and voice guidance scripts.

[1647] output:

[1648] Recorded voice data for identity verification.

[1649] Specific behavior:

[1650] The call center system activates an automated voice guidance system, prompting the customer to speak phrases, which are then recorded and saved as audio data.

[1651] Step 5:

[1652] explanation:

[1653] The voice recognition system analyzes the recorded customer voice and compares it with pre-registered voice data to verify the customer's identity.

[1654] input:

[1655] Recorded voice data for identity verification, pre-registered voice data.

[1656] output:

[1657] The result of identity verification (success / failure).

[1658] Specific behavior:

[1659] The voice recognition system analyzes the recorded voice data and extracts features. These are then compared with pre-registered voice features. If they match, the identity verification is successful; if they do not, it is deemed unsuccessful.

[1660] Step 6:

[1661] explanation:

[1662] Once identity verification is complete, customers can interact with the AI ​​system and request changes to their information or plans.

[1663] input:

[1664] Identity verification result (success), customer's statement.

[1665] output:

[1666] Text data for information change or plan change requests.

[1667] Specific behavior:

[1668] Once identity verification is successful, the AI ​​system will be activated and will receive voice requests from customers to change plans or information, which will be converted into text by a voice recognition system.

[1669] Step 7:

[1670] explanation:

[1671] The server temporarily stores the change information analyzed by the AI ​​system.

[1672] input:

[1673] Text data for information change or plan change requests.

[1674] output:

[1675] Changed data temporarily saved on the server.

[1676] Specific behavior:

[1677] The text data generated by the AI ​​system is sent to a server, which temporarily stores the data.

[1678] Step 8:

[1679] explanation:

[1680] Call center staff will check and double-check the data that has been temporarily saved during business hours.

[1681] input:

[1682] Changed data temporarily saved on the server.

[1683] output:

[1684] Results of double-checked data (confirmation / correction).

[1685] Specific behavior:

[1686] Call center staff retrieve the stored data from the server and verify the accuracy and consistency of the input. Once verified, the data is permanently stored in the final database.

[1687] Step 9:

[1688] explanation:

[1689] A call center staff member will notify the customer that the verification has been completed.

[1690] input:

[1691] Results of double-checked data (confirmation).

[1692] output:

[1693] Notifying customers (calls, emails, etc.).

[1694] Specific behavior:

[1695] Call center staff will contact customers by phone or email to inform them that their information or plan change has been completed.

[1696] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1697] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves more flexible and human-like responses. It also aims to enable customers of all ages, including the elderly, to use the service with peace of mind.

[1698] System Overview

[1699] The system consists of a customer terminal, server, voice recognition system, AI system, emotion engine, and call center department. The customer terminal is a smartphone or dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. The emotion engine recognizes emotions from the customer's voice and the content of the dialogue, and responds accordingly. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[1700] Program processing and specific examples

[1701] Customer voice registration

[1702] User device:

[1703] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[1704] server:

[1705] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[1706] Identity verification and information changes outside business hours

[1707] User device:

[1708] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[1709] server:

[1710] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[1711] User device:

[1712] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the content of the request. At this time, the emotion engine also recognizes the customer's emotions and adjusts the response as necessary. For example, if the customer is feeling stressed, the system will respond in a calmer tone. The changes are then temporarily saved on the server.

[1713] server:

[1714] The temporarily saved change information will be provided to call center staff during the next business hours.

[1715] Data confirmation and double-checking during business hours

[1716] server:

[1717] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[1718] Call Center Staff:

[1719] Our staff will review the data provided and double-check the accuracy and consistency of the information. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[1720] Final confirmation to the customer:

[1721] A staff member will call the customer to let them know the change has been made.

[1722] Specific examples

[1723] Example 1:

[1724] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[1725] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[1726] 3. The customer requests a change of address through a dialogue with the AI ​​system, and the emotion engine recognizes the customer's emotions and adjusts the response accordingly. The change information is temporarily stored on the server.

[1727] 4. On the next business day, call center staff will double-check the data and complete the verification.

[1728] 5. Call the customer to notify them that the address change is complete.

[1729] Example 2:

[1730] 1. The customer registers the voice call when signing the contract.

[1731] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[1732] 3. After a second attempt, the customer is successfully identified and requests a plan change through a dialogue with the AI. The emotion engine recognizes the customer's stress and responds calmly.

[1733] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[1734] 5. Make a confirmation call and notify the customer that the plan change is complete.

[1735] In this way, the entire system works together to provide an environment where customers can use the service with peace of mind. The introduction of an emotion engine makes it possible to provide customer service with a more human touch.

[1736] The processing flow will be explained below.

[1737] Step 1:

[1738] User device: When signing up for a new contract, the customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[1739] Step 2:

[1740] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[1741] Step 3:

[1742] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[1743] Step 4:

[1744] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[1745] Step 5:

[1746] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[1747] Step 6:

[1748] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[1749] Step 7:

[1750] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[1751] Step 8:

[1752] Emotion engine: Recognizes emotions from customer voices and conversations and provides that emotional information to the AI ​​system. For example, if the emotion engine determines that a customer is stressed, the AI ​​system will adjust its responses to use a calmer tone.

[1753] Step 9:

[1754] Server: Temporarily stores data on information changes and plan changes received from the AI ​​system.

[1755] Step 10:

[1756] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[1757] Step 11:

[1758] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[1759] Step 12:

[1760] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[1761] Step 13:

[1762] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[1763] Specific examples

[1764] Example 1:

[1765] 1. Step 1: When a customer signs up for a new service, their voice is recorded and sent to the server.

[1766] 2. Step 2: The server registers the received audio data.

[1767] 3. Step 3-6: The customer calls the call center outside business hours and uses automated voice guidance to verify their identity.

[1768] 4. Step 7: Following the automated voice guidance, the AI ​​system is activated and the customer requests a change of address.

[1769] 5. Step 8: The emotion engine recognizes customer emotions and provides adjustment information to the AI ​​system.

[1770] 6. Step 9: The changes are saved to the server.

[1771] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[1772] 8. Step 12: Call center staff will verify and double-check the data.

[1773] 9. Step 13: A staff member will call the customer to inform them that the address change has been completed.

[1774] Example 2:

[1775] 1. Step 1: The customer registers by voice upon signing up.

[1776] 2. Step 2: The server registers the audio data.

[1777] 3. Step 3-6: You try to change your plan outside of business hours, but the initial identity verification fails.

[1778] 4. Step 7: After successfully verifying your identity, you can request a plan change through a dialogue with the AI.

[1779] 5. Step 8: The emotion engine recognizes the customer's stress level and instructs the AI ​​system to adjust.

[1780] 6. Step 9: The changes are saved to the server.

[1781] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[1782] 8. Step 12: Call center staff will verify and double-check the data.

[1783] 9. Step 13: A staff member will call the customer to inform them that the plan change has been completed.

[1784] This trend allows customers to receive fast and accurate service through automated systems, and the introduction of emotion engines makes responses more flexible and human, improving customer satisfaction.

[1785] Example 2

[1786] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1787] Call center operations require fast and accurate customer identity verification and information changes. However, conventional systems require a lot of manual work, which can lead to errors and delays. Furthermore, conventional automated systems lack emotion recognition and provide mechanical customer service, making it difficult to improve customer satisfaction. Furthermore, few systems are available outside of business hours, resulting in poor customer convenience.

[1788] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request an information change or a plan change in an interactive format with AI after the identity verification is completed; means including an emotion engine for recognizing the customer's emotions and adjusting the response; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results. This makes it possible to realize a system that quickly and accurately changes customer information and verifies the customer's identity, enhances customer service with a more human touch, and is available even outside business hours.

[1789] A "customer voice recording device" is any device or software that allows a customer to digitally record their own voice.

[1790] The "means for transmitting recorded audio data to a server" refers to the functions and protocols for transferring recorded audio files to a server via a network.

[1791] The "means for the server to register the received voice data in the voice recognition system" refers to the process in which the server passes the received voice file to the voice recognition algorithm for analysis and registration.

[1792] "Means for activating automated voice guidance at a call center" refers to a technology that enables a call center system to automatically start voice guidance and play a voice message to a customer.

[1793] "Means for verifying identity based on received voice data" refers to the process of analyzing the recorded voice file and comparing it with pre-registered voice data to verify the identity of the customer.

[1794] "A means for customers to interactively request changes to information or plans through AI after identity verification is complete" refers to a function that, once identity verification is successful, activates the AI ​​system and interactively accepts requests from customers to change information or plans.

[1795] The "means including an emotion engine that recognizes customer emotions and adjusts responses" refers to an algorithm or system that detects customer emotions through voice analysis and adjusts response content according to those emotions.

[1796] The "means for temporarily saving change information on the server" is a function for temporarily saving information change requests from customers on the server.

[1797] The "means for providing temporarily stored data to call center staff" is a function that allows call center staff to access temporarily stored changed data during business hours.

[1798] "Means for call center staff to double-check data" refers to the process by which staff review the data provided and verify its accuracy and consistency.

[1799] "Means for permanently storing confirmed data and informing customers of changes" refers to a database that stores double-checked data for a long period of time and a means of communication for notifying customers of changes.

[1800] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves a more flexible and human-like response. The purpose of this invention is to provide an environment where customers of all generations can use the service with peace of mind.

[1801] System Overview

[1802] The system consists of the following elements:

[1803] 1. Customer terminal (smartphone or dedicated device): Customers record their own voice and send it to the server. Specifically, this provides a means to capture voice using a recording application and send it to the server.

[1804] 2. Server: Responsible for receiving, registering, temporarily storing, and providing voice data. The server is integrated with the voice recognition system and AI system.

[1805] 3. Voice recognition system: Analyzes customer voice data and verifies their identity. It uses algorithms such as deep learning models.

[1806] 4. AI system: Handles information changes and plan changes through customer interactions.

[1807] 5. Emotion engine: Recognizes emotions from customer voice and dialogue and responds accordingly.

[1808] Hardware and Software

[1809] Customer terminal: Smartphone, dedicated device

[1810] Servers: Database servers, application servers

[1811] Speech recognition system: speech analysis software (e.g., Google Cloud Speech-to-Text)

[1812] AI system: Conversational AI platform (e.g., Dialogflow)

[1813] Emotion engine: Emotion analysis software (e.g. IBM Watson Tone Analyzer)

[1814] Example of system operation

[1815] 1. Registering customer feedback

[1816] User device: When a customer signs up for a new service, they launch the application and record their own voice. Once the recording is complete, they press the "Send" button to send the voice data to the server.

[1817] Server: Receives the transmitted voice data and forwards it to the voice recognition system. It registers it along with the customer ID, which is used in the subsequent identity verification process.

[1818] 2. Identity Verification Process (Outside Business Hours)

[1819] User device: When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[1820] Server: Receives the customer's recorded voice, analyzes it with a voice recognition system, and compares it with registered voice data. If the comparison is successful, identity verification is complete.

[1821] User device: Once identity verification is complete, a dialogue interface with the AI ​​system is launched, allowing the user to request changes to information or plans. The emotion engine recognizes the user's emotions and adjusts the response as necessary.

[1822] 3. Data verification and double-checking (during business hours)

[1823] Server: Provides the temporarily stored data to the terminals of call center staff during business hours.

[1824] Call center staff: verify the data provided, double-check the information for accuracy and consistency, permanently store it if correct, and notify the customer of any changes.

[1825] Specific examples

[1826] Example 1:

[1827] 1. When a user signs up for a new contract, they record a voice message and send it to the server.

[1828] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[1829] 3. The user requests a change of address through an interactive dialogue with the AI ​​system, and the emotion engine recognizes the user's emotions and adjusts the response accordingly. The change information is temporarily stored on the server.

[1830] 4. On the next business day, call center staff will double-check the data and complete the verification.

[1831] 5. Call the user to let them know the address change is complete.

[1832] Example prompt sentence:

[1833] Please write a statement:

[1834] 1. Overview of the call center system using voice recognition technology and AI

[1835] 2. The role of an emotion engine that recognizes customer emotions

[1836] 3. Specific examples of each processing step

[1837] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1838] Step 1: Record and send audio data

[1839] User device: When signing up for a new contract, the user launches the recording application. The user presses the "Start Recording" button to record their own voice. When recording is complete, the user presses the "Send" button to send the voice data to the server.

[1840] Input: User's voice

[1841] Output: Audio data file (e.g., wav format)

[1842] What happens: A user uses the application's recording feature to record audio and send the data to the server.

[1843] Step 2: Receiving and registering voice data

[1844] Server: The server receives the voice data, transfers it to the voice recognition system, and registers it in the server's database along with the customer ID.

[1845] Input: Voice data file, customer ID

[1846] Output: Registration information in the database

[1847] How it works: The server receives the voice data, transfers it to a voice recognition system, and registers it in a database along with the customer ID.

[1848] Step 3: Launch the automated voice guidance

[1849] User device: A user calls the call center outside business hours. An automated voice prompt is activated and provides instructions for recording.

[1850] Input: User's call

[1851] Output: Voice guidance to start recording

[1852] How it works: When a user calls, an automated answering system is activated and provides voice prompts.

[1853] Step 4: Record and send customer voice (identity verification)

[1854] User device: Following the automated voice guidance, the user records a voice message for identity verification and sends it to the server.

[1855] Input: User's recorded voice

[1856] Output: Audio data file (for confirmation)

[1857] How it works: The user follows the audio prompts to record audio, then presses the send button to send it to the server.

[1858] Step 5: Verify your identity with voice data

[1859] Server: The server sends the received voice data to the voice recognition system in real time and compares it with the registered data. If the comparison is successful, identity verification is complete.

[1860] Input: Audio data file

[1861] Output: Matching result

[1862] How it works: The server sends the voice data to a voice recognition system, which checks it against the enrollment data and returns a confirmation result.

[1863] Step 6: Interact with the AI ​​system

[1864] User device: Once identity verification is successful, the user begins a dialogue with the AI ​​system, requesting changes to information or plans via voice.

[1865] Input: What the user says

[1866] Output: Dialogue

[1867] How it works: The user gives voice instructions to the AI ​​system, which then analyzes them and proceeds with the dialogue.

[1868] Step 7: Emotion recognition and response adjustment

[1869] Emotion Engine: The AI ​​system analyzes the content and tone of the user's speech, and the emotion engine recognizes the user's emotions and adjusts the tone of the response as needed.

[1870] Input: User's voice data

[1871] Output: Recognized emotions and adjusted responses

[1872] How it works: The emotion engine analyzes the emotion in the user's voice and adjusts the system's response tone.

[1873] Step 8: Temporarily save the changed information data

[1874] Server: The AI ​​system temporarily stores the information changes data obtained as a result of the dialogue on the server.

[1875] Input: Information change data

[1876] Output: Temporarily saved data

[1877] How it works: The server records the information changes in a temporary database.

[1878] Step 9: Provide data

[1879] Server: Provides temporarily stored data to call center staff devices during business hours.

[1880] Input: Temporarily saved data

[1881] Output: Data displayed on the call center staff's terminal

[1882] Operation: The server provides the temporarily stored data to the staff terminal so that the staff can check it.

[1883] Step 10: Double-check your data

[1884] Call center staff: Staff will review the data provided and double-check the information for accuracy and consistency.

[1885] Input: Temporarily saved data

[1886] Output: Double-check result (confirmed data or correction request)

[1887] Action: Staff will check the data and save it if it is correct, or ask for corrections if it is incorrect.

[1888] Step 11: Final confirmation and notification

[1889] Call center staff: Staff will contact you to let you know that the change has been made.

[1890] Input: Confirmed data

[1891] Output: User notification

[1892] What happens: An agent calls or messages the customer to let them know about the changes.

[1893] (Application example 2)

[1894] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1895] Conventional food delivery services have faced problems such as the hassle it takes for customers to change or cancel their orders, and delayed response outside of business hours. Furthermore, it can be difficult to respond appropriately to customers' emotions, leading to lower customer satisfaction. The present invention aims to solve these problems by providing a system that allows customers to change their orders quickly and smoothly without feeling stressed.

[1896] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1897] In this invention, the server includes means for recording the customer's voice, means for transmitting the recorded voice data to the server, means for registering the voice data received by the server in a voice recognition system, means for activating automated voice guidance at the call center, means for verifying the customer's identity based on the received voice data, means for the customer to interactively request changes to information or order details through AI after identity verification is complete, means for recognizing the customer's emotions and adjusting the response, means for temporarily storing the changed information on the server, means for providing the temporarily stored data to call center staff, means for the call center staff to double-check the data, and means for permanently storing the verified data and notifying the customer of the change results. This allows customers to change or cancel their orders at any time, and also enables appropriate responses based on their emotions.

[1898] A "means for recording the customer's voice" is any device or software that electronically records the voice uttered by the customer.

[1899] The "means for transmitting recorded voice data to a server" is a device or software for transferring recorded voice data to a remote server via a communication network.

[1900] "Means for registering voice data received by the server in the voice recognition system" refers to a device or software that stores the voice data received by the server in the voice recognition system for analysis and makes it reusable as needed.

[1901] A "means for activating automated voice guidance in a call center" is a device or software that initiates a system that automatically plays pre-set voice messages in response to calls from customers.

[1902] "Means for verifying identity based on received voice data" refers to a device or software that analyzes the recorded voice data and verifies whether it matches the voice data of the registered customer.

[1903] "A means for customers to request changes to information or order details through an interactive AI dialogue after identity verification has been completed" refers to a device or software that allows verified customers to converse with AI and request changes to information or order details.

[1904] "Means for recognizing customer emotions and adjusting responses" refers to devices or software that analyze emotions from a customer's voice or text and adjust the content and tone of responses based on the results.

[1905] The "means for temporarily storing change information on a server" refers to a device or software that temporarily stores change requests from customers so that they can be verified and processed later.

[1906] "Means for providing temporarily stored data to call center staff" refers to a device or software that presents temporarily stored data to the terminal of a call center staff member for processing or confirmation.

[1907] "Means for call center staff to double-check data" means devices or software that allow call center staff to verify the accuracy of the data provided and correct or re-verify it as necessary.

[1908] "Means for permanently storing confirmed data and notifying customers of the results of changes" refers to a device or software for storing confirmed data for a long period of time and notifying customers of the results.

[1909] This invention provides a system for efficiently changing or canceling orders in food delivery services. The system aims to increase customer satisfaction by combining customer identity verification using voice, conversational order changes using AI, and emotion recognition. The system consists of the following components:

[1910] System configuration

[1911] The system is broadly composed as follows:

[1912] Customer device: A smartphone or dedicated device that records the customer's voice and sends it to the server.

[1913] Server: Responsible for receiving voice data, recognizing the voice, temporarily storing the data, and finally permanently storing the data. The following software components are installed on the server:

[1914] Voice Recognition System

[1915] Emotion Recognition Engine

[1916] AI order processing engine

[1917] Call center system: Double-checks data and notifies customers of changes.

[1918] Program processing

[1919] Audio recording and transmission

[1920] The user starts a dedicated application on their smartphone and records their voice request for an order change. The recorded voice data is sent to the server in real time.

[1921] Voice recognition and identity verification

[1922] The server passes the received voice data to a voice recognition system for analysis, which compares the voice data with registered voice data to verify the user's identity.

[1923] Order changes through AI dialogue

[1924] After completing identity verification, the AI ​​system receives a conversational request from the customer to change their order. An emotion recognition engine analyzes the customer's emotions, recognizing stress, joy, etc., and adjusts the response accordingly.

[1925] Emotion Recognition and Data Storage

[1926] If the customer is feeling stressed, the AI ​​system will respond in a calmer tone. The modified order details are temporarily saved on the server.

[1927] Response and data check

[1928] Call center staff will review the temporarily stored data during business hours, double-check for accuracy, and if there are no issues, store the data permanently and notify the customer of the changes.

[1929] Specific examples

[1930] 1. The customer changes their order by voice, saying, "I'd like one more pizza and one more Coke," and the emotion engine recognizes the customer's joy.

[1931] 2. The AI ​​order processing engine calmly replies, "We've added a pizza and a Coke. Thank you for your order!"

[1932] Prompt Sentence Examples

[1933] "A customer says, 'I'd like one more pizza and one more Coke.' You process this order, and your emotion recognition engine recognizes the pleasure. Respond in the most appropriate order to make the customer happy."

[1934] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1935] Step 1:

[1936] Audio recording and transmission

[1937] Customers use their smartphones to launch a dedicated application and record voice requests for changes to their order, such as "I'd like one more pizza and one more Coke." The recorded voice data is sent to the server in real time.

[1938] Input: Customer voice

[1939] Data processing: Recorded as audio data

[1940] Output: Audio data sent to the server

[1941] Step 2:

[1942] Voice recognition and identity verification

[1943] The server passes the received voice data to a voice recognition system for analysis, which compares the voice data with registered voice data to verify the user's identity.

[1944] Input: Received audio data

[1945] Data calculation: Analysis of voice data and comparison with registered data

[1946] Output: Identity verification result

[1947] Step 3:

[1948] Order changes through AI dialogue

[1949] After identity verification is complete, the AI ​​system receives order change requests from customers in an interactive format.

[1950] Input: Voice data after identity verification

[1951] Data calculation: Analyzes voice and understands order details

[1952] Output: Text format data of the order change request

[1953] Step 4:

[1954] Emotion recognition and response adjustment

[1955] An emotion recognition engine analyzes the customer's voice and text to determine whether they are feeling happy or stressed, and the AI ​​system adjusts the response accordingly.

[1956] Input: Customer voice and text data

[1957] Data Computing: Emotion Recognition Analysis

[1958] Output: Emotion recognition results and corresponding policy

[1959] Step 5:

[1960] Temporarily save order details

[1961] The analyzed order details and emotion recognition results are temporarily stored on the server, allowing call center staff to check them later.

[1962] Input: Order details and emotion recognition results

[1963] Data processing: Temporary storage of data

[1964] Output: Data temporarily saved on the server

[1965] Step 6:

[1966] Data check and storage during business hours

[1967] Call center staff will review the temporarily stored data during business hours, double-check for accuracy, and if there are no issues, store the data permanently.

[1968] Input:Temporarily saved data

[1969] Data calculations: data verification and double-checking

[1970] Output: Verified and persistently stored data

[1971] Step 7:

[1972] Notify customers of the changes

[1973] Finally, the changes are notified to the customer, who can confirm that the order details have been changed.

[1974] Input: Verified permanently stored data

[1975] Data processing: Creating a notification of the changes

[1976] Output: Customer notification

[1977] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1978] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1979] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1980] [Fourth embodiment]

[1981] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1982] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1983] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1984] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1985] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1986] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1987] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1988] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1989] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1990] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1991] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1992] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1993] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1994] This invention utilizes voice recognition technology and AI to provide a system that automates identity verification and various information changes at call centers, aiming to enable customers of all ages, including the elderly, to use the service with peace of mind.

[1995] System Overview

[1996] The system consists of a customer terminal, a server, a voice recognition system, an AI system, and a call center. The customer terminal is a smartphone or a dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[1997] Program processing and specific examples

[1998] Customer voice registration

[1999] User device:

[2000] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[2001] server:

[2002] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[2003] Identity verification and information changes outside business hours

[2004] User device:

[2005] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[2006] server:

[2007] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[2008] User device:

[2009] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the request, then temporarily stores the changes on the server.

[2010] server:

[2011] The temporarily saved change information will be provided to call center staff during the next business hours.

[2012] Data confirmation and double-checking during business hours

[2013] server:

[2014] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[2015] Call Center Staff:

[2016] Our staff will review the data provided and double-check it for accuracy and consistency. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[2017] Final confirmation to the customer:

[2018] A staff member will call the customer to let them know the change has been made.

[2019] Specific examples

[2020] Example 1:

[2021] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[2022] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[2023] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[2024] 4. On the next business day, call center staff will double-check the data and complete the verification.

[2025] 5. Call the customer to notify them that the address change is complete.

[2026] Example 2:

[2027] 1. The customer registers the voice call when signing the contract.

[2028] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[2029] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[2030] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[2031] 5. Make a confirmation call and notify the customer that the plan change is complete.

[2032] This allows customers to use the service with peace of mind even outside of business hours. The entire system is automated, ensuring efficient and accurate support.

[2033] The processing flow will be explained below.

[2034] Step 1:

[2035] User device: The customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[2036] Step 2:

[2037] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[2038] Step 3:

[2039] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[2040] Step 4:

[2041] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[2042] Step 5:

[2043] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[2044] Step 6:

[2045] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[2046] Step 7:

[2047] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[2048] Step 8:

[2049] User: A customer requests an AI system to change information or plans.

[2050] Step 9:

[2051] Server: Temporarily stores changes received from the AI ​​system.

[2052] Step 10:

[2053] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[2054] Step 11:

[2055] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[2056] Step 12:

[2057] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[2058] Step 13:

[2059] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[2060] In this way, the entire system works in cooperation, providing an environment in which customers can use services with peace of mind.

[2061] Example 1

[2062] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2063] Traditionally, identity verification and information change procedures at call centers have been done manually, which not only required a great deal of effort and time, but also posed the risk of human error. Furthermore, the process of identity verification and information change was cumbersome and difficult to use, especially for elderly people and non-technical users. Furthermore, the inability to respond outside of business hours limited the ability for customers to complete the procedures at their desired times.

[2064] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2065] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the received voice data in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request information changes or plan changes interactively with AI after identity verification is complete; means for temporarily storing the change information on the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; a voice recognition system for processing the customer's voice; means for analyzing and understanding the customer's request using an AI system; means for storing the voice data and change information in storage; means for the server to add the stored information to a queue; and means for call center staff to verify and correct the data. This automates identity verification and information change procedures, improving efficiency and convenience. In particular, it provides an environment where customers of all ages, including seniors and non-technical personnel, can use the service with peace of mind, and also realizes a system that can be used outside of business hours.

[2066] "Means for recording the customer's voice" means any device or software that the customer uses to record his or her own voice.

[2067] The "means for transmitting recorded voice data to a server" is a technology for transmitting recorded voice data from a customer's terminal to a server via a network.

[2068] The "means for registering the voice data received by the server in the voice recognition system" refers to the process of sending the voice data received by the server to the voice recognition system and registering the data.

[2069] The "means for starting automated voice guidance at a call center" refers to a system or program for automatically starting voice guidance at a call center.

[2070] "Means for verifying identity based on received voice data" refers to a technology that uses a voice recognition system to verify the identity of a customer by comparing the received voice data with registered voice data.

[2071] "A means for customers to interactively request changes to their information or plans through AI after identity verification has been completed" is a function that uses AI linked to a voice recognition system to enable customers to interactively request changes to their desired information or plans.

[2072] The "means for temporarily storing change information on a server" is a method for temporarily storing change information generated in response to a request from a customer on a server.

[2073] The "means for providing temporarily stored data to call center staff" is a mechanism for providing data temporarily stored on a server to call center staff.

[2074] "Means for call center staff to double-check data" refers to a process by which call center staff manually review the data provided and check for accuracy.

[2075] "Means for permanently storing confirmed data and notifying customers of any changes" refers to procedures for storing data confirmed by call center staff on a long-term basis and notifying customers of any changes.

[2076] A "voice recognition system for processing customer voice" is a system for analyzing and recognizing customer voice data.

[2077] "Means for analyzing and understanding customer requests using an AI system" refers to a method for using AI technology to analyze customer requests and understand their meaning.

[2078] The "means for saving voice data and change information in storage" refers to a technology for saving recorded voice data and change request information from a customer in digital storage.

[2079] The term "means for a server to add information stored in storage to a queue" refers to a technique for adding information stored in storage to a queue in order to process it in a fixed order.

[2080] "Means for call center staff to check and correct data" refers to a method by which call center staff can check the contents of the data and correct it as necessary.

[2081] This invention relates to a system that utilizes voice recognition technology and artificial intelligence (AI) to automate identity verification and various information changes at call centers. This system consists of customer terminals, servers, a voice recognition system, an AI system, and the call center departments.

[2082] System configuration

[2083] Customer terminal

[2084] The customer terminal is a smartphone or other dedicated device that the customer uses to record voice and send it to the server. For example, a dedicated application is installed on the smartphone, and the customer starts the application to record their own voice. The recorded voice data is automatically sent to the server.

[2085] server

[2086] The server receives and stores the voice data sent from the customer's device and links it with the voice recognition system and AI system. The server is responsible for the following processes:

[2087] 1. Receive the voice data sent from the customer terminal.

[2088] 2. The received voice data is temporarily stored and transferred to the voice recognition system.

[2089] 3. The data analyzed by the voice recognition system is stored in a database along with the customer ID.

[2090] 4. Process customer identity verification and information change requests outside of business hours.

[2091] 5. The analysis results and changes from the AI ​​system are saved and provided to call center staff during the next business hours.

[2092] Voice Recognition System

[2093] The voice recognition system analyzes the voice data received from the customer and registers the customer ID and voice data in a database. When verifying the customer's identity, it also plays a role in comparing the registered voice data with newly recorded voice data.

[2094] AI systems

[2095] The AI ​​system understands and analyzes requests for information changes and plan changes through dialogue with customers. The AI ​​system analyzes customer utterances and temporarily stores the request content on the server.

[2096] Call Center

[2097] At the call center, staff double-check the data provided by the server during business hours and permanently store the confirmed data. Staff manually check the accuracy of the data and contact the customer to request corrections if necessary. After confirming the data, they notify the customer that the changes have been made.

[2098] Specific examples

[2099] Example 1: Address change

[2100] 1. When a user signs up for a new contract, they record a voice message and send it to the server.

[2101] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[2102] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[2103] 4. On the next business day, call center staff will double-check the data and complete the verification.

[2104] 5. Call the user to let them know the address change is complete.

[2105] Example 2: Plan change

[2106] 1. The user registers the voice call when signing up.

[2107] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[2108] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[2109] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[2110] 5. Make a confirmation call and notify the user that the plan change is complete.

[2111] Prompt Sentence Examples

[2112] "Please tell me about identity verification systems that use voice recognition technology at call centers."

[2113] This provides customers with an environment where they can use the service with peace of mind even outside of business hours, and because the entire system is automated, efficient and accurate support is achieved.

[2114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2115] Step 1: User records audio

[2116] input:

[2117] A smartphone or dedicated device used by users when signing up for a new contract or verifying their identity

[2118] process:

[2119] 1. The user launches the dedicated application and follows the application's instructions.

[2120] 2. The user records audio. The application starts recording audio data when the user presses the record button, and ends recording when the user presses the stop button.

[2121] output:

[2122] Recorded audio data

[2123] ---

[2124] Step 2: Send the audio data to the server

[2125] input:

[2126] Recorded audio data

[2127] process:

[2128] 1. The user device sends the recorded audio data to the server via a dedicated application.

[2129] 2. The application packets the audio data and sends it using the HTTPS protocol.

[2130] output:

[2131] Audio data sent to the server

[2132] ---

[2133] Step 3: Temporarily store the voice data and transfer it to the voice recognition system

[2134] input:

[2135] Audio data sent to the server

[2136] process:

[2137] 1. The server temporarily stores the received voice data.

[2138] 2. The server transfers the voice data to the voice recognition system. The voice recognition system analyzes the voice data, extracts customer information, and registers it in the database. The server updates the database based on the analysis results.

[2139] output:

[2140] Customer voice data and customer ID stored in a database

[2141] ---

[2142] Step 4: The customer contacts the call center and receives an automated message

[2143] input:

[2144] A database containing customer voice data and customer IDs

[2145] process:

[2146] 1. A user calls the call center outside business hours.

[2147] 2. The server launches an automated voice guidance system, prompting the user to re-record their voice for identity verification.

[2148] output:

[2149] Recorded voice data (for identity verification)

[2150] ---

[2151] Step 5: Verify your identity

[2152] input:

[2153] Recorded voice data for identity verification, customer voice data stored on the server

[2154] process:

[2155] 1. The server sends the recorded voice data back to the voice recognition system.

[2156] 2. The voice recognition system compares the newly recorded voice data with the voice data already registered.

[2157] 3. If the match is successful, the server notifies the user terminal that identity verification is complete.

[2158] output:

[2159] Identity verification result (success / failure)

[2160] ---

[2161] Step 6: The AI ​​system receives requests for information changes

[2162] input:

[2163] Identity verification result (success)

[2164] process:

[2165] 1. After successful identity verification, the server initiates a dialogue with the AI ​​system for the customer.

[2166] 2. The user speaks a request to change information or plan.

[2167] 3. The AI ​​system analyzes the user's speech and understands the request.

[2168] output:

[2169] Parsed request content

[2170] ---

[2171] Step 7: Save the changes

[2172] input:

[2173] Parsed request content

[2174] process:

[2175] 1. The server temporarily stores the change information received from the AI ​​system.

[2176] 2. The saved changes are stored in storage and added to a queue for the next business hours.

[2177] output:

[2178] Temporarily saved changes

[2179] ---

[2180] Step 8: Data verification and double-checking by call center staff

[2181] input:

[2182] Temporarily saved changes

[2183] process:

[2184] 1. During business hours, the server provides the saved changes to the call center staff.

[2185] 2. Call center staff will review the data provided and manually double-check the accuracy of the data.

[2186] output:

[2187] Confirmed Changes

[2188] ---

[2189] Step 9: Persistent data storage and customer notification

[2190] input:

[2191] Confirmed Changes

[2192] process:

[2193] 1. After the staff confirms the accuracy of the data, the server will permanently store the confirmed data.

[2194] 2. The staff member calls the customer to inform them that the change has been completed.

[2195] output:

[2196] Permanently stored data, customer notification

[2197] (Application example 1)

[2198] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2199] Traditional call center identity verification and information change procedures are often dependent on business hours, making them inconvenient for customers. Traditional authentication methods are often manual, requiring time and resources, and risking errors and unauthorized access. Authentication and transaction processing at financial institutions, in particular, require high security, requiring fast and accurate identity verification.

[2200] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2201] In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at a call center; means for verifying the customer's identity based on the received voice data; means for the customer to request changes to their information or plan in an interactive format with AI after identity verification is complete; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results; means for authenticating the financial institution and for users to access accounts using voice; means for the voice recognition system to verify the customer's identity by comparing it with pre-registered voice data; and means for the AI ​​system to process the transaction details and action details if identity verification is successful. This enables customers to safely and quickly verify their identity and process transactions 24 hours a day.

[2202] "Means for recording the customer's voice" refers to a device or software that has the function of recording the customer's voice as digital data.

[2203] The "means for transmitting recorded voice data to a server" is a function or device for transmitting recorded voice data to a server via a network.

[2204] The "means for registering voice data received by the server in the voice recognition system" is a function for registering voice data received by the server in the voice recognition system, making it available for use in the subsequent authentication process.

[2205] "Means for activating automated voice guidance at a call center" refers to a system or software for automatically starting voice guidance in response to a call from a customer.

[2206] "Means for verifying identity based on received voice data" refers to technical means for verifying identity using voice data sent to the server.

[2207] "A means for customers to interact with AI to request changes to their information or plans after identity verification is complete" refers to a function that allows customers to interact with the AI ​​system and request changes to their information or plans after successful identity verification.

[2208] The "means for temporarily storing change information on a server" refers to a system or function for temporarily storing change information requested by a customer on a server.

[2209] The "means for providing temporarily stored data to call center staff" refers to a system or function that provides temporarily stored modified data to call center staff.

[2210] "Means for call center staff to double-check data" refers to the process by which call center staff verify the accuracy and consistency of temporarily stored data.

[2211] "Means for permanently storing confirmed data and notifying customers of the results of changes" refers to a system or function that permanently stores data after double-checking and notifies customers that changes have been completed.

[2212] "Financial institution authentication method" means a system or technology used to verify the identity of a user in order to ensure the security of financial transactions.

[2213] "A means for users to access their accounts using their voice" refers to the ability for users to access their bank accounts or online accounts using their voice.

[2214] "Means for verifying identity by having a voice recognition system compare the user's speech with pre-registered voice data" refers to a technology in which a voice recognition system compares the user's speech with pre-registered data to verify the user's identity.

[2215] "Means for the AI ​​system to process transaction details and action details if identity verification is successful" refers to technology that enables the AI ​​system to analyze the user's request and process the transaction or action after identity verification is successful.

[2216] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. This system not only improves customer convenience but also provides high security for financial institutions and other organizations. An embodiment of the present invention is described below.

[2217] Overall system configuration

[2218] The system mainly consists of the following parts:

[2219] Customer's device

[2220] server

[2221] Voice Recognition System

[2222] AI systems

[2223] Call center departments

[2224] Hardware and software used

[2225] Hardware:

[2226] A microphone built into a smartphone or dedicated device

[2227] software:

[2228] Voice Recording Application

[2229] Python

[2230] speech_recognition library (speech recognition)

[2231] requests library (HTTP communication)

[2232] AI system (imported as ai_system)

[2233] Processing flow

[2234] Customer voice registration and identity verification

[2235] 1. Customer device: The customer launches the voice recording app and speaks a specific phrase to record it. Once recording is complete, the voice data is sent to the server.

[2236] 2. Server: The received voice data is forwarded to the voice recognition system and registered with the customer ID, which is used for the subsequent identity verification process.

[2237] 3. Call center response: When a customer contacts the center outside of business hours, an automated voice guidance system is activated, prompting the customer to record a voice message for identity verification, which is then sent to the server.

[2238] 4. Voice recognition system: The system analyzes the recorded voice of the customer and verifies their identity by comparing it with registered voice data. If the comparison is successful, identity verification is complete.

[2239] AI-based information change processing

[2240] 5. Customer terminal: After completing identity verification, the customer interacts with the AI ​​system to request changes to their information or plans. The AI ​​system analyzes the content of their speech and understands their request.

[2241] 6. Server: The AI ​​system will temporarily store the analyzed changes on the server, and this data will be provided to the call center staff during the next business hours.

[2242] Call center confirmation and notification

[2243] 7. Call center staff: During business hours, they will review the temporarily stored data, double-check the accuracy and consistency of the data, and if there are no problems, store the data permanently.

[2244] 8. Customer Notification: Once verification is complete, a call center staff member will contact the customer to inform them that the change has been completed.

[2245] Specific examples

[2246] Example 1: Address change

[2247] 1. The customer records the audio and sends it to the server.

[2248] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[2249] 3. Request an address change through an interactive AI session, and the change information is temporarily saved on the server.

[2250] 4. On the next business day, call center staff will double-check the data and complete the confirmation.

[2251] 5. Call the customer to notify them that the address change is complete.

[2252] Example 2: Plan change

[2253] 1. The customer registers the voice call when signing the contract.

[2254] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[2255] 3. After a second attempt, the user's identity is successfully verified and they can request a plan change through a dialogue with the AI.

[2256] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[2257] 5. Make a confirmation call and notify the customer that the plan change is complete.

[2258] Prompt Sentence Examples

[2259] The user launches the application and speaks the following phrase:

[2260] "Log in, please use face recognition"

[2261] "Will you pay this month's electricity bill?"

[2262] In this way, customers can rest assured that their services are safe and the entire system can provide efficient and accurate support.

[2263] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2264] Step 1:

[2265] explanation:

[2266] The user starts the voice recording app, speaks a specific phrase, and records it. Once the recording is complete, the voice data is stored on the device.

[2267] input:

[2268] What the user said.

[2269] output:

[2270] Recorded audio data file.

[2271] Specific behavior:

[2272] The user launches the application and speaks a specific phrase following the app's guidance. The speech is recorded through the device's microphone and saved as an audio data file in the device's internal storage or memory.

[2273] Step 2:

[2274] explanation:

[2275] The device sends the recorded voice data to the server via the network, and the server stores the received voice data.

[2276] input:

[2277] Recorded audio data file.

[2278] output:

[2279] Audio data transferred to the server.

[2280] Specific behavior:

[2281] The device creates an HTTP request and sends it to the server with the recorded audio file attached. The server receives the request and stores the audio data in a database.

[2282] Step 3:

[2283] explanation:

[2284] The server transfers the received voice data to the voice recognition system and registers it along with the customer ID.

[2285] input:

[2286] Voice data and customer ID stored on the server.

[2287] output:

[2288] A pair of voice data and customer ID registered in the voice recognition system.

[2289] Specific behavior:

[2290] The server sends the voice data and customer ID to the voice recognition system's API, and the voice recognition system registers the data in its internal database.

[2291] Step 4:

[2292] explanation:

[2293] When a customer contacts the call center outside of business hours, an automated voice guidance system is activated, prompting them to record a voice message to verify their identity.

[2294] input:

[2295] Customer calls and voice guidance scripts.

[2296] output:

[2297] Recorded voice data for identity verification.

[2298] Specific behavior:

[2299] The call center system activates an automated voice guidance system, prompting the customer to speak phrases, which are then recorded and saved as audio data.

[2300] Step 5:

[2301] explanation:

[2302] The voice recognition system analyzes the recorded customer voice and compares it with pre-registered voice data to verify the customer's identity.

[2303] input:

[2304] Recorded voice data for identity verification, pre-registered voice data.

[2305] output:

[2306] The result of identity verification (success / failure).

[2307] Specific behavior:

[2308] The voice recognition system analyzes the recorded voice data and extracts features. These are then compared with pre-registered voice features. If they match, the identity verification is successful; if they do not, it is deemed unsuccessful.

[2309] Step 6:

[2310] explanation:

[2311] Once identity verification is complete, customers can interact with the AI ​​system and request changes to their information or plans.

[2312] input:

[2313] Identity verification result (success), customer's statement.

[2314] output:

[2315] Text data for information change or plan change requests.

[2316] Specific behavior:

[2317] Once identity verification is successful, the AI ​​system will be activated and will receive voice requests from customers to change plans or information, which will be converted into text by a voice recognition system.

[2318] Step 7:

[2319] explanation:

[2320] The server temporarily stores the change information analyzed by the AI ​​system.

[2321] input:

[2322] Text data for information change or plan change requests.

[2323] output:

[2324] Changed data temporarily saved on the server.

[2325] Specific behavior:

[2326] The text data generated by the AI ​​system is sent to a server, which temporarily stores the data.

[2327] Step 8:

[2328] explanation:

[2329] Call center staff will check and double-check the data that has been temporarily saved during business hours.

[2330] input:

[2331] Changed data temporarily saved on the server.

[2332] output:

[2333] Results of double-checked data (confirmation / correction).

[2334] Specific behavior:

[2335] Call center staff retrieve the stored data from the server and verify the accuracy and consistency of the input. Once verified, the data is permanently stored in the final database.

[2336] Step 9:

[2337] explanation:

[2338] A call center staff member will notify the customer that the verification has been completed.

[2339] input:

[2340] Results of double-checked data (confirmation).

[2341] output:

[2342] Notifying customers (calls, emails, etc.).

[2343] Specific behavior:

[2344] Call center staff will contact customers by phone or email to inform them that their information or plan change has been completed.

[2345] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2346] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves more flexible and human-like responses. It also aims to enable customers of all ages, including the elderly, to use the service with peace of mind.

[2347] System Overview

[2348] The system consists of a customer terminal, server, voice recognition system, AI system, emotion engine, and call center department. The customer terminal is a smartphone or dedicated device that is capable of recording and transmitting voice. The server is responsible for receiving, registering, temporarily storing, and providing voice data. The voice recognition system analyzes the voice data and verifies the identity of the customer. The AI ​​system processes changes to information and plans through dialogue with the customer. The emotion engine recognizes emotions from the customer's voice and the content of the dialogue, and responds accordingly. Finally, call center staff double-check the data, permanently store the confirmed data, and contact the customer.

[2349] Program processing and specific examples

[2350] Customer voice registration

[2351] User device:

[2352] When signing up for a new contract, a customer launches a voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[2353] server:

[2354] The server then transfers the received voice data to a voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[2355] Identity verification and information changes outside business hours

[2356] User device:

[2357] When a customer calls the call center outside of business hours, an automated voice guidance system will be activated, prompting them to record a voice message to verify their identity.

[2358] server:

[2359] The recorded customer voice is analyzed by a voice recognition system and compared with registered voice data. If the comparison is successful, identity verification is complete.

[2360] User device:

[2361] After completing identity verification, customers can request changes to their information or plans through dialogue with the AI ​​system. The AI ​​system analyzes the customer's speech and understands the content of the request. At this time, the emotion engine also recognizes the customer's emotions and adjusts the response as necessary. For example, if the customer is feeling stressed, the system will respond in a calmer tone. The changes are then temporarily saved on the server.

[2362] server:

[2363] The temporarily saved change information will be provided to call center staff during the next business hours.

[2364] Data confirmation and double-checking during business hours

[2365] server:

[2366] During business hours, the temporarily stored data is provided to the terminals of call center staff.

[2367] Call Center Staff:

[2368] Our staff will review the data provided and double-check the accuracy and consistency of the information. If the data is correct, we will store it permanently. If the data is incorrect, we will contact the customer and ask them to correct it.

[2369] Final confirmation to the customer:

[2370] A staff member will call the customer to let them know the change has been made.

[2371] Specific examples

[2372] Example 1:

[2373] 1. When a customer signs up for a new contract, their voice is recorded and sent to the server.

[2374] 2. Call the call center outside of business hours and verify your identity using the automated voice guidance.

[2375] 3. The customer requests a change of address through a dialogue with the AI ​​system, and the emotion engine recognizes the customer's emotions and adjusts the response accordingly. The change information is temporarily stored on the server.

[2376] 4. On the next business day, call center staff will double-check the data and complete the verification.

[2377] 5. Call the customer to notify them that the address change is complete.

[2378] Example 2:

[2379] 1. The customer registers the voice call when signing the contract.

[2380] 2. I tried to change my plan outside of business hours, but my identity verification initially failed.

[2381] 3. After a second attempt, the customer is successfully identified and requests a plan change through a dialogue with the AI. The emotion engine recognizes the customer's stress and responds calmly.

[2382] 4. The next business day, call center staff will check the data, and if there are no problems, it will be stored permanently.

[2383] 5. Make a confirmation call and notify the customer that the plan change is complete.

[2384] In this way, the entire system works together to provide an environment where customers can use the service with peace of mind. The introduction of an emotion engine makes it possible to provide customer service with a more human touch.

[2385] The processing flow will be explained below.

[2386] Step 1:

[2387] User device: When signing up for a new contract, the customer launches the voice recording application and records their own voice. Once the recording is complete, the voice data is sent to the server.

[2388] Step 2:

[2389] Server: The server forwards the received voice data to the voice recognition system and registers it along with the customer ID, which is used in the subsequent identity verification process.

[2390] Step 3:

[2391] User: A customer calls the call center outside business hours and an automated voice prompt is triggered, prompting them to record a voice message to verify their identity.

[2392] Step 4:

[2393] User terminal: The customer follows the automated voice guidance, records the audio, and sends it to the server.

[2394] Step 5:

[2395] Server: The received customer voice is analyzed by the voice recognition system and compared with the registered voice data. If the comparison is successful, identity verification is complete.

[2396] Step 6:

[2397] Server: Sends the information after identity verification has been completed to the user's device and prepares to proceed to the next step.

[2398] Step 7:

[2399] User device: The AI ​​system starts up and interacts with the customer to request changes to information or plans. The AI ​​system analyzes the customer's speech and understands the request.

[2400] Step 8:

[2401] Emotion engine: Recognizes emotions from customer voices and conversations and provides that emotional information to the AI ​​system. For example, if the emotion engine determines that a customer is stressed, the AI ​​system will adjust its responses to use a calmer tone.

[2402] Step 9:

[2403] Server: Temporarily stores data on information changes and plan changes received from the AI ​​system.

[2404] Step 10:

[2405] User: Waits during business hours until the temporarily saved data is provided to the call center staff terminal.

[2406] Step 11:

[2407] Server: Once business hours begin, the temporarily stored data is provided to the call center staff terminal.

[2408] Step 12:

[2409] Call center staff: Review the data received and double-check the information for accuracy and consistency. If the data is correct, store it permanently. If the data is incorrect, contact the customer and request a correction.

[2410] Step 13:

[2411] Call center staff: Permanently save the confirmed data, call the customer to inform them of the change results, and notify the customer that the change has been completed.

[2412] Specific examples

[2413] Example 1:

[2414] 1. Step 1: When a customer signs up for a new service, their voice is recorded and sent to the server.

[2415] 2. Step 2: The server registers the received audio data.

[2416] 3. Step 3-6: The customer calls the call center outside business hours and uses automated voice guidance to verify their identity.

[2417] 4. Step 7: Following the automated voice guidance, the AI ​​system is activated and the customer requests a change of address.

[2418] 5. Step 8: The emotion engine recognizes customer emotions and provides adjustment information to the AI ​​system.

[2419] 6. Step 9: The changes are saved to the server.

[2420] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[2421] 8. Step 12: Call center staff will verify and double-check the data.

[2422] 9. Step 13: A staff member will call the customer to inform them that the address change has been completed.

[2423] Example 2:

[2424] 1. Step 1: The customer registers by voice upon signing up.

[2425] 2. Step 2: The server registers the audio data.

[2426] 3. Step 3-6: You try to change your plan outside of business hours, but the initial identity verification fails.

[2427] 4. Step 7: After successfully verifying your identity, you can request a plan change through a dialogue with the AI.

[2428] 5. Step 8: The emotion engine recognizes the customer's stress level and instructs the AI ​​system to adjust.

[2429] 6. Step 9: The changes are saved to the server.

[2430] 7. Steps 10-11: The server provides the data to the call center staff on the next business day.

[2431] 8. Step 12: Call center staff will verify and double-check the data.

[2432] 9. Step 13: A staff member will call the customer to inform them that the plan change has been completed.

[2433] This trend allows customers to receive fast and accurate service through automated systems, and the introduction of emotion engines makes responses more flexible and human, improving customer satisfaction.

[2434] Example 2

[2435] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2436] Call center operations require fast and accurate customer identity verification and information changes. However, conventional systems require a lot of manual work, which can lead to errors and delays. Furthermore, conventional automated systems lack emotion recognition and provide mechanical customer service, making it difficult to improve customer satisfaction. Furthermore, few systems are available outside of business hours, resulting in poor customer convenience.

[2437] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for recording the customer's voice; means for transmitting the recorded voice data to the server; means for registering the voice data received by the server in a voice recognition system; means for activating automated voice guidance at the call center; means for verifying the customer's identity based on the received voice data; means for the customer to request an information change or a plan change in an interactive format with AI after the identity verification is completed; means including an emotion engine for recognizing the customer's emotions and adjusting the response; means for temporarily storing the change information in the server; means for providing the temporarily stored data to call center staff; means for the call center staff to double-check the data; means for permanently storing the verified data and notifying the customer of the change results. This makes it possible to realize a system that quickly and accurately changes customer information and verifies the customer's identity, enhances customer service with a more human touch, and is available even outside business hours.

[2438] A "customer voice recording device" is any device or software that allows a customer to digitally record their own voice.

[2439] The "means for transmitting recorded audio data to a server" refers to the functions and protocols for transferring recorded audio files to a server via a network.

[2440] The "means for the server to register the received voice data in the voice recognition system" refers to the process in which the server passes the received voice file to the voice recognition algorithm for analysis and registration.

[2441] "Means for activating automated voice guidance at a call center" refers to a technology that enables a call center system to automatically start voice guidance and play a voice message to a customer.

[2442] "Means for verifying identity based on received voice data" refers to the process of analyzing the recorded voice file and comparing it with pre-registered voice data to verify the identity of the customer.

[2443] "A means for customers to interactively request changes to information or plans through AI after identity verification is complete" refers to a function that, once identity verification is successful, activates the AI ​​system and interactively accepts requests from customers to change information or plans.

[2444] The "means including an emotion engine that recognizes customer emotions and adjusts responses" refers to an algorithm or system that detects customer emotions through voice analysis and adjusts response content according to those emotions.

[2445] The "means for temporarily saving change information on the server" is a function for temporarily saving information change requests from customers on the server.

[2446] The "means for providing temporarily stored data to call center staff" is a function that allows call center staff to access temporarily stored changed data during business hours.

[2447] "Means for call center staff to double-check data" refers to the process by which staff review the data provided and verify its accuracy and consistency.

[2448] "Means for permanently storing confirmed data and informing customers of changes" refers to a database that stores double-checked data for a long period of time and a means of communication for notifying customers of changes.

[2449] This invention is a system that utilizes voice recognition technology and AI to automate identity verification and various information changes at call centers. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves a more flexible and human-like response. The purpose of this invention is to provide an environment where customers of all generations can use the service with peace of mind.

[2450] System Overview

[2451] The system consists of the following elements:

[2452] 1. Customer terminal (smartphone or dedicated device): Customers record their own voice and send it to the server. Specifically, this provides a means to capture voice using a recording application and send it to the server.

[2453] 2. Server: Responsible for receiving, registering, temporarily storing, and providing voice data. The server is integrated with the voice recognition system and AI system.

[2454] 3. Voice recognition system: Analyzes customer voice data and verifies their identity. It uses algorithms such as deep learning models.

[2455] 4. AI system: Handles information changes and plan changes through customer interactions.

[2456] 5. Emotion engine: Recognize...

Claims

1. a means for recording the customer's voice; means for transmitting the recorded voice data to a server; A means for registering the voice data received by the server into a voice recognition system; A means for activating automated voice guidance at the call center; means for verifying the identity of the person based on the received voice data; After completing identity verification, customers can request changes to their information or plans through a dialogue with AI. A means for temporarily storing the change information on a server; A means for providing the temporarily stored data to call center staff; A means for call center staff to double-check data, A means of permanently storing the confirmed data and informing the customer of the changes; A system including:

2. 2. The system according to claim 1, wherein the means for registering the customer's voice uses an application installed on a smartphone or a dedicated terminal.

3. 2. The system according to claim 1, further comprising a means for responding to customer inquiries even outside the call center's business hours, and allowing customers to request changes to information or plans through automated voice guidance.

Images