Banking agent system and method

Abstract

The application discloses a bank intelligent agent system based on artificial intelligence and a method thereof, and belongs to the technical field of financial technology and artificial intelligence. The system comprises a voice recognition module, a voiceprint verification module, a natural language understanding module, an intention classification module, an entity extraction module, a bank business processing module, a risk control module, a voice synthesis module, a dialogue management module and a core controller. The voice recognition module adopts Azure Speech Service, and the recognition accuracy is greater than 95%; the voiceprint verification module adopts Azure Speaker Recognition, and the verification accuracy is greater than 98%; the natural language understanding module supports 30+ business intentions, and the accuracy is greater than 95%; the bank business processing module covers comprehensive business such as account management, transfer and remittance, investment and financial management, loan service and the like, and the transaction success rate is greater than 99.9%; the risk control module adopts a combination of rule evaluation and machine learning, and the fraud detection accuracy is greater than 99%. The system also supports functions such as intelligent recommendation, customer portrait analysis, anti-fraud detection, multi-factor authentication and REST API interface. The application can provide 24 / 7 uninterrupted intelligent bank services for customers through voice interaction, the end-to-end response time is less than 5s, and meanwhile, the safety and compliance are ensured.

Description

[0001] ## Technical Field This invention belongs to the fields of financial technology and artificial intelligence, specifically relating to an AI-based intelligent banking system and method. More specifically, this invention relates to speech recognition technology, voiceprint verification technology, natural language understanding technology, speech synthesis technology, banking business processing technology, risk control technology, intelligent recommendation technology, customer profiling analysis technology, and anti-fraud detection technology.

[0002] This invention is mainly applied to scenarios such as personal banking services, corporate banking services, intelligent customer service, and financial consulting. It can provide customers with 24 / 7 uninterrupted intelligent banking services through voice interaction, including account inquiry, fund transfer, investment and wealth management, loan application, credit card operation, and other services, while ensuring security and compliance.

[0003] ## Background Technology With the development of fintech and the increasing demand for intelligent services from customers, traditional banking service models are facing enormous challenges. Customers expect more convenient, personalized, and efficient financial services. At the same time, the banking industry has extremely high requirements for security and compliance, and needs to improve the service experience without compromising security standards.

[0004] ### Current Status of Technological Development Banking services have evolved from traditional branch services to online banking, mobile banking, and then to smart banking. Traditional branch services require customers to visit a bank branch in person, which is cumbersome and inefficient. While online and mobile banking have improved convenience, they still require manual operation, have limited interaction methods, and are not user-friendly for special groups such as the elderly.

[0005] In recent years, with the development of artificial intelligence technology, smart banks have begun to emerge. Telephone banking provides services through voice interaction, but its functions are limited and it can only handle simple inquiry business. Intelligent customer service systems can answer common questions, but their ability to handle complex business is weak. The existing smart banking systems mainly have the following problems: (1) The accuracy of voice recognition needs to be improved, especially in the recognition of financial professional terms; (2) There is a lack of effective voiceprint verification mechanism, resulting in insufficient security; (3) The ability to understand natural language is limited, making it difficult to understand complex business needs; (4) The business processing capacity is limited, making it impossible to realize comprehensive banking business; (5) The risk control capacity is insufficient, making it difficult to effectively prevent fraudulent behavior; (6) The personalized service capability is weak, making it difficult to provide customized services according to user needs.

[0006] ### Technical Challenges Banking intelligent agents face the following technical challenges: (1) Voice recognition accuracy: It needs to accurately recognize financial terms and numerical amounts, and maintain a high accuracy rate in noisy environments; (2) Voiceprint verification reliability: It needs to accurately identify the user's identity and prevent unauthorized access and fraudulent behavior; (3) Natural Language Understanding Accuracy: It needs to accurately understand users' business needs and implicit intentions, and support diverse expression methods; (4) Accuracy of business processing: It is necessary to accurately process various banking businesses, including account management, fund transfers, investment and wealth management, loan services, etc. (5) Effectiveness of risk control: It is necessary to assess transaction risks in real time, identify fraudulent activities, and prevent financial risks; (6) System response speed: It needs to respond to user requests quickly and provide a smooth interactive experience; (7) Multi-user support: It needs to support multiple users, each with independent voiceprints and permissions; (8) Personalized services: Personalized recommendations and services need to be provided based on user profiles and transaction history; (9) Compliance requirements: Must meet regulatory requirements such as KYC (Know Your Customer) and AML (Anti-Money Laundering); (10) Data security: It is necessary to protect users’ sensitive information and prevent data leakage and abuse.

[0007] ### Shortcomings of existing technology The existing technology has the following shortcomings: (1) The accuracy of speech recognition needs to be improved, especially in the recognition of financial professional terms and numbers; (2) There is a lack of effective voiceprint verification mechanism, which is not secure enough and is easily subject to impersonation attacks; (3) The natural language understanding ability is limited, making it difficult to understand complex business needs and contextual dependencies; (4) The business processing ability is limited, making it impossible to realize comprehensive banking business, especially complex investment and financial management and loan services; (5) The risk control ability is insufficient, making it difficult to effectively identify fraudulent behavior and prevent financial risks; (6) The personalized service capability is weak, making it difficult to provide customized services according to user needs; (7) The system response speed is slow and the user experience is poor; (8) There is a lack of complete monitoring and analysis functions, making it difficult to optimize performance and troubleshoot problems; (9) The compliance requirements are not met, making it difficult to meet regulatory requirements such as KYC and AML; (10) The data security is insufficient, and there is a risk of data leakage and abuse.

[0008] ## Invention Content ### Technical problems to be solved The main technical problems to be solved by this invention include: (1) how to improve the accuracy and noise resistance of speech recognition, especially the accuracy of recognition of financial professional terms and numbers; (2) how to achieve reliable voiceprint verification, ensure system security, and prevent impersonation attacks; (3) how to accurately understand user intent and support diverse expression methods and contextual understanding; (4) how to achieve precise banking business processing, including account management, fund transfer, investment and wealth management, loan services, etc.; (5) how to achieve effective risk control, identify fraudulent behavior, and prevent financial risks; (6) how to improve system response speed and provide a smooth interactive experience; (7) how to support multi-user and personalized services and improve user satisfaction; (8) how to meet compliance requirements, including KYC, AML and other regulatory requirements; and (9) how to ensure data security and prevent data leakage and abuse.

[0009] ### Technical Solution To address the aforementioned technical problems, this invention provides an artificial intelligence-based bank intelligent agent system, comprising a speech recognition module, a voiceprint verification module, a natural language understanding module, an intent classification module, an entity extraction module, a banking business processing module, a risk control module, a speech synthesis module, a dialogue management module, and a core controller.

[0010] The speech recognition module utilizes Azure Speech Service, supporting real-time and document recognition with an accuracy rate exceeding 95%. It supports mixed Chinese and English speech recognition and is capable of recognizing financial terminology. Recognition performance is optimized through audio preprocessing (noise reduction, VAD, and volume normalization) and model caching mechanisms, achieving a speech recognition time of <1.5 seconds.

[0011] The voiceprint verification module uses Azure Speaker Recognition to extract and compare voiceprint features, including a voiceprint registration unit, a voiceprint verification unit, and a voiceprint management unit. The voiceprint registration unit extracts voiceprint features such as MFCC, i-vector, and x-vector from multiple speech samples and encodes them into fixed-length voiceprint vectors. The voiceprint verification unit calculates the cosine similarity between the test voiceprint and the registered voiceprint, and determines whether they belong to the same user based on the similarity threshold. The verification accuracy is >98%, and the verification time is <0.5s.

[0012] The Natural Language Understanding module combines Azure NLU and a custom rules engine, including an intent classifier and an entity extractor. The intent classifier identifies user business intents with a confidence level >95% and supports 30+ types of business intents. The entity extractor uses regular expressions and named entity recognition technology to extract key business parameters, including amount, account number, date, time, phone number, etc., with an extraction accuracy >95%.

[0013] The intent classification module supports over 30 business intents, including account inquiry (balance inquiry, transaction details, account information), transfer and remittance (intra-bank transfer, inter-bank transfer, real-time arrival), investment and wealth management (fund, wealth management product, stock inquiry and purchase), loan service (loan application, repayment inquiry, credit limit inquiry), credit card (bill inquiry, repayment, points inquiry), and consultation and Q&A (interest rate inquiry, fee explanation, business consultation). It improves classification accuracy through pattern matching, similarity calculation, and contextual understanding.

[0014] The banking business processing module includes an account management unit, a transaction processing unit, a wealth management unit, and a loan management unit. The account management unit is responsible for account information inquiry, balance inquiry, and transaction detail inquiry; the transaction processing unit is responsible for intra-bank transfers, inter-bank transfers, and real-time fund transfers, with a transaction success rate >99.9%; the wealth management unit is responsible for inquiries and purchases of investment products such as funds, wealth management products, and stocks; and the loan management unit is responsible for loan services such as loan applications, repayment inquiries, and credit limit inquiries.

[0015] The risk control module employs a combination of rule-based evaluation and machine learning for risk assessment, comprising a feature extraction unit, a rule-based evaluation unit, a machine learning evaluation unit, and a comprehensive scoring unit. The feature extraction unit extracts features from transactions such as transaction amount, transaction time, transaction frequency, and account balance; the rule-based evaluation unit evaluates transactions based on preset rules; the machine learning evaluation unit uses a machine learning model to evaluate transactions; and the comprehensive scoring unit combines the results of the rule-based and machine learning evaluations to calculate the final risk score and risk level, which is categorized into high, medium, and low risk levels. The risk assessment time is less than 0.5 seconds.

[0016] The speech synthesis module uses Azure TTS to convert text to speech, supports a variety of timbres and intonations, including soft female voices, steady male voices, and lively female voices, supports SSML tagging, and allows control over speech speed, pitch, and emotional expression. The sound quality is excellent and close to that of a real person.

[0017] The dialogue management module includes a dialogue state management unit, a dialogue context management unit, and a multi-turn dialogue processing unit. The dialogue state management unit maintains the dialogue state, including idle, listening, authenticating, processing, executing, and confirming states; the dialogue context management unit maintains the dialogue history and slot information, supporting context understanding and intelligent completion; the multi-turn dialogue processing unit processes the user's continuous dialogue input based on the dialogue state and context information, supporting natural and smooth multi-turn interactions.

[0018] The system also includes an intelligent recommendation module, which recommends suitable financial products to users based on user profiles and transaction history. This module includes a collaborative filtering recommendation unit, a content recommendation unit, and a hybrid recommendation unit. The collaborative filtering recommendation unit makes recommendations based on user behavior similarity; the content recommendation unit makes recommendations based on product features and user preferences; and the hybrid recommendation unit combines the results of collaborative filtering and content recommendation to provide more accurate recommendations.

[0019] The system also includes a customer profile analysis module, which is used to build user profiles, including basic information extraction unit, asset status analysis unit, transaction behavior analysis unit, investment preference analysis unit, and credit status analysis unit. It analyzes user characteristics from multiple dimensions and provides a foundation for personalized services.

[0020] The system also includes an anti-fraud detection module for detecting fraudulent behavior, which includes a behavior analysis unit, a pattern recognition unit, and an anomaly detection unit. The fraud detection accuracy rate is >99%, effectively preventing financial fraud.

[0021] The system also includes a multi-factor authentication module for user authentication, which includes a password authentication unit, a voiceprint authentication unit, a face authentication unit, and an SMS verification code unit. It supports a combination of multiple authentication methods and provides flexible authentication strategies.

[0022] The system also includes a REST API interface module, providing interfaces such as account management API, transaction processing API, query service API, risk management API, customer service API, data analysis API, system management API, and monitoring and statistics API. It supports remote calls and integration with third-party systems. All API requests must carry authentication information, including JWT token and API Key.

[0023] This invention also provides an artificial intelligence-based bank intelligent agent method, comprising the following steps: collecting user voice input and converting the voice signal into text data through a voice recognition model; extracting voiceprint features from the voice signal to verify the user's identity, and continuing execution after successful verification; performing natural language understanding on the text data to identify the user's business intent and extract key business parameters; determining the banking business to be executed based on the identified business intent and parameters; conducting risk assessment on the business operation, including transaction risk, credit risk, fraud risk, and compliance risk assessment; deciding whether to execute the business operation based on the risk assessment results, such as refusing execution and prompting the user if the risk score exceeds a threshold, or executing the corresponding banking business operation if the risk score is within an acceptable range; obtaining the business operation execution result and generating feedback information; converting the feedback information into a voice signal through speech synthesis and broadcasting it to the user; recording operation logs and performance data, and updating user profiles and transaction history; and determining whether to continue the dialogue based on the dialogue status and context information, such as continuing voice input if necessary, otherwise ending the dialogue.

[0024] ### Beneficial Effects Compared with the prior art, the present invention has the following beneficial effects: (1) High speech recognition accuracy: It adopts the advanced Azure Speech Service, with a recognition accuracy of >95%, supports the recognition of financial professional terms and numbers, and has strong anti-noise capability; (2) Reliable voiceprint verification: Employs Azure Speaker Recognition, with a verification accuracy rate of >98%, effectively preventing impersonation attacks and fraudulent activities; (3) Accurate natural language understanding: Combining Azure NLU and a custom rule engine, it supports 30+ business intents with an accuracy rate of >95%, and supports diverse expressions and contextual understanding; (4) Comprehensive banking services: covering account management, fund transfers, investment and wealth management, loan services, credit card operations, and other comprehensive banking services, with a transaction success rate of >99.9%; (5) Effective risk control: Risk assessment is conducted by combining rule evaluation and machine learning, with a fraud detection accuracy rate of >99%, effectively preventing financial risks; (6) Fast system response: speech recognition time <1.5s, voiceprint verification time <0.5s, intent classification time <0.1s, end-to-end response time <5s, and smooth user experience; (7) Strong personalized service: Provides personalized recommendations and services based on user profiles and transaction history to improve user satisfaction; (8) Multi-factor authentication security: Supports a combination of multiple authentication methods such as password, voiceprint, face, and SMS verification code, providing high security; (9) Compliance: Meets regulatory requirements such as KYC (Know Your Customer) and AML (Anti-Money Laundering); (10) Data security: HTTPS / TLS encrypted transmission, AES-256 encrypted storage, data desensitization and other technologies are used to ensure data security.

Claims

1. A bank intelligent agent system based on artificial intelligence, characterized in that... ,include: The speech recognition module is used to collect user voice input in real time and convert the speech signal into text data; The voiceprint verification module is used to extract voiceprint features from speech signals to verify user identity. The Natural Language Understanding module is used to perform intent recognition and entity extraction on text data to understand user business needs; The intent classification module is used to identify user operation intents, supporting more than 30 types of business intents such as account inquiry, fund transfer, investment and wealth management, loan services, and credit card operations; The entity extraction module is used to extract key business parameters from user input, including amount, account number, date, time, phone number, etc. The banking transaction processing module is used to execute corresponding banking transactions based on the identified business intent and parameters. The risk control module is used to conduct real-time risk assessments of transactions, including assessments of transaction risk, credit risk, fraud risk, and compliance risk. The speech synthesis module is used to convert system feedback information into speech signals and broadcast the execution results to the user. The dialogue management module is used to manage multi-turn dialogue states and maintain dialogue context and history. The core controller coordinates the collaborative work of various modules and manages user sessions and task execution.

2. Characterized by The speech recognition module uses Azure Speech Service to implement speech recognition, supports real-time recognition and document recognition, has a recognition accuracy of >95%, supports mixed Chinese and English recognition, has the ability to recognize financial professional terms, and optimizes recognition performance through audio preprocessing (noise reduction, VAD, volume normalization) and model caching mechanisms.

3. Characterized by The voiceprint verification module uses Azure Speaker Recognition to extract and compare voiceprint features. It includes a voiceprint registration unit, a voiceprint verification unit, and a voiceprint management unit. The voiceprint registration unit extracts voiceprint features such as MFCC, i-vector, and x-vector from multiple speech samples and encodes them into fixed-length voiceprint vectors. The voiceprint verification unit calculates the cosine similarity between the test voiceprint and the registered voiceprint and determines whether they are the same user based on the similarity threshold. The verification accuracy is >98%.

4. Characterized by The natural language understanding module combines Azure NLU and a custom rule engine, including an intent classifier and an entity extractor. The intent classifier identifies the user's business intent with a confidence level of >95%. The entity extractor uses regular expressions and named entity recognition technology to extract key business parameters with an extraction accuracy of >95%.

5. Characterized by The intent classification module supports over 30 business intents, including account inquiry (balance inquiry, transaction details, account information), transfer and remittance (intra-bank transfer, inter-bank transfer, real-time arrival), investment and wealth management (fund, wealth management product, stock inquiry and purchase), loan service (loan application, repayment inquiry, credit limit inquiry), credit card (bill inquiry, repayment, points inquiry), and consultation and Q&A (interest rate inquiry, fee explanation, business consultation). It improves classification accuracy through pattern matching, similarity calculation, and contextual understanding.

6. Characterized by The banking business processing module includes an account management unit, a transaction processing unit, a wealth management unit, and a loan management unit. The account management unit is responsible for account information inquiry, balance inquiry, and transaction detail inquiry; the transaction processing unit is responsible for intra-bank transfers, inter-bank transfers, real-time fund transfers, and other transaction operations, with a transaction success rate of >99.9%; the wealth management unit is responsible for inquiries and purchases of investment products such as funds, wealth management products, and stocks; and the loan management unit is responsible for loan applications, repayment inquiries, and credit limit inquiries, among other loan services.

7. Characterized by The risk control module employs a combination of rule-based evaluation and machine learning for risk assessment. It includes a feature extraction unit, a rule evaluation unit, a machine learning evaluation unit, and a comprehensive scoring unit. The feature extraction unit extracts features such as transaction amount, transaction time, transaction frequency, and account balance from transactions. The rule evaluation unit evaluates transactions based on preset rules. The machine learning evaluation unit evaluates transactions using machine learning models. The comprehensive scoring unit combines the results of rule evaluation and machine learning evaluation to calculate the final risk score and risk level, which are divided into three levels: high risk, medium risk, and low risk.

8. Characterized by The speech synthesis module uses Azure TTS to convert text to speech, supports a variety of timbres and intonations, including soft female voices, steady male voices, and lively female voices, supports SSML tagging, and can control speech speed, pitch, and emotional expression. The sound quality is excellent and close to that of a real person.

9. Characterized by The dialogue management module includes a dialogue state management unit, a dialogue context management unit, and a multi-turn dialogue processing unit. The dialogue state management unit maintains the dialogue state, including states such as idle, listening, authenticating, processing, executing, and confirming. The dialogue context management unit maintains the dialogue history and slot information, and supports context understanding and intelligent completion. The multi-turn dialogue processing unit processes the user's continuous dialogue input according to the dialogue state and context information, and supports natural and smooth multi-turn interaction.

10. Characterized by It also includes an intelligent recommendation module, which recommends suitable financial products to users based on user profiles and transaction history. This module includes a collaborative filtering recommendation unit, a content recommendation unit, and a hybrid recommendation unit. The collaborative filtering recommendation unit makes recommendations based on user behavior similarity; the content recommendation unit makes recommendations based on product features and user preferences; and the hybrid recommendation unit combines the results of collaborative filtering and content recommendation to provide more accurate recommendations.

11. Characterized by It also includes a customer profile analysis module for building user profiles, including a basic information extraction unit, an asset status analysis unit, a transaction behavior analysis unit, an investment preference analysis unit, and a credit status analysis unit. The basic information extraction unit extracts basic information such as age, gender, and occupation from user registration information; the asset status analysis unit analyzes user asset size and income level from account data; the transaction behavior analysis unit analyzes user transaction frequency and amount from transaction records; the investment preference analysis unit analyzes user risk preference and product selection from investment records; and the credit status analysis unit analyzes user credit score and repayment record from credit data.

12. Characterized by It also includes an anti-fraud detection module for detecting fraudulent behavior, which includes a behavior analysis unit, a pattern recognition unit, and an anomaly detection unit. The behavior analysis unit analyzes user behavior patterns and identifies abnormal behavior; the pattern recognition unit identifies known fraud patterns; and the anomaly detection unit uses anomaly detection algorithms to identify unknown fraudulent behavior, with a fraud detection accuracy of >99%.

13. Characterized by It also includes a multi-factor authentication module for user authentication, which includes a password authentication unit, a voiceprint authentication unit, a face authentication unit, and an SMS verification code unit. The password authentication unit verifies the user's password; the voiceprint authentication unit verifies the user's voiceprint; the face authentication unit verifies the user's face; and the SMS verification code unit verifies the SMS verification code. It supports combinations of multiple authentication methods and provides flexible authentication strategies.

14. Characterized by It also includes a REST API interface module, providing interfaces such as account management API, transaction processing API, query service API, risk management API, customer service API, data analysis API, system management API, and monitoring and statistics API. It supports remote calls and integration with third-party systems. All API requests need to carry authentication information, including JWT token and API Key.

15. A method for a bank intelligent agent based on artificial intelligence, characterized in that... This includes the following steps: Step S1: Collect user voice input and convert the voice signal into text data using a speech recognition model; Step S2: Extract voiceprint features from the speech signal, verify the user's identity, and continue execution after successful verification; Step S3: Perform natural language understanding on the text data to identify the user's business intent and extract key business parameters; Step S4: Based on the identified business intent and parameters, determine the banking transactions to be performed; Step S5: Conduct a risk assessment of business operations, including assessments of transaction risk, credit risk, fraud risk, and compliance risk; Step S6: Based on the risk assessment results, decide whether to execute the business operation. If the risk score exceeds the threshold, refuse to execute and notify the user. Step S7: If the risk score is within an acceptable range, perform the corresponding banking operations; Step S8: Obtain the execution result of the business operation and generate feedback information; Step S9: Convert the feedback information into a speech signal using speech synthesis and broadcast it to the user; Step S10: Record operation logs and performance data, and update user profiles and transaction history; Step S11: Based on the dialogue status and context information, determine whether the dialogue needs to continue. If so, return to step S1; otherwise, end the dialogue.

16. The method is characterized in that the voiceprint verification in step S2 specifically includes: Extracting voiceprint features such as MFCC, i-vector, and x-vector from speech signals; Voiceprint features are encoded into fixed-length voiceprint vectors; Calculate the cosine similarity between the test voiceprint vector and the registered voiceprint vector; The verification passes if the similarity exceeds a preset threshold; otherwise, the verification fails. When verification fails, the user is prompted to re-authenticate or use another authentication method.

17. The method is characterized by The risk assessment in step S5 specifically includes: Extract features from transactions such as transaction amount, transaction time, transaction frequency, and account balance; Transactions are evaluated based on preset rules, including checks for amount thresholds, time anomalies, and frequency anomalies. Transactions are evaluated using machine learning models trained on historical transaction data and fraud samples. The final risk score is calculated by combining the results of rule-based evaluation and machine learning evaluation. Risk levels are determined based on risk scores: high risk (>0.8), medium risk (0.5-0.8), and low risk (<0.5).