Bank customer insight and portrait updating method and system based on voice interaction analysis

Abstract

The invention discloses a bank customer insight and portrait updating method and system based on voice interaction analysis. The method comprises the following steps: collecting interactive audio of a bank staff and a customer through a portable audio collection device; transmitting to a remote processing end; processing the audio by using a natural language processing model, extracting customer financial demand characteristics based on a predefined financial product label library and a demand intention recognition model, extracting staff financial service performance characteristics based on a preset compliance rule library and a service quality evaluation model, and generating a bidirectional analysis result; generating output information based on the result; receiving feedback input of the staff and adjusting model parameters or strategies according to the feedback input; and automatically sending the customer insight data to the bank CRM system to update the portrait. According to the invention, by fusing hardware acquisition, AI analysis and system integration technologies, the technical problem that offline interaction data are difficult to carry out real-time structured analysis and form an automatic closed loop with an online business system is solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence natural language processing (NLP) and multimodal data acquisition technology, and in particular to an intelligent interactive processing system based on audio streaming processing and analysis, which is applied to customer relationship management scenarios in the banking and financial industry. Background Technology

[0002] Currently, bank employees (such as account managers and financial advisors) generally face the following technical pain points in their interactions with customers: (1) Invisible process and unstructured data: The interaction content exists in the form of unstructured audio, which is difficult to digitize and analyze in real time, forming a "data black box" that cannot be directly used by downstream business systems. (2) Lagging analysis and lack of real-time performance: Existing technical solutions (such as general voice recorders or cloud recording services) mostly adopt a batch processing mode after the fact, which cannot perform low-latency real-time analysis of streaming audio, resulting in the inability to conduct immediate business intervention. (3) Data silos and system fragmentation: Interaction data and analysis results are isolated from core business systems (such as CRM), lacking automated interfaces to transform insightful data into actions that drive business, requiring a large amount of manual intervention, which is inefficient. A search revealed that existing related patents (such as publication number CN112487426A, a customer service quality detection method based on speech recognition) mainly focus on post-event monitoring and evaluation of the one-way service quality of customer service personnel. Their technical solutions cannot achieve simultaneous two-way analysis of customers and employees, nor can they achieve deep automated integration with CRM systems, thus failing to solve the aforementioned technical problems. Summary of the Invention

[0003] I. Purpose of the Invention The present invention aims to overcome the defects of the prior art and solve the technical problem of how to perform real-time, two-way, and structured analysis of multi-channel interactive audio in banking scenarios, and break down data silos to achieve an automated closed loop with business systems. II. Technical Solution To achieve the above objectives, the present invention adopts the following technical solution: A method for bank customer insight and profile update based on voice interaction analysis, applied to the banking and financial industry, comprising the following steps: Collecting audio data from an interaction between a bank employee and a customer using a portable audio acquisition device worn on the employee's person; transmitting the audio data to a remote processing terminal; processing the audio data at the remote processing terminal using a natural language processing model, wherein customer financial demand characteristics are extracted from the audio data based on a predefined financial product tag library and a demand intent recognition model, and employee financial service performance characteristics are extracted from the same audio data based on a pre-set compliance rule library and a service quality assessment model, thereby generating a bidirectional analysis result containing both the customer financial demand characteristics and the employee financial service performance characteristics; generating output information based on the bidirectional analysis result; receiving feedback input from the bank employee based on the output information; adjusting subsequent analysis model parameters or prompting strategies according to the feedback input; automatically sending the customer insight data from the bidirectional analysis result to the bank's customer relationship management (CRM) system, causing the CRM system to update the corresponding user's feature profile. A system for implementing the above method includes: a portable audio acquisition device for acquiring audio of interactions between bank employees and customers; a processing terminal, communicatively connected to the portable audio acquisition device, for receiving and processing the interactive audio, and generating bidirectional analysis results based on a natural language processing model, targeting customer needs and employee performance characteristics; an output module, communicatively connected to the processing terminal, for receiving the bidirectional analysis results and generating output information; an interaction processing module, communicatively connected to the output module and the processing terminal, for receiving feedback input from the bank employee regarding the output information, and triggering the processing terminal to execute new analysis actions or adjust analysis strategies based on the feedback input; and a customer relationship management (CRM) interface module, communicatively connected to the processing terminal, for sending customer insight data from the bidirectional analysis results to the CRM system to update the corresponding user's feature profile. III. Beneficial Effects Compared with the closest prior art, the main differences and technical advancements of this invention are shown in the table below: The beneficial technical effects of this invention are: 1. Due to the adoption of real-time bidirectional analysis technology, the delay in structuring interactive data is reduced, improving data flow efficiency. 2. By adopting a technical solution that is deeply integrated with the business system, the automatic conversion from unstructured data to business actions has been achieved, reducing manual intervention and improving the reliability of data processing.3. The introduction of a feedback loop mechanism improves the accuracy of system output and its adaptability to complex business scenarios. Attached Figure Description Figure 1 is a schematic diagram of the system architecture according to an embodiment of the present invention. In Figure 1, 101 represents a portable audio acquisition device, 102 represents a processing terminal, 103 represents an employee terminal, and 104 represents a bank CRM system. Figure 2 is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation Example 1: The method flow is shown in Figure 2. The method of this invention includes the following steps: Initiating an interaction between a bank employee and a customer; Selecting the channel for this interaction; If it is an offline face-to-face scenario, collecting audio data through the smart badge worn by the employee; If it is an outbound call scenario, obtaining the audio stream by calling the API interface of the bank's outbound call system; If it is an APP communication scenario, obtaining the audio stream by accessing the API interface of instant messaging software (such as WeChat for Business); If it is a voicemail scenario, directly obtaining the voicemail audio file; Transmitting the obtained audio data to a remote processing end (such as a cloud server cluster); The remote processing end calls a natural language processing model to process the audio data. The model can adopt a pre-trained model based on the Transformer architecture (such as BERT, FinBERT), and perform supervised fine-tuning training on dialogue text in the banking field. The training objectives are text classification, named entity recognition, and sentiment analysis. The predefined financial product tag library is a knowledge graph automatically constructed and updated based on the bank's internal product database through natural language processing technologies such as entity recognition (NER) and relation extraction. The pre-built compliance rule base is a structured rule set generated by parsing massive amounts of unstructured documents issued by financial regulatory agencies through text mining and classification algorithms. It can be configured to monitor new rule releases and automatically trigger updates via technical means. The processing includes the following technical steps: a) Preprocessing audio data by noise reduction, segmentation, and Automatic Speech Recognition (ASR); b) Extracting customer financial demand characteristics from the text based on a predefined financial product tag library (e.g., containing product names such as "mortgage," "wealth management," and "credit card," and their synonyms) and a demand intent recognition model; c) Extracting employee financial service performance characteristics from the same text based on the pre-built compliance rule base (e.g., containing a list of prohibited terms explicitly stipulated by financial regulatory agencies) and a service quality assessment model (e.g., containing quantitative assessment algorithms for indicators such as employee speaking speed, response delay, and frequency of professional terminology use); d) Generating bidirectional analysis results and establishing a mapping relationship between customer needs and financial products / services; generating output information based on the analysis results; and determining whether employee feedback has been received. If feedback is received, the system accepts the feedback input and automatically judges its validity based on preset algorithm rules and historical effective data. Then, it adjusts the analysis model parameters or prompt strategies, and then returns to the analysis step to create a new compliance rule base. This is a set of structured rules generated by parsing massive amounts of unstructured documents issued by financial regulatory agencies through text mining and classification algorithms. It can also be configured to monitor the release of new regulations and automatically trigger updates through technical means.The processing steps include the following technical steps: a) Preprocessing the audio data by noise reduction, segmentation, and Automatic Speech Recognition (ASR); b) Extracting customer financial demand characteristics from the text based on a predefined financial product tag library (e.g., containing product names such as "mortgage," "wealth management," and "credit card," and their synonyms) and a demand intent recognition model; c) Extracting employee financial service performance characteristics from the same text based on a pre-defined compliance rule library (e.g., containing a list of prohibited terms explicitly stipulated by financial regulatory agencies) and a service quality assessment model (e.g., containing quantitative assessment algorithms for indicators such as employee speaking speed, response delay, and frequency of use of professional terms); d) Generating bidirectional analysis results and establishing a mapping relationship between customer needs and financial products / services; generating output information based on the analysis results; determining whether employee feedback has been received; if feedback has been received, accepting the feedback input and automatically judging its validity based on preset algorithm rules and historical effective data, then adjusting the analysis model parameters or prompting strategies, and then returning to the analysis step for new analysis; if no feedback has been received, automatically sending the structured insight data to the bank's CRM system through a predefined API interface; the CRM system triggers the customer profile update process; end. Example 2: System architecture as shown in Figure 1. This system includes: Portable audio acquisition device 101 (such as a smart badge, mobile phone, or software system called via API): used to collect or acquire interactive audio. Processing terminal 102 (such as a cloud server): built-in analysis and processing module and interaction processing module, used for core analysis and feedback processing. The analysis and processing module communicates with the interaction processing module via a data interface protocol. Employee terminal 103 (such as a mobile phone or AR glasses): connected to the output module, used to receive prompts and reports. If no feedback is received, the structured insight data is automatically sent to the bank's CRM system through a predefined API interface; the CRM system triggers the customer profile update process; end. Example 2: System architecture as shown in Figure 1. This system includes: Portable audio acquisition device 101 (such as a smart badge, mobile phone, or software system called via API): used to collect or acquire interactive audio. Processing terminal 102 (such as a cloud server): built-in analysis and processing module and interaction processing module, used for core analysis and feedback processing. The analysis and processing module communicates with the interaction processing module via a data interface protocol. Bank CRM System 104: Connects to the CRM interface module for updating customer profiles. Modules communicate via a specific interface protocol, transmitting audio, commands, and structured data. Example 3: Compliance Statement In all scenarios, especially when obtaining data from third-party platforms via API, explicit and separate authorization from the customer must be obtained before the interaction begins, and the customer must be fully informed. The implementation of this invention is premised on strict compliance with the Personal Information Protection Law and other relevant laws and regulations, as well as platform rules.In this embodiment of the invention, the synchronous bidirectional analysis mechanism and the feedback-based iterative optimization mechanism work together to solve the technical problems of 'insufficient data value mining' and 'insufficient model generalization ability' in offline banking interaction scenarios. Specifically, the feedback data originates from the results of bidirectional analysis, and the optimized model directly improves the accuracy of the next bidirectional analysis. The two form a continuously self-reinforcing technical closed loop, which cannot be achieved by single-dimensional analysis or independent optimization steps. The "synchronous execution of bidirectional analysis" is not a simple parallel process of two independent analysis processes, but rather shares the same audio preprocessing pipeline and computing resource scheduling unit, and needs to solve the problem of temporal alignment and fusion of different model output results, which brings new challenges to technical implementation. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for bank customer insight and profile updating based on voice interaction analysis, applied to the banking and financial industry, characterized in that: include: (1) Obtain audio data from bank interaction scenarios; (2) Perform bidirectional analysis processing on the same audio data simultaneously to generate analysis results that include both customer financial needs and employee service performance; (3) Generate interactive strategy information based on the analysis results, and iteratively optimize the analysis process based on the feedback obtained from executing the strategy; (4) Synchronize the optimized analysis results to the banking business system.

2. The method according to claim 1, characterized in that, The portable audio acquisition device is a smart badge, smartphone, or smart earphone.

3. The method according to claim 1, characterized in that, The remote processing terminal is deployed on a cloud server cluster or a local server.

4. The method according to claim 1, characterized in that, The output information includes real-time prompts provided to employees during the interaction; and / or analysis reports generated after the interaction ends.

5. The method according to claim 1, characterized in that, The step of adjusting subsequent analysis model parameters or prompting strategies based on feedback input includes: automatically determining the validity of the feedback input based on preset algorithm rules and historical effective data; if the feedback is valid, adjusting model parameters using gradient descent algorithm or updating the prompting strategy based on rule base according to its content; if the feedback is invalid, recording it as a negative sample for local optimization of the prompting strategy only.

6. The method according to claim 1, characterized in that, In the automatic sending step, the customer insight data is sent in a structured data format and through a predefined API interface protocol.

7. The method according to claim 1, characterized in that, The synchronization to the banking business system refers to: transforming the analysis results into a predefined structured data model, and automatically triggering the customer profile update process by calling the open API interface of the bank's CRM system.

8. A system for implementing the method as described in any one of claims 1-6, characterized in that, include: A portable audio acquisition device for capturing audio of interactions between bank employees and customers; A processing unit, communicatively connected to the portable audio acquisition device, is used to receive and process the interactive audio, and generate bidirectional analysis results based on a natural language processing model, targeting customer needs and employee performance characteristics; an output module, communicatively connected to the processing unit, is used to receive the bidirectional analysis results and generate output information. An interactive processing module, communicatively connected to the output module and the processing terminal, is used to receive feedback input from the bank staff on the output information, and trigger the processing terminal to execute new analysis actions or adjust analysis strategies based on the feedback input; A Customer Relationship Management (CRM) interface module is connected to the processing terminal and is used to send customer insight data from the bidirectional analysis results to the CRM system to update the feature profile of the corresponding user.

9. The system according to claim 7, characterized in that, The processing terminal communicates with the interactive processing module via a first data interface protocol to transmit feedback instructions and optimization parameters; the processing terminal communicates with the CRM interface module via a second data interface protocol to transmit encapsulated structured data.

10. The system according to claim 7, characterized in that, The system also includes an access control module, which is communicatively connected to the processing terminal and the output module, and is used to control the display content of the bidirectional analysis results according to the employee's access level.

Images