Intelligent routing agent switching method and device, electronic equipment and storage medium

Abstract

The invention discloses an intelligent routing agent switching method and device, electronic equipment and a storage medium. The method comprises the following steps: processing a voice interaction signal between a customer and an artificial seat, which is acquired in real time, through a framing strategy to obtain a real-time audio stream; performing multi-dimensional feature extraction on the real-time audio stream to obtain voiceprint features, voice features and text features of the real-time audio stream; fusing the voiceprint feature, the voice feature and the text feature of the real-time audio stream to obtain a multi-modal fusion feature; through a pre-trained intention classification model, determining the intention type of the customer according to the multi-modal fusion features; and according to a preset service intention map, transferring the customer incoming call to a target manual seat corresponding to the intention type for continuous processing. According to the invention, intention prediction is carried out based on rich information included in the multi-modal features, so that the intention recognition accuracy is greatly improved; and automatic seat transfer is carried out according to the identified intention, so that the seat transfer efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and in particular to a smart routing agent transfer method, device, electronic device, and storage medium. Background Technology

[0002] With the accelerated digital transformation of financial services, bank telephone agents serve as an important channel for customers to conduct business, inquire about issues, and provide feedback. Their service efficiency and quality directly affect customer satisfaction and the market competitiveness of financial institutions.

[0003] Currently, bank telephone agent transfers mainly fall into the traditional manual transfer mode, which relies on predefined business rules and keyword lists, with agents selecting the transfer recipient based on the customer's stated business scenario. However, this method has certain shortcomings: traditional manual transfers suffer from rigid responses due to rule-driven mechanisms, resulting in low agent transfer efficiency and a poor customer experience. Therefore, a new agent transfer method is urgently needed. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, storage medium, and computer program product for intelligent routing agent switching.

[0005] According to one aspect of the present invention, a smart routing agent transfer method is provided, comprising: By using a frame-segmentation strategy, the voice interaction signals between the customer and the human agent acquired in real time are processed to obtain a real-time audio stream; Multi-dimensional feature extraction is performed on the real-time audio stream to obtain the speaker's features, speech features, and text features. The speaker features, speech features, and text features of real-time audio streams are fused to obtain multimodal fusion features; By using a pre-trained intent classification model, the type of customer intent is determined based on multimodal fusion features; Based on the preset service intent map, customer calls are transferred to the target human agent corresponding to the intent type for further processing.

[0006] According to another aspect of the present invention, a smart routing agent switching device is provided, comprising: The frame processing module is used to process the real-time voice interaction signals between the customer and the human agent through a frame segmentation strategy to obtain a real-time audio stream. The feature extraction module is used to extract multi-dimensional features from the real-time audio stream to obtain the speaker features, speech features and text features of the real-time audio stream. The feature fusion module is used to fuse the speaker features, speech features and text features of the real-time audio stream to obtain multimodal fusion features; The intent recognition module is used to determine the type of customer intent based on multimodal fusion features using a pre-trained intent classification model. The transfer module is used to transfer customer calls to the target human agent corresponding to the intent type for further processing, based on a preset service intent map.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the intelligent routing agent transfer method of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the intelligent routing agent transfer method of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps in the above-described method.

[0010] The technical solution of this invention predicts intent based on the rich information included in multimodal features, which greatly improves the accuracy of intent recognition; and then automatically transfers agents according to the recognized intent, thereby improving the efficiency of agent transfer.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating the intelligent routing agent transfer method provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating another intelligent routing agent transfer method provided in an embodiment of the present invention; Figure 3This is a schematic diagram of the structure of the intelligent routing agent transfer device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the intelligent routing agent transfer method of this invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] Example 1 Figure 1 This is a flowchart of an intelligent routing agent transfer method provided by an embodiment of the present invention. This embodiment can be applied to the scenario of transferring bank telephone agents. The method can be executed by an intelligent routing agent transfer device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0016] like Figure 1 As shown, the intelligent routing agent transfer method includes: S101. By using a frame-segmentation strategy, the voice interaction signals between the customer and the human agent acquired in real time are processed to obtain a real-time audio stream.

[0017] Among them, the voice interaction signal is the raw audio signal generated during the call between the customer and the human agent, which includes voice content, emotion-related acoustic information and background noise; the framing strategy refers to the processing rule of splitting the continuous voice interaction signal into discrete audio frames according to a preset time window, the core of which is to balance the continuity of the signal and the real-time processing; the real-time audio stream refers to the set of discrete audio frames that are continuously transmitted in time order after framing processing, which can support subsequent real-time feature extraction and analysis.

[0018] In some embodiments, the real-time voice interaction signal between the customer and the agent can be discretized using a preset time window splitting strategy to form a continuous audio data sequence that can be transmitted and analyzed in real time. Specifically, a sliding window-based framing strategy can be used, for example, a sliding window strategy with a 20ms frame length and a 10ms frame shift. The first framing starts from the moment the voice interaction signal between the customer and the agent is captured. The first framing involves extracting an audio segment within the 0ms-20ms time period as the first frame. After a 10ms interval (i.e., the frame shift duration) between the window sliding and subsequent framing, the window moves backward by 10ms, extracting an audio segment within the 10ms-30ms time period as the second frame; after another 10ms interval, a segment within the 20ms-40ms time period is extracted as the third frame, and so on, to obtain the real-time audio stream. In addition, a dynamic frame length segmentation strategy can be adopted, which mainly adjusts the frame length adaptively based on the energy of the voice signal. When the voice energy is high (such as during normal calls), a frame length of 20ms is used, and when the energy is low (such as during silent periods), the frame length is switched to 100ms to reduce the amount of invalid data processing.

[0019] It should be noted that the real-time voice interaction signals collected in the embodiments of this invention are information and data authorized by the customer or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, and necessary confidentiality measures have been taken. This does not violate public order and good morals, and a corresponding operation entry point is provided for users to choose to authorize or refuse. The key feature of this invention is that it utilizes the collected real-time voice interaction signals and, based on artificial intelligence technology, automates decision-making to intelligently transfer incoming calls to customers. To protect the personal rights of customers, this invention also provides a corresponding operation entry point for customers to choose to agree to or refuse the automated decision result; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0020] S102. Perform multi-dimensional feature extraction on the real-time audio stream to obtain the speaker features, speech features and text features of the real-time audio stream.

[0021] Multi-dimensional feature extraction refers to extracting features representing different dimensions of information from the audio stream, covering three core dimensions: identity recognition (voiceprint), acoustic attributes (speech), and semantic content (text). Voiceprint features are feature vectors representing the unique acoustic attributes of a user's voice, possessing unique identity characteristics, and are used for user identity association and emotion-assisted judgment. Speech features refer to multi-dimensional data describing the acoustic attributes of speech, including prosodic features, fundamental frequency features, energy features, spectral features, emotional features, and customer behavior patterns during the call (e.g., silence duration / number of interruptions), fundamentally reflecting the physical characteristics and emotional state of the speech. Text features are semantic representation vectors extracted from speech-to-text, carrying the core semantic information of the customer's needs and serving as the core basis for intent recognition.

[0022] In some embodiments, multi-dimensional feature extraction is performed on the real-time audio stream to obtain the speaker's features, speech features, and text features, including: The quantized voiceprint feature recognition model extracts voiceprint features from the real-time audio stream. The extracted voiceprint features can be encrypted and cached so that the customer's voiceprint features do not need to be extracted again when the customer calls a human agent again.

[0023] Speech features are extracted from real-time audio streams using a speech feature recognition model. According to the preset duration, the real-time audio stream is segmented and processed. For each audio segment obtained, the audio segment is encoded and compressed. The encoded and compressed audio segment is then streamed to the speech recognition engine in the cloud for speech-to-text processing. It is understandable that segmenting the audio stream and encoding and compressing each segment can ensure transmission efficiency and save bandwidth. The text content corresponding to the audio segments fed back by the speech recognition engine is obtained, and a text feature recognition model adapted for the financial field is used to perform text feature recognition and financial entity recognition on the text content, and the recognized financial entities are embedded into the text features.

[0024] Understandably, compared to intent recognition based solely on text features, this application breaks through the limitations of single text features by supplementing language information such as emotion and identity through voiceprint and speech features, providing multi-source data support for accurate intent recognition; features of each dimension are specifically adapted to different recognition needs, achieving information complementarity; the voiceprint feature recognition model, speech feature recognition model, and text feature recognition model are all lightweight models, which can balance feature extraction accuracy and real-time performance.

[0025] In some embodiments, after obtaining the text content of the audio segment from the voice recognition response, the text content corresponding to the audio segment from the voice recognition engine can be matched with sensitive words in a pre-built financial-specific sensitive word library (including 12 categories of high-profile legal / medical topics). Based on the matching result, it is determined whether to block the customer call or transfer it to a dedicated human agent. For example, if the text content matches a sensitive word, the call can be directly transferred to a dedicated human agent. This achieves effective blocking of sensitive topics. In addition, conflict detection can be performed between the text content corresponding to the audio segment from the voice recognition engine and key information (such as amount and date) recorded by the human agent. If the deviation exceeds a threshold, a review is initiated.

[0026] S103. The speaker features, speech features and text features of the real-time audio stream are fused to obtain multimodal fusion features.

[0027] Multimodal feature fusion refers to transforming heterogeneous features—voiceprint, speech, and text—into a unified-dimensional feature vector through dimensional adaptation and information integration, thereby eliminating modal differences and achieving information synergy. Multimodal fusion features are unified representation vectors carrying multi-dimensional information such as identity, emotion, and semantics. They are the core input of intent classification models and possess an information richness unmatched by single-modal features.

[0028] In some embodiments, text features, speech features, and voiceprint features can be integrated into a unified multimodal fusion feature through dimension alignment and splicing fusion strategies, eliminating modal differences and achieving information complementarity. During dimension alignment, the three types of features can be linearly transformed to ensure dimensional consistency; for example, speech features and voiceprint features can be mapped to match the dimensions of text features through a fully connected layer. The three types of features with the same dimensions are then spliced ​​together to obtain the modal fusion feature. For example, a 768-dimensional text semantic vector, a 768-dimensional mapped speech feature, and a 768-dimensional mapped voiceprint feature can be spliced ​​dimensionally to form a 768×3=2304-dimensional multimodal fusion feature.

[0029] In other embodiments, the weights of the three types of features are first calculated using a modal attention mechanism (e.g., text feature weight 0.6, speech feature weight 0.3, and voiceprint feature weight 0.1). After weighting each feature vector, dimensional alignment and splicing are performed to highlight the role of the core modality (text).

[0030] Furthermore, batch normalization can be used to process multimodal fusion features to reduce the distribution differences between modalities.

[0031] Understandably, this step can eliminate the modal heterogeneity of the three types of features, enabling deep collaboration of identity, emotion, and semantic information, and solving the problem of one-sided information in single-modal features; through normalization and weight optimization, it can improve the stability and discriminative ability of fused features, laying the foundation for improving the accuracy of intent recognition; and by unifying feature dimensions and forms, it can provide standardized input for subsequent intent classification models, reducing the complexity of model adaptation.

[0032] S104. Using a pre-trained intent classification model, determine the customer's intent type based on multimodal fusion features.

[0033] The pre-trained intent classification model refers to a deep learning model pre-trained on large-scale general corpora and financial domain corpora, adaptable to multimodal fusion feature inputs, used to determine the category of customer business requests. Intent type refers to predefined categories of customer business requests, corely covering regular business, legal consultation, complaint escalation, and other core banking service scenarios, corresponding to different agent transfer needs.

[0034] In some embodiments, a pre-trained intent classification model is used to determine the customer's intent type based on multimodal fusion features, including S1041-S1045: S1041. Through the encoding layer in the intent classification model, the multimodal fusion features are processed to obtain the global semantic feature vector and word vector sequence.

[0035] The encoding layer consists of a pre-trained semantic recognition model. This model is pre-trained with a large amount of bank customer service corpus for domain adaptation. It adopts a strategy of freezing the first 6 layers and fine-tuning the last 6 layers. The first 6 layers retain general semantic capabilities, while the last 6 layers are fine-tuned to enhance multimodal information fusion and adaptability to financial scenarios, thereby avoiding overfitting and improving recognition accuracy.

[0036] The global semantic feature vector carries global information of the entire multimodal sequence and is a semantic summary of customer needs (integrating multi-dimensional information such as core text semantics, voice emotion, and voiceprint identity); the word vector sequence refers to the local feature vector of each position corresponding to the text word segmentation, which contains the semantic information of the segmentation and is also bound to the corresponding voice and voiceprint features, providing a foundation for subsequent entity recognition and context modeling.

[0037] S1042. Using the first bidirectional long short-term memory network and conditional random field in the intent classification model, entity annotation is performed on the word vector sequence to generate entity feature vectors.

[0038] The first bidirectional long short-term memory network (BSSN) is responsible for capturing the temporal dependencies of word vector sequences (such as the association between "financial management" and "product"), ensuring accurate entity boundary recognition. The output of the first BSSN is the entity prediction corresponding to each word. The conditional random field (CRF) corrects the prediction errors of the first BSSN by learning the labeling rules of domain entities, ultimately outputting an accurate sequence of entity labels. Finally, the word vectors corresponding to the labeled entity words are averaged and pooled to obtain entity feature vectors of a unified dimension. This vector serves as the business anchor for intent recognition, directly related to the core requirements of financial scenarios.

[0039] S1043. By using the second bidirectional long short-term memory network in the intent classification model, the contextual dependence of the word vector sequence is captured, and a bidirectional temporal feature vector is obtained.

[0040] In some embodiments, the second bidirectional long short-term memory network includes a forward long short-term memory network and a backward long short-term memory network. The forward long short-term memory network captures the historical context of the word vector sequence in text order, while the backward long short-term memory network captures the future context of the word vector sequence in reverse order. The two outputs are concatenated to form a bidirectional temporal feature vector, which can completely preserve the semantic dependencies in long sequences and solve the problem of ambiguous intent in multi-turn dialogues or complex expressions.

[0041] S1044. By using the attention mechanism in the intent classification model, the global semantic feature vector and the bidirectional temporal feature vector are processed to obtain the contextual intent vector.

[0042] In some embodiments, the global semantic feature vector is used as the query vector and serves as a reference standard for measuring semantic importance; the bidirectional temporal feature vector is used as the key / value vector, and the similarity between the temporal feature at each location and the global semantics is calculated. Attention weights are obtained by normalization; each bidirectional temporal feature vector is multiplied by its corresponding weight and then summed to obtain the contextual intent vector. This vector is a high-level aggregation of the core semantics and accurately reflects the core direction of customer demands.

[0043] S1045. The contextual intent vector and entity feature vector are fused together, and the classifier in the intent classification model is used to identify the fused feature vector to obtain the customer's intent type.

[0044] The intent types include routine business, legal consultation, and escalation of complaints. By fusing the contextual intent vector and entity feature vector through dimensional concatenation, the fused feature vector ensures that it retains the core semantic logic while enhancing the business attributes within the financial sector. The allocator in the intent recognition model maps the fused high-dimensional feature vector to a predefined intent type space (routine business, legal consultation, escalation of complaints, etc.). A predefined activation function outputs the probability distribution of each intent type, and the type with the highest probability is selected as the final recognition result.

[0045] Understandably, this embodiment relies on the strong semantic understanding capability and rich information of multimodal features of the pre-trained model to significantly improve the accuracy of intent recognition; the model is adapted to financial scenarios and can accurately identify specific intent types, solving the semantic bias problem of general models.

[0046] S105. Based on the preset service intent map, transfer the customer's call to the target human agent corresponding to the intent type for further processing.

[0047] The service intent graph refers to a predefined tree-structured knowledge base containing over 300 intent nodes in the banking service domain and their corresponding transfer rules, clearly defining the mapping relationship between various intents and target agents. Target agents refer to service resources matched to the customer's intent type, including current regular agents, the pool of senior representatives, and third-party gateways, all possessing the capability to process the corresponding intent. Intelligent transfer refers to the automatic allocation of incoming call routes based on intent recognition results and the service intent graph, without manual intervention.

[0048] In some embodiments, the service intent graph is traversed based on the intent type to locate the corresponding target agent type. Target agents with the capability to handle that intent are then selected. Specifically, if the intent type is escalation of a complaint, currently available senior agent pool agents are selected as target agents; if the intent type is legal consultation, the online customer service of the connected third-party agency gateway is selected as the target agent; if the intent type is routine business, the current agent is selected as the target agent, in which case no transfer is needed. The agent transfer process automatically switches the call link, seamlessly transferring the customer's call from the current agent (or incoming line) to the target agent. During the transfer, a voice announcement (such as "Your request has been identified as an escalation of a complaint; we are transferring you to a senior agent") informs the customer, improving the experience. In this way, this step achieves accurate matching of intent type and service resources, avoids invalid transfers, and improves the efficiency of resolving customer issues.

[0049] The present invention provides an embodiment that predicts intent based on the rich information included in multimodal features, overcoming the limitations of single-modal processing and significantly improving the accuracy of intent recognition; furthermore, it transfers agents based on the recognized intent, thereby improving the efficiency of agent transfer.

[0050] Example 2 Figure 2 A flowchart of an intelligent routing agent transfer method is provided for an embodiment of the present invention. See also... Figure 2 The method includes the following steps: S201: In response to a customer call being connected by a human agent, the voice interaction signal between the customer and the human agent is acquired in real time, and the voice interaction signal is denoised using a lightweight noise reduction model; the denoised voice interaction signal is then processed using a frame-segmentation strategy to obtain a real-time audio stream.

[0051] In this embodiment, the step is triggered by the establishment of a call connection between the agent and the customer. The voice interaction signals of both parties are collected in real time. After the signal quality is optimized by a lightweight noise filtering model, a preset time window splitting strategy is used for discretization to form a continuous audio data sequence that can be transmitted and analyzed in real time. For example, a sliding window framing strategy of 20ms frame length + 10ms frame shift is used to discretize the noise-reduced voice signal. Each frame of data is associated with a unique timestamp to ensure time sequence consistency.

[0052] Understandably, processing is initiated only after the call is connected to avoid invalid data collection and resource waste, adapting to the business process of agent transfer; the lightweight noise reduction model filters out noise with low latency, ensuring the accuracy of subsequent feature extraction and reducing noise interference with intent recognition. Frame-by-frame processing converts continuous speech signals into discrete audio streams, preserving key speech information while adapting to real-time processing and transmission requirements.

[0053] S202. Perform multi-dimensional feature extraction on the real-time audio stream to obtain the speaker features, speech features and text features of the real-time audio stream.

[0054] S203. The speaker features, speech features and text features of the real-time audio stream are fused to obtain multimodal fusion features.

[0055] S204. Using a pre-trained intent classification model, determine the customer's intent type based on multimodal fusion features.

[0056] S205. Based on the preset service intent map, transfer the customer's call to the target human agent corresponding to the intent type for further processing.

[0057] The specific implementation process of steps S202-S205 can be found in the description of the above embodiments, and will not be repeated here.

[0058] S206. Obtain the full-link call data between the customer and the agent; perform behavioral tracking, quality tracking, and business tracking on the full-link call data, and store the tracked data in columnar format.

[0059] The call end-to-end data covers the entire process of a call, from connection to termination / transfer, including raw audio signals, noise-reduced audio streams, frame / segment data, voiceprint / voice / text features, intent recognition results, transfer paths, and agent operation records. Behavioral tracking refers to data collection of key interactions between customers and agents during a call, focusing on behavioral signals related to intent and risk. Quality tracking refers to data collection of performance indicators for technical aspects such as voice processing, feature extraction, and intent recognition, used to monitor system technical quality. Business tracking refers to data collection of business indicators such as call workflow status and processing efficiency, supporting business optimization and decision-making. Columnar storage refers to a storage method organized by data fields (columns) rather than entire records (rows), adapting to the efficient storage and retrieval of multi-dimensional tracking data.

[0060] In some embodiments, data from the entire call chain between the customer and the human agent is collected through a data acquisition interface; Furthermore, the process of behavioral tracking includes: using keyword matching technology to identify key phrases (such as "complaint" or "lawsuit") directly related to business intent in customer statements from the entire call data; when the above key phrases are detected, the precise timestamp of the key phrases is automatically recorded, and the corresponding audio clips, text clips, and voiceprint / voice features are associated to form a record of the association between phrases, time, and multimodal data; the captured key phrase content, timestamps, and associated data are archived in a columnar storage format to ensure efficient data storage and fast retrieval.

[0061] The quality tracking process includes: In the stage of transmitting audio segments to the speech recognition engine, the word error rate of the transcribed text for each audio segment is calculated in real time. Specifically, by comparing the transcribed text with the subsequently manually reviewed and corrected text, the proportion of erroneous characters to the total number of characters is calculated, and word error rate data is recorded by segment dimension. After the intent analysis module outputs the recognition results, the results of subsequent business processing (such as customer confirmation of intent accuracy, agent correction records) are correlated to calculate the recognition accuracy rate for each type of intent (routine business, legal consultation, complaint escalation, etc.), and accuracy rate data is recorded by call dimension. Quality indicators such as word error rate and intent classification accuracy are bound to the unique identifier of the corresponding call and stored in a columnar format to support model quality evaluation.

[0062] The process of business event tracking includes: recording the final processing path of a call, distinguishing between scenarios such as no transfer required (the current agent completes the service), transfer to the advanced agent pool, and transfer to a third-party gateway, and calculating the percentage of each scenario (i.e., the transfer rate to human agents), and summarizing by time period or business type; starting from when a customer enters the agent's line and stopping when the call ends (or the transfer is completed and the target agent confirms acceptance), recording the complete processing time of a single call; calculating the average processing time by dimensions such as intent type and agent type, forming multi-dimensional duration statistics; binding business indicators such as the transfer rate to human agents and the average processing time with the customer intent type, transfer path, and service result (whether the problem is resolved) corresponding to the call, archiving them in the business event tracking database, and saving them in a columnar storage manner.

[0063] Understandably, end-to-end data collection ensures no critical information is missed, providing comprehensive data support for subsequent model optimization and business analysis. The three types of tracking points correspond to behavioral signals, technical indicators, and business status, respectively, ensuring highly targeted data and avoiding redundant or invalid data. Columnar storage combined with high compression ratios significantly saves storage space while supporting fast query and analysis by field, ensuring efficient operation of the feedback loop.

[0064] S207. Add pseudo-labels to the newly added customer-human agent call data using a teacher model, and select high-value training samples based on uncertainty sampling.

[0065] Among them, the teacher model refers to a stable, high-performance intent recognition model selected online, used to generate credible labels for unlabeled data. Pseudo-labels are labels generated by the teacher model after predicting unlabeled data; they possess a certain degree of credibility and can be used as temporary labels for training samples. Uncertainty sampling refers to a sampling method that selects the most valuable samples for improving model performance based on the degree of uncertainty in the model's prediction results for unlabeled data.

[0066] High-value training samples refer to samples that are difficult for the model to accurately identify, contain new scenario / new phrasing information, and can significantly improve the model's generalization ability after training.

[0067] In some embodiments, the optimal version of the online intent recognition model is selected as the teacher model, all its parameters are frozen, and it is used only for inference and prediction, without participating in subsequent training and updates. Newly added unlabeled full-link call data (core being multimodal fusion features) is input into the teacher model, and the model outputs the probability distribution of various intents (routine business, legal consultation, complaint escalation, etc.). A confidence threshold is set, and pseudo-labels are generated only for samples with a prediction confidence higher than the threshold; samples with a confidence lower than the threshold are not labeled. The prediction uncertainty of unlabeled samples is calculated using the entropy method; the higher the entropy value, the more ambiguous the model's judgment of that sample. Samples are sorted from high to low entropy values, and the top preset proportion are selected as high-value candidate samples. Simultaneously, representative sampling is combined to ensure that candidate samples cover all intent types and business scenarios, avoiding sample concentration in a single scenario.

[0068] By generating pseudo-labels using a teacher model, the workload of manual annotation is reduced, especially the processing cost of massive amounts of unlabeled data. Uncertainty sampling accurately selects the high-value samples that the model most needs to learn, avoiding invalid samples from wasting training resources and improving model iteration efficiency.

[0069] S208. In response to the event that triggers the update of the intent recognition model, the intent recognition model is incrementally trained using the selected high-value training samples to obtain the updated intent recognition model.

[0070] Update events refer to preset conditions that trigger incremental training of the model, including timed triggers (such as daily nighttime), data drift triggers (such as detecting changes in feature distribution), and new business rule injection triggers. Incremental training refers to a training method that updates local parameters based on the parameters of the original model using only newly added high-value training samples, rather than retraining on the entire dataset.

[0071] In some embodiments, high-value training samples are mixed with a small number of historical core samples to form an incremental training set, preventing the model from forgetting historical knowledge. During incremental training, an elastic weight consolidation mechanism can be used to calculate the importance weights of parameters in the original model that are crucial to the historical task. Regularization constraints are added to these parameters during incremental training to prevent catastrophic forgetting caused by significant parameter modifications.

[0072] After incremental training is completed, the pre-updated intent recognition model and the post-updated intent recognition model are evaluated according to preset evaluation metrics. These metrics include problem resolution rate and user satisfaction. Based on the evaluation results, it is determined whether to deploy the post-updated intent recognition model. For example, if the complaint rate of the post-updated intent recognition model exceeds the aforementioned threshold, the post-updated intent recognition model will not be deployed, and the pre-updated intent recognition model will continue to be used.

[0073] Understandably, incremental training only updates local parameters, eliminating the need for retraining on the entire dataset, significantly shortening training time and addressing the pain point of traditional models' 3-6 month update cycles. Through an elastic weight consolidation mechanism and mixed training with historical core samples, it ensures that the model learns new knowledge while retaining its existing historical business logic and intent recognition capabilities.

[0074] In this embodiment of the invention, through multimodal fusion, full-link data closed loop and adaptive learning mechanism, the core technical problems existing in traditional agent transfer schemes, such as rigid response, one-sided intent recognition and lagging model iteration, are systematically solved.

[0075] Example 3 Figure 3 This is a schematic diagram of a smart routing agent transfer device provided in an embodiment of the present invention. This device can execute any of the smart routing agent transfer methods of the present invention. For example... Figure 3 As shown, the intelligent routing agent transfer device includes: The frame processing module 301 is used to process the voice interaction signal between the customer and the human agent in real time through a frame strategy to obtain a real-time audio stream. The feature extraction module 302 is used to extract multi-dimensional features from the real-time audio stream to obtain the speaker features, speech features and text features of the real-time audio stream. The feature fusion module 303 is used to fuse the speaker features, speech features and text features of the real-time audio stream to obtain multimodal fusion features; The intent recognition module 304 is used to determine the customer's intent type based on multimodal fusion features using a pre-trained intent classification model. The transfer module 305 is used to transfer customer calls to the target human agent corresponding to the intent type for further processing based on a preset service intent map.

[0076] In some embodiments, the intent recognition module 304 is specifically used to determine the customer's intent type based on multimodal fusion features using a pre-trained intent classification model. The multimodal fusion features are processed through the encoding layer in the intent classification model to obtain a global semantic feature vector and a word vector sequence; the encoding layer is composed of a pre-trained semantic recognition model. Entity annotation is performed on word vector sequences using the first bidirectional long short-term memory network and conditional random field in the intent classification model to generate entity feature vectors. By using the second bidirectional long short-term memory network in the intent classification model, the contextual dependence of the word vector sequence is captured, and bidirectional temporal feature vectors are obtained. By using the attention mechanism in the intent classification model, the global semantic feature vector and the bidirectional temporal feature vector are processed to obtain the contextual intent vector; The contextual intent vector is fused with the entity feature vector, and the classifier in the intent classification model is used to identify the fused feature vector to obtain the customer's intent type; the intent type includes routine business, legal consultation and complaint escalation.

[0077] In some embodiments, in processing the real-time acquired voice interaction signal between the customer and the human agent using a framing strategy to obtain a real-time audio stream, the framing processing module 301 is specifically used for: In response to a customer's call being connected by a human agent, the system acquires the voice interaction signal between the customer and the human agent in real time and performs noise reduction processing on the voice interaction signal using a lightweight noise reduction model. By employing a frame-segmentation strategy, the denoised voice interaction signal is processed to obtain a real-time audio stream.

[0078] In some embodiments, in extracting multi-dimensional features from a real-time audio stream to obtain its speakerprint features, speech features, and text features, the feature extraction module 302 is specifically used for: Voiceprint features are extracted from real-time audio streams using a voiceprint feature recognition model. Speech features are extracted from real-time audio streams using a speech feature recognition model. These features include prosodic features, fundamental frequency features, energy features, spectral features, emotional features, and customer behavior patterns during the call. According to the preset duration, the real-time audio stream is segmented and processed. For each audio segment obtained, the audio segment is encoded and compressed, and the encoded and compressed audio segment is streamed to the speech recognition engine in the cloud for speech-to-text processing. The system obtains the text content corresponding to the audio segments fed back by the speech recognition engine, performs text feature recognition and financial entity recognition on the text content, and embeds the recognized financial entities into the text features.

[0079] In some embodiments, the device further includes a data entry and model update module for: Obtain full-link data of the call between the customer and the human agent; Behavioral, quality, and business data are collected across the entire call chain, and the collected data is stored in columnar format. The teacher model adds pseudo-labels to the newly added customer-human agent call data and selects high-value training samples based on uncertainty sampling. In response to the event that triggers the update of the intent recognition model, the intent recognition model is incrementally trained using the selected high-value training samples to obtain the updated intent recognition model.

[0080] In some embodiments, the device further includes a model evaluation module for: The intent recognition model before and after the update are evaluated based on preset evaluation indicators; the evaluation indicators include problem resolution rate and user satisfaction. Based on the evaluation results, determine whether to deploy the updated intent recognition model.

[0081] In some embodiments, a sensitive interception module is also included, for: The text content corresponding to the audio segment returned by the speech recognition engine is matched with sensitive words in a pre-built financial-specific sensitive word database. Based on the matching results, determine whether to block customer calls or transfer them to a dedicated human agent.

[0082] The intelligent routing agent transfer device provided in the embodiments of the present invention can execute the intelligent routing agent transfer method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0083] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0084] Example 4 Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0085] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or loaded from storage unit 18 into the random access memory 13. The random access memory 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output interface 15 is also connected to the bus 14.

[0086] Multiple components in electronic device 10 are connected to input / output interface 15, including: input unit 16; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disks, optical disks, etc.; and communication unit 19, such as network interface cards, modems, wireless transceivers, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as performing intelligent routing agent transfer methods.

[0088] In some embodiments, the intelligent routing agent transfer method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via read-only memory 12 and / or communication unit 19. When the computer program is loaded into random access memory 13 and executed by processor 11, one or more steps of the intelligent routing agent transfer method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the intelligent routing agent transfer method by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable intelligent routing agent, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device or liquid crystal display for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet. The computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having client-server relationships with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service system, addressing the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0094] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0095] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for intelligent routing agent transfer, characterized in that, include: By using a frame-segmentation strategy, the voice interaction signals between the customer and the human agent acquired in real time are processed to obtain a real-time audio stream; Multi-dimensional feature extraction is performed on the real-time audio stream to obtain the speaker features, speech features, and text features of the real-time audio stream; The speaker features, speech features, and text features of the real-time audio stream are fused to obtain multimodal fusion features; The customer's intent type is determined using a pre-trained intent classification model based on the multimodal fusion features. Based on the preset service intent map, the customer's call is transferred to the target human agent corresponding to the intent type for further processing.

2. The method according to claim 1, characterized in that, The pre-trained intent classification model determines the customer's intent type based on the multimodal fusion features, including: The multimodal fusion features are processed through the encoding layer in the intent classification model to obtain a global semantic feature vector and a word vector sequence; wherein, the encoding layer is composed of a pre-trained semantic recognition model; The word vector sequence is labeled with entities using the first bidirectional long short-term memory network and conditional random field in the intent classification model to generate entity feature vectors. By using the second bidirectional long short-term memory network in the intent classification model, the contextual dependencies of the word vector sequence are captured to obtain a bidirectional temporal feature vector; The global semantic feature vector and the bidirectional temporal feature vector are processed through the attention mechanism in the intent classification model to obtain the contextual intent vector; The contextual intent vector and the entity feature vector are fused, and the classifier in the intent classification model is used to identify the fused feature vector to obtain the customer's intent type; wherein, the intent type includes routine business, legal consultation and complaint escalation.

3. The method according to claim 1, characterized in that, The process of processing the real-time voice interaction signal between the customer and the human agent using a frame-segmentation strategy to obtain a real-time audio stream includes: In response to a customer's call being connected by a human agent, the voice interaction signal between the customer and the human agent is acquired in real time, and the voice interaction signal is processed for noise reduction using a lightweight noise reduction model. By employing a frame-segmentation strategy, the denoised voice interaction signal is processed to obtain a real-time audio stream.

4. The method according to claim 1, characterized in that, The step of performing multi-dimensional feature extraction on the real-time audio stream to obtain the speakerprint features, speech features, and text features of the real-time audio stream includes: Voiceprint features are extracted from the real-time audio stream using a voiceprint feature recognition model. Speech features are extracted from the real-time audio stream using a speech feature recognition model; wherein, the speech features include prosodic features, fundamental frequency features, energy features, spectral features, emotional features, and the customer's behavioral patterns during the call. The real-time audio stream is segmented according to a preset duration. For each audio segment obtained, the audio segment is encoded and compressed. The encoded and compressed audio segment is then streamed to the speech recognition engine in the cloud for speech-to-text processing. The text content corresponding to the audio segment fed back by the speech recognition engine is obtained, and text feature recognition and financial entity recognition are performed on the text content, and the recognized financial entities are embedded into the text features.

5. The method according to claim 1, characterized in that, Also includes: Obtain full-link data of the call between the customer and the human agent; The entire call data is subjected to behavioral, quality, and service tracking points, and the tracked data is stored in columnar format. The teacher model adds pseudo-labels to the newly added customer-human agent call data and selects high-value training samples based on uncertainty sampling. In response to an event that triggers an update of the intent recognition model, the intent recognition model is incrementally trained using selected high-value training samples to obtain an updated intent recognition model.

6. The method according to claim 5, characterized in that, Also includes: The intent recognition model before and after the update are evaluated according to preset evaluation indicators; wherein the evaluation indicators include problem resolution rate and user satisfaction. Based on the evaluation results, determine whether to deploy the updated intent recognition model.

7. The method according to claim 4, characterized in that, Also includes: The text content corresponding to the audio segments returned by the speech recognition engine is matched with sensitive words in a pre-built financial-specific sensitive word database. Based on the matching results, determine whether to block customer calls or transfer them to a dedicated human agent.

8. A smart routing agent transfer device, characterized in that, include: The frame processing module is used to process the real-time voice interaction signals between the customer and the human agent through a frame segmentation strategy to obtain a real-time audio stream. The feature extraction module is used to perform multi-dimensional feature extraction on the real-time audio stream to obtain the speaker features, speech features and text features of the real-time audio stream. The feature fusion module is used to fuse the speaker features, speech features and text features of the real-time audio stream to obtain multimodal fusion features; An intent recognition module is used to determine the customer's intent type based on the multimodal fusion features using a pre-trained intent classification model. The transfer module is used to transfer the customer's call to the target human agent corresponding to the intent type for further processing, based on a preset service intent map.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the method of any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images