Core competency characteristic analysis system for telephone collection personnel based on large model

Abstract

The invention discloses a telephone collection personnel core competency characteristic analysis system based on a large model. The system integrates a voice data acquisition module, a preprocessing module, a large model core competency analysis module, an evaluation result output module and a model updating module. The voice data acquisition module acquires voice in a high-fidelity manner and digitalizes the voice; the voice preprocessing module provides high-quality data for a large model through noise reduction, segmentation and standardization processing. And the large model core competency analysis module quantitatively evaluates the competency of the collection personnel in a multi-dimensional manner based on a specific Transform architecture variant. And the evaluation result output module generates a detailed report to assist a company in managing decisions. And the model updating module optimizes the model regularly according to new data and service feedback. The system achieves real-time, comprehensive and accurate evaluation, improves the collection service efficiency and quality, formulates coping strategies for potential risks, and has remarkable innovativeness and practicability.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent speech analysis, and specifically relates to a system for accurately analyzing the core competency characteristics of telephone debt collectors using a large model. The system is mainly used in debt collection business scenarios, and aims to improve the management efficiency and quality of debt collection business. Background Art

[0002] In the current collection business, the existing methods for evaluating the capabilities of telephone collection personnel have many defects that are difficult to ignore. Traditional evaluation methods mostly rely on subjective evaluations after the fact, and supervisors often judge the performance of collection personnel based on limited call record fragments or collection performance data. This method lacks real-time performance and cannot detect problems and provide guidance in a timely manner during the collection process. For example, in actual collection scenarios, collection personnel may cause customer disgust due to improper communication methods, but traditional evaluation methods cannot correct them in real time, resulting in poor collection results. Moreover, the evaluation results are easily interfered by subjective factors, and accuracy is difficult to guarantee. The evaluation criteria of different supervisors are different, which makes the evaluation results lack objectivity and fairness.

[0003] Although some evaluation methods based on quantitative indicators can pay attention to result data such as collection rate and call duration, they ignore the key capabilities demonstrated by debt collectors during the call. Core competency behaviors such as communication skills, emotional management, and problem solving have not been fully considered. This makes it impossible to fully and deeply understand the true ability level of debt collectors, which is not conducive to the company's targeted training and improving the overall capabilities of the team. From a macro perspective, it is difficult for companies to accurately grasp their own strengths and weaknesses in the debt collection business, which seriously restricts the efficient development and optimization of the business. For example, the company may misjudge the ability of debt collectors and assign more difficult tasks to personnel with mismatched abilities, resulting in an increase in the failure rate of tasks. Through a survey of common debt collection personnel evaluation systems on the market, it was found that most systems can only implement simple data statistics and basic voice analysis functions. They cannot perform real-time, comprehensive and accurate evaluations from multiple core competency dimensions like the system of the present invention. For example, a well-known debt collection evaluation system mainly focuses on the statistical analysis of the collection rate, and lacks in-depth analysis of key factors such as the clarity of language expression and emotional stability of debt collectors in communication. This highlights the necessity of the present invention in terms of technological innovation and practical application. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art and provide a core competency feature analysis system for telephone debt collectors based on a large model, so as to realize real-time, objective and comprehensive evaluation of the debt collectors' capabilities and provide strong support for company decision-making.

[0005] To achieve the above object, the present invention adopts the following technical solutions: Voice data acquisition module: It uses high-sensitivity audio acquisition equipment to collect voice signals in telephone collection calls in real time at a sampling frequency of 44.1kHz, ensuring complete acquisition of voice information from both the collection staff and the customer. The collected analog voice signals are converted into digital signal format through a built-in 24-bit A / D converter to ensure high fidelity and accuracy of the voice data for subsequent precise analysis and processing.

[0006] **Voice preprocessing module**: - **Noise removal submodule**: Using an adaptive filtering algorithm based on the minimum mean square error (LMS), the algorithm continuously adjusts the coefficients of the filter to minimize the mean square error between the output signal and the expected signal, so that it can dynamically identify and filter out background noise during calls, such as environmental noise, line current noise, etc., so that the signal-to-noise ratio of the voice signal is increased to no less than 30dB. - **Voice segmentation submodule**: Based on the voice activity detection (VAD) technology of the hidden Markov model (HMM), by analyzing the energy, zero-crossing rate and other characteristic parameters of the voice signal, the voice fragments of the debt collector and the customer are accurately separated, and the segmentation accuracy reaches more than 95%, providing a basis for the subsequent separate analysis of the debt collector's voice. - **Standardization processing submodule**: Normalize the volume of the segmented debt collectors’ voice data to stabilize its average volume within the range of 65dB to 75dB, and control the speech rate between 180 and 220 words per minute through a speech rate adjustment algorithm based on dynamic time warping (DTW), ensuring that the voice data input into the large model meets unified standards and improves the accuracy and stability of model analysis.

[0007] Big model core competency analysis module: - Model architecture: A variant of the Transformer architecture based on the multi-head attention mechanism is used, which includes 6 encoder and decoder layers, and the number of model parameters reaches 100 million. This architecture can effectively capture the long-term and short-term dependencies and semantic features in voice data, so as to more accurately analyze the language behavior performance of debt collectors. Through the multi-head attention mechanism, the model can focus on different positions of the input sequence in parallel to obtain richer semantic information. For relevant principles, please refer to the paper "Attention Is All You Need" published by Vaswani et al. in 2017.

[0008] - Input and output: The input is the text representation of the pre-processed voice data of the debt collector. The voice is converted into text through speech recognition technology based on deep neural networks, and the text is structured and input into the model by combining natural language processing technologies such as part-of-speech tagging and syntactic analysis. The output is the quantitative evaluation score of the debt collector in multiple core competency dimensions, including communication skills (such as language clarity, logical coherence, effectiveness of speech, etc.), emotional management (such as emotional stability, proportion of positive emotional expression, ability to suppress negative emotions, etc.), problem-solving ability (such as effectiveness of response to customer objections, ability to guide repayment plans, etc.), and application of professional knowledge (such as accurate reference to laws and regulations, financial knowledge, etc.). The score range of each dimension is 0 to 100 points, and a comprehensive competency score is output at the same time, which is used to comprehensively evaluate the performance of the debt collector during the call.

[0009] - Training process: - Dataset source: A large amount of historical collection call voice data from our company and cooperative collection agencies is collected as training samples. The number of samples is no less than 100,000, covering call records in different regions, different customer types, and different collection scenarios to ensure the diversity and representativeness of the data.

[0010] - Labeling process: Invite professional experts with more than 10 years of collection business experience to label the collection personnel behaviors in the sample in detail to form the core competency characteristic labels corresponding to each sample. The labeling process follows strict internal labeling specifications to ensure the accuracy and consistency of the labeling.

[0011] - Training algorithm: supervised learning is adopted, cross entropy is used as the loss function, and the model is trained by stochastic gradient descent (SGD) algorithm combined with Adagrad adaptive learning rate strategy. The number of training rounds is not less than 50 rounds, and the model parameters are continuously adjusted to enable it to accurately predict the competency feature scores of debt collectors. During the training process, L2 regularization and Dropout technology are used to prevent overfitting, and the Dropout rate is set to 0.2. The accuracy of the trained model on the validation set is over 85%, ensuring that the model has high reliability and generalization ability.

[0012] Evaluation result output module: Generate a detailed collection personnel competency evaluation report based on the output results of the large model. The report is presented in a combination of visual charts (such as bar charts, line charts, radar charts, etc.) and text descriptions, intuitively showing the scores of collection personnel in each competency dimension, and comparing and analyzing with the excellent standard line set by the company (the excellent standards for each dimension are: communication skills 90 points, emotional management 85 points, problem-solving ability 85 points, professional knowledge application 95 points) and the average level to clarify their strengths and weaknesses. At the same time, this module can summarize and count the evaluation results of all collection personnel according to different time periods (such as daily, weekly, monthly, etc.), generate a company-level collection capability characteristics report, and show the company's overall distribution, average score, standard deviation and other statistical indicators in each core competency dimension, as well as the correlation analysis between the dimensions, so as to provide detailed data support for the company's management to fully understand the company's collection team's capabilities, so as to formulate accurate personnel training plans, business process optimization strategies, and team formation and division of labor plans.

[0013] Model update module: - Update timing: Regularly (weekly) collect new collection call voice data and corresponding actual business feedback information, such as customer complaint rate, collection success rate changes, etc. When the amount of new data reaches 5,000 or the business feedback indicators fluctuate significantly (such as customer complaint rate changes exceeding 10%, collection success rate changes exceeding 15%), the model update process is triggered.

[0014] - Update method: Use active learning strategies based on uncertainty sampling to select samples with great value for model improvement from new data. The specific screening criteria are samples with information entropy greater than 0.8, which are integrated with the original training data at a ratio of 1:9. Incremental training updates are performed on the large model, and the weight parameters of the model are adjusted to enable the model to adapt to new scenarios, new problems, and new speech in the collection business in a timely manner, continuously improve the analysis accuracy and adaptability of the core competency characteristics of telephone collection personnel, and maintain the advancement and effectiveness of the model. In the process of model updating, use the Early Stopping method to prevent overfitting. When the loss on the validation set does not decrease for 5 consecutive epochs, stop training.

[0015] The present invention has the following beneficial effects: Real-time evaluation and feedback: Different from the traditional post-evaluation method, the present invention can analyze the behavior of the collector in real time during the telephone collection call, identify problems in time and give feedback, so that the collector can adjust the communication strategy at the moment and improve the collection effect. This has achieved an important innovative breakthrough in the field of collection business capability assessment, greatly improving the timeliness and practicality of the assessment. For example, in a collection call, the system detected in real time that the collector was emotionally agitated. Through instant feedback, the collector adjusted his tone in time, successfully avoided the customer's resistance, and improved the collection success rate.

[0016] Multi-dimensional comprehensive evaluation: Different from the previous evaluation methods of single dimension or simple quantitative indicators, this system conducts a comprehensive evaluation of debt collectors from multiple core competency dimensions, comprehensively covering key aspects such as communication, emotions, problem solving, and professional knowledge, and can more deeply and accurately reflect the actual ability level of debt collectors. Providing the company with richer and more valuable personnel ability information will help the company implement refined talent management and business optimization strategies. For example, based on the evaluation results, the company can carry out special training for the weak links of debt collectors to improve the overall team quality.

[0017] Accurate analysis based on large models: Using advanced large model technology, through learning and deep mining of massive collection voice data, it is possible to accurately identify and quantify the various behavioral characteristics and competency performance of collection personnel during calls, overcoming the limitations of traditional analysis methods in processing complex voice data and semantic understanding, and significantly improving the accuracy and reliability of the assessment. In actual applications, compared with traditional assessment methods, the accuracy of the system of the present invention in assessing the ability of collection personnel has increased by more than 30%, providing a more reliable basis for corporate decision-making. DETAILED DESCRIPTION

[0018] Taking a collection company with 500 telephone collection personnel as an example, the implementation process of the system of the present invention is described in detail. The core competency feature analysis system of telephone collection personnel based on a large model of the present invention is integrated into the company's telephone collection business system. First, a voice data acquisition module is installed on the telephone equipment of each collection seat to ensure its normal operation and establish a stable data transmission connection with the subsequent modules. When the collection personnel talk to the customer, the voice data acquisition module is immediately started, and the voice signals of both parties are collected in real time at a sampling frequency of 44.1kHz, and converted into digital signals and transmitted to the voice preprocessing module. The noise removal submodule in the voice preprocessing module uses an adaptive filtering algorithm based on LMS to reduce the background noise in the call to less than 30dB, and then the voice segmentation submodule separates the voice fragments of the collection personnel with an accuracy of 95% through the VAD technology based on HMM. Finally, the standardization processing submodule adjusts the average volume of the collection personnel's voice to 65dB-75dB, and the speech speed is controlled at 180 words per minute-220 words. The processed voice data is sent to the core competency analysis module of the large model. The core competency analysis module of the large model analyzes the input voice and text data of debt collectors based on a pre-trained Transformer architecture variant model based on a multi-head attention mechanism (with 100 million model parameters).

[0019] For example, in a call between Xiao Li, a debt collector, and a customer, the model analyzed Xiao Li's language expression, emotional tone, and response to questions, and gave him a score of 80 in the communication skills dimension, 70 in the emotion management dimension, 75 in the problem-solving ability dimension, and 85 in the application of professional knowledge dimension, with a comprehensive competency score of 78. Based on these scores, the evaluation result output module generates a detailed evaluation report for Xiao Li's call, using a visual radar chart to display the scores of each dimension, and compares them with the company's average level (the average scores of each dimension are: 70 points for communication skills, 65 points for emotion management, 70 points for problem-solving ability, and 80 points for application of professional knowledge), clearly presenting Xiao Li's strengths (such as application of professional knowledge) and weaknesses (such as emotion management). After a day's collection work is completed, the evaluation result output module summarizes and counts the evaluation results of all 500 debt collectors on that day to generate a company-level collection capability feature report. The report shows that the company's average score in the communication skills dimension is 72 points, with a standard deviation of 5 points; the average score in the emotion management dimension is 68 points, with a standard deviation of 6 points; the average score in the problem-solving ability dimension is 73 points, with a standard deviation of 4 points; the average score in the professional knowledge application dimension is 82 points, with a standard deviation of 3 points. At the same time, through correlation analysis, it is found that there is a strong positive correlation between communication skills and problem-solving ability (the correlation coefficient is 0.7), while the correlation between emotion management and other dimensions is relatively weak. Based on this report, the company's management has formulated a special training plan for improving emotion management capabilities, and adjusted the team division of labor according to the ability characteristics of each collection staff, assigning personnel with strong communication skills and problem-solving skills to more difficult collection tasks to improve the overall collection efficiency and success rate.

[0020] At the same time, the model update module collects at least 5,000 new collection call voice data and corresponding business feedback information every week, and uses an active learning algorithm based on uncertainty sampling to screen out 500 representative new samples, which are integrated with the original 4,500 training data, and the large model is incrementally trained and updated. After 50 rounds of training, the accuracy of the model on new data has increased from the original 80% to 85%, effectively improving the model's adaptability to new business scenarios and collection scripts, and ensuring the continuous optimization and advancement of the system.

[0021] Risks and limitations 1. Data privacy and security risks: Since the system involves the collection and storage of a large amount of voice data from debt collection calls, there is a risk of data leakage. To address this risk, strict data encryption technology is used to encrypt the transmitted and stored data. At the same time, a complete data access rights management mechanism is established. Only authorized personnel can access relevant data to ensure data security and privacy.

[0022] 2. Model bias risk: The model’s training data may be biased, resulting in inaccurate model assessments of certain groups or scenarios. To reduce this risk, during the data collection phase, we try to ensure the diversity and representativeness of the samples, covering call data from different regions, different types of customers, and different collection scenarios. At the same time, we regularly conduct manual review and calibration of the model, and revise and optimize the model’s output results based on actual business experience.

[0023] 3. Risk of technology upgrade: With the continuous development of speech analysis technology and debt collection business, new algorithms and technologies may cause the system of the present invention to face the risk of falling behind. In order to meet this challenge, the company has established a special technical research and development team to continuously pay attention to the latest technological developments in the industry, upgrade and improve the system in a timely manner, and explore the application of new technologies such as reinforcement learning and semi-supervised learning to the model to improve the performance and competitiveness of the system. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 :System framework diagram: This diagram clearly shows the system architecture and the relationship between each module. The voice data acquisition module collects voice signals at a sampling frequency of 44.1kHz and converts them into digital signals through a 24-bit A / D converter to provide raw data for subsequent processing. The voice preprocessing module includes a noise removal submodule (using an LMS-based adaptive filtering algorithm to reduce noise), a voice segmentation submodule (segmenting voice segments based on HMM-based VAD technology) and a standardization processing submodule (adjusting speech speed using a DTW-based algorithm) to comprehensively preprocess the collected data. The large model core competency analysis module analyzes the preprocessed voice data based on a specific Transformer architecture variant (6 encoder and decoder layers, 100 million parameters) and outputs the quantitative assessment scores and comprehensive competency scores of debt collectors in multiple core competency dimensions. The evaluation result output module generates an evaluation report containing visual charts (such as bar charts, line charts, radar charts, etc.) and text descriptions based on the output of the large model, providing intuitive and detailed information to the company's management. The model update module regularly collects new data and business feedback information. After data screening and fusion, it performs incremental training updates on the large model, adjusts model parameters, and ensures continuous optimization of the system.

[0024] Figure 2:Overall business process of the system: This figure shows the complete business process of the core competency feature analysis system for telephone debt collectors based on a large model. The first step is voice data collection to obtain voice information in the collection call. The collected data enters the voice preprocessing stage, and through noise removal, voice segmentation and standardization processing, it provides high-quality data for subsequent analysis. Then, a full voice analysis is carried out, and the core competency analysis module of the large model is used to evaluate the debt collectors from multiple dimensions such as communication skills and emotional management. A detailed evaluation report is generated based on the evaluation results, and then the report is summarized and counted. Finally, determine whether the model needs to be updated based on the set conditions. If the update conditions are met, the model update process is started so that the system can adapt to the ever-changing business scenarios and ensure the accuracy and effectiveness of the evaluation.

[0025] Figure 3 :Data preprocessing, model training and verification process: This figure shows the key steps of data processing before large model training and model training and verification. First, collect no less than 100,000 historical collection call voice data to ensure sufficient data volume. Then professional experts annotate the core competency feature labels, and the annotations follow internal specifications to ensure the accuracy and consistency of the data and provide a reliable data foundation for subsequent training. The model adopts a variant of the Transformer architecture based on a multi-head attention mechanism, with 6 encoder and decoder layers and 100 million parameters. During training, supervised learning is used, with cross entropy as the loss function, combined with the stochastic gradient descent (SGD) algorithm and the Adagrad adaptive learning rate strategy, and the number of training rounds is no less than 50 rounds. L2 regularization and Dropout technology (Dropout rate is set to 0.2) are also used to prevent overfitting. After training, the model is verified by the validation set. If the accuracy of the validation set reaches more than 85%, it means that the model is reliable and has good generalization ability; if it does not meet the standard, the model parameters are readjusted to continue training until the requirements are met.

[0026] Figure 4:Model update process: This figure focuses on the specific process of model update. Newly generated collection call voice data and related business feedback information, such as changes in customer complaint rate and collection success rate, are continuously collected every week. When the amount of new data reaches 5,000, or when business indicators fluctuate significantly (for example, customer complaint rate changes by more than 10%, and collection success rate changes by more than 15%), the model update is triggered. During the update, an active learning strategy based on uncertainty sampling is used to screen out samples with information entropy greater than 0.8, which have great value for model improvement. The screened samples are fused with the original training data at a ratio of 1:9, and then the large model is incrementally trained and updated. During the training process, the early stopping method is used to prevent overfitting. When the loss on the validation set has not decreased for 5 consecutive epochs, the training is stopped, so that the model can adapt to new scenarios, new problems, and new words in a timely manner, and maintain the advancement and effectiveness of the model.

Claims

1. A core competency feature analysis system for telephone debt collectors based on a large model, characterized by: include: - Voice data acquisition module, used to collect voice signals in telephone collection calls at a sampling frequency of 44.1kHz and convert them into digital signals through a 24-bit A / D converter; - Voice pre-processing modules, including a module for noise reduction, which uses an adaptive filtering algorithm to increase the signal-to-noise ratio of voice signals to no less than 30 dB, a module for voice segmentation, which uses model-based voice activity detection technology to separate the voice segments of debt collectors with a segmentation accuracy of more than 95%, and a module for standardization processing, which uses a specific algorithm to adjust the average volume of debt collectors' voices to 65dB to 75dB and control the speaking speed to 180 to 220 words per minute; - The core competency analysis module of the large model adopts a variant of the Transformer architecture based on the multi-head attention mechanism, including 6 encoder and decoder layers, and the number of model parameters reaches 100 million. The input is the text representation of the pre-processed speech data of the debt collectors, and the output is the quantitative evaluation score and comprehensive competency score of the debt collectors in the core competency dimensions such as communication skills, emotion management, problem-solving ability, and application of professional knowledge; - Evaluation result output module, which generates a collection personnel competency evaluation report based on the output results of the big model, and generates a company-level collection capability characteristic report by summarizing and counting the results in different time periods; - Model update module, which regularly collects new data and business feedback information. When the amount of new data reaches 5,000 or the business feedback indicators fluctuate significantly, an active learning strategy based on uncertainty sampling is used to screen samples and merge them with the original training data to perform incremental training updates on the large model.

2. The system according to claim 1, characterized in that The training process of the core competency analysis module of the large model includes: collecting no less than 100,000 historical collection call voice data as training samples, annotating them by professional experts with more than 10 years of collection business experience, adopting supervised learning, using cross entropy as the loss function, and training through the stochastic gradient descent (SGD) algorithm combined with the Adagrad adaptive learning rate strategy. The number of training rounds is no less than 50 rounds, and L2 regularization and Dropout technology are used in the training process. The Dropout rate is set to 0.

2. The accuracy of the trained model on the validation set reaches more than 85%.

3. The system according to claim 1, characterized in that The noise reduction in the speech preprocessing module adopts an adaptive filtering algorithm based on the minimum mean square error (LMS), the speech activity detection technology is based on the hidden Markov model (HMM), and the speech rate adjustment in the standardization process adopts an algorithm based on dynamic time warping (DTW).

Images