Service call compliance intelligent quality inspection system based on key semantic anchor point triggering

Abstract

The invention belongs to the technical field of artificial intelligence and natural language processing, and relates to a service call compliance intelligent quality inspection system based on key semantic anchor point triggering, and the system comprises a data obtaining and preprocessing module which is used for extracting client audio acoustic features and streaming texts; the acoustic baseline and anomaly detection module constructs an individual acoustic normal baseline and monitors deviation to lock anomaly; the semantic bias instruction generation module is used for generating a semantic analysis bias instruction containing a time anchor point in response to the exception; the semantic feature extraction module is used for reconstructing the attention weight of the model by using the instruction, performing biased extraction on the text and outputting a feature vector; the intention verification risk scoring module is used for generating a single-point risk score based on the feature vector in combination with the abnormal strength; and the comprehensive portrait generation module is used for aggregating verification results to generate a comprehensive compliance portrait. According to the method, the problem that the existing call quality inspection technology is insufficient in the aspect of verifying the real intention of a customer by fusing acoustics and semantic information is solved.

Description

Technical Field

[0001] This invention belongs to the technical field of artificial intelligence and natural language processing, and relates to an intelligent quality inspection system for business call compliance based on key semantic anchor points. Background Technology

[0002] In business scenarios that rely heavily on telephone communication, such as finance and customer service, compliance checks on business calls are a crucial step in ensuring service quality and controlling operational risks. Traditional quality control methods rely on manual sampling, where quality inspectors listen to recordings and evaluate them according to standards. This method requires significant human resources and has limited sampling coverage, making it difficult to comprehensively monitor massive amounts of call data. Furthermore, manual evaluation results are easily influenced by the personal experience of quality inspectors, and the consistency and objectivity of quality control standards need improvement.

[0003] To overcome the limitations of manual quality inspection, the industry has introduced automated quality inspection technologies. Early automation solutions were mainly based on keyword search technology, identifying potentially problematic calls by matching pre-set lists of risky or sensitive words in the call transcript. With technological advancements, some solutions have incorporated analysis of the overall mood of the call, using acoustic parameters such as volume or speech rate, as well as text sentiment analysis, to determine the overall atmosphere of the call. These automation methods have, to some extent, improved the coverage and data processing capabilities of quality inspection.

[0004] However, existing automated solutions still fall short in deeply understanding customers' true intentions. Simple keyword retrieval solutions are prone to misjudgment due to lack of context and struggle to identify implicit violations without explicitly using risky vocabulary. Analysis solutions based on overall emotion or acoustic characteristics have a coarse-grained analysis, potentially misinterpreting a customer's inherent speaking habits as emotional agitation or overlooking brief changes in state at crucial moments. Current technologies typically treat acoustic and semantic analysis as two separate processes, failing to effectively establish a deep connection between acoustic performance and specific speech content. This makes it difficult to reveal the hidden true intentions or risks when a customer's speech does not align with their acoustic performance. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides an intelligent quality inspection system for business call compliance based on key semantic anchor points.

[0006] A business call compliance intelligent quality inspection system based on key semantic anchors includes:

[0007] The data acquisition and preprocessing module acquires dual-track recording data of the target business call, separates the customer-side audio stream to extract the temporal acoustic feature sequence, and converts the customer-side audio stream into streaming text data.

[0008] The acoustic baseline and anomaly detection module constructs an individual acoustic baseline for each customer based on the feature distribution at the initial stage of a call, and monitors in real time the deviation of the temporal acoustic feature sequence from the individual acoustic baseline in order to identify abnormal acoustic fluctuations.

[0009] The semantic bias instruction generation module, in response to locked acoustic anomaly fluctuations, generates semantic parsing bias instructions containing anomaly type encoding and time anchors.

[0010] The semantic feature extraction module dynamically reconstructs the internal attention weights of the semantic understanding model using semantic parsing bias instructions, performs biased feature extraction on the streaming text data with associated time anchors, and outputs text feature vectors.

[0011] The intent verification risk scoring module outputs intent verification results based on text feature vectors and generates a single-point risk score by combining the intensity of acoustic anomaly fluctuations.

[0012] The comprehensive profile generation module aggregates multiple intent verification results from the entire call process to generate a comprehensive compliance profile of the customer.

[0013] A further aspect of the present invention includes a data acquisition and preprocessing module, which performs the following steps:

[0014] The system separates the agent-side and customer-side audio channels in dual-track recording data in real time, marks the silent segments and performs noise reduction on the customer-side audio channel, and obtains the noise-reduced customer voice data stream.

[0015] The noise-reduced customer voice data stream is segmented into frames, and multi-dimensional features including pitch change rate, frequency domain formant stability and speech rate local acceleration are calculated and arranged in time order to form a temporal acoustic feature sequence.

[0016] The speech recognition engine is invoked to transcribe the noise-reduced customer voice data stream into text, and a time-aligned mapping between the streaming text data and the temporal acoustic feature sequence is established based on the timestamp index.

[0017] A further aspect of the present invention includes an acoustic baseline and anomaly detection module, used to perform the following operations:

[0018] The temporal acoustic feature sequence with a preset duration before the call is selected as the sample set. The mean vector and covariance matrix of each dimension feature are calculated, and a multidimensional Gaussian distribution model is constructed as the individual acoustic normal baseline.

[0019] A sliding window mechanism is used to extract the current temporal acoustic feature sequence, and the Mahalanobis distance between the instantaneous feature vector within the window and the center of the individual acoustic normal baseline is calculated.

[0020] The calculated Mahalanobis distance is compared with the dynamic adaptive threshold. If the Mahalanobis distance exceeds the dynamic adaptive threshold, it is determined that there is an acoustic abnormal fluctuation within the current sliding window.

[0021] A further aspect of the present invention includes a semantic bias instruction generation module, which performs the following steps:

[0022] Feature clustering is performed on locked acoustic anomalous fluctuations to identify corresponding anomalous patterns of emotional agitation, hesitation, pauses, or cognitive overload.

[0023] The identified abnormal patterns are used to retrieve a pre-set acoustic-semantic mapping table to determine the corresponding semantic parsing bias type;

[0024] It encapsulates the semantic parsing bias type and the time anchor point of acoustic anomaly fluctuations, and generates semantic parsing bias instructions to guide the direction of subsequent semantic analysis.

[0025] A further aspect of the present invention includes a semantic feature extraction module, which performs the following operations:

[0026] The semantic parsing bias instruction is parsed, and the specific semantic vector cluster associated with the semantic parsing bias type is located by querying the internal mapping table;

[0027] Generate an attention bias mask and dynamically increase the attention weight coefficients for specific semantic vector clusters in the attention layer of the semantic understanding model, so that the model enters a biased activation state.

[0028] Based on the time anchor points in the semantic parsing bias instructions, dynamic text windows containing the preceding and following context are extracted from the streaming text data and input into the semantic understanding model after state reconstruction for feature calculation.

[0029] A further aspect of this invention aims to verify the risk scoring module, which performs the following steps:

[0030] Input the text feature vector into the classification layer, output the intent classification probability distribution, and select the category corresponding to the highest probability as the true intent;

[0031] The Mahalanobis distance corresponding to the acoustic anomaly fluctuations triggered by retrospective analysis is used as the anomaly intensity value, and the anomaly intensity value is mapped to a risk multiplier through an intensity scaling function.

[0032] A single-point risk score is calculated using a preset baseline risk coefficient and risk multiplier, and this score, along with the true intent, is encapsulated as an intent verification result.

[0033] A further embodiment of the present invention integrates a portrait generation module, which is used to perform the following operations:

[0034] All intent verification results triggered during the call are serialized and spliced ​​together in chronological order to construct a time-series graph reflecting the evolution of the customer's psychological state.

[0035] By combining the business rule base, high-risk nodes in the time series graph are identified and single-point risk scores are weighted and accumulated to calculate the customer's comprehensive repayment willingness index and potential loss of contact risk value;

[0036] Based on the numerical range of the comprehensive repayment willingness index and the potential loss of contact risk value, a comprehensive compliance profile including the rating conclusion is output.

[0037] A further aspect of the present invention includes a method for calculating multidimensional features, comprising:

[0038] The pitch change rate is calculated by dividing the difference between the fundamental frequency of the current frame and the previous frame by the frame shift.

[0039] The stability of frequency domain resonant peaks is obtained by calculating the variance of the center frequencies of the first three resonant peaks within a certain time window;

[0040] The local acceleration of speech rate is obtained by calculating the first and second derivatives of the number of syllables per unit time.

[0041] In a further embodiment of the present invention, the dynamic adaptive threshold is set according to the preset percentile value of the Mahalanobis distance distribution of all data points in the sample set of the individual acoustic normal baseline.

[0042] The preset duration is the effective voice duration at the beginning of the call. If the total call duration is less than the preset duration, the system's preset global average baseline will be used to replace the individual acoustic normal baseline.

[0043] A further aspect of this invention is that an acoustic-semantic mapping table establishes a correlation between acoustic-level anomalous patterns and semantic-level parsing bias types; wherein the correlation includes at least: mapping emotional arousal patterns to high-arousal anxiety bias types, instructing the model to focus on words expressing urgency or difficulty; and mapping hesitation and pause patterns to uncertain evasion bias types, instructing the model to focus on words expressing uncertainty or ambiguity.

[0044] In summary, the present invention has the following beneficial technical effects:

[0045] 1. This invention establishes a personalized acoustic behavior reference standard for calls by constructing a baseline of normal acoustic behavior for each customer and monitoring deviations in real time. This mechanism utilizes a comparison of the customer's previous and subsequent speech characteristics to distinguish between the customer's inherent speaking habits and acoustic fluctuations caused by changes in emotion or cognitive state. This approach reduces the risk of misjudgment due to individual differences in speech characteristics and, to a certain extent, improves the targeting and accuracy of acoustic anomaly detection.

[0046] 2. This invention generates semantic parsing bias instructions in response to acoustic anomalies and dynamically reconstructs the internal attention weights of the semantic understanding model using these instructions, thus transforming acoustic cues into guidance for semantic analysis tasks. When the system detects specific acoustic features, it can adjust the semantic model's attention weights for specific types of words in the text accordingly. This mechanism establishes a technical link between acoustic performance and semantic content, helping to uncover deeper intentions hidden beneath the literal meaning and optimizing the problem of the disconnect between acoustic and semantic analysis.

[0047] 3. This invention provides a dynamic customer status assessment solution by aggregating multiple intent verification results throughout the entire call process and combining them with a business rule base to generate a comprehensive customer compliance profile. The system correlates discrete intent judgment points in chronological order to construct a time-series graph reflecting the evolution of the customer's psychological state. Based on this graph and the comprehensive scoring performed according to business rules, a quantitative assessment of the customer's overall willingness to cooperate and potential risks is achieved, thus providing more comprehensive and objective data support for business decisions. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention.

[0049] Figure 1 This is a schematic diagram of the framework in the embodiments of this application.

[0050] Figure 2 This is a flowchart illustrating an embodiment of this application. Detailed Implementation

[0051] The following is in conjunction with the appendix Figure 1 - Figure 2 A preferred description of the present invention is provided below.

[0052] See attached document Figure 1 - Figure 2 This invention proposes an intelligent quality inspection system for business call compliance based on key semantic anchor points, comprising the following modules:

[0053] The data acquisition and preprocessing module acquires dual-track recording data of the target business call, separates the customer-side audio stream to extract the temporal acoustic feature sequence, and converts the customer-side audio stream into streaming text data.

[0054] The acoustic baseline and anomaly detection module constructs an individual acoustic baseline for each customer based on the feature distribution at the initial stage of a call, and monitors in real time the deviation of the temporal acoustic feature sequence from the individual acoustic baseline in order to identify abnormal acoustic fluctuations.

[0055] The semantic bias instruction generation module, in response to locked acoustic anomaly fluctuations, generates semantic parsing bias instructions containing anomaly type encoding and time anchors.

[0056] The semantic feature extraction module dynamically reconstructs the internal attention weights of the semantic understanding model using semantic parsing bias instructions, performs biased feature extraction on the streaming text data with associated time anchors, and outputs text feature vectors.

[0057] The intent verification risk scoring module outputs intent verification results based on text feature vectors and generates a single-point risk score by combining the intensity of acoustic anomaly fluctuations.

[0058] The comprehensive profile generation module aggregates multiple intent verification results from the entire call process to generate a comprehensive compliance profile of the customer.

[0059] In one embodiment of the present invention, the data acquisition and preprocessing module is used to perform the following steps:

[0060] The system separates the agent-side and customer-side audio channels from the dual-track recording data in real time. It then performs silence segment marking and noise reduction on the customer-side channel to obtain the noise-reduced customer voice data stream. The noise-reduced customer voice data stream is then processed by frame segmentation, calculating multi-dimensional features including pitch change rate, frequency domain formant stability, and local acceleration of speech rate, and arranging them in chronological order to form a temporal acoustic feature sequence. Finally, the system calls a speech recognition engine to transcribe the noise-reduced customer voice data stream into text, and establishes a time-aligned mapping between the streaming text data and the temporal acoustic feature sequence based on timestamp indexes.

[0061] Specifically, this invention provides a semantic intent verification method driven by acoustic preamble signals. In a specific implementation scenario, this method is executed by an intelligent quality inspection system deployed on a cloud server or local data center. First, dual-track recording data of the target business call is acquired in real time from the call center platform via a network interface. It should be noted that dual-track recording data refers to a stereo audio data format, where the left channel records the voice of one participant in the call, such as an agent, and the right channel records the voice of another participant, such as a customer. The sampling rate is typically set between 8 kHz and 16 kHz to balance speech clarity and data storage overhead.

[0062] The dual-track recording data is encoded in pulse code modulation format and contains independent audio streams on the agent side and the customer side. The data processing module first decapsulates the dual-track recording data, accurately separating the customer-side audio stream. Subsequently, a deep learning-based speech activity detection model, such as a recurrent neural network structure like the LSTM-VAD model, performs frame-by-frame analysis of the customer-side audio stream, identifying and marking non-speech silence segments. Simultaneously, a pre-trained noise suppression model, such as algorithms based on deep complex convolutional networks or spectral subtraction, or more advanced deep spectrogram masking models, is applied to the non-silent segments to filter out line noise, background voices, and other non-target signals, thereby generating a high signal-to-noise ratio, denoised customer speech data stream.

[0063] Next, the acoustic feature extraction engine and the automatic speech recognition engine operate on the denoised customer speech data stream in parallel. The acoustic feature extraction engine divides the denoised customer speech data stream into continuous and partially overlapping analysis frames, for example, each frame is 25 ms long with a frame shift of 10 ms. This is an empirical value widely used in the field of speech signal processing to balance temporal resolution and computational efficiency. For each analysis frame, the engine calculates a multi-dimensional feature vector, which contains at least three key acoustic dimensions:

[0064] The first dimension is the pitch change rate, which is calculated by the ratio of the difference between the fundamental frequency F0 of the current frame and the previous frame to the frame shift. This indicator reflects the arousal level of the customer's emotions. When calculating, unvoiced frames with F0 of 0 need to be removed to avoid singular values.

[0065] The second dimension is the stability of the frequency domain formants. The center frequencies of the first three formants F1, F2, and F3 are estimated through linear predictive coding analysis, and the variance of the formant frequencies within a fixed-length sliding time window is calculated. This index is related to the tension or relaxation state of the vocalization.

[0066] The third dimension is the local acceleration of speech rate, obtained by calculating the first and second derivatives of the number of syllables per unit time. This indicator is related to the customer's cognitive load or the degree of hesitation in speech. The feature vectors of all analysis frames are arranged in chronological order, forming a complete temporal acoustic feature sequence used to quantitatively describe the changes in the customer's speaking style over time.

[0067] Meanwhile, the automatic speech recognition engine, typically a streaming recognition service based on Transformer or Long Short-Term Memory network architecture, continuously transcribes the noise-reduced client speech data stream into text. The engine's output not only includes the recognized words or sentences but also precisely labels each recognized basic language unit with its corresponding start and end timestamps from the original audio stream. This timestamp-indexed text sequence constitutes the streaming text data, its core feature being that each text word is bound to a timestamp accurate to the millisecond level. Finally, the system's data alignment module, based on the timestamp information, precisely aligns the temporal acoustic feature sequence with the streaming text data, establishing a mapping relationship between each text unit and its corresponding acoustic feature vector set during its utterance period, providing a data foundation for subsequent cross-modal analysis.

[0068] In this embodiment, the rate of change of pitch in the temporal acoustic feature sequence In discrete time frames The calculation at this point can be expressed by the following formula:

[0069]

[0070] in, Representative at the The pitch change rate of each analysis frame, measured in Hz / s. The system uses the YIN algorithm to calculate the first... The fundamental frequency value of each analysis frame, in Hz. For the system in the first The fundamental frequency value calculated from each analysis frame. It is the time interval between analyzed frames, i.e., frame shift, which is usually set to a constant value.

[0071] For example, suppose the system acquires dual-track recording data of a target business call and separates a 2-second client-side audio stream. First, this audio stream is processed to filter out approximately -35 dBFS of background fan noise, generating a noise-reduced client voice data stream. Then, the acoustic feature extraction engine analyzes this data stream with a 10 ms frame shift. At time t=1.52 s, corresponding to the 152nd analysis frame, the fundamental frequency of that frame is calculated using the YIN algorithm. It is 215 Hz. And in the previous frame, at t=1.51 s, the fundamental frequency of the 151st analysis frame... The frequency is 210Hz. According to the formula for calculating the rate of change of pitch, Hz / s. Simultaneously, calculations show that the variance of the first resonance peak F1 frequency is 1200 Hz within the time window from t=1.40 s to t=1.60 s. 2The local acceleration of the speech rate is -15 syllables / s². These values ​​collectively constitute the acoustic feature vector of the 152nd analysis frame. Simultaneously, the automatic speech recognition engine transcribes this 2-second audio stream and outputs streaming text data, such as {(token:"cost", start: 1.48s, end: 1.85s)}. Finally, based on the timestamp, the acoustic feature vector [500, 1200, -15] of the 152nd frame is time-aligned with the text "cost," completing the data preparation for this step. The temporal acoustic feature sequence and streaming text data are then fed into the next processing step.

[0072] In one embodiment of the present invention, the acoustic baseline and anomaly detection module is used to perform the following operations:

[0073] A temporal acoustic feature sequence of a preset duration before the call is selected as the sample set. The mean vector and covariance matrix of each dimension of the feature are calculated, and a multidimensional Gaussian distribution model is constructed as the individual acoustic normal baseline. A sliding window mechanism is used to extract the current temporal acoustic feature sequence, and the Mahalanobis distance between the instantaneous feature vector within the window and the center of the individual acoustic normal baseline is calculated. The calculated Mahalanobis distance is compared with a dynamic adaptive threshold. If the Mahalanobis distance exceeds the dynamic adaptive threshold, it is determined that there is an acoustic abnormal fluctuation within the current sliding window.

[0074] Specifically, the process begins with the baseline construction phase, which is executed automatically at the start of the call. It should be understood that the initial phase is preferably set to the effective voice duration between 15 and 45 seconds of the first 15 seconds of the call. This range is chosen because it ensures sufficient data samples are collected to obtain stable statistical characteristics while minimizing the possibility of emotional fluctuations that might occur after the call enters substantive business discussions, thus maintaining the purity of the baseline. A set of feature vectors is cached, accumulating to the preset effective voice duration from the beginning of the temporal acoustic feature sequence. For example, it continuously collects data until 30 seconds of effective customer speech are accumulated. If the total call duration is less than 30 seconds, the system's preset global average baseline is used instead of the personalized baseline to ensure system availability. This set of feature vectors is organized into an N×M sample matrix, where N is the number of analysis frames and M is the dimension of the acoustic features.

[0075] Based on this sample matrix, the mean vector of the M-dimensional features is estimated using standard statistical calculation methods. and the covariance matrix of M×M These two parameters together define a multidimensional Gaussian distribution model, which is established as an individual acoustic baseline characterizing the customer's normal, stable speaking state during this call. This baseline is essentially a mathematical model that captures the customer's average acoustic performance in an emotionally neutral state and the fluctuations and correlations between its various characteristic dimensions.

[0076] After the baseline is established, a seamless transition to the real-time monitoring phase is achieved. In this phase, a fixed-length sliding window mechanism is used to continuously process subsequently received temporal acoustic feature sequences. In this embodiment, the width of the sliding window is set to 300 ms to 800 ms to strike a balance between smoothing transient noise and maintaining sensitivity to rapid emotional changes. For each time window, the mean of all feature vectors within the window is calculated to obtain the instantaneous feature vector representing the acoustic performance of the current time period. Next, using the established individual acoustic baseline, instantaneous eigenvectors are calculated. The Mahalanobis distance between the current customer's vocal pattern and the center of the baseline Gaussian distribution. This distance quantifies how much the current customer's vocal pattern deviates from their personal norm.

[0077] Finally, the calculated Mahalanobis distance is compared in real time with the dynamic adaptive threshold. It is worth noting that the setting of this dynamic adaptive threshold is directly related to the individual acoustic baseline. When constructing the baseline, the Mahalanobis distance distribution from all data points in the baseline sample set to their own centers is calculated, and the 98th percentile value of this distribution is used, or... The established statistical boundaries are set as thresholds for anomaly detection. This approach ensures that the threshold adapts to the dispersion of different customer baseline models, making subsequent anomaly detection statistically robust. If the Mahalanobis distance exceeds this threshold, an acoustic anomaly fluctuation is determined to exist within the current time window, and the center timestamp of that window is recorded. This determination serves as a trigger signal to initiate subsequent deep semantic analysis.

[0078] Calculate the Mahalanobis distance for the deviation of the current temporal acoustic feature sequence. The formula is as follows:

[0079]

[0080] in, instantaneous feature vector The Mahalanobis distance is a dimensionless scalar value. To prevent errors in calculation... Irreversible; typically, a small perturbation value is added to the diagonal. , such as 10 -5 . It is the mean vector calculated from the temporal acoustic feature sequence within the current sliding window. It is the mean vector obtained from the individual acoustic normal baseline. It is the covariance matrix obtained from the individual acoustic normal baseline. Denotes the inverse of the covariance matrix. This represents the transpose operation of a vector.

[0081] For example, the processing of the temporal acoustic feature sequence begins. First, 3000 three-dimensional feature vectors corresponding to the customer's valid speech in the first 30 seconds of the call are collected as baseline samples. After calculation, the parameters of the individual acoustic normal baseline are obtained, assuming the mean vector μ is [20, 800, 5] and the covariance matrix Σ is [[150, 100, -10], [100, 40000, 50], [-10, 50, 25]]. Simultaneously, the Mahalanobis distance of these 3000 samples is calculated, and the 98th percentile value of 2.5 is set as the dynamic adaptive threshold for this call. Next, at the 55th second of the call, a 500ms sliding window centered at t=55.2s is processed, and the average feature vector within this window is calculated. The value is [55, 1350, -5]. Then, Substituting μ and Σ into the Mahalanobis distance formula, we calculate ([55,1350,-5]-[20,800,5]) and the inverse of the covariance matrix to obtain the Mahalanobis distance. The calculated result is 3.8. Finally, a comparison is made. Since the calculated distance of 3.8 is greater than the preset threshold of 2.5, it is determined that there is a significant acoustic anomaly fluctuation within the time window around t=55.2 s, and an event record containing this timestamp is generated for subsequent processing.

[0082] In one embodiment of the present invention, the semantic bias instruction generation module is used to perform the following steps:

[0083] The system performs feature clustering on locked acoustic anomalies to identify corresponding abnormal patterns of emotional agitation, hesitation, pauses, or cognitive overload. It then uses the identified abnormal patterns to retrieve a pre-defined acoustic-semantic mapping table to determine the corresponding semantic parsing bias type. Finally, it encapsulates the semantic parsing bias type and the time anchor point of the acoustic anomaly occurrence to generate semantic parsing bias instructions to guide the direction of subsequent semantic analysis.

[0084] Specifically, the temporal acoustic feature sequences within the time window associated with abnormal acoustic fluctuations are first extracted as independent anomalous feature segments. Next, a pre-trained feature clustering model is invoked to perform pattern recognition on these anomalous feature segments, such as K-means clustering or a Gaussian mixture model. This clustering model is obtained through unsupervised or semi-supervised learning on thousands of hours of collection call recordings labeled with emotional states, ensuring the objectivity and business relevance of the classification. Several cluster centers are predefined within the model; for example, K=5, corresponding to: extreme anger, anxiety masking, hesitation, mechanical repetition, and abnormal silence. Each cluster center represents a typical acoustic anomaly pattern, i.e., a qualitative classification of a customer's abnormal vocalization at a specific time point. The anomalous feature segments, typically in the form of a mean vector, are input into the clustering model. The model identifies the specific abnormal pattern of the acoustic anomaly by calculating the distance between the input vector and each preset cluster center, such as Euclidean distance, and classifying the input vector into the cluster represented by the nearest cluster center. For example, the emotional excitement pattern is usually manifested by the pitch change rate and frequency domain formant stability values ​​both significantly exceeding the baseline, while the cognitive overload pattern may be manifested by negative local acceleration of speech rate and a significant increase in the duration of silence.

[0085] After identifying anomaly patterns, a pre-defined acoustic-semantic mapping table is queried. This mapping table is constructed based on psychological theories and the knowledge of experts in the debt collection field, aiming to establish a bridge between purely acoustic behavioral patterns and potential customer psychological intentions. The acoustic-semantic mapping table is a database or configuration file in key-value pair form, where the key is a unique identifier of the anomaly pattern output by the clustering model, and the value is a predefined semantic parsing bias type. The semantic parsing bias type is a set of predefined enumerated values, each indicating a specific semantic category that the semantic analysis model should focus on. For example, the identifier for the emotional agitation pattern might map to a high-arousal anxiety bias type, indicating that the model should focus on the semantics of words related to "difficulty," "urgent," and "uncontrollable," while the hesitation and pause pattern might map to a non-deterministic evasion bias type.

[0086] Using the identified anomaly pattern identifier as the query key, the corresponding semantic parsing bias type is retrieved from the acoustic-semantic mapping table. Finally, the retrieved semantic parsing bias type is encapsulated with the time anchor point of the acoustic anomaly fluctuation that triggered the analysis—that is, the center timestamp of the anomaly occurrence time window. These two pieces of information are combined into a structured data object, namely the semantic parsing bias instruction. This instruction is the core output of this step, and its structure typically includes a timestamp field and a bias type encoding field, ensuring the accuracy and unambiguity of information transmission. This instruction is serialized into a format such as JSON or XML and sent to the downstream semantic analysis module via an internal message queue or direct function call, providing clear directional guidance for its subsequent analysis work.

[0087] For example, following the output of the previous step, we receive the acoustic anomaly event detected at t=55.2 s, and the mean feature vector [450, 1100, -10] corresponding to this time window. This vector is input into a pre-trained K-means clustering model. Assume the model pre-sets three cluster centers, each representing one of the three anomaly modes. The coordinates of cluster center C1 ("emotional agitation") are [500, 1000, 0], the coordinates of cluster center C2 ("hesitation and pause") are [10, 500, -25], and the coordinates of cluster center C3 ("smooth statement") are [20, 800, 5]. We calculate the Euclidean distance from the input vector [450, 1100, -10] to these three centers. The calculation shows that the distance to C1 is the smallest. Therefore, this acoustic anomaly is identified as the "emotional agitation" mode. Subsequently, the identifier of the "emotional agitation" mode is used to query the pre-set acoustic-semantic mapping table. The table reveals the following mapping: the "emotional agitation" pattern corresponds to the "high arousal anxiety bias" type. Finally, this information is encapsulated to generate a structured semantic parsing bias instruction, which can be represented as {timestamp:55.2, bias_type:"HIGH_AWAKENING_ANXIETY"}. This instruction is then sent to the semantic understanding model, instructing it to pay special attention to semantic content related to high anxiety when processing text near t=55.2s.

[0088] In one embodiment of the present invention, the semantic feature extraction module is used to perform the following steps:

[0089] The semantic parsing bias instruction is parsed, and the specific semantic vector clusters associated with the semantic parsing bias type are located by querying the internal mapping table; an attention bias mask is generated, and the attention weight coefficients for specific semantic vector clusters in the attention layer of the semantic understanding model are dynamically increased, so that the model enters a biased activation state; according to the time anchor in the semantic parsing bias instruction, a dynamic text window containing the context is extracted from the streaming text data and input into the semantic understanding model after state reconstruction for feature calculation.

[0090] Specifically, after generating and sending semantic parsing bias instructions, the semantic understanding model deployed in the system acts as the receiver and executor of these instructions. The semantic understanding model is a deep neural network trained on massive amounts of general text and text specific to business domains, such as BERT or RoBERTa based on the Transformer architecture, possessing powerful contextual understanding capabilities. Upon receiving the semantic parsing bias instructions, it first parses them, extracting key bias type codes and temporal anchors. Based on these bias type codes, the model queries its internally pre-built semantic vector mapping table. This mapping table associates each bias type with a specific set of semantic vector clusters in the model's vocabulary. It should be understood that a specific semantic vector cluster is a set of words that are predefined based on business knowledge and are strongly related to a certain customer's psychological state or intention. For example, for a high arousal anxiety bias, its vector cluster may contain vectors of words such as "urgent", "what to do", "trouble", and "immediately". In order to ensure the completeness of the vector cluster, this embodiment can use the cosine similarity of the word embedding space for expansion: select a number of seed words, calculate the average cosine similarity between other words in the vocabulary and the seed words, and if the similarity is greater than a preset threshold, such as 0.75, then the word is also automatically included in the vector cluster of this bias type.

[0091] Next, the model enters the internal state reconstruction phase. In this phase, the model does not change its inherent network parameters, but dynamically adjusts the computational logic in its attention layer. Internal attention weights refer to the quantified representation of the importance of each element to other elements, dynamically calculated by the model when processing sequence data. Specifically, the model generates an attention bias mask with the same length as the input sequence. For words belonging to a specific semantic vector cluster in the sequence, the mask value at the corresponding position is a preset attention weight coefficient; for other words, the mask value is zero. The attention weight coefficient is a hyperparameter controlling the intensity of biased guidance. The value typically ranges from 0.5 to 2.0, with a recommended value of 1.0, which indicates doubling or significantly enhancing the attention score. This value is determined through experimental tuning on the validation set.

[0092] In the standard self-attention score calculation process, this bias mask is added to the attention score matrix, thereby systematically improving the attention scores of words in specific semantic vector clusters before Softmax normalization. This allows the model to pay more attention to these biased semantic information in subsequent context representation learning, which is the model entering a preset biased activation state. Simultaneously, based on the time anchor in the semantic parsing bias instruction, a dynamic text window containing the preceding and following context is extracted from the streaming text data. In this embodiment, the range of this dynamic text window is set to -3s to +5s centered on the time anchor. This asymmetric setting is based on the empirical observation that customer acoustic anomalies usually occur before their expression of key semantics, i.e., acoustics precedes semantics, and the 5s post-window can cover a complete short sentence at a normal speaking speed. Finally, this extracted dynamic text window is fed as input into the semantic understanding model that has completed state reconstruction, performing biased feature extraction.

[0093] When reconstructing attention weights, the corrected attention logic score The calculation can be expressed as:

[0094]

[0095] in, It is the first in the input sequence The word pair The corrected attention logic score for each lexical unit. It is the first A query vector of 1 word elements It is the first The key vector of each word. It represents the dimension of the key vector, used for scaling. It focuses on the weighting coefficients, which are positive real numbers. It is a binary mask value, when the first When a word element belongs to a specific semantic vector cluster activated by the current semantic parsing bias instruction... The value is 1 if it is 1, otherwise it is 0.

[0096] For example, following the previous step, the semantic understanding model receives a semantic parsing bias instruction with the content {timestamp:55.2, bias_type:"HIGH_AWAKENING_ANXIETY"}. First, the model parses the bias type as "high arousal anxiety bias". Next, the model queries its internal mapping table to locate a specific semantic vector cluster associated with this type, which contains indices of words such as "difficulty", "fee", and "what to do". From the streaming text data, a dynamic text window centered at t=55.2s is extracted, yielding the text content "I'm having trouble with my bills next month, what should I do about this fee?" The model then prepares to process this text window. When calculating the contextual representation of the word "fee" in the text, it needs to calculate its attention weight to all other words within the window. Assume that in the original model, the attention score of "fee" to "what to do" is 0.8. After bias reconstruction, since "what to do" belongs to the activated specific semantic vector cluster, its mask value... The value is 1. Assume we are concerned with the weighting coefficient. The preset value is 1.2, so the corrected attention score is: In this way, the model pays more attention to the word "what to do," which expresses anxiety and a need for help. As a result, the semantic components related to anxiety and the focus of the problem are strengthened in the final text feature vector, thus completing the biased feature extraction.

[0097] In one embodiment of the present invention, a risk scoring module is intended to perform the following steps:

[0098] The text feature vector is input into the classification layer, and the output intent classification probability distribution is obtained. The category corresponding to the highest probability is selected as the true intent. The Mahalanobis distance corresponding to the acoustic anomaly fluctuation that triggers the analysis is used as the anomaly intensity value. The anomaly intensity value is mapped to the risk multiplier through the intensity scaling function. The single-point risk score is calculated using the preset benchmark risk coefficient and risk multiplier. The score and the true intent are encapsulated into the intent verification result.

[0099] Specifically, the biased semantic understanding model processes the dynamic text window and outputs a text feature vector reinforced with specific semantic cues. This step utilizes this text feature vector to complete the final intent determination. First, the text feature vector is input into the fully connected classification layer at the end of the model. This classification layer is trained to map the input vector to a predefined intent category space. Through the Softmax activation function, the classification layer outputs a probability distribution vector, where each dimension corresponds to the probability of a customer's true intent category. The set of true intent categories is predefined based on typical customer reaction patterns in debt collection scenarios, such as genuine difficulty, excuses for delay, or malicious resistance, and is trained and validated using a large amount of real call data. The dimension with the highest probability value in this probability distribution vector is selected, and its corresponding intent category is taken as the customer's true intent at that point in time.

[0100] Next, to quantify the credibility and severity of this intent, the monitoring results from the acoustic baseline and anomaly detection module are traced back to retrieve the original acoustic anomaly intensity value that triggered this analysis, i.e., the calculated Mahalanobis distance. Then, the determined true intent is combined with this anomaly intensity value. A single-point risk score is calculated, which is a quantitative measure of the risk exhibited by the customer at a specific dialogue node, and its value is normalized to, for example, between 0 and 100.

[0101] Finally, all the key information from this analysis, including the time anchor, the original text of the input dynamic text window, the determined true intent, the probability distribution of each intent, and the calculated individual risk scores, are packaged into a structured data record, namely the intent verification result. This result is stored and passed to subsequent aggregation analysis steps as an independent evidence point for assessing the overall customer status.

[0102] This step is used to calculate the single-point risk score. The formula can be expressed as:

[0103]

[0104] in, This is the final generated single-point risk score, which is usually a standardized value. Generally, if the calculated result exceeds 100, it is truncated to 100. Is it consistent with the determined true intention? The associated baseline risk coefficient is obtained from a pre-set risk rule base and is set by business experts based on the potential risk level of different intentions. For example, the baseline coefficient for "malicious confrontation" will be much higher than that for "real difficulty". The Mahalanobis distance value that triggered this analysis was obtained from the acoustic baseline and anomaly detection module. This is the intensity scaling function, used to map the original Mahalanobis distance values ​​to a standardized risk multiplier. Specifically, this function employs a threshold-based linear growth model, as shown in the following formula: in, For dynamic adaptive threshold, This is the scaling sensitivity constant, which defaults to 1. This function ensures that the risk multiplier is 1 when the anomaly intensity is exactly equal to the threshold, and increases linearly with the degree of deviation, thereby achieving quantitative amplification of risk. Its parameters are determined based on the statistical distribution analysis of Mahalanobis distance values ​​in historical data. The purpose is to convert unevenly distributed Mahalanobis distance values ​​into a more meaningful and smoothly changing multiplier for risk assessment.

[0105] For example, following the previous step, the model outputs a text feature vector for the text "I'm really having trouble with my bill next month, what should I do about this expense?", weighted by "high arousal anxiety bias". This vector is then fed into the Softmax classification layer, and the model outputs an intent classification probability distribution of {Genuine difficulty: 0.85, Excuses for delay: 0.10, Malicious resistance: 0.05}. Since "Genuine difficulty" has the highest probability of 0.85, the customer's genuine intent is determined as "Genuine difficulty". Subsequently, the acoustic baseline and anomaly detection module is consulted, and the Mahalanobis distance value triggering this analysis is found to be 3.8, with a threshold of 2.5. The risk rule base is then queried to obtain the baseline risk coefficient corresponding to the "Genuine difficulty" intent. The value is 20. The intensity scaling function used is the linear model described above, and the sensitivity constant is set to 20. ,Right now Substitute the numerical values ​​to calculate the risk multiplier. Ultimately, the single-point risk score was calculated as follows: This information is integrated to generate the intent verification result for the dialogue node, which includes a timestamp of 55.2 seconds, a true intent determination of "real difficulty", and a single-point risk score of 30.4. This result is then archived for the final comprehensive profile generation.

[0106] In one embodiment of the present invention, the comprehensive portrait generation module is used to perform the following steps:

[0107] All intent verification results triggered during the call are sequentially assembled to construct a time-series graph reflecting the evolution of the customer's psychological state. Combined with the business rule base, high-risk nodes in the time-series graph are identified and single-point risk scores are weighted and accumulated to calculate the customer's comprehensive repayment willingness index and potential loss of contact risk value. Based on the numerical range of the comprehensive repayment willingness index and potential loss of contact risk value, a comprehensive compliance profile including rating conclusions is output.

[0108] Specifically, after the entire call process concludes or reaches the preset analysis endpoint, the final comprehensive evaluation phase begins. The main execution unit is the customer profile generation module. This module first collects all intent verification results generated by the aforementioned intent verification and risk scoring modules throughout the entire call. These results are arranged strictly in chronological order according to their respective time anchors and pieced together into a time-series sequence. This sequence constitutes a time-series graph of the customer's psychological state evolution. It is a data visualization or structured representation that clearly shows the customer's emotional nodes and intent turning points during the call, recording at which points in time the customer experienced emotional or intent fluctuations, as well as the specific content, intensity, and analysis conclusions of each fluctuation.

[0109] Next, a pre-built business rule base is loaded. This rule base, jointly developed by financial risk control experts and debt collection experts, transforms abstract business logic into executable calculation rules, which is key to achieving automated and standardized assessments. The rule base associates specific intent verification results with business risk indicators and defines different weights for different types of risk nodes. Iterating through each intent verification result in the time-series graph, for records marked as high-risk nodes by the business rule base, such as nodes judged as "malicious resistance" or "excuses for delay," their single-point risk scores are weighted and accumulated.

[0110] In this way, two core comprehensive indicators are calculated: the customer's overall repayment willingness index and the potential loss of contact risk value. The overall repayment willingness index is a quantitative indicator used to predict the subjective likelihood of the customer making timely repayments in the future. It is calculated by adding points to nodes expressing positive willingness and deducting points to nodes expressing negative willingness. The potential loss of contact risk value is used to assess the probability that the customer may subsequently break contact or evade debt. It is obtained by accumulating risk scores related to intentions such as "evasion" and "threat of loss of contact," and the value usually ranges from 0 to 100.

[0111] Finally, based on the calculated comprehensive repayment willingness index and potential loss of contact risk value, customers are categorized into a pre-set rating system, generating a comprehensive compliance profile. This comprehensive compliance profile is the final evaluation report of the customer's overall creditworthiness and cooperation demonstrated during the collection call. The rating system typically includes four levels: Excellent, Good, Poor, and Very Poor, each corresponding to a specific numerical range. For example, a customer with a comprehensive repayment willingness index greater than 80 and a potential loss of contact risk value less than 10 is rated Excellent, while a customer with an index less than 30 or a risk value greater than 60 is rated Very Poor. The final comprehensive compliance profile is then generated and, along with the original indicator values ​​and rating conclusions, output to a quality control report or directly pushed to the collection strategy decision-making system to guide subsequent collection actions.

[0112] Overall repayment willingness index The calculation can be simplified to the following form:

[0113]

[0114] Potential risk of losing contact The calculation can be expressed as:

[0115]

[0116] in, This represents the overall willingness to repay loans index. This is the basic willingness score set by the system for each customer, usually 50. It is the first A single-point risk score for the result of an intent verification. Is with the first The weight coefficients corresponding to the true intentions of each node are positive for positive intentions and negative for negative intentions. It represents the total number of intent verification results during the call. This represents the potential risk of losing contact. It is the first A single-point risk score for a node identified by the business rule base as being related to the risk of losing contact. That is the corresponding risk weighting coefficient. It is the total number of nodes associated with the risk of losing contact.

[0117] For example, suppose that in a complete collection call, three intent verification results are generated. The first node occurs at t=55.2 s, judged as "genuine hardship," with a single-point risk score of 30.4. The second node occurs at t=183.5 s, judged as "excuses for delay," with a single-point risk score of 55.0. The third node occurs at t=241.1 s, judged as "commitment to repayment," with a single-point risk score of 15.0. Loading the business rule base, the intent weights for "genuine hardship" are found to be -0.5, "excuses for delay" -1.5, and "commitment to repayment" +2.0. Simultaneously, the rule base indicates that "excuses for delay" belongs to the node related to the risk of being out of contact, with a risk weighting coefficient of 1.2. The calculation of the comprehensive repayment intent index begins. Potential risk of losing contact Finally, the rating is mapped according to the rating rules. Since the customer's overall repayment willingness index is -17.7, which is far below 30, and its potential risk of being out of contact is 66.0, which is higher than 60, the customer's overall compliance profile is ultimately rated as poor, and a report containing detailed indicators and rating conclusions is generated.

[0118] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A business call compliance intelligent quality inspection system based on key semantic anchor points, characterized in that: include: The data acquisition and preprocessing module acquires dual-track recording data of the target business call, separates the customer-side audio stream to extract the temporal acoustic feature sequence, and converts the customer-side audio stream into streaming text data. The acoustic baseline and anomaly detection module constructs an individual acoustic baseline for each customer based on the feature distribution at the initial stage of a call, and monitors in real time the deviation of the temporal acoustic feature sequence from the individual acoustic baseline in order to identify abnormal acoustic fluctuations. The semantic bias instruction generation module, in response to locked acoustic anomaly fluctuations, generates semantic parsing bias instructions containing anomaly type encoding and time anchors. The semantic feature extraction module dynamically reconstructs the internal attention weights of the semantic understanding model using semantic parsing bias instructions, performs biased feature extraction on the streaming text data with associated time anchors, and outputs text feature vectors. The intent verification risk scoring module outputs intent verification results based on text feature vectors and generates a single-point risk score by combining the intensity of acoustic anomaly fluctuations. The comprehensive profile generation module aggregates multiple intent verification results from the entire call process to generate a comprehensive compliance profile of the customer.

2. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 1, characterized in that, The data acquisition and preprocessing module is used to perform the following steps: The system separates the agent-side and customer-side audio channels in dual-track recording data in real time, marks the silent segments and performs noise reduction on the customer-side audio channel, and obtains the noise-reduced customer voice data stream. The noise-reduced customer voice data stream is segmented into frames, and multi-dimensional features including pitch change rate, frequency domain formant stability and local speech rate acceleration are calculated and arranged in time order to form a temporal acoustic feature sequence. The speech recognition engine is invoked to transcribe the noise-reduced customer voice data stream into text, and a time-aligned mapping between the streaming text data and the temporal acoustic feature sequence is established based on the timestamp index.

3. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 1, characterized in that, The acoustic baseline and anomaly detection module is used to perform the following operations: The temporal acoustic feature sequence with a preset duration before the call is selected as the sample set. The mean vector and covariance matrix of each dimension feature are calculated, and a multidimensional Gaussian distribution model is constructed as the individual acoustic normal baseline. A sliding window mechanism is used to extract the current temporal acoustic feature sequence, and the Mahalanobis distance between the instantaneous feature vector within the window and the center of the individual acoustic normal baseline is calculated. The calculated Mahalanobis distance is compared with the dynamic adaptive threshold. If the Mahalanobis distance exceeds the dynamic adaptive threshold, it is determined that there is an acoustic abnormal fluctuation within the current sliding window.

4. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 1, characterized in that, The semantic bias instruction generation module is used to perform the following steps: Feature clustering is performed on locked acoustic anomalous fluctuations to identify corresponding anomalous patterns of emotional agitation, hesitation, pauses, or cognitive overload. The identified abnormal patterns are used to retrieve a pre-set acoustic-semantic mapping table to determine the corresponding semantic parsing bias type; It encapsulates the semantic parsing bias type and the time anchor point of acoustic anomaly fluctuations, and generates semantic parsing bias instructions to guide the direction of subsequent semantic analysis.

5. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 1, characterized in that, The semantic feature extraction module is used to perform the following steps: The semantic parsing bias instruction is parsed, and the specific semantic vector cluster associated with the semantic parsing bias type is located by querying the internal mapping table; Generate an attention bias mask and dynamically increase the attention weight coefficients for specific semantic vector clusters in the attention layer of the semantic understanding model, so that the model enters a biased activation state. Based on the time anchor points in the semantic parsing bias instructions, dynamic text windows containing the preceding and following context are extracted from the streaming text data and input into the semantic understanding model after state reconstruction for feature calculation.

6. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 1, characterized in that, The intent verification risk scoring module is used to perform the following steps: Input the text feature vector into the classification layer, output the intent classification probability distribution, and select the category corresponding to the highest probability as the true intent; The Mahalanobis distance corresponding to the acoustic anomaly fluctuations triggered by retrospective analysis is used as the anomaly intensity value, and the anomaly intensity value is mapped to a risk multiplier through an intensity scaling function. A single-point risk score is calculated using a preset baseline risk coefficient and risk multiplier, and this score, along with the true intent, is encapsulated as an intent verification result.

7. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 1, characterized in that, The comprehensive portrait generation module is used to perform the following steps: All intent verification results triggered during the call are serialized and spliced ​​together in chronological order to construct a time-series graph reflecting the evolution of the customer's psychological state. By combining the business rule base, high-risk nodes in the time series graph are identified and single-point risk scores are weighted and accumulated to calculate the customer's comprehensive repayment willingness index and potential loss of contact risk value; Based on the numerical range of the comprehensive repayment willingness index and the potential loss of contact risk value, a comprehensive compliance profile including the rating conclusion is output.

8. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 2, characterized in that, Methods for calculating multidimensional features include: The pitch change rate is calculated by dividing the difference between the fundamental frequency of the current frame and the previous frame by the frame shift. The stability of frequency domain resonant peaks is obtained by calculating the variance of the center frequencies of the first three resonant peaks within a certain time window; The local acceleration of speech rate is obtained by calculating the first and second derivatives of the number of syllables per unit time.

9. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 3, characterized in that, The dynamic adaptive threshold is set according to the preset percentile value of the Mahalanobis distance distribution of all data points in the sample set of the individual acoustic normal baseline. The preset duration is the effective voice duration at the beginning of the call. If the total call duration is less than the preset duration, the system's preset global average baseline will be used to replace the individual acoustic normal baseline.

10. The intelligent quality inspection system for business call compliance based on key semantic anchor points as described in claim 4, characterized in that, The acoustic-semantic mapping table establishes the association between anomalous patterns at the acoustic level and parsing bias types at the semantic level; wherein, the association includes at least: mapping emotional arousal patterns to high arousal anxiety bias types, instructing the model to focus on words expressing urgency or difficulty; mapping hesitant pause patterns to nondeterministic evasion bias types, instructing the model to focus on words expressing uncertainty or ambiguity.

Images