Call compliance detection method, device, equipment, medium and product

Abstract

The invention discloses a call compliance detection method, device and equipment, a medium and a product. The method comprises the following steps: extracting acoustic features according to collected call data; performing voice conversion and semantic analysis on the call data to obtain semantic features; performing cross-modal time sequence alignment on the acoustic features and the semantic features, and performing feature fusion to obtain a joint representation matrix; calculating an emotion energy index according to the joint representation matrix and a preset emotion energy model; and determining a blocking strategy according to the emotion energy index. According to the scheme, the precision of semantic compliance detection can be improved, and the false alarm rate and the missing report rate are reduced.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and more specifically to a call compliance detection method, apparatus, device, medium, and product. Background Technology

[0002] In today's ever-evolving market, despite the increasing diversity of communication methods, telephone communication remains an indispensable tool in the procurement process. However, in scenarios with extremely high compliance requirements, such as procurement, finance, and customer service, regulating telephone calls in compliance has always been a challenge.

[0003] In existing technologies, there are two main implementation schemes for call compliance supervision: First, a series of sensitive words, prohibited words, or specific phrases are preset in the text stream output from the speech-to-text converter. Once any preset phrase is identified in the call text, a blocking or warning mechanism is immediately triggered. Second, the acoustic characteristics of the speech are analyzed to identify the emotional state of the call participants. When a specific emotion is identified, a blocking or warning mechanism is triggered.

[0004] The aforementioned single-modal recognition methods using text semantics or emotion recognition have limitations. Even when using multimodal fusion, there are problems such as information redundancy, omission of key information, or simple fusion strategies. It is also difficult to capture the dynamic relationship between acoustic and textual information in complex situations and its profound impact on compliance risks. The false positive rate and false negative rate are high, and the accuracy of semantic compliance detection is low. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a call compliance detection method, apparatus, device, medium, and product that can improve the accuracy of semantic compliance detection and reduce false alarm and false negative rates.

[0006] This invention provides a call compliance detection method, including: Acoustic features are extracted from the collected call data; The call data is subjected to speech conversion and semantic analysis to obtain semantic features; The acoustic features and semantic features are cross-modal temporally aligned and then fused to obtain a joint representation matrix. The emotional energy index is calculated based on the joint representation matrix and the preset emotional energy model. The blocking strategy is determined based on the emotional energy index.

[0007] Preferably, the step of extracting acoustic features based on the collected call data includes: The call voice stream of the device to be monitored is acquired in the form of voice packets and used as the call data. The call audio stream is segmented into frames to obtain different audio frames; Different speech frames are standardized to generate a normalized acoustic feature sequence, thus obtaining the acoustic features.

[0008] Preferably, before performing frame segmentation processing on the call audio stream, the method further includes: The call audio stream is preprocessed, including echo cancellation and background noise suppression.

[0009] Preferably, the step of performing frame segmentation on the call audio stream to obtain different audio frames includes: The call audio stream is divided into frames according to a preset audio length and a preset overlap between adjacent frames to obtain different audio frames.

[0010] Preferably, before standardizing different speech frames, the method further includes: Acoustic gain compensation is performed on different voice frames according to the device type of the device to be monitored. By monitoring the current ambient noise, environmental noise correction is performed on different speech frames.

[0011] Preferably, the step of performing acoustic gain compensation on different speech frames according to the device type of the device to be monitored includes: Identify the device type of the device to be monitored; Based on the device type, match the corresponding compensated acoustic features in a pre-trained database of typical acoustic fingerprints of different acquisition devices; Acoustic gain compensation is performed on different speech frames based on the compensated acoustic features.

[0012] Preferably, the step of correcting the environmental noise of different speech frames by monitoring the current environmental noise includes: Continuously monitor non-speech segments and / or low-volume segments during the call to estimate the spectral density and power spectral density of the current ambient noise; Based on the spectral density and the power spectral density, noise suppression techniques are used to correct the acoustic features of different speech frames in real time.

[0013] Preferably, the acoustic feature sequence includes at least one of fundamental frequency jitter deviation, energy entropy, Mel frequency cepstral coefficient, speech rate, and a specific harmonic ratio.

[0014] Preferably, the standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The Z-score normalization method is used for different speech frames to calculate the degree of deviation of the fundamental frequency jitter, and the degree of fundamental frequency jitter deviation is obtained. The degree of base frequency jitter deviation ; in, This refers to the fundamental frequency jitter value obtained by real-time measurement of different speech frames; This is the baseline average value of baseband jitter, obtained through statistical analysis of a large amount of normal call data. This represents the standard deviation of the normal fluctuation range.

[0015] Preferably, the standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The energy entropy was calculated using the Z-score normalization method for different speech frames; The energy entropy ; in, The energy entropy value is obtained by real-time measurement of different speech frames; This represents the baseline mean of the average energy distribution complexity. The standard deviation is used to define the critical threshold for energy mutation.

[0016] Preferably, the standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The spectral envelope information of different speech frames is detected, and the Mel frequency cepstral coefficients are calculated as feature values ​​of the acoustic feature sequence.

[0017] Preferably, the standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The number of speech words or syllables per unit time in different speech frames is detected, and the speech rate is calculated as the feature value of the acoustic feature sequence.

[0018] Preferably, the standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The proportion of harmonics in a specific frequency range in different speech frames is detected, and a specific harmonic ratio is calculated as the feature value of the acoustic feature sequence.

[0019] Preferably, the step of performing speech conversion and semantic analysis on the call data to obtain semantic features includes: The voice stream in the call data is semantically transcribed to obtain a text sequence; Sentiment semantic analysis was performed on the text sequence to obtain the analysis results; The analysis results are then transformed into semantic features.

[0020] Preferably, the step of semantically transcribing the stream of call data to obtain a text sequence includes: The call audio stream in the call data is cut into different audio frames according to a preset frame length, and a preset number of Mel spectrum features are extracted from each frame. The Mel spectrum features extracted are processed frame by frame by an encoder composed of multiple unidirectional LSTMs, and the acoustic features of each frame are output. The language features of the call voice stream in the call data are extracted using a preset speech recognition model. The text sequence is obtained by calculating the word probability distribution based on the extracted linguistic features and the acoustic features using a pre-defined joint network.

[0021] Preferably, the step of performing sentiment semantic analysis on the text sequence to obtain the analysis results includes: Based on a pre-built emotional semantic dictionary and risk word database, different emotional scenarios and risk words are identified accordingly; Determine the emotional intensity and risk level labels corresponding to the emotional scenarios and the risk words; The emotional tendency score is determined based on the identified emotional scenarios, risky words, and corresponding emotional intensity and risk level labels; The identified emotional scenarios, risky words, corresponding emotional intensity and risk level labels, and emotional tendency scores are used as the analysis results.

[0022] Preferably, the step of converting the analysis results into semantic features includes: The different feature values ​​in the analysis results are transformed into a semantic feature sequence of text sentiment tendency, which is then used as the semantic feature.

[0023] Preferably, the step of performing cross-modal temporal alignment of the acoustic features and the semantic features, and then performing feature fusion to obtain a joint representation matrix, includes: A dynamic time warping strategy is used to find the optimal nonlinear mapping path between the acoustic features and the semantic features, which serves as the optimal alignment path. Based on the optimal alignment path, the binding relationship between the acoustic features of different frames and the semantic features of different frames is determined, and a mapping relationship is constructed. Based on the constructed mapping relationship, the fusion weights are determined through a cross-modal attention mechanism; Based on the binding relationship between the acoustic features and semantic features of different frames, and the determined fusion weights, a spatiotemporally related joint representation matrix is ​​generated.

[0024] Preferably, the step of using a dynamic time warping strategy to find the optimal nonlinear mapping path between the acoustic features and the semantic features includes: The sequence representation of the acoustic features is divided into different speech frames according to a preset frame length, and Mel-spectral features are extracted from each frame to obtain the dimensional acoustic sequence. Perform deep text analysis on the sequence of semantic features to extract keywords; Each keyword is converted into a multi-dimensional semantic vector using word embedding technology to obtain a text sequence; The dimensionality of the acoustic sequence is reduced to match the embedding dimension of the text sequence, and the Euclidean distance between the acoustic features of different frames and different keywords in the text sequence is calculated. Based on the calculated Euclidean distance, calculate the minimum cumulative distance between each acoustic feature and each keyword; Tracing back from the key point to the first acoustic feature and the first keyword, the neighboring nodes with the smallest cumulative distance are selected to form the optimal nonlinear mapping path.

[0025] Preferably, the step of calculating the emotional energy index based on the joint representation matrix and a preset emotional energy model includes: Identify the stage of the call data; The weighting coefficients of the emotional energy model are determined based on the stage of the call. The emotional energy index is calculated based on the joint representation matrix and the emotional energy model with confirmed weight coefficients.

[0026] Preferably, identifying the stage of the call data includes: Based on the call data, identify call duration, silent ratio, keyword frequency, and completed business process nodes; The call stage is identified based on the call duration, the muting rate, the frequency of keyword occurrence, and the completed business process nodes.

[0027] Preferably, the call phase includes: a needs confirmation phase, a solution submission and clarification phase, a negotiation phase, and a signing and confirmation phase.

[0028] Preferably, the step of basing the emotion energy model includes: ; in, , and These are the weighting coefficients confirmed based on the i-th call phase; The degree of fundamental frequency jitter deviation in the joint characterization matrix. Let S be the energy entropy in the joint characterization matrix, and let S be the sentiment score in the joint characterization matrix.

[0029] Preferably, determining the blocking strategy based on the emotional energy index includes: Identify the stage of the call data; Match the corresponding blocking threshold based on the identified call phase; The blocking strategy is determined based on the emotional energy index and the blocking threshold.

[0030] Preferably, the step of matching the corresponding blocking threshold based on the identified call phase includes: Monitor the rate of change of the emotional energy index over a preset period of time; An adjustment coefficient is determined based on the rate of change, and the blocking threshold is calculated based on the adjustment coefficient and the base threshold of the identified call phase.

[0031] Preferably, determining the blocking strategy based on the emotional energy index and the blocking threshold includes: When the emotional energy index is not greater than the blocking threshold, it is identified as compliant; When the emotional energy index is found to be greater than the blocking threshold, a mild compliance risk is identified, and a compliance reminder is sent to both parties in the call. Real-time acquisition of call data and continuous monitoring of emotional energy index; When the monitored emotional energy index is not lower than the blocking threshold within the preset monitoring period, or when the rate of decline of the monitored emotional energy index is not less than the preset decay rate threshold, a serious compliance risk is identified, and an early warning notification is sent to the preset compliance monitoring device. When the emotional energy index monitored within the preset monitoring period is not lower than the preset circuit breaker threshold, a serious violation is identified, and the communication link is cut off.

[0032] This invention also provides a call compliance detection device, the device comprising: The acoustic module is used to extract acoustic features based on the collected call data; The semantic module is used to perform speech conversion and semantic analysis on the call data to obtain semantic features; The joint module is used to perform cross-modal temporal alignment of the acoustic features and the semantic features, and to perform feature fusion to obtain a joint representation matrix; The index module is used to calculate the emotional energy index based on the joint representation matrix and the preset emotional energy model. The blocking module is used to determine the blocking strategy based on the emotional energy index.

[0033] This invention also provides a communication device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the call compliance detection method as described in any of the above embodiments.

[0034] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the call compliance detection method as described in any of the above embodiments.

[0035] This invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0036] This invention provides a method, apparatus, device, medium, and product for call compliance detection. The method involves extracting acoustic features from collected call data; performing speech conversion and semantic analysis on the call data to obtain semantic features; aligning the acoustic and semantic features across modal time sequences and fusing them to obtain a joint representation matrix; calculating an emotional energy index based on the joint representation matrix and a preset emotional energy model; and determining a blocking strategy based on the emotional energy index. This solution improves the accuracy of semantic compliance detection and reduces false positive and false negative rates. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the call compliance detection method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the execution process of the blocking strategy provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a call compliance detection device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a communication device provided in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] To address the aforementioned technical issues, this case provides a call compliance detection method, see [link to relevant documentation]. Figure 1 This is a flowchart illustrating the call compliance detection method provided in an embodiment of the present invention. The method includes: Step S1: Extract acoustic features based on the collected call data; Step S2: Perform speech conversion and semantic analysis on the call data to obtain semantic features; Step S3: Perform cross-modal temporal alignment of the acoustic features and the semantic features, and perform feature fusion to obtain a joint representation matrix; Step S4: Calculate the emotional energy index based on the joint representation matrix and the preset emotional energy model; Step S5: Determine the blocking strategy based on the emotional energy index.

[0040] In this specific implementation, based on data acquisition and feature fusion, and through emotion quantification and strategy output, emotion signals are captured through both acoustic and semantic dimensions. After cross-modal alignment and fusion, accurate emotion assessment is achieved, which ultimately drives the execution of the blocking strategy.

[0041] Acoustic features are extracted from the collected call data. The acoustic aspect of the call data includes non-verbal signals of emotion, and physical attributes that reflect emotional fluctuations are extracted.

[0042] For example, the key feature types extracted include: fundamental frequency F0, i.e. pitch, which generally rises when a person is angry; speech rate, which increases when a person is emotionally agitated; energy, which is volume and fluctuation, which fluctuates when a person is anxious; spectral features MFCC and Mel spectrum, which capture timbre changes, which are low when a person is sad; and pause features, i.e. silence duration and pause frequency, which are frequent pauses when a person is nervous.

[0043] By using signal processing algorithms such as short-time Fourier transform and Mel filtering, the original audio waveform is transformed into a structured feature vector, thereby quantifying the physical signal of emotion.

[0044] The call data is subjected to speech-to-speech and semantic analysis to obtain semantic features. The semantic level of the call data includes linguistic signals of emotion, and the emotional tendency in the text content is analyzed.

[0045] Speech-to-text technology converts spoken conversations into text, which requires processing accents, noise, and colloquial expressions.

[0046] Based on pre-trained language models such as BERT and RoBERTa, the text is encoded with emotional semantics, keywords and semantic dependencies (such as conflicting sentence structures) are extracted, and semantic feature vectors are output.

[0047] Capture emotional intent in text through Natural Language Processing (NLP).

[0048] The temporal granularity of acoustic features and semantic features does not match. This application's solution uses the call timeline as a benchmark, employing Dynamic Time Warping (DTW) or attention mechanisms to map semantic features to corresponding acoustic feature segments, ensuring a one-to-one correspondence between the two in the temporal dimension and achieving matching between emotion and speech segments. This allows acoustic features reflecting emotional intensity to complement semantic features reflecting emotional intent, generating a more comprehensive emotional representation.

[0049] By employing concatenation and attention weighting, the aligned acoustic feature vectors and semantic feature vectors are fused into a joint representation matrix. Each element in the matrix simultaneously contains the speech emotion intensity and the corresponding text emotion intent for that time segment, achieving the joint representation of non-verbal and verbal signals.

[0050] Finally, the emotional energy index is calculated, which is a quantitative indicator of the intensity, duration, and potential harm of emotions. Specifically, a pre-set emotional energy model is used, with a joint representation matrix as input. The model learns the mapping relationship between emotional characteristics and intensity in historical data and outputs a quantitative score.

[0051] A threshold triggering mechanism based on the emotional energy index enables tiered blocking. For example, multiple thresholds can be set, each corresponding to a different strategy.

[0052] The blocking strategy can dynamically adjust the threshold according to the actual scenario.

[0053] This application addresses the limitations of single-modal approaches. Traditional methods relying solely on acoustic features are susceptible to environmental noise and accents, while semantic features alone can overlook emotional nuances in speech. By complementing acoustic and semantic features, the accuracy of emotion recognition is improved, particularly in accurately identifying implicit emotions, such as seemingly calm but semantically threatening calls and complex emotions. This proposal achieves real-time, precise early warning and tiered blocking of potential compliance risks through refined extraction of acoustic and textual features, construction of a specific cross-modal temporal correlation fusion model, and calculation of a compliance-oriented emotion energy index. This significantly enhances regulatory efficiency and compliance assurance capabilities in high-risk call scenarios.

[0054] In another embodiment provided by the present invention, the process of extracting acoustic features in step S1 is specifically as follows: Call data access and real-time monitoring: The mobile phone numbers of procurement personnel are strongly linked to the work numbers assigned by the company, ensuring that all calls involving the procurement process can be monitored and collected by the system in real time without omission.

[0055] All call audio streams are transmitted to the backend processing system in real time as voice packets, ensuring the continuity and integrity of the data stream. The system employs efficient data stream transmission protocols and encryption technologies to guarantee the security and real-time nature of call information.

[0056] Subsequently, the cleaned audio signal was subjected to precise frame segmentation processing.

[0057] A series of acoustic features optimized for emotional intensity and risk sensitivity are extracted from each speech frame, and then standardized in a unified manner to eliminate the differences in the units of measurement between different features and the influence of individual speaking habits, thereby generating a normalized and highly comparable acoustic feature sequence.

[0058] In another embodiment of the present invention, during a call, especially a hands-free or speakerphone call, one's own voice is played through the speaker and then re-captured by the other party's microphone, resulting in the other party's voice stream being mixed with one's own echo signal. This echo can mask the other party's true voice characteristics, leading to distortion in subsequent acoustic feature extraction.

[0059] Before performing frame-by-frame processing on the call audio stream, the method further includes a data preprocessing step. This step resolves interference signals in the call audio stream, providing target audio signals for subsequent feature extraction and emotion recognition.

[0060] The voice stream of the call is first preprocessed, including echo cancellation and background noise suppression.

[0061] For example, an adaptive filtering algorithm, such as NLMS or RLS, is used to separate the target speech from the echo signal: the speech signal output by the speaker is obtained, the transmission path from the reference signal to the echo signal is learned through an adaptive filter, and an estimated echo signal that is consistent with the actual echo is simulated and generated; the estimated echo signal is subtracted from the speech stream of the other party, and finally the background noise signal of the target speech is obtained, thus achieving accurate echo cancellation.

[0062] There is a lot of environmental noise in the call scenario. This noise will be superimposed on the target speech, resulting in a decrease in the signal-to-noise ratio (SNR) of the speech signal, which in turn affects the accuracy of semantic feature extraction.

[0063] Noise frequency components are identified and suppressed using spectral subtraction and Wiener filtering. In the silent segments of a call, the spectral characteristics of environmental noise are extracted to establish a noise model. The noisy speech signal is then subjected to spectral analysis. The noise frequency components are identified through the noise model. The noise spectrum is then subtracted by spectral subtraction or enhanced by Wiener filtering to finally output a clean target speech signal with a high signal-to-noise ratio.

[0064] Echoes and noise can cause temporal fluctuations in acoustic features. After preprocessing, the temporal trajectory of acoustic features is smoother, the speech quality is higher, and the temporal matching degree with semantic features is higher.

[0065] In another embodiment of the present invention, when performing frame segmentation processing on the call voice stream, specifically: The call audio stream is divided into frames according to a preset audio length and a preset overlap between adjacent frames, and the cleaned audio signal is subjected to precise frame segmentation processing.

[0066] For example, each speech frame is set to a length of 25 milliseconds (ms), with 50% overlap between adjacent frames. This framing strategy effectively captures the short-term characteristics of the speech signal and ensures the continuity and stability of acoustic features through inter-frame overlap, avoiding information loss and thus obtaining more detailed information.

[0067] In a typical procurement call, the system collects real-time audio signals of user A (the buyer) and user B (the supplier) discussing contract details. Assuming that at a critical moment, the system processes 2 seconds of the call audio, calculating data points with a 25ms frame length and a 50% overlap rate to provide sufficient, fine-grained data for subsequent feature extraction.

[0068] In another embodiment of the present invention, before standardizing different speech frames, in order to improve the system's robustness to different calling devices and complex calling environments, the system also performs adaptive device compensation and environmental noise correction when extracting acoustic features. This solves the problems of inconsistent speech intensity collected by different devices and feature distortion caused by changes in environmental noise over time, further purifying the speech frame data and providing a device-independent, noise-stable unified input for subsequent standardization and feature extraction.

[0069] The method further includes: Acoustic gain compensation is performed on different speech frames according to the device type of the device to be monitored; the interference of device differences is eliminated and the speech energy collected by different devices is unified to a standard scale to avoid misjudgment of emotional intensity due to differences in device hardware. By monitoring the current ambient noise, environmental noise correction is applied to different speech frames. This accurately suppresses dynamic noise, and the real-time updated noise model captures noise changes during the call, avoiding feature distortion caused by sudden noise or environmental changes, ensuring that the features of each speech frame truly reflect the emotional state.

[0070] In another embodiment of the present invention, the step of performing acoustic gain compensation on different voice frames according to the device type of the device to be monitored includes: By pre-setting or training, typical acoustic fingerprints of different acquisition devices are learned, including frequency response curves and signal-to-noise ratio baselines. During feature extraction, corresponding compensating acoustic features are matched based on the identified device type to compensate for the original acoustic features, eliminating biases introduced by device differences. For example, low-frequency gain compensation is performed on microphones with insufficient low-frequency response.

[0071] By mapping the speech energy collected from different devices to the energy scale of a standard device, speech with the same emotional intensity has consistent energy characteristics regardless of the device it comes from.

[0072] In another embodiment of the present invention, the step of correcting environmental noise for different speech frames by monitoring the current environmental noise includes: Continuously monitor non-speech segments and / or low-volume segments during calls, and dynamically estimate the spectrum and power spectral density of current ambient noise.

[0073] Based on this, spectral subtraction or deep learning noise suppression techniques are used to correct acoustic features in real time to ensure that emotion-related features are not excessively disturbed by background noise.

[0074] In another embodiment of the present invention, a series of acoustic features optimized for emotional intensity and risk sensitivity are extracted from each speech frame, and then subjected to uniform standardization processing (such as Z-score standardization) to eliminate the differences in dimensions between different features and the influence of individual speaking habits, thereby generating a normalized and highly comparable acoustic feature sequence. , where N is the number of frames.

[0075] The acoustic feature sequence includes at least one of the following: fundamental frequency jitter deviation, energy entropy, Mel frequency cepstral coefficient, speech rate, and specific harmonic ratio.

[0076] In another embodiment of the present invention, the generated acoustic feature sequence further includes fundamental frequency jitter, used to measure minute fluctuations in the periodic changes of the fundamental frequency. It is highly sensitive to reflecting emotional tension, excitement, vocal tremor, or uncertainty. Outliers are typically closely associated with emotional instability or inner struggle.

[0077] Fundamental frequency jitter (F0) normalization: The deviation of fundamental frequency jitter is calculated using the Z-score normalization method. The specific formula is as follows: ; in, This represents the current real-time measured baseband jitter value; This is the baseline average value of baseband jitter, obtained through statistical analysis of a large amount of normal call data. For example, =200Hz represents the average base frequency for normal calls; The standard deviation of the normal fluctuation range, for example, =30Hz, defining the fundamental frequency fluctuation under normal emotional conditions.

[0078] In another embodiment of the present invention, the generated acoustic feature sequence further includes: energy entropy, used to measure the uniformity or disorder of the distribution of speech energy at different frequencies or times. It can effectively indicate the energy concentration of speech and the explosiveness of emotions; for example, a significant decrease in energy entropy may indicate a sudden outburst or suppression of emotions.

[0079] Similarly, the energy entropy (E) standardization also adopts Z-score standardization: ; in, This is the current real-time measured energy entropy value; For example, the benchmark mean of average energy distribution complexity. =0.5; To define the standard deviation of the critical threshold for energy mutation, for example, =0.2. By using Z-score normalization, acoustic features with different dimensions are unified to a dimensionless scale, enabling them to participate fairly in subsequent weighted fusion and effectively suppressing transient interference from environmental noise.

[0080] In another embodiment of the present invention, the generated acoustic feature sequence further includes: Mel frequency cepstral coefficients.

[0081] The spectral envelope information of different speech frames is detected, and the Mel frequency cepstral coefficients are calculated as feature values ​​of the acoustic feature sequence.

[0082] Mel frequency cepstral coefficients can capture the spectral envelope information of speech and reflect subtle changes in speech timbre, which is crucial for the perception of emotions such as pleasure, anger, and sadness.

[0083] In another embodiment of the present invention, the generated acoustic feature sequence further includes: speech rate.

[0084] Detect the number of speech words or syllables per unit time in different speech frames and calculate the speech rate.

[0085] Speech rate is used to measure the number of words or syllables spoken per unit of time. It is an important indicator of emotional excitement, anxiety (faster speech rate), or frustration (slower speech rate).

[0086] In another embodiment of the present invention, the generated acoustic feature sequence further includes a specific harmonic ratio.

[0087] Detect the proportion of harmonics in a specific frequency range in different speech frames and calculate the specific harmonic ratio.

[0088] A specific harmonic ratio is used to reflect the clarity and intensity of voiced components in a speech signal. It is associated with emotional stability or hoarseness.

[0089] In another embodiment provided by the present invention, step S2 specifically includes: Performing semantic transcription on the voice stream in the call data to obtain a text sequence; Performing emotional tendency semantic analysis on the text sequence to obtain an analysis result; Converting the analysis result into semantic features.

[0090] Specifically, the voice stream collected in real time is subjected to high-precision real-time transcription through a preset deep learning-based speech recognition model. This proposal preferably uses the RNN-Transducer model, which has been大规模 pre-trained and fine-tuned for the specific vocabulary, phrases, and spoken language habits in the procurement field, such as industry terms, abbreviations, negotiation tones, etc., to ensure the transcription accuracy and robustness in complex dialogue scenarios and control the real-time delay within 200 ms.

[0091] Performing in-depth emotional tendency semantic analysis on the transcribed text sequence. A pre-trained large-scale Chinese language model is used to perform in-depth semantic analysis on the transcribed text. This step is a significant innovation point of this proposal compared with traditional emotion recognition and general speech classification. It breaks through the limitation of simple keyword matching and can combine more complex context understanding and emotion intensity recognition, especially for analyzing specific risk semantics in the procurement scenario.

[0092] Converting the analysis result into a text emotional tendency semantic feature sequence to obtain semantic features.

[0093] In another embodiment provided by the present invention, when generating the text sequence, specifically: Cutting the call voice stream in the call data into different voice frames according to a preset frame length. When the user starts a call, the audio stream is cut into 10 ms segments.

[0094] Extracting a preset number of Mel spectrum features for each frame. 80-dimensional Mel spectrum features are extracted for each frame. A dedicated high-speed transmission thread ensures that the acoustic features are continuously input into the processing pipeline at a rhythm of 10 ms / frame, realizing seamless connection of the data stream. For example, when the user emits the first vibration wave of the character "合", the first 20 frames have quickly entered the encoder.

[0095] Subsequently, the acoustic features are processed frame by frame through an encoder composed of multiple layers of unidirectional LSTMs. This encoder adopts a causal convolution design to strictly avoid leakage of future information and ensure the real-time performance and accuracy of the inference. For example, when processing the 150th frame, corresponding to the pronunciation segment of the character "修", the network outputs a 512-dimensional acoustic vector. At this time, the acoustic pattern of the first half sentence "合同需要" has been efficiently abstractly represented. The GPU acceleration module ensures that the single-frame processing time is less than 5 ms, thus ensuring that the acoustic feature extraction rate far exceeds the audio input speed.

[0096] The prediction network running in parallel with the encoder continuously analyzes the output text sequence. For example, when the system successfully transcribes "The contract requires", the network generates a language feature vector based on the formed text context. For the prediction of subsequent words, such as the character "xiū", the network can learn the grammatical structure that verbs often follow after "The contract requires", and increase the probability weights of verb candidates such as "modify" and "adjust" by more than 30%, thus improving the transcription accuracy.

[0097] Finally, the joint network fuses the acoustic features from the encoder and the language features from the prediction network in real time, and dynamically calculates the lexical probability distribution. For example, when processing the 150th frame corresponding to the character "xiū", the network will simultaneously evaluate various lexical options such as "xiū" (probability 0.41), "xū" (probability 0.32), blank character (probability 0.27), etc. Through the tensor core accelerator, the probability calculation for a vocabulary of tens of thousands can be completed within 1 millisecond, providing ultra-low latency support for instant lexical decision-making.

[0098] This process ensures real-time, high-precision, and low-latency conversion from speech to text, providing high-quality text input for subsequent in-depth semantic analysis and sentiment tendency recognition.

[0099] In another embodiment provided by the present invention, in-depth sentiment tendency semantic analysis is performed on the transcribed text sequence. This step is a significant innovation point of this proposal compared with traditional sentiment recognition and general speech classification. It breaks through the limitation of simple keyword matching, and can combine more complex context understanding and sentiment intensity recognition, especially for analyzing specific risk semantics in the procurement scenario. Specifically, it includes: Construct a domain-specific emotion semantic dictionary and a risk word library. The system pre-constructs and continuously maintains a high-precision dictionary specialized for the procurement scenario. This dictionary not only contains words with strong sentiment tendencies, such as anger and surprise, but also covers specific words, phrases highly associated with compliance risks, and their precise sentiment intensity and risk level labels. For example, words and phrases such as "discount", "fee", "kickback", "flexibility", "benefit", "private transaction", "inside information", "go through the motions", "we know" will be assigned different negative sentiment weights and risk coefficients, and even identify their implied illegal intentions in specific contexts.

[0100] According to the pre-constructed emotion semantic dictionary and risk word library, identify different emotion scenarios and risk words correspondingly; Determine the sentiment intensity and risk level labels corresponding to the emotion scenarios and the risk words; Determine the sentiment tendency score according to the identified emotion scenarios, risk words, and the corresponding sentiment intensity and risk level labels; The core of this proposal is to use a pre-trained large-scale Chinese language model to conduct in-depth semantic analysis of transcribed text. This AI model can not only identify the surface sentiment and literal meaning of words, but also infer their true implied intentions and emotional intensity through powerful contextual understanding. For example, for ironic expressions like "This matter is really 'too' smooth," euphemistic expressions like "Our matter is easy to discuss," or suggestive expressions like "The contract needs to be amended in terms of payment, let's handle this 'specially'," the large model can accurately capture their potential negative sentiments or implications of violations and assign them corresponding sentiment scores S∈[-1, 1], where negative values ​​represent negative and positive values ​​represent positive. This far surpasses the recognition capabilities of traditional rule-based or simple statistical models.

[0101] In addition to individual words, the system can also identify specific sentence structures, word combinations, or more complex semantic patterns in the text. These patterns are often strong signals of compliance risks, and identifying them further enhances the system's ability to issue early warnings of potential violations.

[0102] In another embodiment of the present invention, the step of converting the analysis result into semantic features includes: The different feature values ​​in the analysis results are transformed into a semantic feature sequence of text sentiment, which is then used as the semantic feature. Each feature vector contains not only the emotional intensity and polarity of a word or phrase, but also its potential compliance risk weights and pattern matching strength, providing accurate input for the subsequent calculation of the emotional energy index Q.

[0103] In another embodiment provided by the present invention, step S3 specifically includes: The Dynamic Time Warping (DTW) strategy is used to accurately solve the problem of asynchrony between speech signals and text semantics in the time dimension.

[0104] DTW ensures precise temporal alignment between acoustic feature sequences and text semantic feature sequences by finding the optimal nonlinear mapping path between them. Compared to simple feature concatenation or general attention fusion, DTW ensures precise temporal synchronization of cross-modal information, enabling subsequent sentiment assessment to comprehensively consider the matching degree between the speech intonation at a certain moment and the text content at the same time, significantly improving the accuracy and reliability of the judgment.

[0105] Based on the DTW alignment relationship, the node (i, j) on the path represents the binding of the acoustic feature of the i-th frame to the j-th text word, forming a many-to-one mapping. For example, a word may correspond to multiple frames of speech, or multiple frames of speech may correspond to a long word or phrase.

[0106] This proposal takes aligned acoustic information, such as fundamental frequency jitter and dynamic changes in energy entropy, and text sentiment, such as the negative semantics and potential risk scores of prohibited words obtained after deep analysis by a large AI model, as core inputs and performs deep fusion through a cross-modal attention mechanism.

[0107] The attention mechanism is required to dynamically learn and compute which modality and which part contributes more to the judgment of emotion and compliance risk at a specific time sequence. For example, when acoustic features show a trembling voice and the text semantics are threatening, the attention mechanism will assign higher fusion weights to both, thereby generating a spatiotemporally correlated joint representation matrix. This joint representation matrix not only contains independent acoustic and semantic information, but more importantly, it captures the emotion-risk resonance pattern in which the two reinforce or corroborate each other in the temporal dimension, providing a more accurate and comprehensive input for the subsequent calculation of the emotion energy index Q.

[0108] In another embodiment provided by the present invention, the process of determining the optimal nonlinear mapping path specifically includes: The acoustic feature sequence is processed by framing the speech stream in 10ms increments, for example, 200 frames within 2 seconds. Mel-spectral features are extracted from each frame, with particular emphasis on acoustic features sensitive to emotional fluctuations such as fundamental frequency jitter and energy entropy. These features constitute a dimensional acoustic sequence.

[0109] The next step is to process the semantic sequence of the text, extracting keywords or phrases through real-time text conversion and deep analysis by a large AI model.

[0110] For example, terms like "price," "violation," "discount," "contract," and "payment," as well as more complex risk-implying semantics, are used. Each word is converted into a word embedding technique. 3D semantic vector, forming × A text sequence.

[0111] Distance matrix construction and dimensionality matching: The acoustic features are reduced in dimensionality or encoded so that their dimensions match the dimensions of the text word embeddings, so that distance calculations can be performed in the same space.

[0112] Calculate the acoustic features A of the i-th frame i Embedded with the j-th word T j Euclidean distance: ; Build × The local distance matrix is ​​used to quantify the similarity between acoustic frames and text words.

[0113] Cumulative distance calculation: Initialize the cumulative distance from the starting point to D. .

[0114] For each position Calculate the minimum cumulative distance: ; At the same time, boundary constraints are applied, allowing only unidirectional extension in the first row / column to ensure that the path covers the entire sequence and avoids unreasonable jumps.

[0115] Optimal path backtracking and alignment verification: trace back from the endpoint to the starting point (1,1), select the adjacent node with the smallest cumulative distance (diagonal, horizontal, and vertical movement are allowed) to form the optimal alignment path.

[0116] Furthermore, the path must satisfy temporal monotonicity, with both i and j being non-decreasing, ensuring that the semantic order of acoustic frames and text words is consistent, conforming to the expression habits of natural language.

[0117] For example, when the path passes through (80, 3), the acoustic features of frame 80 may show significant changes in fundamental frequency jitter and energy entropy, reflecting emotional fluctuations aligned with the third word "violation." The cumulative distance D(80, 3) = 8.2 reflects the degree of matching. This alignment precisely links acoustic features, such as abrupt changes in tone and increases in volume, to the sensitive word "violation" with negative semantic meaning, greatly enhancing the contextual consistency and accuracy of subsequent emotion determination.

[0118] In yet another embodiment provided by the present invention, step S4 specifically includes: The Emotional Energy Index Q is the core innovation of this proposal in the quantitative assessment of compliance risks. It is a comprehensive indicator designed to accurately quantify the potential compliance risks arising from the combined effects of emotional fluctuations and semantic intent in a current call, rather than simply identifying emotional categories or making general classifications. This step, based on extracted acoustic features (fundamental frequency jitter, energy entropy) and semantic features of text sentiment derived from deep analysis using a large AI model, standardizes and weights these features to generate the emotional energy index Q. This proposal particularly emphasizes dynamically adjusting the calculation weights of the emotional energy index Q and the blocking threshold Q based on the specific business stage of the call, such as different stages in the procurement process. th This provides a business-driven adaptive risk assessment mechanism that can effectively avoid misjudgments caused by reasonable emotional fluctuations at different stages and ensure the refinement and rationality of supervision.

[0119] The emotional energy index Q is multi-stage weighted fusion. The weighting of the emotional energy index Q is not fixed, but can be fine-tuned according to the specific business stage of the current call. This solution emphasizes acoustic emotion and textual semantic risks differently at different stages, thereby more accurately capturing the compliance risks at that stage.

[0120] The weighting coefficients of the emotional energy model are determined based on the stage of the call. The emotional energy index is calculated based on the joint representation matrix and the emotional energy model with confirmed weight coefficients.

[0121] In another embodiment of the present invention, when performing call phase identification, specifically: By analyzing information such as call duration, silent ratio, keyword frequency, and completed business process nodes; The stage of the call is automatically identified in real time based on the call duration, the silence ratio, the frequency of keyword occurrence, and the completed business process nodes.

[0122] In another embodiment of the present invention, the call phase includes: a demand confirmation phase, a solution submission and clarification phase, a negotiation phase, and a signing confirmation phase.

[0123] Specifically: Based on the context of the call and pre-defined business process nodes, such as in the procurement process, It can be subdivided into T1: requirement confirmation stage; T2: solution submission and clarification stage; T3: business negotiation stage; T4: contract signing confirmation stage, etc., to dynamically identify the current stage.

[0124] Each identified call phase Set a specific set of weight coefficients .

[0125] The weighted fusion weights of the emotional energy index Q are not fixed, but can be fine-tuned according to the specific business stage of the current call. This solution can emphasize different aspects of acoustic emotion and textual semantic risks at different stages, thereby more accurately capturing the compliance risks at that stage.

[0126] In yet another embodiment of the present invention, the step of following the emotional energy model includes: ; In the T1 requirement confirmation phase, more emphasis may be placed on the accuracy of the text semantics, and the weight may be set to ( =0.3, =0.2, =0.5).

[0127] In the T3 business negotiation stage, emotional fluctuations may be significant and often accompanied by sensitive words. At this point, a more balanced consideration of acoustic and semantic information may be needed, or specific emotions (such as anger) may be given higher weight. The weighting might be set as follows: =0.4, =0.3, =0.3).

[0128] The formula for calculating the emotional energy index Q is: ; The essence of this formula lies in its organic combination of the physiological performance of speech with the semantic connotation of language, and its dynamic adjustment of the importance of each modality according to the current business stage of the call, forming a comprehensive, quantitative, and business-adaptive compliance risk indicator.

[0129] Specific calculation example: Assume that the measured... ≈0.666, =1.75, AI large model semantic score S= 0.7.

[0130] The current call is in the T1 demand confirmation phase. Assume the weight of this phase is set to ( =0.3, =0.2, =0.5). ; The current call is in the T3 business negotiation stage. Assume the weight of this stage is set to ( =0.4, =0.3, =0.3). .

[0131] As the example shows, even if the acoustic and semantic feature values ​​are the same, the Q value will change due to weight adjustments at different business stages, reflecting the system's adaptability to the emphasis on stage-specific risks.

[0132] In another embodiment of the present invention, the trigger threshold for blocking is not fixed, but is dynamically adjusted in a more refined manner according to the specific business stage of the call. This solution can adapt to the risk tolerance of different stages in the procurement process, avoid misjudgments caused by reasonable emotional fluctuations in negotiation scenarios, and ensure the accuracy and rationality of supervision.

[0133] The phased threshold setting is applied to each identified call phase. Set a specific dynamic threshold ( ).

[0134] The thresholds for each stage can be finely set based on historical violation data for that stage, business expert experience, and risk exposure for that stage.

[0135] For example, in the T1 demand confirmation phase, the risk is usually low, and the threshold can be set to a lower sensitivity level, such as... ( =0.4.

[0136] During the T2 proposal submission and clarification phase, information opacity issues may arise. A threshold of medium sensitivity can be set, such as... ( =0.6.

[0137] During the T3 business negotiation phase, due to the involvement of profit sharing, emotional fluctuations and the risk of violations may increase significantly. Therefore, the threshold can be set to a higher sensitivity level, such as... ( =0.8.

[0138] During the pre-contract signing confirmation phase of T4, a final attempt at violation may occur; the threshold can be set to the highest sensitivity level, such as... ( =0.9.

[0139] The system detects the current stage of the call in real time and dynamically loads the corresponding threshold parameters. ( When the real-time calculated emotional energy index Q exceeds the preset dynamic threshold for the current stage. ( When this occurs, subsequent real-time tiered blocking operations will be triggered immediately.

[0140] The multi-stage, adaptive threshold management mechanism is a significant advantage of this proposal in achieving intelligent and refined compliance supervision.

[0141] In another embodiment of the present invention, the step of matching the corresponding blocking threshold based on the identified call phase includes: This solution achieves a complete closed loop from risk identification to real-time intervention. It can execute graded, intelligent, and automated intervention and blocking strategies based on the real-time calculated emotional energy index Q and dynamically adjusted blocking thresholds, and is supplemented by a feedback monitoring mechanism, which significantly improves the risk control capabilities in scenarios with high compliance requirements such as procurement.

[0142] Dynamic blocking threshold Application and fine-tuning, through dynamic setting of blocking thresholds This threshold is not fixed, but depends on the specific business phase of the call. The intervention was finely adjusted, and the rate of change in mood was introduced as an auxiliary adjustment factor to adapt to the risk tolerance at different stages, avoid false alarms, and ensure the accuracy of the intervention.

[0143] By analyzing information such as call duration, silence rate, keyword frequency, and completed business process nodes, the system can automatically identify the stage of the call in real time. For example: During the T1 connection establishment / initial communication phase, emotional fluctuations may be significant, but the risk is generally low, and the threshold can be appropriately relaxed. For example... ,in The system's preset general basic threshold, such as 0.8, Let be the adjustment coefficient for this stage, such as 0.5. =0.4.

[0144] T2 - Stable Communication / Negotiation Phase: This phase involves core business communication, where emotional fluctuations become more strongly correlated with compliance risks, and thresholds tighten. For example, (in If it is 1.0, then =0.8.

[0145] In the T3-Dispute / End Phase, emotions may fluctuate again, and the threshold needs to be dynamically adjusted according to the risk situation. For example... (in It could be 0.9, then =0.72.

[0146] By assessing the rate of emotion change and fine-tuning the threshold, the system continuously monitors the upward slope of the emotion energy index Q(t). If Q(t) spikes rapidly in a short period of time, for example, A preset threshold rate is set even if the current Q(t) has not yet fully reached the stage threshold. ( The system will also appropriately lower the actual trigger threshold, for example, by 10%-20%, to provide early warning or intervention and prevent sudden escalation of risks. This is a threshold adaptive capability based on the rate of change.

[0147] The system detects the current stage of the call in real time and dynamically loads corresponding threshold parameters based on the rate of emotion change. (t). When the real-time calculated emotional energy index Q exceeds the current dynamic threshold. When (t), subsequent graded blocking operations will be triggered immediately.

[0148] In yet another embodiment provided by the present invention, see Figure 2 This is a schematic diagram of the execution process of the blocking strategy provided in an embodiment of the present invention.

[0149] The step of determining the blocking strategy based on the emotional energy index and the blocking threshold includes: An automated, tiered blocking process is adopted, with progressive intervention measures taken based on real-time changes in risk levels, ranging from mild warnings to mandatory circuit breakers, forming a robust risk control system.

[0150] When the emotional energy index is not greater than the blocking threshold, it is identified as compliant; Primary blocking (mild risk / warning) trigger condition: When the real-time calculated standardized emotional energy index Q(t) first exceeds the current dynamic threshold. When (t), after identifying the risk, For example, providing compliance alerts to both parties in a call includes silently injecting a pre-recorded, low-volume, non-intrusive compliance warning tone into the audio streams of both parties. The content and duration of this warning tone can be customized based on the specific type of risk detected, such as emotional instability, innuendo of violations, or verbal threats. For instance, when emotional fluctuations are detected, the tone might say, "Please be mindful of call compliance and maintain rational communication"; when sensitive words are detected, it might say, "Please verify relevant information." Furthermore, a compliance reminder text message will pop up on both parties' devices via instant messaging or a client interface, such as, "This call may involve compliance risks; please be cautious in your words and actions." Intermediate-level blocking (moderate risk / early warning upgrade): If, after the initial blocking is implemented, real-time monitoring indicates that the initial intervention has failed, Q(t) continues to rise or fails to decrease effectively after the initial blocking. For example, if the rate of decrease of Q(t) does not reach the preset minimum decay rate within 30 seconds, or Q(t) breaks through the initial threshold of the current stage again. (t). Or Q(t) spikes directly, breaking through the intermediate threshold for the first time. (t) (t) identifies serious compliance risks and sends an early warning notification to the preset compliance monitoring equipment.

[0151] For example, after sending a warning notification, the call can be forcibly interrupted to insert a stronger and clearer voice prompt, such as: "This call may have serious compliance risks, please stop communication immediately!", or the call can be briefly interrupted for a few seconds, such as 3-5 seconds, to give both parties a forced buffer time.

[0152] Simultaneously, the system automatically sends real-time alerts to pre-defined compliance supervisors or direct supervisors. The alerts include: information about both parties in the call, a risk summary, the current Q index value, and the associated ID of the call event, facilitating manual review.

[0153] Advanced blocking (high-risk / forced circuit breaker) trigger condition: reaching the highest threshold Q(t) and exceeding the advanced threshold. (t) (t). A clear and irreversible serious violation pattern is detected. For example, explicitly prohibited words such as "benefits" appear directly in the transcribed text, and the contextual meaning is clear. Immediately disconnect the communication link, forcibly terminate the call, interrupt the communication link, and prevent any further unauthorized communication.

[0154] The system generates and automatically archives encrypted event logs, including complete call recordings, detailed transcripts, real-time curves of the emotional energy index Q, details of identified risk patterns, and all blocking decision chains: trigger time, Q value, threshold, and executed actions. These logs are processed with high-strength encryption and automatically archived to the enterprise-level compliance audit system for subsequent compliance audits, accountability, and legal evidence collection, ensuring data integrity, immutability, and traceability.

[0155] Ensure that when serious violations or high-risk out-of-control situations are confirmed, the strongest intervention measures are taken immediately to minimize losses and impacts and provide a comprehensive and reliable chain of evidence for subsequent investigations.

[0156] Real-time feedback monitoring and adaptive decision-making. After the initial intervention is implemented, the system continuously monitors the subsequent trend of the Q value, such as whether the Q value decreases and the rate of decrease, and determines whether the initial intervention is effective based on this. If the Q value decays below a preset threshold within a certain period of time, such as 30 seconds, the initial intervention is deemed to have failed.

[0157] Automated upgrade and circuit breaker. When the monitoring results meet the upgrade conditions, the primary intervention fails and the Q value continues to rise, or a secondary over-threshold event occurs, the system will automatically and seamlessly switch to the next level of blocking strategy until the highest level of circuit breaker operation is reached.

[0158] The entire decision-making process, from calculating the Q-value, applying dynamic thresholds, triggering tiered blocking, to the final execution of circuit breakers and generation of encrypted logs, is fully automated and meticulously documented. This provides unprecedented real-time monitoring capabilities, automated risk control capabilities, and robust post-audit support for highly compliant scenarios such as procurement and finance.

[0159] This application does not construct a single, fused emotional energy index Q. Instead, it maintains multiple independent feature subsets or scores representing different risk dimensions in parallel, such as acoustic arousal scores, textual negative semantic intensity scores, and specific risk pattern matching degrees. These independent scores are not explicitly weighted and fused based on call stages to form a single Q value, but are directly fed as input feature vectors into a pre-trained deep neural network, such as an MLP, decision head. This network implicitly captures the importance of each risk dimension at different stages through end-to-end learning and directly outputs blocking decisions or risk levels, thus avoiding explicit, formulaic Q value calculations and dynamic weights. It uses a language model to extract word embeddings, not entirely relying on the direct contextual inference capabilities of a large model to identify implicit risks. Instead, it constructs a knowledge graph or semantic graph from entities, concepts, and their relationships in the call text. Then, it uses a graph neural network to propagate information on the graph structure, learning complex relationships between entities and potential risk transmission paths. For example, a seemingly neutral word is assigned a risk weight through connections with other high-risk words in the graph. Risk identification relies more on the analysis and reasoning of the graph structure than on the contextual understanding of a single text sequence.

[0160] This proposed solution, deeply integrated with 5G New Voice products, will significantly enhance the commercial value and application prospects of 5G New Voice in highly compliant scenarios such as procurement. By leveraging the open data channels and business capabilities of 5G New Voice, it can not only achieve real-time monitoring and intelligent blocking of voice calls, improving user experience, but also, as an intelligent value-added service of 5G New Voice, significantly enhance its competitiveness in the government and enterprise market, open up new business growth points, and promote the development of communication services towards intelligence and compliance.

[0161] See Figure 3 This is a schematic diagram of a call compliance detection device provided in an embodiment of the present invention. The device includes: The acoustic module is used to extract acoustic features based on the collected call data; The semantic module is used to perform speech conversion and semantic analysis on the call data to obtain semantic features; The joint module is used to perform cross-modal temporal alignment of the acoustic features and the semantic features, and to perform feature fusion to obtain a joint representation matrix; The index module is used to calculate the emotional energy index based on the joint representation matrix and the preset emotional energy model. The blocking module is used to determine the blocking strategy based on the emotional energy index.

[0162] The call compliance detection device provided in this embodiment can execute all the steps and functions of the call compliance detection method provided in any of the above embodiments. The specific functions of the device will not be described in detail here.

[0163] See Figure 4 This is a schematic diagram of a communication device provided in an embodiment of the present invention. The communication device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a call compliance detection program. When the processor executes the computer program, it implements the steps in each of the above-described call compliance detection method embodiments, for example... Figure 1 The steps shown. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described device embodiments.

[0164] For example, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the call compliance detection device. For example, the computer program can be divided into several modules, the specific functions of which have been described in detail in the call compliance detection method provided in any of the above embodiments; therefore, the specific functions of the device will not be repeated here.

[0165] The communication device described can be a desktop computer, laptop, handheld computer, or cloud server, etc. The communication device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the schematic diagram is merely an example of a communication device and does not constitute a limitation on a call compliance detection device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the communication device may also include input / output devices, network access devices, buses, etc.

[0166] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the call compliance detection device, connecting all parts of the device via various interfaces and lines.

[0167] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the call compliance detection device by running or executing the computer programs and / or modules stored in the memory and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0168] If the module integrated into the call compliance detection device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0169] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the functional network element implementing the method described in the above embodiments.

[0170] The computer program product provided in this embodiment can execute all the steps and functions of the call compliance detection method provided in any of the above embodiments. The specific functions of the product will not be described in detail here.

[0171] It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered to be within the scope of protection of this invention.

Claims

1. A method for call compliance detection, characterized in that, The method includes: Acoustic features are extracted from the collected call data; The call data is subjected to speech conversion and semantic analysis to obtain semantic features; The acoustic features and semantic features are cross-modal temporally aligned and then fused to obtain a joint representation matrix. The emotional energy index is calculated based on the joint representation matrix and the preset emotional energy model. The blocking strategy is determined based on the emotional energy index.

2. The call compliance detection method according to claim 1, characterized in that, The extraction of acoustic features based on the collected call data includes: The call voice stream of the device to be monitored is acquired in the form of voice packets and used as the call data. The call audio stream is segmented into frames to obtain different audio frames; Different speech frames are standardized to generate a normalized acoustic feature sequence, thus obtaining the acoustic features.

3. The call compliance detection method according to claim 2, characterized in that, Before performing frame segmentation on the call audio stream, the method further includes: The call audio stream is preprocessed, including echo cancellation and background noise suppression.

4. The call compliance detection method according to claim 2, characterized in that, The step of performing frame segmentation on the call audio stream to obtain different audio frames includes: The call audio stream is divided into frames according to a preset audio length and a preset overlap between adjacent frames to obtain different audio frames.

5. The call compliance detection method according to claim 2, characterized in that, Before standardizing different speech frames, the method further includes: Acoustic gain compensation is performed on different voice frames according to the device type of the device to be monitored. By monitoring the current ambient noise, environmental noise correction is performed on different speech frames.

6. The call compliance detection method according to claim 5, characterized in that, The step of performing acoustic gain compensation on different speech frames according to the device type of the device to be monitored includes: Identify the device type of the device to be monitored; Based on the device type, match the corresponding compensated acoustic features in a pre-trained database of typical acoustic fingerprints of different acquisition devices; Acoustic gain compensation is performed on different speech frames based on the compensated acoustic features.

7. The call compliance detection method according to claim 5, characterized in that, The step of correcting environmental noise for different speech frames by monitoring the current environmental noise includes: Continuously monitor non-speech segments and / or low-volume segments during the call to estimate the spectral density and power spectral density of the current ambient noise; Based on the spectral density and the power spectral density, noise suppression techniques are used to correct the acoustic features of different speech frames in real time.

8. The call compliance detection method according to claim 2, characterized in that, The acoustic feature sequence includes at least one of the following: fundamental frequency jitter deviation, energy entropy, Mel frequency cepstral coefficient, speech rate, and a specific harmonic ratio.

9. The call compliance detection method according to claim 2, characterized in that, The standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The Z-score normalization method is used for different speech frames to calculate the degree of deviation of the fundamental frequency jitter, and the degree of fundamental frequency jitter deviation is obtained. The degree of base frequency jitter deviation ; in, This refers to the fundamental frequency jitter value obtained by real-time measurement of different speech frames; This is the baseline average value of baseband jitter, obtained through statistical analysis of a large amount of normal call data. This represents the standard deviation of the normal fluctuation range.

10. The call compliance detection method according to claim 2, characterized in that, The standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The energy entropy was calculated using the Z-score normalization method for different speech frames; The energy entropy ; in, The energy entropy value is obtained by real-time measurement of different speech frames; This represents the baseline mean of the average energy distribution complexity. The standard deviation is used to define the critical threshold for energy mutation.

11. The call compliance detection method according to claim 2, characterized in that, The standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The spectral envelope information of different speech frames is detected, and the Mel frequency cepstral coefficients are calculated as feature values ​​of the acoustic feature sequence.

12. The call compliance detection method according to claim 2, characterized in that, The standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The number of speech words or syllables per unit time in different speech frames is detected, and the speech rate is calculated as the feature value of the acoustic feature sequence.

13. The call compliance detection method according to claim 2, characterized in that, The standardization process for different speech frames to generate a normalized acoustic feature sequence includes: The proportion of harmonics in a specific frequency range in different speech frames is detected, and a specific harmonic ratio is calculated as the feature value of the acoustic feature sequence.

14. The call compliance detection method according to claim 1, characterized in that, The process of performing speech conversion and semantic analysis on the call data to obtain semantic features includes: The voice stream in the call data is semantically transcribed to obtain a text sequence; Sentiment semantic analysis was performed on the text sequence to obtain the analysis results; The analysis results are then transformed into semantic features.

15. The call compliance detection method according to claim 14, characterized in that, The step of semantically transcribing the stream of call data to obtain a text sequence includes: The call audio stream in the call data is cut into different audio frames according to a preset frame length, and a preset number of Mel spectrum features are extracted from each frame. The Mel spectrum features extracted are processed frame by frame by an encoder composed of multiple unidirectional LSTMs, and the acoustic features of each frame are output. The language features of the call voice stream in the call data are extracted using a preset speech recognition model. The text sequence is obtained by calculating the word probability distribution based on the extracted linguistic features and the acoustic features using a pre-defined joint network.

16. The call compliance detection method according to claim 14, characterized in that, The sentiment semantic analysis of the text sequence, to obtain the analysis results, includes: Based on a pre-built emotional semantic dictionary and risk word database, different emotional scenarios and risk words are identified accordingly; Determine the emotional intensity and risk level labels corresponding to the emotional scenarios and the risk words; The emotional tendency score is determined based on the identified emotional scenarios, risk words, and corresponding emotional intensity and risk level labels. The identified emotional scenarios, risky words, corresponding emotional intensity and risk level labels, and emotional tendency scores are used as the analysis results.

17. The call compliance detection method according to claim 14, characterized in that, The process of converting the analysis results into semantic features includes: The different feature values ​​in the analysis results are transformed into a semantic feature sequence of text sentiment tendency, which is then used as the semantic feature.

18. The call compliance detection method according to claim 1, characterized in that, The process of performing cross-modal temporal alignment of the acoustic features and the semantic features, and then fusing the features to obtain a joint representation matrix, includes: A dynamic time warping strategy is used to find the optimal nonlinear mapping path between the acoustic features and the semantic features, which serves as the optimal alignment path. Based on the optimal alignment path, the binding relationship between the acoustic features of different frames and the semantic features of different frames is determined, and a mapping relationship is constructed. Based on the constructed mapping relationship, the fusion weights are determined through a cross-modal attention mechanism; Based on the binding relationship between the acoustic features and semantic features of different frames, and the determined fusion weights, a spatiotemporally related joint representation matrix is ​​generated.

19. The call compliance detection method according to claim 18, characterized in that, The step of using a dynamic time warping strategy to find the optimal nonlinear mapping path between the acoustic features and the semantic features includes: The sequence representation of the acoustic features is divided into different speech frames according to a preset frame length, and Mel-spectral features are extracted from each frame to obtain the dimensional acoustic sequence. Perform deep text analysis on the sequence of semantic features to extract keywords; Each keyword is converted into a multi-dimensional semantic vector using word embedding technology to obtain a text sequence; The dimensionality of the acoustic sequence is reduced to match the embedding dimension of the text sequence, and the Euclidean distance between the acoustic features of different frames and different keywords in the text sequence is calculated. Based on the calculated Euclidean distance, calculate the minimum cumulative distance between each acoustic feature and each keyword; Tracing back from the key point to the first acoustic feature and the first keyword, the neighboring nodes with the smallest cumulative distance are selected to form the optimal nonlinear mapping path.

20. The call compliance detection method according to claim 1, characterized in that, The calculation of the emotional energy index based on the joint representation matrix and the preset emotional energy model includes: Identify the stage of the call data; The weighting coefficients of the emotional energy model are determined based on the stage of the call. The emotional energy index is calculated based on the joint representation matrix and the emotional energy model with confirmed weight coefficients.

21. The call compliance detection method according to claim 20, characterized in that, The process of identifying the stage of the call data includes: Based on the call data, identify call duration, silent ratio, keyword frequency, and completed business process nodes; The call stage is identified based on the call duration, the muting rate, the frequency of keyword occurrence, and the completed business process nodes.

22. The call compliance detection method according to claim 20, characterized in that, The call phase includes: the needs confirmation phase, the solution submission and clarification phase, the negotiation phase, and the signing and confirmation phase.

23. The call compliance detection method according to claim 20, characterized in that, The emotional energy model includes: ; in, , and These are the weighting coefficients confirmed based on the i-th call phase; The degree of fundamental frequency jitter deviation in the joint characterization matrix. Let S be the energy entropy in the joint characterization matrix, and let S be the sentiment score in the joint characterization matrix.

24. The call compliance detection method according to claim 1, characterized in that, The step of determining the blocking strategy based on the emotional energy index includes: Identify the stage of the call data; Match the corresponding blocking threshold based on the identified call phase; The blocking strategy is determined based on the emotional energy index and the blocking threshold.

25. The call compliance detection method according to claim 20, characterized in that, The step of matching the corresponding blocking threshold based on the identified call phase includes: Monitor the rate of change of the emotional energy index over a preset period of time; An adjustment coefficient is determined based on the rate of change, and the blocking threshold is calculated based on the adjustment coefficient and the base threshold of the identified call phase.

26. The call compliance detection method according to claim 20, characterized in that, The step of determining the blocking strategy based on the emotional energy index and the blocking threshold includes: When the emotional energy index is not greater than the blocking threshold, it is identified as compliant; When the emotional energy index is found to be greater than the blocking threshold, a mild compliance risk is identified, and a compliance reminder is sent to both parties in the call. Real-time acquisition of call data and continuous monitoring of emotional energy index; When the monitored emotional energy index is not lower than the blocking threshold within the preset monitoring period, or when the rate of decline of the monitored emotional energy index is not less than the preset decay rate threshold, a serious compliance risk is identified, and an early warning notification is sent to the preset compliance monitoring device. When the emotional energy index monitored within the preset monitoring period is not lower than the preset circuit breaker threshold, a serious violation is identified, and the communication link is cut off.

27. A call compliance detection device, characterized in that, The device includes: The acoustic module is used to extract acoustic features based on the collected call data; The semantic module is used to perform speech conversion and semantic analysis on the call data to obtain semantic features; The joint module is used to perform cross-modal temporal alignment of the acoustic features and the semantic features, and to perform feature fusion to obtain a joint representation matrix; The index module is used to calculate the emotional energy index based on the joint representation matrix and the preset emotional energy model. The blocking module is used to determine the blocking strategy based on the emotional energy index.

28. A communication device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the call compliance detection method as described in any one of claims 1 to 26.

29. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the call compliance detection method as described in any one of claims 1 to 26.

30. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 26.

Images