User classification system based on speech intent recognition

Abstract

The application relates to the technical field of voice user classification, and discloses a user classification system based on voice intention recognition. A voice collection module of the system acquires original voice data and corresponding text transcription data, and divides the data into aligned voice segment units according to a unified time stamp; a signal enhancement module performs environment noise suppression on the voice data and grammar correction on the text data, and outputs an enhanced voice text data set; a semantic fusion module inputs the data set into a deep neural network based on context constraints, and obtains a voice text fusion semantic vector; a confidence evaluation module calculates voice recognition and semantic understanding accuracy quantitative values according to the fusion semantic vector, and obtains a confidence parameter in combination with user historical behavior parameters; and a classification generation module divides user risk levels according to the confidence parameter, embeds a behavior preference mapping function into a classification algorithm, iteratively adjusts a classification boundary to be within a stability threshold, and obtains an initial user classification result.

Description

Technical Field

[0001] This invention relates to the field of voice user classification technology, specifically a user classification system based on voice intent recognition. Background Technology

[0002] In today's rapidly evolving landscape of digital services, user segmentation has become a crucial tool for platforms to optimize services and accurately match resources. Voice-based user segmentation, aligning with users' natural interaction habits, is increasingly becoming a hot research topic in the industry. However, existing voice-based user segmentation systems still face numerous unresolved issues in practical applications.

[0003] In the voice data acquisition stage, existing systems often only acquire voice information in a single dimension, or while acquiring both voice and text data simultaneously, they fail to achieve precise alignment between the two in the temporal dimension. This leads to difficulties in effectively associating the features of the voice signal with the semantics of the text information during subsequent processing. When a user's voice exhibits changes in speaking speed or pauses, mismatches between voice and text segments can easily occur, affecting the accuracy of subsequent semantic analysis. For example, in customer service scenarios, if a user speaks faster or pauses midway due to emotional excitement, the voice segments acquired by existing systems may not correspond to the transcribed text segments, making it difficult for the system to accurately capture the core needs expressed by the user.

[0004] Existing systems have significant limitations in preprocessing capabilities for collected speech and text data. Speech data is highly susceptible to environmental noise, such as crowds in public places and mechanical noise from equipment. Existing noise suppression algorithms often only handle noise within a specific frequency range, offering limited effectiveness against complex and varied mixed noise. This results in a significant amount of redundant information in the processed speech signal, impacting the quality of feature extraction. For text transcription data, existing systems have relatively simple grammar correction functions, typically only correcting common spelling errors. They struggle to effectively correct grammatical confusion and semantic ambiguity caused by speech recognition deviations, making it difficult for transcribed text to accurately reflect the user's true intentions.

[0005] At the semantic fusion and analysis level, most existing systems employ traditional machine learning models, which struggle to fully uncover the deep semantic connections inherent in speech and text data. Traditional models often simply concatenate speech and text features before inputting them into the model, failing to consider their semantic complementarity and correlation. This results in semantic vectors that cannot comprehensively and accurately represent the user's speech intent. Furthermore, existing systems lack effective evaluation mechanisms for semantic understanding results, making it impossible to quantify the accuracy of speech recognition and semantic understanding. They also fail to integrate historical user behavior data for comprehensive analysis, leaving the system's judgment of user intent without reliable basis and prone to misjudgment.

[0006] In the user classification generation stage, existing systems typically categorize users solely based on a single semantic analysis result, failing to consider the differences in behavioral preferences among users. Furthermore, the boundary adjustments of classification algorithms lack scientific basis and often rely on human experience, resulting in poor stability of classification results. When user group characteristics change, the classification boundaries cannot be adaptively adjusted in a timely manner, causing the classification results to become disconnected from the actual user situation. This makes it difficult to meet the need for accurate user classification in different scenarios. For example, in e-commerce recommendation scenarios, existing systems may recommend products that do not match the user's actual needs due to inaccurate classification, affecting user experience and the platform's service quality. Summary of the Invention

[0007] The purpose of this invention is to provide a user classification system based on voice intent recognition to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides a user classification system based on voice intent recognition, the system comprising:

[0009] The system includes a voice acquisition module, a signal enhancement module, a semantic fusion module, a confidence assessment module, and a classification generation module.

[0010] The voice acquisition module acquires the user's raw voice data and corresponding text transcription data, and divides it into aligned voice segment units based on a unified timestamp.

[0011] The signal enhancement module performs environmental noise suppression processing on the raw speech data of each speech segment unit, performs grammatical error correction processing on the text transcription data, and outputs an enhanced speech-text dataset.

[0012] The semantic fusion module inputs the enhanced speech-text dataset into a context-constrained deep neural network to obtain a speech-text fusion semantic vector.

[0013] The confidence assessment module calculates the speech recognition accuracy metric and the semantic understanding accuracy metric based on the speech-text fusion semantic vector, and obtains the confidence parameter by combining the user's historical behavior parameters.

[0014] The classification generation module: classifies user risk levels based on the confidence parameters, embeds the behavior preference mapping function into the classification algorithm, and iteratively adjusts the classification boundary to within the stability threshold to obtain the initial user classification result.

[0015] Preferably, the processing procedure of the voice acquisition module includes the following:

[0016] The system collects multi-channel raw speech data containing user identification features through a microphone array, and generates corresponding text transcription data through a speech recognition engine.

[0017] A time alignment mechanism is used to align multi-channel raw speech data and text transcription data to a unified time reference;

[0018] Transform the transcribed text data from the text coordinate system to the user identity coordinate system to achieve semantic alignment with the multi-channel raw speech data;

[0019] The continuous multi-channel raw speech data stream and text transcription data stream are divided into discrete speech segment units at fixed time intervals.

[0020] Noise reduction preprocessing is performed on a subset of multi-channel raw speech data, and word segmentation is performed on a subset of text transcribed data to extract key intent information.

[0021] Preferably, the processing procedure of the signal enhancement module includes the following:

[0022] For multi-channel raw speech data, the signal-to-noise ratio distribution of the speech signal is calculated by simulating the propagation characteristics of acoustic signals in a noisy environment, the attenuation coefficient of each frequency band is determined, and the energy value of the speech data is adjusted according to the attenuation coefficient and the length of the speech segment to generate enhanced speech data. Subsequently, phoneme segmentation and feature extraction are performed on the enhanced speech data to generate a speech feature set that reflects the speech content.

[0023] Preferably, the processing of the signal enhancement module further includes the following:

[0024] For text transcription data, the parsing process of language model in grammatical structure is simulated to calculate the probability distribution of grammatical errors, determine the correction weight of error type, and adjust the word sequence of text data according to the correction weight to generate enhanced text data. Then, semantic role labeling and dependency analysis are performed on the enhanced text data to generate a text semantic set that reflects the intention structure.

[0025] Finally, the speech feature set and the text semantic set are integrated into an enhanced speech-text dataset.

[0026] Preferably, the processing procedure of the semantic fusion module includes the following:

[0027] The enhanced speech-text dataset is input into a deep neural network, which includes a feature encoder and a semantic decoder. The feature encoder receives the speech feature set and the text semantic set, extracts deep features from the speech feature set and the text semantic set through a convolutional neural network, and uses an attention mechanism to perform weighted fusion of the deep features to generate a speech-text fused semantic vector.

[0028] Preferably, the semantic fusion module's processing procedure further includes the following:

[0029] During the training of the deep neural network, the loss function of the feature encoder consists of reconstruction loss and context constraint loss. The context constraint loss is calculated based on semantic consistency residuals, and the reconstruction loss measures the probability that the speech-text fusion semantic vector is reconstructed into the original input by the semantic decoder.

[0030] The loss function of the semantic decoder is used to optimize the ability to reconstruct the original input;

[0031] After training is completed, the speech-text fusion semantic vector is output.

[0032] Preferably, the processing procedure of the confidence assessment module includes the following:

[0033] Extract speech-related feature subsets from the speech-text fusion semantic vector, calculate the overall clarity intensity of the speech feature subsets, and generate a quantified value of speech recognition accuracy by combining the signal-to-noise ratio coefficient;

[0034] Extract text-related feature subsets from the speech-text fusion semantic vector, calculate the overall coherence strength of the text feature subsets, and generate a semantic understanding accuracy quantification value by combining the grammatical error coefficient;

[0035] Extract the user's historical interaction frequency and operation preferences from the speech-text fusion semantic vector to generate user historical behavior parameters;

[0036] The accurate metric values ​​of speech recognition, semantic understanding, and user historical behavior parameters are combined into a feature vector, which is then input into a pre-trained logistic regression model. The feature vector is weighted and summed using the weight vector and bias value of the logistic regression model and mapped to a range of 0 to 1 to generate a confidence parameter.

[0037] Preferably, the processing procedure of the classification generation module includes the following:

[0038] User risk levels are classified based on the confidence level parameters. The user feature space is divided into grid cells. The risk level radius is calculated based on the confidence level value. A spherical region centered on the user behavior feature and with the risk level radius as its radius is defined as a sensitive region. Grid cells falling into the sensitive region are marked as high-risk user regions.

[0039] Preferably, the processing steps of the classification generation module further include the following:

[0040] Embed a behavior preference mapping function in the classification algorithm to calculate the preference degree and corresponding stability value of user classification nodes, remove candidate nodes whose stability value exceeds the maximum stability threshold, and retain candidate nodes whose stability value is within the maximum stability threshold;

[0041] After generating the initial classification, nodes with instability exceeding the limit are identified. Local boundary adjustments are made to these nodes and their neighboring nodes to reduce local preference differences until the stability values ​​of all nodes are within the maximum stability threshold or the maximum number of iterations is reached, thus obtaining the initial user classification results.

[0042] Preferably, the processing steps of the classification generation module further include the following:

[0043] The continuity of preferences in the initial user classification results is verified by calculating the local preference degree and preference change rate of each node, identifying nodes with discontinuous preferences, and using a clustering smoothing method to adjust the node affiliation of these nodes until the preference change rate of all nodes is less than or equal to the set preference change threshold.

[0044] The method detects stability mutations in smooth classification by calculating the stability value and stability change rate of each node, identifies nodes with stability mutations, and uses hierarchical clustering to adjust the classification boundary for these nodes until the stability change rate of all nodes is less than or equal to the set stability change threshold. Finally, the classification result is output to the end user.

[0045] Compared with the prior art, the beneficial effects of the present invention are:

[0046] In the voice data acquisition and preprocessing stage, the system's voice acquisition module can simultaneously acquire the user's raw voice data and corresponding text transcription data, and divide them into aligned voice segment units based on a unified timestamp. This precise alignment in the time dimension allows the features of the voice signal to be closely related to the semantics of the text information. Even with complex situations such as changes in user speech rate and pauses, it can ensure accurate matching between voice segments and text segments, laying a solid foundation for subsequent semantic fusion analysis. Simultaneously, the signal enhancement module performs targeted processing on voice and text data separately. For voice data, its environmental noise suppression processing effectively copes with complex and varied environmental noise, effectively filtering out both the noise of crowds in public places and the mechanical noise generated by equipment operation, reducing redundant information in the voice signal and improving the quality of voice feature extraction. For text transcription data, its grammar correction processing not only corrects common spelling errors but also optimizes for problems such as grammatical confusion and semantic ambiguity caused by speech recognition deviations, enabling the transcribed text to more accurately reflect the user's true expressive intent and providing high-quality data support for subsequent semantic analysis.

[0047] In the semantic fusion and confidence assessment stages, the semantic fusion module employs a context-constrained deep neural network. By inputting the enhanced speech-text dataset into the model, it fully leverages the semantic complementarity and correlation between speech and text data, rather than simply concatenating their features. The powerful feature learning capabilities of deep neural networks capture the deep connections between emotional features such as intonation and tone in speech signals and semantic logic in text information, thereby generating comprehensive and accurate speech-text fusion semantic vectors that more precisely represent the user's speech intent. The confidence assessment module further enhances the reliability of the system's judgment of user intent. It not only calculates the accuracy metrics for speech recognition and semantic understanding but also performs comprehensive analysis based on the user's historical behavior parameters to generate confidence parameters. These historical behavior parameters include information such as the user's past interaction habits and preferences. Combining them with the results of speech recognition and semantic understanding allows the system to make more comprehensive and reasonable judgments of user intent, avoiding misjudgments due to insufficient information from a single dimension and providing a reliable basis for subsequent user classification.

[0048] In the user classification generation stage, the classification module divides users into risk levels based on confidence parameters. It categorizes users into different risk levels according to the reliability of their voice intent recognition, allowing the platform to adopt differentiated service strategies for users at different risk levels. Simultaneously, this module embeds a behavioral preference mapping function into the classification algorithm. This fully considers differences in user behavioral preferences, transforming historical user behavior data into quantifiable preference features and integrating them into the classification algorithm. This ensures that the classification results better match the actual needs and characteristics of users. Furthermore, the classification algorithm iteratively adjusts the classification boundary to within a stability threshold, achieving adaptive optimization of the classification boundary without relying on human experience. When user group characteristics change, the classification boundary can be adjusted promptly, ensuring that the classification results maintain high stability and accuracy, thus providing precise user classification support for applications in different scenarios. Whether accurately identifying user needs and improving service efficiency in customer service scenarios, or accurately matching user preferences and optimizing recommendation effects in e-commerce scenarios, this system plays a crucial role in effectively improving the platform's service quality and user experience. Attached Figure Description

[0049] Figure 1 This is a timing diagram of the user classification system based on voice intent recognition described in this invention;

[0050] Figure 2 A flowchart of the voice acquisition module's processing flow;

[0051] Figure 3 This is a flowchart of the signal enhancement module process. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Please see Figure 1 This invention provides a user classification system based on voice intent recognition. The system includes: a voice acquisition module, a signal enhancement module, a semantic fusion module, a confidence assessment module, and a classification generation module. Specific implementation details are as follows:

[0054] The speech acquisition module acquires the user's raw speech data and corresponding text transcription data, and divides it into aligned speech segment units based on a unified timestamp. The signal enhancement module performs environmental noise suppression on the raw speech data of each speech segment unit and grammatical error correction on the text transcription data, outputting an enhanced speech-text dataset. The semantic fusion module inputs the enhanced speech-text dataset into a context-constrained deep neural network to obtain a speech-text fused semantic vector. The confidence assessment module calculates the speech recognition accuracy metric and the semantic understanding accuracy metric based on the speech-text fused semantic vector, and combines this with user historical behavior parameters to obtain a confidence parameter. The classification generation module classifies users' risk levels based on the confidence parameter, embeds a behavior preference mapping function into the classification algorithm, and iteratively adjusts the classification boundary to within a stability threshold to obtain the initial user classification result.

[0055] Example 1: See Figure 2 Sound is captured using a six-microphone array arranged in a ring, with each microphone unit synchronously recording the audio stream at a 16kHz sampling rate. This configuration effectively captures speech signals from different directions while preserving spatial acoustic characteristics. The speech recognition engine employs an end-to-end automatic speech recognition model based on a deep neural network. This model directly maps the original audio waveform to a text sequence, avoiding the error accumulation caused by multiple independent processing modules in traditional methods. The transcription process is performed in real time and is kept in hardware-level synchronization with the audio acquisition. The time alignment mechanism uses a dynamic time warping algorithm to handle the temporal correspondence between speech frames and text characters. The algorithm first processes the audio signal into frames, each 25 milliseconds long with a frame shift of 10 milliseconds. Then, it calculates the cost matrix between the audio features and text characters of each frame, and finally achieves frame-level alignment by finding the minimum cost path. The conversion from the text coordinate system to the user identity coordinate system requires the establishment of a user-specific semantic mapping dictionary. This dictionary contains the user's identity information provided during registration, a commonly used vocabulary, and personalized expression habits. The conversion process replaces general text tags with user-specific semantic tags by looking up the mapping dictionary.

[0056] The fixed time interval division employs a sliding window mechanism with a window length of 500 milliseconds and a step size of 250 milliseconds. This segmentation process divides the continuous multi-channel raw speech data stream and text transcription data stream into segments. Each generated speech segment unit contains audio data from all microphones and a corresponding text transcription fragment within the time window. Noise reduction preprocessing of the multi-channel raw speech data subset uses an adaptive filtering algorithm. This algorithm constructs a noise spectrum template by referencing environmental noise samples collected by the microphones, then subtracts noise components from the main microphone signal, retaining the clean speech signal. Word segmentation of the text transcription data subset uses a maximum probability word segmentation algorithm based on a statistical language model. The algorithm first constructs a dictionary containing commonly used words, then calculates the probability values ​​of all possible word segmentation paths in the text sequence, and finally selects the segmentation scheme with the highest probability as the result. Key intent information extraction is achieved through a pattern matching algorithm. The algorithm predefines a series of intent templates, each containing a keyword sequence and grammatical structure pattern for a specific intent. User intent is identified by matching the word segmentation results with the templates. Synchronous acquisition of multi-channel voice data requires a precise clock synchronization mechanism. The main controller ensures that all microphone units start recording simultaneously through hardware trigger signals, with sampling clock deviation controlled within microseconds. The speech recognition engine is deployed using a distributed architecture. Audio data is transmitted to the computing cluster via a high-speed network for parallel processing. Each computing node is responsible for processing the recognition task of one audio channel, and the final results are summarized and verified by the main node.

[0057] The dynamic time warping algorithm is implemented using a recursive computation method. To avoid excessive computational complexity, constraints are used in practical applications to limit the path search range, transforming the global alignment problem into a local optimum search. The establishment of the user identity coordinate system requires continuous updating and maintenance. The system records new words and expressions in each user interaction and periodically expands and optimizes the semantic mapping dictionary. The segmentation of speech segments needs to consider semantic integrity. In addition to fixed time intervals, the system adds an adaptive adjustment mechanism based on semantic boundaries, immediately completing the current segmentation when a sentence end marker is detected. The adaptive filtering algorithm in the noise reduction preprocessing uses an LMS adaptive filter. The filter coefficients are updated in real time according to noise statistical characteristics to achieve the best noise reduction effect.

[0058] The language model used in word segmentation employs a ternary grammar model, with model parameters trained from a large-scale corpus, effectively handling ambiguous segmentation issues in Chinese text. The intent template library is categorized and managed according to application domains, with each domain containing dozens of finely defined intent templates. The template matching algorithm supports fuzzy matching and partial matching modes. The multi-microphone array's geometry is optimized, with six microphones arranged in an equally spaced ring with a diameter of 15 cm. This configuration provides good spatial resolution while maintaining portability. Audio acquisition utilizes a 24-bit analog-to-digital converter with a dynamic range of 96 dB, ensuring the capture of complete signals from subtle speech details to high-intensity sounds. The speech recognition model's training data includes samples from various dialects and accents. The model structure employs a hybrid architecture of convolutional neural networks and recurrent neural networks, with convolutional layers extracting spectral features and recurrent layers handling temporal relationships. The cost function in temporal alignment comprehensively considers multiple factors such as spectral distance, phoneme duration ratio, and contextual relevance, ensuring that the alignment results conform to acoustic linguistic principles.

[0059] The user identity dictionary update mechanism employs incremental learning. New words are automatically identified through statistical analysis, and only those reaching a certain importance threshold are formally added to the dictionary. Semantic boundary detection during segmentation is based on pause features in speech signals and punctuation marks in text. The detection algorithm combines energy thresholding and statistical language model prediction. The order of the adaptive filter is dynamically adjusted according to environmental noise characteristics, using lower-order filters in steady-state noise environments and higher-order filters in non-steady-state noise environments. The word segmentation algorithm's dictionary includes general vocabulary and domain-specific vocabulary. Domain-specific vocabulary enjoys higher priority during matching to ensure correct segmentation of technical terms. During intent extraction, a confidence score is calculated for each matching result. Only intents with scores exceeding a threshold are adopted; cases below the threshold are transferred to a manual review process.

[0060] Example 2: See Figure 3The implementation of the signal enhancement module involves the collaborative operation of two independent processing paths: speech and text. For the processing of multi-channel raw speech data, an enhancement model is first established by simulating the propagation characteristics of acoustic signals in a noisy environment. This model constructs an indoor sound field propagation map based on ray tracing and calculates the signal-to-noise ratio (SNR) distribution of the speech signal at different spatial locations. SNR calculation employs band energy analysis, dividing the speech signal into multiple critical frequency bands and calculating the energy ratio of signal to noise in each band to form a full-band SNR distribution map. The attenuation coefficient is determined based on the absorption characteristics of sound waves in air and the environmental reflection coefficient, dynamically adjusting the gain parameters of each frequency band in conjunction with the speech segment duration. The speech data energy value adjustment uses an adaptive gain control algorithm. This algorithm calculates the gain curve in real time based on the attenuation coefficient and speech segment length, performing dynamic range compression and frequency band equalization on the speech signal. The enhanced speech data undergoes phoneme segmentation, and a Hidden Markov Model (HMM) is used for phoneme boundary detection. The model input is the Mel-frequency cepstral coefficient feature sequence, and the output is the phoneme start and end timestamps. The feature extraction stage calculates various acoustic features, including linear prediction coefficients and perceptual linear prediction coefficients. These features together constitute a speech feature set that reflects the speech content. The feature dimensions are uniformly normalized to a fixed length for subsequent processing.

[0061] The augmentation of text transcription data leverages the parsing capabilities of language models within grammatical structures. Abnormal structures in the text are identified by calculating the probability distribution of grammatical errors. Grammatical error detection employs an n-gram language model to calculate the perplexity index of lexical sequences, with high-perplexity regions marked as potential error areas. An error type classifier determines the specific error type based on context, including word order errors, lexical errors, and semantic errors. Correction weights are determined based on the severity and frequency of error types, using a combination of expert scoring and statistical learning to assign corresponding weight values ​​to different error types. Lexical sequence adjustment utilizes a sequence-to-sequence neural machine translation model. This model takes text with grammatical errors as input and outputs grammatically corrected text. Model training uses a large-scale parallel corpus containing the correspondence between correct and commonly erroneous texts. The augmented text data undergoes semantic role labeling, employing a transition-based parsing algorithm to identify predicate-argument structures and label the semantic role of each word in the sentence, such as agent, patient, and time. In the dependency parsing stage, a graph-based parsing algorithm constructs a syntactic dependency tree, analyzes the grammatical modification relationships between words, and ultimately generates a text semantic set reflecting the intent structure.

[0062] The integration of the speech feature set and the text semantic set employs a time-aligned multimodal fusion strategy. First, the speech feature sequence and the text semantic sequence are aligned by timestamp. During alignment, a dynamic time warping algorithm is used to resolve temporal differences between the two modalities. The aligned feature vectors are then concatenated into a multimodal feature vector, with the vector dimension remaining fixed to facilitate subsequent neural network processing. The entire signal enhancement module adopts a pipelined architecture design, with speech and text processing performed in parallel. Finally, modal fusion is achieved in the integration stage. Multiple quality checkpoints are set within the module to verify the validity of intermediate results and ensure the reliability of the output data. The establishment of the acoustic signal propagation model requires the pre-collection of environmental acoustic parameters, including room reverberation time and background noise spectrum characteristics. The model parameters are calibrated based on actual environmental measurements to ensure that the simulation results are consistent with the actual sound field characteristics. The signal-to-noise ratio (SNR) distribution calculation uses a sliding window method, with the window length consistent with the speech segment division. The calculation results are stored as a frequency band-SNR matrix for subsequent processing. The calculation of attenuation coefficient takes into account multiple factors such as air absorption, obstacle blockage and interface reflection. Different attenuation models are used for different frequency bands. Low frequency bands mainly consider air absorption loss, while high frequency bands consider scattering and diffraction effects more.

[0063] The adaptive gain control algorithm employs a feedforward structure, first analyzing the short-term energy distribution of the input signal, then calculating the required gain value based on the attenuation coefficient. Gain adjustment uses a soft threshold method to avoid signal distortion. The hidden Markov model in phoneme segmentation uses a three-state left-to-right structure, with each state corresponding to the initial, stable, and final stages of a phoneme. Model parameters are obtained through training on a large amount of labeled data. The feature extraction process includes feature transformation and dimensionality reduction steps. First, various primary acoustic features are calculated, then principal component analysis is used to retain the main feature components, ultimately forming a compact and representative feature vector. Syntax error detection employs a multi-layered processing strategy: first, spell checking corrects obvious spelling errors; then, grammar rule matching detects structures that violate grammar rules; and finally, semantic consistency analysis detects semantic errors. The error type classifier uses a combination of rule-based and statistical methods. Common error types are directly identified using a rule base, while rare error types are classified using a machine learning model. Weight allocation considers the impact of errors on understanding, assigning higher weights to serious errors and lower weights to minor errors.

[0064] The sequence-to-sequence model employs an encoder-decoder architecture. The encoder maps the input text to a latent representation, and the decoder generates revised text based on the latent representation. Model training utilizes an attention mechanism to improve long-distance dependency issues. Semantic role labeling employs a feature-engineering-based machine learning approach, extracting multiple features such as vocabulary, part-of-speech, and syntax, and using a conditional random field model for sequence labeling. Dependency analysis uses a dynamic programming-based parsing algorithm to find the dependency tree structure with the highest probability, considering both syntactic and semantic constraints during the parsing process. Multimodal alignment employs a bidirectional alignment strategy, considering both speech-to-text and text-to-speech alignment, ultimately taking the intersection of the two alignment results as the final alignment point. Feature concatenation uses an early fusion method, connecting speech feature vectors and text semantic vectors along the feature dimension to form an enhanced speech-text dataset.

[0065] Example 3: The implementation of the semantic fusion module is based on a deep neural network architecture to process the enhanced speech-text dataset. This network structure includes two core components: a feature encoder and a semantic decoder. The feature encoder receives the speech feature set and text semantic set output from the signal enhancement module as input data. The speech feature set includes acoustic feature vectors such as Mel-frequency cepstral coefficients and linear prediction coefficients, while the text semantic set includes grammatical feature vectors generated by semantic role labeling and dependency analysis. These input data are first standardized to eliminate dimensional differences before being input into the convolutional neural network layer of the feature encoder. The convolutional neural network uses a multi-layer one-dimensional convolutional structure to process temporal feature sequences. The kernel size is set to 5 for the speech feature set and 3 for the text semantic set. Pooling layers are connected after the convolutional layers for feature dimensionality reduction. Deep feature extraction is achieved by alternately stacking convolutional and pooling layers. The attention mechanism adopts a multi-head self-attention calculation method. First, the deep features are mapped to the query, key, and value space, then the attention weight distribution is calculated, and finally, feature fusion is achieved through weighted summation to generate a speech-text fused semantic vector.

[0066] The training process of deep neural networks employs an end-to-end approach to optimize parameters. The loss function of the feature encoder consists of reconstruction loss and contextual constraint loss. The reconstruction loss measures the semantic decoder's ability to reconstruct the original input from the fused semantic vectors, while the contextual constraint loss is calculated based on the semantic consistency residual. The semantic consistency residual is obtained by comparing the feature similarity between speech and text modalities, and its specific calculation method is as follows:

[0067]

[0068] in: Indicates loss of contextual constraints. Indicates batch size, Indicates the first The speech feature vector of each sample, Indicates the first The text feature vector of each sample, operators Represents the vector dot product. Represents the vector norm.

[0069] The semantic decoder's loss function focuses on optimizing reconstruction quality, using mean squared error to calculate the difference between the original input and the reconstructed output. During training, stochastic gradient descent is used to update network parameters, and the learning rate employs an exponential decay strategy. Network initialization uses the Xavier method to set initial weights, with the bias term initialized to zero. Training data is divided into batches, with the batch size dynamically adjusted based on GPU memory capacity. The feature encoder's convolutional layers use the ReLU activation function to introduce non-linear transformations, and the pooling layers employ max pooling to preserve salient features. The attention mechanism uses eight heads, each responsible for attention calculations in different feature subspaces. The semantic vector for speech-text fusion is fixed at 512 dimensions. This dimension design balances representational power and computational efficiency, ensuring semantic continuity in the vector space, where similar semantic inputs are mapped to adjacent regions.

[0070] The gradient update during training employs the Adam optimization algorithm, which combines momentum and adaptive learning rate adjustment to effectively handle sparse gradients. Gradient clipping prevents gradient explosion. The weight allocation of the loss function is determined through grid search, with a 2:1 weight ratio for reconstruction loss and contextual constraint loss, demonstrating optimal balance in multiple validation set experiments. Network regularization combines Dropout and weight decay, with a Dropout ratio of 0.2 and a weight decay coefficient of 0.001, effectively preventing overfitting. An early stopping mechanism determines termination based on validation set loss changes, automatically stopping training when the validation set loss stops decreasing for several consecutive epochs. Batch normalization layers are inserted after convolutional layers and before activation functions to accelerate training convergence and improve gradient flow. Normalization statistics are calculated differently in training and inference phases. The moving average calculated during training is used directly in the inference phase to ensure output stability.

[0071] The input pipelines for the speech feature set and the text semantic set employ an asynchronous loading method. Data from both modalities are preprocessed separately and then merged into batch data. Data augmentation techniques are applied during the training phase, including temporal distortion of speech features and random replacement of text features. Model checkpoints periodically save optimal parameters, allowing resumption from the most recent checkpoint after training interruptions. Training logs record loss changes and evaluation metrics for subsequent analysis. The trained feature encoder maps multimodal inputs to a unified semantic vector space. The semantic decoder, as an auxiliary training component, does not participate in computation during the inference phase. The output speech-text fusion semantic vector serves as the input features for subsequent modules.

[0072] Example 4: The implementation of the confidence assessment module requires the extraction of multi-dimensional features from the semantic vector of speech-text fusion for comprehensive analysis. This module first processes a subset of speech-related features, including the variance statistics of Mel-frequency cepstral coefficients, the stability index of the fundamental frequency profile, and the harmonic-to-noise ratio (HNR) feature. The overall intelligibility strength is calculated based on a weighted combination of these features, where the weights of each feature are determined through principal component analysis, and the HNR coefficient is directly taken from the environmental noise assessment results provided by the signal enhancement module. The quantified value of speech recognition accuracy is finally obtained by multiplying the intelligibility strength by the HNR coefficient. The processing of the text-related feature subset focuses on linguistic characteristics. Feature extraction includes the transition probability of part-of-speech sequences, the depth distribution of syntactic trees, and the consistency index of semantic role labeling. The overall coherence strength is obtained by calculating the weighted sum of the entropy values ​​of these features. The grammatical error coefficient is derived from the average value of the correction weights recorded during the text enhancement process. The quantified value of semantic understanding accuracy is determined by the ratio of coherence strength to the grammatical error coefficient. This calculation process ensures that the text quality assessment considers both surface coherence and deep semantic rationality.

[0073] The generation of user historical behavior parameters requires the analysis of long-term interaction data. Historical interaction frequency is calculated by statistically analyzing the number of user sessions and active duration within a certain time window, while operational preferences are derived by analyzing the types of commonly used commands and their frequency of occurrence. These parameters are combined with the speech and text features of the current session to form a complete profile of user behavior characteristics. All feature parameters are normalized to eliminate dimensional differences. The feature vector is composed by concatenating the accurate metric values ​​of speech recognition, semantic understanding, and user historical behavior parameters in a specific order to form a fixed-dimensional input vector. The logistic regression model is trained using a large number of labeled samples. Model parameters are learned through maximum likelihood estimation. The weight vector reflects the degree of influence of each feature on the confidence level, the bias term adjusts the overall output level, and the final output value is mapped to a confidence parameter between 0 and 1 using the sigmoid function.

[0074] The classification generation module categorizes users into risk levels based on confidence parameters. A kd-tree structure is used to establish a multi-dimensional index in the user feature space, dividing the space into uniform grid cells. The risk level radius is calculated based on a quadratic function of the confidence parameters; lower confidence levels correspond to larger radius values. The sensitive region is defined as a hypersphere centered on the user behavior feature vector and encompassing the risk level radius. The spatial relationship between the grid cells and the hypersphere is calculated using Euclidean distance. Cells falling within the hypersphere are marked as high-risk user regions, forming clusters of high-risk users in the feature space. Table 1 illustrates the correspondence between confidence parameters and risk level classifications for different user samples.

[0075] Table 1: User Confidence Parameters and Risk Level Classification

[0076] User ID Speech recognition accuracy quantification Semantic understanding accurate metric Historical interaction frequency Operational Preference Index Confidence parameter Risk level radius Risk level U1001 0.87 0.92 0.75 0.82 0.84 0.15 Low risk U1002 0.62 0.78 0.45 0.63 0.62 0.28 Medium risk U1003 0.41 0.53 0.32 0.47 0.43 0.42 High risk U1004 0.75 0.81 0.68 0.74 0.75 0.21 Low risk U1005 0.58 0.65 0.52 0.59 0.59 0.31 Medium risk

[0077] In practical applications, taking user U1003 as an example, this user has a low speech recognition accuracy metric (0.41), indicating difficulties in speech signal quality or the recognition process. The semantic understanding accuracy metric is also relatively low (0.53), reflecting potential obstacles in language expression or semantic understanding. The historical interaction frequency is 0.32, indicating low user activity, and the operation preference index of 0.47 indicates a significant difference between the user's operational behavior and the usual pattern. These factors collectively lead to a low confidence parameter (0.43), resulting in a large risk level radius (0.42), ultimately classifying the user as a high-risk user. For user U1001, all indicators perform well, with a speech recognition accuracy metric of 0.87, a semantic understanding accuracy metric of 0.92, and both historical interaction frequency and operation preference index remaining at high levels. These characteristics collectively contribute to a high confidence parameter (0.84) and a small risk level radius (0.15), ultimately classifying the user as a low-risk user. During the risk level classification process, the system updates user feature vectors and confidence parameters in real time. When user behavior characteristics change, the risk level may be adjusted accordingly. The labeling of high-risk user areas adopts a progressive strategy: first, obvious high-risk areas are labeled, and then the risk level classification of boundary areas is gradually refined through iterative calculation.

[0078] Example 5: First, a behavior preference mapping function is embedded in the classification algorithm. This function constructs a multi-dimensional feature space based on the user's historical behavior data and calculates the preference index for each user classification node. The preference calculation comprehensively considers the user's operation frequency, command type preference, and interaction pattern characteristics, and obtains a standardized value between 0 and 1 through a weighted summation method. The stability value is calculated by analyzing the fluctuation of the classification results of the node within a continuous time window, and using variance calculation to measure the classification consistency of the node. Taking a practical application scenario as an example, the system initially generates 20 candidate classification nodes. After stability evaluation, it is found that the stability value of 7 nodes exceeds the maximum stability threshold of 0.05, and these nodes are removed from the candidate set. The remaining 13 nodes enter the initial classification stage, with each node representing a potential user category. After generating the initial classification, the system identifies 3 nodes with stability exceeding the limit, and these nodes have a large difference in preference between themselves and their neighboring nodes. Local boundary adjustment uses the gradient descent method to optimize the classification boundary function, and gradually reduces the local preference difference by iteratively adjusting the decision boundary between nodes. During the adjustment process, the system recalculates the stability value of all nodes in each iteration until the stability value of all nodes is controlled within 0.05 or reaches the preset limit of 100 iterations.

[0079] In the preference continuity verification phase, the system calculates the local preference degree and preference change rate of each node, discovering a significant preference discontinuity between two adjacent nodes. The preference change rate of these two nodes reaches 0.015, exceeding the set preference change threshold of 0.01. To address this, the system employs a clustering smoothing method to redistribute node affiliations, analyzing the density distribution of the node feature space using the k-nearest neighbor algorithm to merge discrete nodes into the nearest dense cluster. After preference continuity adjustment, the system enters the stability mutation detection phase. Calculating the stability value and stability change rate of each node reveals that the stability change rate of a boundary node reaches 0.006, exceeding the stability change threshold of 0.005. This node is located at the boundary of two classification regions, and its stability value fluctuates significantly during continuous iterations. The system then uses a hierarchical clustering method to readjust the classification boundary, reconstructing the classification tree structure through a bottom-up aggregation strategy, merging the node with stability mutations with its adjacent nodes.

[0080] In the final output stage, the system generates a tree-structured final user classification result. Each leaf node represents a specific user category, containing the category's feature description and boundary definition. Internal nodes store the parameter set of the classification boundary function, including weight vectors and bias terms. The entire classification tree forms eight final user categories, each with a clearly defined preference range and stability index. Taking one user category as an example, this category contains users with a preference between 0.7 and 0.8, and a stability value controlled within 0.03. Users in this category exhibit highly consistent operational behavior characteristics, preferring to use specific types of voice commands, and maintaining a medium-to-high interaction frequency. The classification boundary function ensures that the user feature distribution of this category has good cohesion and maintains appropriate separation from neighboring categories.

[0081] Another user category includes users with a preference score between 0.4 and 0.5. These users exhibit relatively dispersed behavior and have slightly higher stability scores, still controlled within 0.04. The system sets a more lenient classification boundary for this category to accommodate a more diverse group of users with varied behavioral characteristics. Through hierarchical clustering adjustments, the boundary region of this category is optimized, avoiding overlap with adjacent categories. The entire implementation process employs an incremental update mechanism. When new user data reaches a certain scale, the system automatically triggers a reclassification process. During reclassification, the original classification structure is retained as a foundation, with only local adjustments made to affected nodes to improve classification efficiency. The final user classification results are pushed to the user management system, providing a basis for personalized services and risk control. The system continuously monitors classification quality during operation, periodically evaluating the continuity and stability indicators of preferences for each category. When an indicator exceeds a threshold range, the corresponding adjustment process is automatically initiated. This dynamic maintenance mechanism ensures that user classification results can adapt to changes in data distribution, maintaining the accuracy and usability of the classification model. All adjustment records and classification parameters are stored in the system log for subsequent analysis and optimization.

[0082] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0083] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A user classification system based on voice intent recognition, characterized in that, include: The system includes a voice acquisition module, a signal enhancement module, a semantic fusion module, a confidence assessment module, and a classification generation module. The voice acquisition module acquires the user's raw voice data and corresponding text transcription data, and divides it into aligned voice segment units based on a unified timestamp. The signal enhancement module performs environmental noise suppression processing on the raw speech data of each speech segment unit, performs grammatical error correction processing on the text transcription data, and outputs an enhanced speech-text dataset. The semantic fusion module inputs the enhanced speech-text dataset into a context-constrained deep neural network to obtain a speech-text fusion semantic vector. The confidence assessment module calculates the speech recognition accuracy metric and the semantic understanding accuracy metric based on the speech-text fusion semantic vector, and obtains the confidence parameter by combining the user's historical behavior parameters. The classification generation module: classifies user risk levels according to the confidence parameters, embeds the behavior preference mapping function into the classification algorithm, and iteratively adjusts the classification boundary to within the stability threshold to obtain the initial user classification result; The processing steps of the signal enhancement module include the following: For multi-channel raw speech data, the signal-to-noise ratio distribution of the speech signal is calculated by simulating the propagation characteristics of acoustic signals in a noisy environment, the attenuation coefficient of each frequency band is determined, and the energy value of the speech data is adjusted according to the attenuation coefficient and the length of the speech segment to generate enhanced speech data. Then, phoneme segmentation and feature extraction are performed on the enhanced speech data to generate a speech feature set that reflects the speech content. The processing steps of the signal enhancement module also include the following: For text transcription data, the parsing process of language model in grammatical structure is simulated to calculate the probability distribution of grammatical errors, determine the correction weight of error type, and adjust the word sequence of text data according to the correction weight to generate enhanced text data. Then, semantic role labeling and dependency analysis are performed on the enhanced text data to generate a text semantic set that reflects the intention structure. Finally, the speech feature set and the text semantic set are integrated into an enhanced speech-text dataset; The semantic fusion module's processing procedure includes the following: The enhanced speech-text dataset is input into a deep neural network, which includes a feature encoder and a semantic decoder. The feature encoder receives the speech feature set and the text semantic set, extracts deep features from the speech feature set and the text semantic set through a convolutional neural network, and uses an attention mechanism to perform weighted fusion of the deep features to generate a speech-text fused semantic vector. The semantic fusion module's processing also includes the following: During the training of the deep neural network, the loss function of the feature encoder consists of reconstruction loss and context constraint loss. The context constraint loss is calculated based on semantic consistency residuals, and the reconstruction loss measures the probability that the speech-text fusion semantic vector is reconstructed into the original input by the semantic decoder. The loss function of the semantic decoder is used to optimize the ability to reconstruct the original input; After training is completed, the speech-text fusion semantic vector is output.

2. The user classification system based on voice intent recognition according to claim 1, characterized in that, The processing steps of the voice acquisition module include the following: The system collects multi-channel raw speech data containing user identification features through a microphone array, and generates corresponding text transcription data through a speech recognition engine. A time alignment mechanism is used to align multi-channel raw speech data and text transcription data to a unified time reference; Transform the transcribed text data from the text coordinate system to the user identity coordinate system to achieve semantic alignment with the multi-channel raw speech data; The continuous multi-channel raw speech data stream and text transcription data stream are divided into discrete speech segment units at fixed time intervals. Noise reduction preprocessing is performed on a subset of multi-channel raw speech data, and word segmentation is performed on a subset of text transcribed data to extract key intent information.

3. The user classification system based on voice intent recognition according to claim 1, characterized in that, The processing steps of the confidence assessment module include the following: Extract speech-related feature subsets from the speech-text fusion semantic vector, calculate the overall clarity intensity of the speech feature subsets, and generate a quantified value of speech recognition accuracy by combining the signal-to-noise ratio coefficient; Extract text-related feature subsets from the speech-text fusion semantic vector, calculate the overall coherence strength of the text feature subsets, and generate a semantic understanding accuracy quantification value by combining the grammatical error coefficient; Extract the user's historical interaction frequency and operation preferences from the speech-text fusion semantic vector to generate user historical behavior parameters; The accurate metric values ​​of speech recognition, semantic understanding, and user historical behavior parameters are combined into a feature vector, which is then input into a pre-trained logistic regression model. The feature vector is weighted and summed using the weight vector and bias value of the logistic regression model and mapped to a range of 0 to 1 to generate a confidence parameter.

4. The user classification system based on voice intent recognition according to claim 3, characterized in that, The processing steps of the classification generation module include the following: User risk levels are classified based on the confidence level parameters. The user feature space is divided into grid cells. The risk level radius is calculated based on the confidence level value. A spherical region centered on the user behavior feature and with the risk level radius as its radius is defined as a sensitive region. Grid cells falling into the sensitive region are marked as high-risk user regions.

5. The user classification system based on voice intent recognition according to claim 4, characterized in that, The processing steps of the classification generation module also include the following: Embed a behavior preference mapping function in the classification algorithm to calculate the preference degree and corresponding stability value of user classification nodes, remove candidate nodes whose stability value exceeds the maximum stability threshold, and retain candidate nodes whose stability value is within the maximum stability threshold; After generating the initial classification, nodes with instability exceeding the limit are identified. Local boundary adjustments are made to these nodes and their neighboring nodes to reduce local preference differences until the stability values ​​of all nodes are within the maximum stability threshold or the maximum number of iterations is reached, thus obtaining the initial user classification results.

6. The user classification system based on voice intent recognition according to claim 5, characterized in that, The processing steps of the classification generation module also include the following: The continuity of preferences in the initial user classification results is verified by calculating the local preference degree and preference change rate of each node, identifying nodes with discontinuous preferences, and using a clustering smoothing method to adjust the node affiliation of these nodes until the preference change rate of all nodes is less than or equal to the set preference change threshold. The method detects stability mutations in smooth classification by calculating the stability value and stability change rate of each node, identifies nodes with stability mutations, and uses hierarchical clustering to adjust the classification boundary for these nodes until the stability change rate of all nodes is less than or equal to the set stability change threshold. Finally, the classification result is output to the end user.

Images