Speech recognition control system driven by user behavior prediction

Abstract

The invention provides a voice recognition control system driven by user behavior prediction. The voice recognition control system comprises a voice acquisition and preprocessing module, a voice feature extraction module, a user behavior prediction module, a voice recognition module and a decision fusion module. According to the method, voice feature extraction is combined, user behavior prediction based on LSTM-curved surface fitting-large language model fusion and a Transform voice recognition technology are combined, and the defects of a traditional method are effectively overcome by the system. The voice recognition accuracy in a complex environment test is improved, the error recognition rate is greatly reduced, and it is ensured that the system can accurately understand the voice instruction of the user.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition control system driven by user behavior prediction. Background Art

[0002] With the rapid development of artificial intelligence technology, voice recognition control systems have been widely used in smart homes, smart cars, smart customer service and other fields. Traditional voice recognition control systems mainly rely on the voice signal itself for recognition. After pre-processing, the collected voice signal is directly input into the voice recognition algorithm to obtain the text result, and then the corresponding operation is performed based on it. However, this approach has obvious limitations:

[0003] Poor environmental adaptability: In complex real-world environments, such as noisy public places or scenes with multiple people speaking simultaneously, voice signals are easily affected by noise and reverberation, resulting in a significant drop in recognition accuracy. For example, in noisy environments like subway stations, the recognition error rate for voice control commands in smart homes can be as high as 40%.

[0004] Lack of understanding of user behavior: Operations are executed solely based on voice content, without considering the user's historical behavior habits, operation context, and other information. For example, if a user habitually turns on the living room lights and plays soothing music at 7 p.m. every night, traditional systems cannot anticipate user needs and can only execute operations after receiving clear voice commands, resulting in a less intelligent and convenient interactive experience.

[0005] The problem of ambiguity is prominent: Voice commands are often ambiguous, meaning the same command can have different meanings in different scenarios. For example, "open it" can refer to different objects in different device operation scenarios. Traditional systems struggle to accurately understand its specific meaning, making it prone to misoperation.

[0006] At the same time, single algorithmic models face numerous challenges when handling speech recognition and user behavior analysis. For example, HMM-based speech recognition models are insufficiently capable of processing long-term dependencies, while simple statistical models struggle to mine complex behavioral patterns for user behavior prediction. Therefore, a new speech recognition control system is urgently needed that comprehensively considers multiple factors to improve recognition accuracy and intelligence. Summary of the Invention

[0007] The present invention provides a user behavior prediction-driven speech recognition control system to solve one or more of the above problems.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A speech recognition control system driven by user behavior prediction, comprising:

[0010] The speech acquisition and preprocessing module is used to acquire time-domain speech signals, perform noise reduction and frame processing on the time-domain speech signals, and convert them into frequency-domain signals;

[0011] The speech feature extraction module is connected to the speech acquisition and preprocessing module and is used to convert the frequency domain signal into the Mel frequency domain, extract the Mel spectrum and perform logarithmic and discrete cosine transform to obtain the Mel frequency cepstral coefficient feature vector;

[0012] The user behavior prediction module is connected to the speech feature extraction module and is used to predict user behavior based on the long short-term memory network, surface fitting algorithm and large language model, combined with the Mel-frequency cepstral coefficient feature vector and historical behavior data;

[0013] The speech recognition module connects the speech feature extraction module and the user behavior prediction module and is used to convert the Mel-frequency cepstral coefficient feature vector into speech-to-text recognition results based on the Transformer model;

[0014] The decision fusion module connects the user behavior prediction module and the speech recognition module, and is used to perform weighted fusion of the user behavior prediction results and the speech text recognition results, and output the final recognition results to control external devices.

[0015] In this specification, the voice collection and preprocessing module uses Wiener filtering or wavelet noise reduction algorithm to perform noise reduction on the voice signal.

[0016] In this specification, the speech feature extraction module processes the frequency domain signal through a Mel filter bank consisting of 20 to 40 Mel filters to obtain a Mel spectrum.

[0017] In this specification, the historical behavior data obtained by the user behavior prediction module includes device control records, voice command records and operation timestamps.

[0018] In this specification, the key numerical features extracted by the user behavior prediction module from historical behavior data include operation frequency, operation time interval and operation duration.

[0019] In this manual, the user behavior prediction module first uses a long short-term memory network to process the Mel-frequency cepstral coefficient feature vector and the historical behavior data sequence to obtain a hidden layer output feature vector; then the hidden layer output feature vector is surface fitted with the key numerical features to obtain the fitting coefficient and the predicted trend value; finally, the semantic understanding vector of the preliminary text information of the speech and the historical behavior description text is combined with the large language model to generate a fusion feature vector for behavior prediction.

[0020] In this specification, the speech recognition module generates preliminary speech text information through an initial recognition model based on a hidden Markov model and inputs it into a large language model.

[0021] In this manual, surface fitting adopts polynomial fitting method. By training the data set composed of hidden layer output feature vector and key numerical features, the fitting coefficient of the polynomial is obtained. The degree of the polynomial is set according to the characteristics of the data set. The predicted trend value obtained by fitting reflects the changing trend of user behavior in the numerical feature dimension.

[0022] In this specification, the decision fusion module determines the weight parameters of weighted fusion based on historical data training, and the weight parameters range from 0 to 1.

[0023] In this manual, external devices include smart home devices, service robots, and in-vehicle voice control systems.

[0024] In summary, the present invention has at least the following beneficial effects:

[0025] High-Accuracy Recognition: Combining speech feature extraction, user behavior prediction based on a fusion of LSTM, surface fitting, and a large language model, and Transformer speech recognition technology, the system effectively overcomes the shortcomings of traditional methods. This improves speech recognition accuracy in complex testing environments, significantly reduces false positives, and ensures the system accurately understands user voice commands.

[0026] Intelligent Behavior Prediction: By leveraging historical behavioral data and large language models to predict user behavior, the system can proactively detect user needs. For example, in a smart home scenario, based on past user behavior, the system can automatically turn on lights and adjust air conditioning temperature before a user enters a room. This makes device control more intelligent and proactive, improves user operation efficiency, and significantly enhances the user interaction experience.

[0027] Strong adaptability across multiple scenarios: By integrating multiple algorithms and considering multi-dimensional data, the system can flexibly adapt to diverse application scenarios, including smart homes, smart cars, and smart offices. In a vehicle environment, it can quickly adapt to changes in interior noise levels and the driver's varying speaking habits. In office settings, it can accurately identify meeting commands and execute corresponding actions based on schedules, broadening the application boundaries of voice recognition control systems.

[0028] More accurate semantic understanding: The introduction of a large language model effectively resolves the ambiguity of voice commands. Combined with user behavior predictions, the system can accurately infer the specific meaning of voice commands in specific scenarios. For example, for the ambiguous command "start," the system can accurately determine whether it means starting music, starting meeting minutes, or other operations based on the user's historical behavior and the current context, making the interaction more natural and smooth. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 Schematic diagram of a speech recognition control system driven by user behavior prediction involved in the present invention.

[0031] Figure 2 It is a flow chart of the voice collection and preprocessing module involved in the present invention.

[0032] Figure 3 Schematic diagram of the flow of the speech feature extraction module involved in the present invention.

[0033] Figure 4 Schematic diagram of the flow of the user behavior prediction module involved in the present invention. DETAILED DESCRIPTION

[0034] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the embodiments of the present invention. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0035] The disclosure below provides many different embodiments or examples for implementing different structures of the embodiments of the present invention. In order to simplify the disclosure of the embodiments of the present invention, the components and configurations of specific examples are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. In addition, the embodiments of the present invention may repeat reference numerals and / or reference letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or configurations discussed.

[0036] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0037] like Figure 1 As shown, this embodiment provides a user behavior prediction-driven speech recognition control system, including:

[0038] The speech acquisition and preprocessing module is used to acquire time-domain speech signals, perform noise reduction and frame processing on the time-domain speech signals, and convert them into frequency-domain signals;

[0039] The speech feature extraction module is connected to the speech acquisition and preprocessing module and is used to convert the frequency domain signal into the Mel frequency domain, extract the Mel spectrum and perform logarithmic and discrete cosine transform to obtain the Mel frequency cepstral coefficient feature vector;

[0040] The user behavior prediction module is connected to the speech feature extraction module and is used to predict user behavior based on the long short-term memory network, surface fitting algorithm and large language model, combined with the Mel-frequency cepstral coefficient feature vector and historical behavior data;

[0041] The speech recognition module connects the speech feature extraction module and the user behavior prediction module and is used to convert the Mel-frequency cepstral coefficient feature vector into speech-to-text recognition results based on the Transformer model;

[0042] The decision fusion module connects the user behavior prediction module and the speech recognition module, and is used to perform weighted fusion of the user behavior prediction results and the speech text recognition results, and output the final recognition results to control external devices.

[0043] In some embodiments, the speech collection and preprocessing module uses Wiener filtering or wavelet noise reduction algorithm to perform noise reduction on the speech signal.

[0044] In some embodiments, the speech feature extraction module processes the frequency domain signal through a Mel filter bank consisting of 20 to 40 Mel filters to obtain a Mel spectrum.

[0045] In some embodiments, the historical behavior data acquired by the user behavior prediction module includes device control records, voice command records, and operation timestamps.

[0046] In some embodiments, the key numerical features extracted by the user behavior prediction module from the historical behavior data include operation frequency, operation time interval, and operation duration.

[0047] In some embodiments, the user behavior prediction module first uses a long short-term memory network to process the Mel-frequency cepstral coefficient feature vector and the historical behavior data sequence to obtain a hidden layer output feature vector; then performs surface fitting on the hidden layer output feature vector and the key numerical features to obtain the fitting coefficient and the predicted trend value; finally, the semantic understanding vector of the preliminary text information of the speech and the historical behavior description text is combined with the large language model to generate a fusion feature vector for behavior prediction.

[0048] In some embodiments, the speech recognition module generates preliminary speech text information through an initial recognition model based on a hidden Markov model and inputs the information into a large language model.

[0049] In some embodiments, surface fitting adopts a polynomial fitting method. By training a data set consisting of the hidden layer output feature vector and key numerical features, the fitting coefficient of the polynomial is obtained. The degree of the polynomial is set according to the characteristics of the data set. The predicted trend value obtained by fitting reflects the changing trend of user behavior in the numerical feature dimension.

[0050] In some embodiments, the decision fusion module determines a weight parameter of weighted fusion based on historical data training, and the weight parameter ranges from 0 to 1.

[0051] In some embodiments, the external devices include smart home devices, service robots, and in-vehicle voice control systems.

[0052] The technical concept of the present invention is as follows:

[0053] This user behavior prediction-driven speech recognition control system includes:

[0054] 1. Voice collection and preprocessing module (process as follows Figure 2 shown)

[0055] In practical applications, the voice acquisition and preprocessing module acts as the system's "auditory receptor," undertaking the critical task of acquiring and initially processing voice signals. This module uses a highly sensitive microphone to capture user voice information in real time. The raw time-domain voice signal x(t) is captured, where t represents time in seconds (s) and x(t) represents the amplitude of the voice signal at time t, typically in volts (V). This continuous waveform records the temporal changes in the strength of the voice wave.

[0056] However, actual collected speech signals are often mixed with various environmental noises, such as the sounds of running appliances and people chatting indoors, and traffic noise outdoors. To obtain a pure speech signal, this module first applies the Wiener filter algorithm to reduce noise on the original signal. Based on the statistical properties of the signal and noise, Wiener filtering effectively suppresses noise by minimizing the mean square error (MSE) to preserve the key components of the speech signal. If the noise environment is more complex, a wavelet noise reduction algorithm can also be used. This algorithm uses a wavelet transform to decompose the signal into different frequency scales. Then, thresholding is used to remove the coefficients of the noise-relevant frequency bands, achieving more refined noise reduction.

[0057] After noise reduction is completed, the speech signal needs to be framed because it has short-term stability, that is, its spectral characteristics are relatively stable in a short period of time (generally 10-30 milliseconds). The framing operation is like cutting a continuous speech stream into small time segments, which facilitates the subsequent independent analysis of each segment. The framed speech signal is converted from the time domain to the frequency domain through the short-time Fourier transform (STFT) to obtain the frequency domain signal X(f,t). Formula In the equation, f represents frequency, the unit is Hertz (Hz), which is used to describe the distribution of different frequency components in the signal; τ is the integral variable, which performs an integral operation on the signal over the entire time axis; w(τ) is the window function, common ones include Hanning window and Hamming window, which is used to intercept the local speech signal so that the local signal can be assumed to be stable during analysis; e -j2πf is a complex exponential function, j is an imaginary unit, and satisfies j 2 = -1. After short-time Fourier transform, the frequency components of the speech signal are clearly displayed. These frequency domain signals X(f, t) will serve as input data for the subsequent speech feature extraction module.

[0058] 2. Speech feature extraction module (process as follows Figure 3 shown)

[0059] The speech feature extraction module is a key link in the system's understanding of speech content. It acts like a "feature extractor" to extract the most representative speech features from the frequency domain signal. This module uses the Mel-Frequency Cepstral Coefficient (MFCC) algorithm, the design of which is inspired by the human auditory system's perception of sound frequency. The human auditory system's perception of sounds of different frequencies is not linear, but conforms to the Mel frequency scale. The MFCC algorithm uses this characteristic to convert the frequency domain signal into a Mel frequency domain that is more in line with the human ear's perception rules. In the specific process, the frequency domain signal X(f,t) is first passed through a Mel filter bank H consisting of M Mel filters. m (f) (m=1,2,…,M. In practical applications, the value of M is usually between 20 and 40. For example, when M=26, it means there are 26 Mel filters with different frequency response ranges.) Each Mel filter H m (f) has a specific frequency response curve, which performs weighted summation of the energy of the frequency domain signal X(f,t) in the corresponding frequency range to obtain the Mel spectrum S m (t), Formula S m (t)=∑ f |X(f,t)| 2 H m (f) depicts the energy accumulation process. Through the processing of the Mel filter bank, the frequency components important to human ear perception are highlighted and the influence of irrelevant frequency information is weakened.

[0060] Get Mel spectrum S mAfter (t), the Mel-spectrum spectrum is logarithmically processed to compress the dynamic range of the spectrum and make speech signal features of varying intensities easier to process and compare. A discrete cosine transform (DCT) is then performed to convert the logarithmic Mel-spectrum spectrum into an MFCC feature vector C(n), where n represents the dimensional index of the MFCC feature vector, n = 1, 2, …, N. Generally, N ranges from 12 to 16. For example, when N = 13, a 13-dimensional MFCC feature vector is obtained. This 13-dimensional feature vector contains important information, such as the spectral envelope of the speech signal, and can effectively characterize the acoustic characteristics of speech. The processed MFCC feature vector C(n) is transmitted simultaneously to the user behavior prediction module and the speech recognition module, providing basic data for subsequent user behavior prediction and speech content recognition.

[0061] 3. User behavior prediction module (process as follows Figure 4 shown)

[0062] data:

[0063] Historical behavior data series B t-n:t-1 :This sequence is a treasure trove of user operation data accumulated during the long-term operation of the system. The system continuously collects various types of information on user interactions with devices through the built-in sensors, control interfaces, and software logging functions of the devices. For example, in a smart home scenario, every time a user switches on or off a smart light, adjusts the temperature and wind speed of a smart air conditioner, it will record; in a voice interaction device, every voice command issued by the user and the result of the command execution will be recorded; at the same time, the timestamps of these operations will be accurately recorded. These data are arranged in chronological order to form a historical behavior data sequence B. t-n1:t-1 , where n1 represents the length of the historical data window. For example, when n1 = 5, it means that the user operation data within the past 5 time units are selected. These data reflect the user's behavioral habits and operation patterns over a period of time.

[0064] Key numerical features in historical behavior data: Quantitative indicators further extracted from historical behavior data sequences can more accurately characterize user behavior characteristics. For example, operation frequency counts the number of times a user operates a device in a unit of time (such as per minute), reflecting how frequently the user uses the device; time interval records the time difference between two consecutive user operations, which helps analyze the time patterns of user operations; operation duration indicates the time it takes a user to complete an operation, which can be used to judge the complexity of the operation. In addition, other numerical features may also be included, such as the change in device status parameters (such as the degree of air conditioning temperature adjustment). These features provide a quantitative basis for user behavior analysis from different angles.

[0065] Initial text information obtained by the speech recognition module: After receiving the MFCC feature vector, the speech recognition module first quickly processes it through an initial, simple recognition model (such as one based on the Hidden Markov Model (HMM)). This initial model can initially decode the speech signal and convert it into preliminary text information in the form of text. Although this preliminary text may contain some recognition errors, it contains the general semantic direction of the speech content and serves as an important clue for the larger language model to understand the user's intent.

[0066] Historical behavior description text: The system pre-establishes a text database that records past user actions in natural language. Examples include "Closed the study curtains at 3:00 PM on June 29, 2024" and "Played Jay Chou's songs last night." These text descriptions not only include the object and action of the operation but also contextual information such as time and location. This provides rich semantic information for the large language model to understand the context and intent of user actions.

[0067] LSTM feature extraction:

[0068] In the user behavior prediction module, the LSTM network serves as the core processing unit, responsible for mining the potential connection between speech signals and user behavior. t It is a carefully processed composite feature vector, which is composed of the MFCC feature vector C(n) and the historical behavior data sequence B t-n1:t-1 This fusion method combines the acoustic features of speech with the user's historical behavior patterns, providing more comprehensive input information for the LSTM network.

[0069] The LSTM network is implemented by forgetting the gate f t , input gate i t , cell state update and output gate o t Structures such as , can effectively process time series data. t By formula f t =σ(W f ·[h y-1 ,x t ]+b f ) is calculated, where σ is the Sigmoid activation function, which maps the input to between 0 and 1 and is used to control the cell state C at the previous moment t-1 What information needs to be forgotten? f is the weight matrix of the forget gate, b f are bias vectors, which determine the degree of influence of the input data on the output of the forget gate; [h t-1 ,x t ] means to hide the state h at the previous momentt-1 With the current input x t Perform splicing. Input gate i t and cell state updates Through i t =σ(W i ·[h t-1 ,x t ]+b i and Calculation is used to determine what information in the current input needs to be added to the cell state. t pass Update, where ⊙ represents element multiplication, which realizes the selective update of cell state. Output gate o t By o t =σ(W o ·[h t-1 ,x t ]+b o ) calculation, controlling which information in the cell state will be output as the hidden state h t , h t =o t ⊙tanh(C t ), the final H t =h t That is, the output feature vector of the LSTM network at time t, which contains the abstract representation of the current input data and the memory of historical information. f ,W i ,W C ,W o are the weight matrices of the corresponding gates, b f ,b i ,b C ,b o is the bias vector,

[0070] Surface fitting feature association:

[0071] Get the feature vector H output by the LSTM network t Then, it is combined with the key numerical features in the historical behavior data to form a data set where t i represents the i-th time sampling point (i=1,2,…,N1, N1 is the number of samples, for example N1=100, that is, data samples of 100 time points are selected), t i The key numerical features corresponding to the moment. Since there may be a complex nonlinear relationship between voice features and user behavior numerical features, a polynomial surface fitting method is used to explore this potential connection. The formula is obtained by polynomial surface fitting. In the equation, d is the degree of the polynomial. In practical applications, the appropriate degree can usually be selected according to the complexity of the data. For example, d = 2 represents a quadratic polynomial fit, which can capture relatively simple nonlinear relationships. j is the fitting coefficient of the jth term, which is calculated by training the data set and determines the shape of the polynomial curve; ∈ is the fitting error, which reflects the difference between the fitting result and the actual data. The fitting coefficient β and the predicted trend value y are obtained by polynomial surface fitting. pred , this information can further reveal the correlation pattern between voice features and user behavior. pred With H n Merge to form a new feature vector H′ n , providing richer feature information for subsequent analysis.

[0072] Semantic enhancement of large language models:

[0073] As an advanced technology in the field of natural language processing, the large language model plays an important role in understanding the user's semantic intent in this module. It receives the text information initially obtained from the speech recognition module and the historical behavior description text as input. This text information contains the general content of the user's voice command and the semantic description of past behavior. The large language model processes it through its powerful language understanding capabilities and attention mechanism. Attention mechanism formula In the above equation, Q is the query vector, which is used to find information related to the current task in the input data; K is the key vector, which serves as the basis for information retrieval; V is the value vector, which contains the actual information content; d k is the dimension of the key vector, which is used to normalize the attention score to ensure the stability of the calculation; T represents transposition. Through the multi-head attention mechanism, the large language model can capture the semantic information in the text from different angles and understand the contextual relationship and logical connection between texts. After processing, the large language model outputs the semantic understanding vector S n , which contains a deep understanding of user intentions and semantic abstract representation. n and H′ t Splice and get the fusion feature vector F n =[H′ n ;S t ], achieving a deep fusion of speech acoustic features, user behavior numerical features and semantic features.

[0074] Comprehensive forecast:

[0075] Fusion feature vector F t Input the fully connected layer, through the formula P t =softmax(W p·F t +b p ) to make the final user behavior prediction. p is the weight matrix of the fully connected layer, which is used to perform linear transformation on the fused feature vector and adjust the weight distribution of the features; b p is the bias vector used to adjust the offset of the output result. After being processed by the fully connected layer, the output is converted into the user behavior prediction result P in the form of probability distribution through the softmax function. t .P t It is a probability vector, each element of which corresponds to a possible user behavior, and the value of the element represents the probability of the user performing the behavior at the current moment, for example, P t The element P in t,k It represents the probability that the user performs the kth behavior at time t, thereby providing the system with a quantitative prediction of user behavior.

[0076] The core function of the user behavior prediction module is to achieve accurate prediction of user behavior through multi-dimensional feature fusion and dynamic association. The specific functions are as follows:

[0077] 1. LSTM Network: Capturing Temporal Dependencies and Dynamic Features

[0078] Core Function: Through recursive operations of the forget gate, input gate, and output gate, this algorithm models the time series of MFCC feature vectors and historical behavior data sequences, addressing the inability of traditional neural networks to handle long-distance dependencies. Its formula operations (such as cell state updates and hidden layer state transfer) dynamically preserve historical user behavior patterns (such as operating habits and time patterns) and map current speech features and historical behavior sequences into abstract feature vectors, providing basic time-series correlation features for subsequent predictions.

[0079] 2. Surface Fitting Algorithm: Mining Nonlinear Feature Associations and Trend Prediction

[0080] Core Function: Polynomial fitting is performed on the feature vectors output by the LSTM to key numerical features in historical behavior (such as operation frequency and time interval). This formula captures the nonlinear mapping relationship between speech features and behavior values. For example, quadratic fitting can reveal a nonlinear positive correlation between rising voice intonation and faster operation frequency. The fitting coefficient and trend value are then output, supplementing the quantitative correlation between features that cannot be represented by traditional LSTM, thereby enhancing the numerical accuracy of the prediction.

[0081] 3. Large Language Model: Injecting Semantic Understanding and Contextual Logic

[0082] Core Function: Utilizing an attention mechanism to process preliminary speech text and historical behavior descriptions, the algorithm converts the semantic information of the speech signal (e.g., the intention to "turn on the lights") and the context of historical behavior (e.g., "often close the curtains in the evening") into a semantic vector. Its formulaic operations (e.g., multi-head attention) can understand the underlying intent of user commands (e.g., "play music" may imply a desire to "relax") and combine it with the feature vector derived from LSTM and surface fitting. This overcomes the lack of semantic logic in acoustic features alone, ensuring that predictions are more consistent with the user's true intent.

[0083] 4. Algorithm Collaboration: Multi-dimensional Feature Fusion Drives Prediction Optimization

[0084] Dynamic association mechanism: The temporal features output by LSTM are used as the input for surface fitting, and numerical associations are supplemented through nonlinear fitting. The fitted features are fused with the semantic vectors of the large language model to form a three-dimensional feature vector containing "acoustics, behavioral values, and semantics." Finally, the three-dimensional feature vector is converted into a behavioral probability distribution through a fully connected layer and a softmax function, achieving collaborative prediction of "temporal patterns + numerical regularities + semantic intent," improving robustness in complex scenarios (such as accurate recognition of ambiguous user commands or non-standard speech).

[0085] 4. Speech Recognition Module

[0086] The speech recognition module adopts an advanced speech recognition model based on Transformer, which has excellent performance and efficient processing capabilities in the field of speech processing. It takes MFCC feature vectors as input. The encoder part of the Transformer model uses a multi-head attention mechanism to process the input speech features in parallel, while capturing long-distance dependencies and rich contextual information in the speech signal. Compared with traditional recurrent neural networks (RNNs), the Transformer model avoids the gradient vanishing and gradient exploding problems that may occur when RNNs process long sequences, and can use computing resources more efficiently. After the encoder performs multi-level feature extraction and transformation on the MFCC feature vectors, the processed features are passed to the decoder. The decoder gradually generates the corresponding speech text recognition results R based on the features output by the encoder, combined with information such as the language model. t For example, when a user says the sentence "play pop music", the speech recognition module converts the speech signal into the text "play pop music" after the above processing, providing clear text information for subsequent decision-making.

[0087] 5. Decision Fusion Module

[0088] The decision fusion module is the "decision center" of the entire system. It combines the prediction results P output by the user behavior prediction module t The text recognition result R obtained by the speech recognition modulet To make more accurate and reliable decisions, we adopt weighted fusion strategy and use formula O t =α·P t +(1-α)·R t Calculate the final recognition result O t α is a weight parameter trained based on historical data, with a value range of [0, 1]. The value of α reflects the relative importance of user behavior prediction and speech-to-text recognition in the final decision. For example, when α = 0.6, user behavior prediction accounts for 60% of the final decision, while speech-to-text recognition accounts for 40%. By training and optimizing a large amount of historical data, the optimal value of α can be determined, allowing the system to make decisions that are more consistent with the user's true intentions in different application scenarios.

[0089] Final recognition result O t It will be used to control external devices, such as smart lights, smart air conditioners, smart door locks in smart home devices, as well as service robots, car voice control systems, etc. t After outputting the corresponding control instructions, the external device performs the corresponding operations according to the instructions, realizing intelligent control based on user behavior prediction and voice recognition, providing users with a more convenient and intelligent interactive experience.

[0090] Standardized data interfaces enable efficient data transmission between modules, ensuring the smooth flow of information within the system. The frequency domain signal processed by the speech acquisition and preprocessing module is transmitted to the speech feature extraction module. The MFCC feature vectors generated by the speech feature extraction module are then transmitted to the user behavior prediction module and the speech recognition module. The outputs of the user behavior prediction module and the speech recognition module are then fed into the decision fusion module. The control commands output by the decision fusion module are then transmitted to external devices, forming a complete speech recognition control system driven by user behavior prediction.

[0091] The above embodiments are intended to illustrate the present invention, not to limit the present invention. Therefore, changes in illustrative values ​​or substitutions of equivalent components should still fall within the scope of the present invention.

[0092] From the above detailed description, it will be clear to those skilled in the art that the present invention can indeed achieve the aforementioned objectives and is in compliance with the provisions of the Patent Law.

[0093] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all changes and modifications that fall within the scope of the invention. The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

[0094] It should be noted that the above description of the relevant processes is for illustration and purpose only and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the processes under the guidance of this specification. However, such modifications and changes are still within the scope of this specification.

[0095] The basic concepts have been described above. It will be apparent to those skilled in the art after reading this application that the above disclosures are merely illustrative and do not constitute limitations on this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and amendments to this application. Such modifications, improvements, and amendments are suggested in this application and remain within the spirit and scope of the exemplary embodiments of this application.

[0096] At the same time, this application uses specific terms to describe the embodiments of this application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or more in different places in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0097] In addition, it will be understood by those skilled in the art that various aspects of the present application can be illustrated and described by a number of patentable categories or situations, including any new and useful combination of processes, machines, products or substances, or any new and useful improvements thereto. Therefore, various aspects of the present application can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can all be referred to as "units", "modules" or "systems". In addition, various aspects of the present application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0098] The computer program code required for the operation of each part of the application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, conventional procedural programming languages ​​such as C programming language, VisualBasic, Fortran2103, Perl, COBOL2102, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy or other programming languages. The program code can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on a remote computer, or run completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or be connected to an external computer (such as by the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0099] In addition, unless expressly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above disclosure discusses some embodiments of the invention that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a pure software solution, for example, installation on an existing server or mobile device.

[0100] Similarly, it should be noted that in order to simplify the presentation of this disclosure and thereby facilitate understanding of one or more of the invention's embodiments, the foregoing descriptions of the embodiments of this disclosure sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this approach should not be interpreted as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject matter of the invention may possess fewer features than the single embodiment described above.

Claims

1. A speech recognition control system driven by user behavior prediction, characterized in that: include: The speech acquisition and preprocessing module is used to acquire time-domain speech signals, perform noise reduction and frame processing on the time-domain speech signals, and convert them into frequency-domain signals; The speech feature extraction module is connected to the speech acquisition and preprocessing module and is used to convert the frequency domain signal into the Mel frequency domain, extract the Mel spectrum and perform logarithmic and discrete cosine transform to obtain the Mel frequency cepstral coefficient feature vector; The user behavior prediction module is connected to the speech feature extraction module and is used to predict user behavior based on the long short-term memory network, surface fitting algorithm and large language model, combined with the Mel-frequency cepstral coefficient feature vector and historical behavior data; The speech recognition module connects the speech feature extraction module and the user behavior prediction module and is used to convert the Mel-frequency cepstral coefficient feature vector into speech-to-text recognition results based on the Transformer model; The decision fusion module connects the user behavior prediction module and the speech recognition module, and is used to perform weighted fusion of the user behavior prediction results and the speech text recognition results, and output the final recognition results to control external devices.

2. The user behavior prediction-driven speech recognition control system according to claim 1, characterized in that: The speech acquisition and preprocessing module uses Wiener filtering or wavelet noise reduction algorithm to reduce the noise of the speech signal.

3. The user behavior prediction-driven speech recognition control system according to claim 1, characterized in that: The speech feature extraction module processes the frequency domain signal through a Mel filter bank consisting of 20 to 40 Mel filters to obtain a Mel spectrum.

4. The user behavior prediction-driven speech recognition control system according to claim 1, characterized in that: The historical behavior data obtained by the user behavior prediction module includes device control records, voice command records, and operation timestamps.

5. The user behavior prediction-driven speech recognition control system according to claim 1, characterized in that: The user behavior prediction module extracts key numerical features from historical behavior data, including operation frequency, operation time interval, and operation duration.

6. The user behavior prediction-driven speech recognition control system according to claim 5, characterized in that: The user behavior prediction module first uses the long short-term memory network to process the Mel-frequency cepstral coefficient feature vector and the historical behavior data sequence to obtain the hidden layer output feature vector; then the hidden layer output feature vector is surface fitted with the key numerical features to obtain the fitting coefficient and predicted trend value; finally, the semantic understanding vector of the preliminary text information of the speech and the historical behavior description text is combined with the large language model to generate a fusion feature vector for behavior prediction.

7. The user behavior prediction-driven speech recognition control system according to claim 6, characterized in that: The speech recognition module generates preliminary text information of speech through an initial recognition model based on the hidden Markov model and inputs it into the large language model.

8. The user behavior prediction-driven speech recognition control system according to claim 6, characterized in that: Surface fitting uses a polynomial fitting method. By training a data set consisting of the hidden layer output feature vector and key numerical features, the fitting coefficients of the polynomial are obtained. The degree of the polynomial is set according to the characteristics of the data set. The predicted trend value obtained by fitting reflects the changing trend of user behavior in the numerical feature dimension.

9. The user behavior prediction-driven speech recognition control system according to claim 1, characterized in that: The decision fusion module determines the weight parameters of weighted fusion based on historical data training, and the weight parameters range from 0 to 1.

10. The user behavior prediction-driven speech recognition control system according to claim 1, characterized in that: External devices include smart home devices, service robots and in-vehicle voice control systems.

Images