A Deep Learning-Based Intent Prediction Model and Construction Method for Voice Robots

Abstract

This invention provides a deep learning-based intention prediction model and construction method for voice robots, belonging to the field of artificial intelligence technology. It includes: a data acquisition module that captures speech through a high-resolution microphone array and converts it into digital audio data, preprocessing the digital audio data; a feature extraction module that extracts spectral features and prosodic features from the preprocessed speech data, and processes these features to obtain deep feature data; a deep learning module that, based on the deep feature data, fuses multiple deep learning algorithms to capture temporal dependencies and semantic information in the speech data; and a prediction module that accelerates the training process for temporal dependencies and semantic information, and performs intention prediction on the dataset to construct an intention prediction model. This invention achieves efficient and accurate intention prediction by integrating technologies such as speech capture, data preprocessing, and deep learning.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a deep learning-based intention prediction model for voice robots. Background Technology

[0002] With the rapid development and widespread application of voice interaction technology, it has become an important way for people to interact with smart devices in their daily lives. Users interact with robots through voice, expecting the robots to accurately understand their intentions and perform corresponding operations or provide the required information. However, traditional voice intent recognition methods have some problems when faced with complex and varied voice signals, failing to meet the growing needs of users. The recognition accuracy of traditional methods is often low. Voice signals are affected by various factors, such as noise, accent, and speech rate, which make feature extraction and pattern recognition of voice signals difficult. Traditional methods are usually based on hand-designed feature extractors, which struggle to capture deep information and complex patterns in voice signals, leading to a decrease in recognition accuracy.

[0003] Traditional methods suffer from slow response times. Voice interaction requires robots to respond to user commands or queries in real time, but traditional methods often involve complex calculations and reasoning when processing voice signals, leading to response delays and impacting user experience. Furthermore, traditional methods struggle to adapt to environmental changes. The characteristics and patterns of voice signals change over time, in different contexts, and with different users. Traditional methods, typically trained on static models, cannot effectively handle these changes, resulting in performance degradation in real-world applications. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a deep learning-based intention prediction model and construction method for voice robots. By integrating technologies such as speech capture, data preprocessing, and deep learning, it achieves efficient and accurate intention prediction and supports adaptive learning to continuously optimize performance.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] Firstly, a deep learning-based intention prediction model for a voice robot, the model comprising:

[0007] The data acquisition module captures speech through a high-resolution microphone array and converts it into digital audio data, and then preprocesses the digital audio data.

[0008] The feature extraction module extracts spectral features and prosodic features from the preprocessed speech data, and processes the spectral features and prosodic features to obtain deep feature data.

[0009] The deep learning module, based on deep feature data, uses the fusion of multiple deep learning algorithms to capture temporal dependencies and semantic information in speech data;

[0010] The prediction module accelerates the training process by taking into account temporal dependencies and semantic information, and performs intent prediction on the dataset to build an intent prediction model.

[0011] Furthermore, a method for constructing a deep learning-based intention prediction model for a voice robot, the method comprising:

[0012] Capture speech signals in the environment, convert the captured speech signals into digital audio data format, and obtain digital audio data;

[0013] The digital audio data is preprocessed to obtain preprocessed digital audio data;

[0014] Extract spectral features and prosodic features from preprocessed digital audio data, and process the extracted spectral features and prosodic features to obtain deep feature data.

[0015] Based on deep feature data, a deep learning model is constructed to capture temporal dependencies and semantic information in speech data;

[0016] Based on the temporal dependencies and semantic information in the speech data, a deep learning model is trained to obtain a well-trained deep learning model.

[0017] Based on the trained deep learning model, the test set is used to test the model's ability to predict intent in order to obtain a prediction model;

[0018] Based on the prediction model, the intention of the voice robot can be predicted.

[0019] The above-described solution of the present invention has at least the following beneficial effects:

[0020] By capturing clearer speech signals through high-resolution microphone arrays and performing efficient preprocessing and feature extraction, the model can more accurately capture user intent. The application of deep learning algorithms effectively handles temporal dependencies and semantic information, thereby improving prediction accuracy and response speed. Advanced noise reduction techniques maintain the quality of speech input in various noisy environments, ensuring the voice robot can accurately understand user commands and inquiries even in noisy settings. Equalization and compression of audio data, along with amplitude adjustments, enhance the model's adaptability and generalization ability to different speech features. Utilizing deep learning structures such as deep belief networks to learn spectral and prosodic features allows for the extraction of more abstract and deeper features. The application of the Transformer model allows for effective training on large-scale datasets, supporting online learning and model fine-tuning, ensuring the model gradually optimizes over time to adapt to changes in user behavior. The fusion of multi-model prediction results, through a weighted averaging strategy, increases prediction reliability, reduces the possibility of misjudgments, and ensures the accuracy and stability of decision-making. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a deep learning-based speech robot intent prediction model provided by an embodiment of the present invention.

[0022] Figure 2 This is a flowchart illustrating a method for constructing a deep learning-based speech robot intent prediction model, as provided in an embodiment of the present invention. Detailed Implementation

[0023] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0024] like Figure 1 As shown, an embodiment of the present invention proposes a deep learning-based intention prediction model for voice robots, the method comprising:

[0025] Data acquisition module 11: The data acquisition module captures speech through a high-frequency microphone array and converts it into digital audio data, and preprocesses the digital audio data.

[0026] The feature extraction module 12 extracts spectral features and prosodic features from the preprocessed speech data, and processes the spectral features and prosodic features to obtain deep feature data.

[0027] Deep learning module 13, based on deep feature data, uses the fusion of multiple deep learning algorithms to capture temporal dependencies and semantic information in speech data;

[0028] Prediction module 14 accelerates the training process by considering temporal dependencies and semantic information, and performs intent prediction on the dataset to build an intent prediction model.

[0029] In this embodiment of the invention, a high-resolution microphone array can accurately capture speech signals, ensuring the clarity and accuracy of the speech data. The speech signal is efficiently converted into digital audio data, facilitating subsequent processing and analysis. Preprocessing of the digital audio data, such as noise reduction and standardization, improves data quality. Extracting spectral and prosodic features from the preprocessed speech data comprehensively reflects the inherent characteristics of speech. Further processing of these features yields deep feature data, helping to reveal implicit information within the speech data. The extracted features are then optimized and selected to remove redundant information. The fusion of multiple deep learning algorithms effectively captures temporal dependencies in the speech data, enhancing the model's temporal modeling capabilities. Deep learning algorithms can deeply mine semantic information in speech data, helping to understand the speaker's true intentions and expressed content. The fusion of multiple deep learning algorithms improves the model's adaptability and generalization ability. By accelerating training techniques, the training time of the model can be shortened and the training efficiency improved. Based on rich feature sets and powerful deep learning models, it is possible to accurately predict the intent of the dataset and build a high-performance intent prediction model. By analyzing the prediction results, the model parameters and structure can be continuously optimized and adjusted to improve the model's prediction performance and stability.

[0030] In a preferred embodiment of the present invention, speech is captured by a high-resolution microphone array and converted into digital audio data. Preprocessing of the digital audio data includes:

[0031] Voice is captured using a high-resolution microphone array, and the voice signal captured by the microphone array is converted into digital audio data.

[0032] Digital audio data through Noise reduction processing is performed to obtain the denoised audio data, where S(ω) is the power spectrum estimate of the clean speech signal, N(ω) is the power spectrum estimate of the noise, X(ω) is the complex spectrum of the speech signal affected by noise, Y(ω) is the filtered output spectrum, a is a weighting coefficient, β is the offset, and γ is the weighting coefficient of the noise power spectrum. It is a positive number;

[0033] The noise-reduced audio data is then subjected to equalization and compression processing to obtain data-enhanced audio data.

[0034] The amplitude of the data-enhanced audio data is adjusted to a uniform range to achieve the pre-processed data.

[0035] In this embodiment of the invention, a high-resolution microphone array can capture speech signals more accurately, ensuring high-quality and detailed speech. Noise reduction processing significantly reduces background noise and interference signals, improving the clarity and purity of the speech signal, which helps to accurately identify speech content in noisy environments and improves the performance of speech recognition models. Equalization and compression processing improve the dynamic range and spectral characteristics of audio data, making the speech signal more uniform and consistent. Adjusting the amplitude of data-enhanced audio data to a uniform range can eliminate the influence of different recording environments and devices on the amplitude of the speech signal. The preprocessed high-quality audio data can be used as input for machine learning or deep learning models, thereby improving the training efficiency and prediction accuracy of the models. The preprocessing step can highlight key information in the speech signal, reduce the complexity of the model, and make it easier for the model to learn effective speech feature representations.

[0036] In a specific embodiment of the present invention, a high-performance microphone array is designed, including factors such as the number of microphones, their layout, sensitivity, and frequency response. The microphone array is installed in an appropriate location, its hardware and software parameters are configured, and the array is started and placed in standby mode to prepare for capturing speech signals. When speech signals appear, the microphone array captures these signals and converts them into analog electrical signals. An analog-to-digital converter (ADC) is used to sample the analog speech signals, and the sampled discrete signals are quantized, mapping them to a finite numerical range. The digital audio data is divided into short frames, and a window function (such as a Hamming window) is applied to each frame. A Fast Fourier Transform (FFT) is performed on each frame to convert it from the time domain to the frequency domain, obtaining a complex spectrum. The power spectrum estimates of the clean speech signal and noise are calculated based on the complex spectrum. Based on actual conditions and experience, parameters such as weighting coefficients, offsets, and small positive numbers are set in the denoising formula. Using the provided denoising formula, the complex spectrum of each frame is calculated, resulting in the denoised complex spectrum of each frame. The denoised complex spectrum is subjected to an inverse fast Fourier transform (I-FFT) to convert it back from the frequency domain to the time domain, obtaining the denoised time-domain audio data. Based on the time-domain audio data, a parametric equalizer or graphic equalizer is selected, and the equalizer parameters, such as gain and frequency response, are adjusted according to the spectral characteristics of the audio signal and the target effect. The equalizer is then applied to the denoised audio data. A dynamic range compressor is selected, and parameters such as the compressor threshold, ratio, attack time, and release time are set. The compressor is then applied to the equalized audio data to reduce its dynamic differences and improve the overall volume level. The equalized and compressed audio data is scanned to find its maximum amplitude value. Based on the maximum amplitude value and a predefined upper limit (e.g., 1.0), a scaling factor is set and applied to the entire audio data to normalize its amplitude range to within the predefined upper limit.

[0037] In a preferred embodiment of the present invention, the noise-reduced audio data is subjected to equalization and compression processing to obtain processed audio data, including:

[0038] The denoised audio data is analyzed using the FFT algorithm to obtain spectral analysis results;

[0039] Based on the spectrum analysis results, the frequency response of the audio is adjusted using an equalizer to adjust the equalizer settings;

[0040] Apply equalizer settings to the entire audio data and play the equalized audio to achieve equalization of the audio data.

[0041] Import the equalized audio file into the compression tool and adjust the compression parameters, including bitrate, sampling rate, and compression level.

[0042] Based on the adjusted compression parameters, a compression operation is performed to generate a compressed audio file.

[0043] In this embodiment of the invention, the FFT algorithm can analyze the spectral characteristics of audio data in detail. By performing FFT analysis on the denoised audio, the various frequency components in the audio can be identified more accurately. Spectral analysis helps to discover potential problem areas in the audio; finely adjusting the frequency response of the audio through an equalizer can effectively improve sound quality, making the audio sound more natural and balanced. Applying equalizer settings to the entire audio data ensures consistent sound quality and listening experience from beginning to end. By adjusting compression parameters, such as bit rate and sampling rate, the size of the audio file can be effectively controlled, facilitating storage and transmission. Compressed audio files occupy less storage space, making it easier to save more audio content on limited devices or in cloud storage.

[0044] In one specific embodiment of the present invention, a suitable audio processing library (such as librosa, scipy, or numpy in Python) is used to read the denoised audio file. A Fast Fourier Transform (FFT) algorithm is applied to the audio sample array. FFT is an algorithm that converts a time-domain signal into a frequency-domain signal, decomposing complex waveforms into sine waves of different frequencies. Through FFT, a complex array is obtained, where the modulus of each complex number represents the amplitude of the corresponding frequency, and the argument represents the phase. From the FFT result, the amplitude of each frequency component can be calculated, thus obtaining the audio spectrum. Observing the spectrum, it is possible to identify which frequency regions have excessively high or low energy. Depending on the audio content, the energy of certain frequency regions is enhanced, or the energy of other regions is reduced to achieve a more balanced auditory effect. The equalizer function of the audio processing software or library is used to adjust the gain or attenuation of each frequency band based on the results of the spectrum analysis. The equalizer typically provides control over multiple frequency bands, allowing for separate adjustment of the volume of different frequency bands. By performing point-by-point multiplication on the audio samples, the value of each sample is adjusted according to the gain or attenuation settings in the equalizer, and the adjusted equalizer settings are applied to the entire audio data. The equalized audio is then played through an audio playback library or software. The equalized audio file is imported into an audio compression tool, and compression parameters are adjusted, including bitrate (which determines the data size and sound quality), sampling rate (which determines the fidelity and file size), and compression level (which controls the strength of the compression algorithm and the size of the generated file). The adjusted parameters are applied in the compression tool, and the compression operation is performed. During compression, the tool uses specific audio encoding algorithms (such as MP3, AAC, etc.) to reduce the file size while maintaining sound quality as much as possible. After compression, the tool generates a new, smaller audio file.

[0045] In a preferred embodiment of the present invention, spectroscopic features and prosodic features are extracted from the preprocessed speech data, and the spectroscopic features and prosodic features are processed to obtain deep feature data, including:

[0046] The preprocessed speech signal is divided into frames of fixed size. Each frame is windowed, and each windowed frame is subjected to a Fast Fourier Transform to convert the time-domain signal into a frequency-domain signal, so as to obtain the spectrum of each frame.

[0047] Based on the spectrum of each frame, various spectral features are extracted, including Mel frequency cepstral coefficients, linear prediction coefficients, and linear prediction cepstral coefficients.

[0048] The preprocessed speech signal is then processed through... Calculations are performed to obtain speech prosodic features, where E i is the energy of the i-th frame, x[n] is the n-th sample of the speech signal, w[ni] is the window function centered on the i-th sample, and N is the number of samples in each frame;

[0049] The extracted spectral and prosodic features are dimensionality reduced to remove redundant information, resulting in dimensionality-reduced features.

[0050] By learning from the dimensionality-reduced features through deep belief networks, more abstract and deeper features are extracted to obtain deep feature data.

[0051] In this embodiment of the invention, by dividing the speech signal into fixed-size frames and windowing each frame, the signal processing can be refined and localized, which helps to capture the short-term characteristics of the speech signal. The Fast Fourier Transform (FFT) can quickly convert the time-domain signal into a frequency-domain signal, thereby obtaining the spectrum of each frame, which helps to analyze the frequency components of the speech signal more intuitively. By extracting various spectral features such as Mel-frequency cepstral coefficients, linear prediction coefficients, and linear prediction cepstral coefficients, and calculating speech prosodic features, the characteristics of the speech signal can be comprehensively and multi-dimensionally described. Dimensionality reduction of the extracted spectral and prosodic features removes redundant information, retains key features, helps reduce the computational complexity of subsequent models, improves processing efficiency, and avoids overfitting. Learning from the dimensionality-reduced features through a deep belief network can extract more abstract and deeper features, capturing the essential information and potential structure in the speech data.

[0052] In a specific embodiment of the present invention, based on the preprocessed speech signal, the signal is divided into frames of a fixed size (e.g., 256 or 512 sample points), and the overlap (frame shift) between adjacent frames is determined. According to the determined frame length and frame shift, the preprocessed speech signal is cut into multiple short-time frames. A suitable window function, such as a Hamming window or Hanning window, is selected. The window function is used to reduce discontinuities between frames and reduce spectral leakage. Its parameters are set according to the type of the selected window function. For each frame of signal, the signal frame is weighted, and the window function is multiplied by it. FFT is an efficient algorithm used to calculate the Discrete Fourier Transform (DFT), which converts a time-domain signal into a frequency-domain signal. In the frequency domain, the signal is represented as a set of different frequency components. An FFT is performed on each frame of windowed signal to generate the spectrum of each frame, i.e., the amplitude and phase information of the signal at different frequencies. The amplitude spectrum is obtained by taking the absolute value of the FFT result and performing a logarithmic transformation (e.g., logarithmic amplitude spectrum). Mel-scale is a nonlinear frequency scale based on the human auditory system. It has high resolution in the low-frequency region and low resolution in the high-frequency region. Therefore, the linear spectrum obtained by FFT is converted into a Mel-scale spectrum. Then, a Discrete Cosine Transform (DCT) is applied to the Mel-scale spectrum to obtain the MFCC, which is a commonly used feature in speech recognition and speech synthesis. Linear prediction is a method for estimating speech signals. It assumes that the current sample can be predicted by a linear combination of past samples. The coefficients of the linear prediction model are solved by minimizing the prediction error; these coefficients are called LPC. LPC reflects the vocal tract characteristics of the speech signal. Cepstrum is the inverse Fourier transform of the logarithm of the signal's spectrum. In speech processing, cepstrum analysis helps separate the sound source and vocal tract characteristics. The cepstrum is calculated based on the LPC, and then further processed by DCT or other transforms to obtain LPCC, which is another effective feature for speech recognition. To reduce the impact of noise and inter-frame variations, the extracted features can be smoothed, such as by using a moving average filter, and the features can be normalized, such as by using mean and variance normalization. The extracted spectral features (such as MFCC, LPC, LPCC) can be applied to various speech processing tasks, such as speech recognition, speaker recognition, and sentiment analysis.

[0053] In another specific embodiment of the present invention, for the preprocessed speech signal, the prosodic features are obtained by calculating the energy of each frame. By calculating the energy of each frame, the prosodic features of the speech signal, such as intensity and duration, can be obtained. The extracted spectroscopic and prosodic features are then subjected to dimensionality reduction processing using principal component analysis to remove redundant information. After dimensionality reduction, a more compact and effective feature set is obtained. A deep belief network model is constructed, which consists of multiple stacked Restricted Boltzmann Machines (RBMs), capable of learning deep representations in the data. The dimensionality-reduced features are used as input to the deep belief network model, and more abstract and deeper features are extracted by training the RBMs layer by layer. Each layer of the RBM learns a representation of the data and passes it to the next layer. After learning by the deep belief network model, more abstract and deeper feature data can be obtained.

[0054] In a preferred embodiment of the present invention, based on deep feature data, a fusion of multiple deep learning algorithms is used to capture temporal dependencies and semantic information in speech data, including:

[0055] Deep feature data are fused by concatenation and weighted summation to obtain fused features, which include short-term information, long-term dependencies and contextual information of speech data;

[0056] Based on the fused features, construct a Transformer deep learning model;

[0057] Train the Transformer deep learning model to obtain the trained model;

[0058] Based on the trained model, predictions and inferences are made on new speech data to capture temporal dependencies and semantic information.

[0059] In this embodiment of the invention, deep feature data is fused through methods such as concatenation and weighted summation. This integrates the advantages of different features to form a richer and more comprehensive feature representation, encompassing both short-term information of the speech data and capturing long-term dependencies and contextual information. The Transformer deep learning model is renowned for its excellent self-attention mechanism, effectively capturing temporal dependencies in sequential data. In speech processing, the model can better understand the continuity and dynamic changes of speech signals, thereby improving the accuracy of speech recognition and reasoning. By training the Transformer deep learning model, information from a large amount of speech data can be fully utilized, enabling the model to learn the complex mapping relationship between speech signals and semantic content. This helps improve the model's generalization ability, allowing it to maintain good performance even when faced with new speech data. The trained Transformer model can accurately predict and reason about new speech data. By capturing temporal dependencies and semantic information in speech data, prediction results closely related to the speech content can be generated.

[0060] In a specific embodiment of the present invention, deep feature data is extracted through a deep belief network (DBN). Deep feature data from different time steps or different levels are concatenated. For example, features from adjacent frames can be concatenated to capture short-term information; or output features from different levels of DBN can be concatenated to fuse information at different levels of abstraction. Each feature is assigned a weight based on its importance or contribution, and then all features are summed with weights to obtain a fused feature vector. Through concatenation and weighted summation, a feature vector that integrates short-term information, long-term dependencies, and contextual information is finally obtained. The architecture of the Transformer model is designed, including an input layer, multiple Transformer encoder layers, and an output layer. Each Transformer layer contains a self-attention mechanism and a feedforward neural network. The parameters of the Transformer model are initialized, including weights and biases. The fused feature vector is paired with corresponding labels to form a training dataset. An appropriate loss function is defined according to the task type, and the Adam optimizer is selected to adjust the model parameters during training to minimize the loss function. Training data is fed into the Transformer model in batches, and the loss is calculated through forward propagation. Then, the model parameters are updated through backpropagation and the optimizer. This process is repeated until the model performance reaches a satisfactory level or no longer shows significant improvement. The model's performance is evaluated on the validation set. If the model performance improves, the current model parameters are saved for later use. The trained Transformer model is loaded, and new speech data undergoes the same preprocessing steps as the training data, including frame segmentation, windowing, and FFT, to extract the corresponding feature vectors. The feature vectors of the new data are fused, and then the fused features are input into the trained Transformer model. The Transformer model performs forward propagation to capture temporal dependencies and semantic information in the input sequence.

[0061] In a preferred embodiment of the present invention, speech data is input into a Transformer deep learning model for model training via forward propagation to obtain a trained model, including:

[0062] The speech data is input into the Transformer model, and forward propagation is performed to obtain the prediction results of the deep learning model.

[0063] Based on the prediction results of the deep learning model and the actual labels, through The parameters of the deep learning model are optimized during training, where L is the value of the loss function, C is the total number of classes, and y c It is the target output, p c It is the probability of class C predicted by the model;

[0064] Based on the optimized parameters of the deep learning model, the gradient is propagated from the output layer to the input layer through the chain rule to update the model's weights and biases, thereby obtaining the gradient of the deep learning model parameters.

[0065] Based on the obtained gradient, the parameters of the deep learning model are updated through gradient descent to minimize the loss function value and achieve parameter optimization.

[0066] The parameters are continuously optimized through iteration until the preset number of training rounds is reached, thus completing the training of the model.

[0067] In this embodiment of the invention, the Transformer model can generate prediction results based on the input speech data through forward propagation. Due to its powerful temporal processing capabilities and self-attention mechanism, it can capture key information in the speech, thereby generating accurate predictions. Calculating the loss function value based on the model's prediction results and actual labels quantifies the accuracy of the model's predictions. By optimizing the model parameters during training to minimize the loss function value, it ensures that the model gradually learns more accurate mapping relationships, improving prediction performance. Using the chain rule to propagate gradients from the output layer to the input layer efficiently calculates the gradients of the model parameters, allowing the model to quickly adjust its internal parameters during training to adapt to the characteristics of the input data, accelerating the model's convergence speed. Updating the model parameters through gradient descent ensures that the model moves in the direction of reducing the loss function value in each iteration. This update strategy helps avoid the model getting trapped in local optima, improving the model's generalization ability and stability. By setting a preset number of training rounds, the model's training process can be controlled, helping to complete the model training within a reasonable time and avoiding excessive training time that leads to resource waste or model performance degradation.

[0068] In a specific embodiment of the present invention, necessary preprocessing is performed on the speech data, including framing, windowing, and feature extraction steps, resulting in a feature vector sequence that can be directly input into the Transformer model. The preprocessed speech data is then input into the Transformer model, where it undergoes a self-attention mechanism to capture dependencies in the sequence, and a nonlinear transformation is performed through a feedforward neural network to output a prediction result. The model's prediction result and the actual labels are then substituted into the loss function to calculate the loss value for the current batch of data. Before starting backpropagation, the gradients of all trainable parameters in the model need to be initialized to zero. Using the chain rule, backpropagation is performed layer by layer from the output layer to the input layer. During backpropagation, based on the derivative of the loss function with respect to the model output and the local derivatives of each layer, a learning rate is set before updating the model parameters. The gradient of each parameter is obtained using a formula. Using the calculated gradient values ​​and learning rate, the model parameters (weights and biases) are updated using the gradient descent algorithm. In each iteration, the model parameters are updated based on the calculated gradients to optimize them. These steps are repeated for multiple training rounds. Each round iterates through the entire training dataset once, until the preset number of training rounds is reached, at which point the model training ends.

[0069] In a preferred embodiment of the present invention, based on the parameters of the optimized deep learning model, the gradient is propagated layer by layer from the output layer to the input layer using a chain rule to update the model's weights and biases, thereby obtaining the gradient of the deep learning model parameters, including:

[0070] Based on the parameters of the optimized model, the gradient is propagated from the output layer to the input layer layer by layer using the chain rule to update the model's weights and biases.

[0071] Based on the updated model's weights and biases, through To obtain the gradient of the model parameters, where Wl is the weight matrix of the l-th layer, η is the learning rate, and δ is the gradient of the model parameters. (l+1) W is the error gradient of the (l+1)th layer. (l+1) It is the weight matrix of the (l+1)th layer, f′(z (l) ) is the derivative of the activation function of the l-th layer, and ⊙ is element-wise multiplication. This represents the weighted input z of the l-th layer. (l) For weight W (l) The partial derivatives of .

[0072] In this embodiment of the invention, the chain rule allows neural networks to efficiently calculate the gradient of each parameter. By passing the gradient layer by layer, the chain rule can accurately calculate the gradient of the loss function with respect to each parameter, thus making the update of model parameters more accurate and helping to improve model performance. Each layer only focuses on the information of the previous and next layers directly related to itself, realizing the modularization of gradient calculation. Although backpropagation requires more memory to store intermediate values ​​for reuse when calculating gradients, this strategy avoids redundant calculations and improves the overall efficiency of the training process. Through continuous iterative optimization, the model's weights and biases are dynamically adjusted, allowing the model to gradually adapt to the training data and improve prediction accuracy. The chain rule is not only applicable to single-output neural networks but can also be extended to the calculation of partial derivatives of multivariate functions, making it suitable for various complex neural network structures. The gradient calculated by the chain rule can be combined with various optimization algorithms to achieve rapid convergence and optimization of model parameters.

[0073] In a specific embodiment of the present invention, the structure of the neural network is determined, including the number of layers, the number of neurons in each layer, and the activation function. The weight matrix of each layer is randomly initialized, and a suitable learning rate is set. The learning rate is a positive number used to control the step size of parameter updates. For each layer l, the weighted input z is calculated. (l) =W (l) a (l-1) +b (l) , where a (l-1) It is the activation output of the previous layer (i.e., the (l-1)th layer), b (l) This is the bias of this layer. The activation output a of this layer is calculated using the activation function f. (l) =f(z) (l) Define a loss function to measure the difference between the network's predictions and the actual values, and calculate the error gradient δ of the output layer based on the loss function. (L) Where L is the total number of layers in the network. Starting from the output layer, the error gradient δ of each layer is calculated layer by layer towards the input layer. (L) Through the formula δ(L)=((W) (l+1) ) T δ (l+1) )⊙f(z (l) Completed, for each layer l, the weight matrix W is calculated according to the formula. (l) The gradient. Through Complete the calculation of the gradient of the bias vector. Update the weights and biases according to the gradient descent algorithm, and repeat the above steps of forward propagation, calculation of error gradient, backpropagation and parameter update until the stopping condition is met (the preset number of iterations is reached).

[0074] In a preferred embodiment of the present invention, the training process is accelerated by processing temporal dependencies and semantic information, and intent prediction is performed on the dataset to obtain prediction results, including:

[0075] Based on the trained model, fine-tuning is performed on temporal dependency and semantic information datasets to obtain a fine-tuned trained model;

[0076] Based on the fine-tuned training model, the learning rate is adjusted, and the model's performance is trained using a test set to obtain a predictive model.

[0077] Based on the prediction model, intent prediction is performed on the temporal dependency and semantic information datasets to obtain multiple prediction results.

[0078] In this embodiment of the invention, by fine-tuning on a specific dataset, the model can better adapt to the characteristics of temporal dependencies and semantic information, thereby improving its performance on tasks in that domain. Fine-tuning allows the model to learn task-specific knowledge while retaining a large amount of general knowledge learned during the pre-training phase. Compared to training the model from scratch, fine-tuning typically requires less training data and computational resources, thus reducing training costs. Adjusting the learning rate helps the model learn more effectively during training, avoiding convergence problems caused by learning too quickly or inefficient training caused by learning too slowly. Evaluating the model's performance using a test set ensures that the model not only performs well on training data but also maintains good generalization ability on unseen data. Performance feedback from the test set can serve as a basis for adjusting the model structure, parameters, or training strategies. Accurate intent prediction can improve the user's interaction experience with the system. In automated tasks or decision-making assistance systems, accurate intent prediction can significantly improve the success rate and efficiency of tasks.

[0079] In a specific embodiment of the present invention, a dataset containing temporal dependencies and semantic information is collected. This dataset may include text sequences and time-series data, which contain rich temporal relationships and semantic context. The data is cleaned, standardized, and formatted to meet the input requirements of the model. For example, for text data, word segmentation, stop word removal, and word vectorization may be required. A suitable pre-trained model is selected as a starting point. The pre-trained model is a model trained on a large amount of general-purpose data, such as BERT or GPT. The weights and configurations of the pre-trained model are loaded. The structure of the pre-trained model is fine-tuned according to the needs of the specific task, such as adding or removing certain layers. To preserve the general knowledge in the pre-trained model, the weights of some layers can be frozen so that they are not updated during fine-tuning. The model is fine-tuned using the prepared temporal dependency and semantic information dataset, undergoing multiple training iterations. In each iteration, the model attempts to minimize the loss function to learn specific patterns in the dataset. An initial learning rate is set, and as training progresses, a learning rate scheduling strategy is used to dynamically adjust the learning rate, such as learning rate decay or periodic adjustment. A portion of the original dataset is partitioned as a validation or test set to evaluate the model's performance. Appropriate performance metrics, such as accuracy, recall, F1 score, and mean squared error, are selected based on the task type. After each training iteration, the validation set is used to evaluate the model's performance. The learning rate or other hyperparameters are adjusted based on changes in the performance metrics. The best-performing model on the validation set is selected as the final prediction model. Based on the temporal dependencies and semantic information of the previous predictions, the dataset shares a similar structure and features with the training set; therefore, the prediction dataset undergoes the same preprocessing steps as the training set. The optimal model weights and configuration selected during the fine-tuning phase are loaded, and the preprocessed prediction dataset is input into the model to obtain its output. This output may represent the probability distribution of different classes, specific predicted values, etc. Based on the model's output and the nature of the task, the final prediction result is parsed. For example, in a classification task, the class with the highest probability is selected as the prediction result.

[0080] In a preferred embodiment of the present invention, the learning rate is adjusted according to the fine-tuned training model, and the model's performance is trained using a test set to obtain a prediction model, including:

[0081] Based on the fine-tuned training model, initialize the learning rate to obtain the parameter-adjusted training model;

[0082] The trained model with adjusted parameters is fine-tuned using the training set, and the model's performance is periodically evaluated on the validation set to obtain evaluation results.

[0083] The hyperparameters are adjusted based on the evaluation results to obtain the prediction model. The hyperparameters include the learning rate, regularization strength, or optimizer parameters.

[0084] In this embodiment of the invention, by fine-tuning the training model and adjusting hyperparameters based on performance evaluation on the validation set, the accuracy, efficiency, and generalization ability of the model can be significantly improved. Adjusting the regularization strength hyperparameter helps control the complexity of the model, thereby reducing overfitting on the training data. The model can then better generalize to unseen data, improving predictive performance in practical applications. Learning rate and optimizer parameter settings can significantly accelerate model training. By adjusting these hyperparameters, computational resources can be utilized more effectively, shortening the time required for the model to reach ideal performance. Careful adjustment of hyperparameters can build models that are more robust to changes in input data. Reasonable hyperparameter settings can also optimize model resource consumption. Sometimes, adjusting model hyperparameters can also yield simpler, more interpretable model structures or outputs. A carefully fine-tuned and evaluated model can serve as a benchmark, providing valuable reference and a starting point for subsequent research and development.

[0085] In one specific embodiment of the present invention, a pre-tuned model is obtained, an initial learning rate is set for model training, and a pre-defined training and validation set is ensured. The training set is used to fine-tune the model. After each iteration, the validation set is used to evaluate the model's performance. Based on the validation set, the model's accuracy and loss values ​​are evaluated. The changes in model performance are analyzed based on the validation set evaluation results. If the performance does not meet expectations, or if overfitting or underfitting occurs, hyperparameter adjustment is necessary. Based on the analysis results, hyperparameters such as the learning rate, regularization strength, and optimizer parameters are adjusted. For example, if the model performs well on the training set but its performance declines on the validation set, it may be necessary to reduce the learning rate, decrease model complexity, or increase the regularization strength. The adjusted hyperparameters are applied to the model, and the fine-tuning process on the training set is restarted. After each hyperparameter adjustment, the fine-tuning and performance evaluation on the validation set are repeated until the model performance reaches a satisfactory level or no longer shows significant improvement. During the iteration process, the best-performing model parameters and configurations are saved for subsequent use or further analysis. After the model performance reaches a stable or optimal state, a final comprehensive evaluation is performed, and the final fine-tuned model is exported.

[0086] like Figure 2 As shown, a method 20 for constructing a deep learning-based speech robot intent prediction model includes:

[0087] Step 21: Capture the speech signal in the environment and convert the captured speech signal into a digital audio data format to obtain digital audio data;

[0088] Step 22: Preprocess the digital audio data to obtain preprocessed digital audio data;

[0089] Step 23: Extract spectrogram features and prosodic features from the preprocessed digital audio data, and process the extracted spectrogram features and prosodic features to obtain deep feature data.

[0090] Step 24: Based on the deep feature data, construct a deep learning model to capture the temporal dependencies and semantic information in the speech data;

[0091] Step 25: Train the deep learning model based on the temporal dependencies and semantic information in the speech data to obtain a trained deep learning model.

[0092] Step 26: Based on the trained deep learning model, test the model's intent prediction capability using a test set to obtain the prediction model;

[0093] Step 27: Based on the prediction model, the intention of the voice robot is predicted.

[0094] In this embodiment of the invention, the model can capture speech signals in the environment in real time and quickly convert them into digital audio data. By extracting spectral features and prosodic features and performing feature fusion, the model can generate deep feature data containing multifaceted speech information, which helps to comprehensively capture the inherent information in the speech data. By integrating multiple deep learning algorithms, the model can effectively capture temporal dependencies and semantic information in the speech data, helping the model to understand the speech content more deeply and providing strong support for intent prediction. By accelerating the training process, the model can complete model training in a short time and quickly predict intent on the dataset, improving the model's response speed and efficiency. By using a weighted average method to fuse multiple prediction results, the model can comprehensively consider the opinions of different prediction results, improving the accuracy and stability of the final prediction. Post-processing the fused prediction results can further optimize the prediction results, enhancing the model's scalability and adaptability. Through data storage and adaptive learning, the model can continuously accumulate new knowledge and optimize its performance.

[0095] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A deep learning-based intention prediction model for voice robots, characterized in that, The model includes: The data acquisition module captures speech through a high-resolution microphone array and converts it into digital audio data, and then preprocesses the digital audio data. The feature extraction module extracts spectral features and prosodic features from the preprocessed speech data, and processes these features to obtain deep feature data. This includes: dividing the preprocessed speech signal into fixed-size frames; windowing each frame; performing a Fast Fourier Transform on each windowed frame to convert the time-domain signal to the frequency-domain signal, thus obtaining the spectrum of each frame; extracting various spectral features based on the spectrum of each frame, including Mel-frequency cepstral coefficients, linear prediction coefficients, and linear prediction cepstral coefficients; calculating the prosodic features from the preprocessed speech signal; performing dimensionality reduction on the extracted spectral and prosodic features to remove redundant information, resulting in dimensionality-reduced features; and learning from the dimensionality-reduced features using a deep belief network to extract deeper features. The deep learning module, based on deep feature data, uses the fusion of multiple deep learning algorithms to capture temporal dependencies and semantic information in speech data. This includes: fusing deep feature data through concatenation and weighted summation to obtain fused features containing short-term information, long-term dependencies, and contextual information of the speech data; constructing a Transformer deep learning model based on the fused features; training the Transformer deep learning model to obtain a trained model; and using the trained model to predict and infer new speech data, capturing its temporal dependencies and semantic information. The prediction module accelerates the training process by taking into account temporal dependencies and semantic information, and performs intent prediction on the dataset to build an intent prediction model.

2. The deep learning-based speech robot intent prediction model according to claim 1, characterized in that, Speech is captured using a high-resolution microphone array and converted into digital audio data. The digital audio data is then preprocessed, including: Voice is captured using a high-resolution microphone array, and the voice signal captured by the microphone array is converted into digital audio data. The digital audio data is denoised to obtain the denoised audio data. The noise-reduced audio data is then subjected to equalization and compression processing to obtain the processed audio data. The amplitude of the data-enhanced audio data is adjusted to a uniform range to obtain preprocessed data.

3. The deep learning-based speech robot intent prediction model according to claim 2, characterized in that, The noise-reduced audio data undergoes equalization and compression processing to obtain processed audio data, including: The denoised audio data is analyzed using the FFT algorithm to obtain spectral analysis results; Based on the spectrum analysis results, the frequency response of the audio is adjusted using an equalizer to adjust the equalizer settings; Apply equalizer settings to the entire audio data and play the equalized audio to achieve equalization of the audio data. Import the equalized audio file into the compression tool and adjust the compression parameters, including bitrate, sampling rate, and compression level. Based on the adjusted compression parameters, a compression operation is performed to generate a compressed audio file.

4. The deep learning-based speech robot intent prediction model according to claim 3, characterized in that, Speech data is input into the Transformer deep learning model and trained through forward propagation to obtain the trained model, which includes: The speech data is input into the Transformer model, and forward propagation is performed to obtain the prediction results of the deep learning model. Based on the prediction results of the deep learning model and the actual labels, the parameters of the deep learning model are optimized during the training process. Based on the optimized parameters of the deep learning model, the gradient is propagated from the output layer to the input layer through the chain rule to update the model's weights and biases, thereby obtaining the gradient of the deep learning model parameters. Based on the obtained gradient, the parameters of the deep learning model are updated through gradient descent to minimize the loss function value and achieve parameter optimization. The parameters are continuously optimized through iteration until the preset number of training rounds is reached, thus completing the training of the model.

5. The deep learning-based speech robot intent prediction model according to claim 4, characterized in that, Based on the optimized parameters of the deep learning model, the gradient is propagated layer by layer from the output layer to the input layer using the chain rule, updating the model's weights and biases to obtain the gradients of the deep learning model parameters, including: Based on the parameters of the optimized model, the gradient is propagated from the output layer to the input layer layer by layer using the chain rule to update the model's weights and biases. The gradient of the model parameters is calculated based on the weights and biases of the updated model.

6. The deep learning-based speech robot intent prediction model according to claim 5, characterized in that, The training process is accelerated by incorporating temporal dependencies and semantic information, and intent prediction is performed on the dataset to obtain prediction results, including: Based on the trained model, fine-tuning is performed on temporal dependency and semantic information datasets to obtain a fine-tuned trained model; Based on the fine-tuned training model, the learning rate is adjusted, and the model's performance is trained using a test set to obtain a predictive model. Based on the prediction model, intent prediction is performed on the temporal dependency and semantic information datasets to obtain multiple prediction results.

7. The deep learning-based speech robot intent prediction model according to claim 6, characterized in that, Based on the fine-tuned training model, the learning rate is adjusted, and the model's performance is trained using a test set to obtain a prediction model, including: Based on the fine-tuned training model, initialize the learning rate to obtain the parameter-adjusted training model; The trained model with adjusted parameters is fine-tuned using the training set, and the model's performance is periodically evaluated on the validation set to obtain evaluation results. The hyperparameters are adjusted based on the evaluation results to obtain the prediction model. The hyperparameters include the learning rate, regularization strength, or optimizer parameters.

Images