Voice robot intention prediction model based on deep learning and construction method

Abstract

The invention provides a voice robot intention prediction model based on deep learning and a construction method, and relates to the technical field of artificial intelligence, and the model comprises a data collection module which captures voice through a high microphone array, converts the voice into digital audio data, and carries out the preprocessing of the digital audio data; the feature extraction module is used for extracting sound spectrum features and voice rhythm features from the preprocessed voice data and processing the sound spectrum features and the voice rhythm features to obtain deep feature data; the deep learning module is used for capturing time sequence dependence and semantic information in the voice data through fusion of various deep learning algorithms according to the deep feature data; and the prediction module is used for carrying out an accelerated training process on the time sequence dependence and semantic information and carrying out intention prediction on the data set so as to construct an intention prediction model. According to the method, efficient and accurate intention prediction is realized by integrating voice capture, data preprocessing, deep learning and other technologies.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a voice robot intention prediction model based on deep learning. Background Art

[0002] With the rapid development and widespread application of voice interaction technology, it has become an important way for people to interact with smart devices in their daily lives. Users interact with robots through voice, expecting robots to accurately understand their intentions and perform corresponding operations or provide required information. However, traditional voice intention recognition methods have some problems when facing complex and changeable voice signals and cannot meet the growing needs of users. The recognition accuracy of traditional methods is often low. Speech signals are affected by many factors, such as noise, accent, speaking speed, etc., which make feature extraction and pattern recognition of speech signals difficult. Traditional methods are usually based on manually designed feature extractors, which are difficult to capture deep information and complex patterns in speech signals, resulting in a decrease in recognition accuracy.

[0003] Traditional methods have a slow response speed. Voice interaction requires robots to respond to user commands or queries in real time, but traditional methods often need to perform complex calculations and reasoning processes when processing voice signals, resulting in response delays and affecting user experience. Traditional methods are also difficult to adapt to environmental changes. The characteristics and patterns of voice signals change with time, scenes, and users. Traditional methods are usually trained based on static models and cannot effectively cope with such changes, resulting in performance degradation in actual applications. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a voice robot intention prediction model and construction method based on deep learning. By integrating voice capture, data preprocessing, deep learning and other technologies, efficient and accurate intention prediction is achieved, and adaptive learning is supported to continuously optimize performance.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] In a first aspect, a voice robot intention prediction model based on deep learning, the model comprises:

[0007] A data acquisition module, which captures speech through a high-resolution microphone array, converts it into digital audio data, and pre-processes the digital audio data;

[0008] The feature extraction module extracts the spectral features and the speech prosody features from the preprocessed speech data, and processes the spectral features and the speech prosody features to obtain deep feature data;

[0009] The deep learning module captures the temporal dependency and semantic information in speech data through the fusion of multiple deep learning algorithms based on deep feature data;

[0010] The prediction module accelerates the training process of temporal dependency and semantic information and performs intent prediction on the dataset to build an intent prediction model.

[0011] Furthermore, a method for constructing a voice robot intention prediction model based on deep learning includes:

[0012] Capturing a voice signal in an environment, and converting the captured voice signal into a digital audio data format to obtain digital audio data;

[0013] Preprocessing the digital audio data to obtain preprocessed digital audio data;

[0014] Extracting spectral features and speech prosody features from the preprocessed digital audio data, and processing the extracted spectral features and speech prosody features to obtain deep feature data;

[0015] Based on the deep feature data, a deep learning model is constructed to capture the temporal dependencies and semantic information in the speech data;

[0016] According to the temporal dependency and semantic information in the speech data, the deep learning model is trained to obtain a trained deep learning model;

[0017] Based on the trained deep learning model, the test set tests the model's intent prediction ability to obtain a prediction model;

[0018] Based on the prediction model, the voice robot intention prediction can be realized.

[0019] The above solution of the present invention includes at least the following beneficial effects:

[0020] By capturing clearer voice signals through a high microphone array and performing efficient preprocessing and feature extraction, the model can more accurately capture the user's intentions. The application of deep learning algorithms can effectively process temporal dependencies and semantic information, thereby improving the accuracy of predictions and the speed of response. Advanced noise reduction technology can maintain the quality of voice input in various noise environments, ensuring that the voice robot can accurately understand the user's instructions and inquiries in noisy environments. Equalizing and compressing audio data, as well as adjusting its amplitude, can enhance the model's adaptability and generalization capabilities to different voice features. Using deep learning structures such as deep belief networks to learn spectral and prosodic features can extract more abstract and deep features. The application of the Transformer model allows for effective training of large-scale data sets, while supporting online learning and model fine-tuning, ensuring that the model is gradually optimized over time and adapts to changes in user behavior; the fusion of multi-model prediction results increases the reliability of predictions through a weighted average strategy, reduces the possibility of misjudgment, and ensures the accuracy and stability of decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of a voice robot intention prediction model based on deep learning provided by an embodiment of the present invention.

[0022] Figure 2 It is a flow chart of a method for constructing a voice robot intention prediction model based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in a form and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0024] like Figure 1 As shown, an embodiment of the present invention proposes a voice robot intention prediction model based on deep learning, and the method includes:

[0025] The data acquisition module 11 captures the voice through a high microphone array and converts it into digital audio data, and pre-processes the digital audio data;

[0026] The feature extraction module 12 extracts the spectral features and the speech prosody features from the pre-processed speech data, and processes the spectral features and the speech prosody features to obtain deep feature data;

[0027] The deep learning module 13 captures the temporal dependency and semantic information in the speech data through the fusion of multiple deep learning algorithms based on the deep feature data;

[0028] The prediction module 14 accelerates the training process of temporal dependency and semantic information, and performs intent prediction on the data set to build an intent prediction model.

[0029] In an embodiment of the present invention, through a high microphone array, it is possible to accurately capture voice signals, ensure the clarity and accuracy of voice data, and efficiently convert voice signals into digital audio data, which is convenient for subsequent processing and analysis, and pre-process the digital audio data, such as noise reduction and standardization, to improve data quality. Extracting spectral features and speech prosody features from the pre-processed voice data can fully reflect the intrinsic characteristics of the voice. By further processing the spectral features and speech prosody features, deep feature data is obtained, which helps to reveal the implicit information in the voice data. The extracted features are optimized and selected as necessary to remove redundant information. Through the fusion of multiple deep learning algorithms, the temporal dependencies in the voice data can be effectively captured, and the temporal modeling ability of the model can be improved. The deep learning algorithm can deeply mine the semantic information in the voice data, which helps to understand the speaker's true intentions and expression content. The fusion of multiple deep learning algorithms improves the adaptability and generalization ability of the model. By accelerating training technology, the training time of the model is shortened and the training efficiency is improved. Based on a rich feature set and a powerful deep learning model, it is possible to accurately predict the intent of the data set and build a high-performance intent prediction model. Through feedback and analysis of the prediction results, the model parameters and structure can be continuously optimized and adjusted to improve the model's prediction performance and stability.

[0030] In a preferred embodiment of the present invention, voice is captured by a microphone array and converted into digital audio data, and the digital audio data is preprocessed, including:

[0031] Capturing speech through a microphone array, and converting the speech signal captured through the microphone array into digital audio data;

[0032] Digital audio data is transmitted through Perform noise reduction processing to obtain the denoised audio data, where S(ω) is the power spectrum estimate of the clean speech signal, N(ω) is the power spectrum estimate of the noise, X(ω) is the complex spectrum of the speech signal interfered by the noise, Y(ω) is the output spectrum after filtering, a is a weight coefficient, β is the offset, γ is the weight coefficient of the noise power spectrum, is a positive number;

[0033] The denoised audio data is equalized and compressed to obtain data-enhanced audio data;

[0034] The amplitude of the data-augmented audio data is adjusted to a uniform range to achieve the preprocessed data.

[0035] In an embodiment of the present invention, a high-resolution microphone array can capture speech signals more accurately, ensuring that the captured speech is of high quality and rich in details; noise reduction processing can significantly reduce background noise and interference signals, improve the clarity and purity of speech signals, and help accurately identify speech content in noisy environments, thereby improving the performance of speech recognition models. Equalization and compression processing can improve the dynamic range and spectral characteristics of audio data, making speech signals more uniform and consistent. Adjusting the amplitude of data-enhanced audio data to a uniform range can eliminate the impact of different recording environments and equipment on the amplitude of speech signals. Preprocessed high-quality audio data can be used as input for machine learning or deep learning models, thereby improving the training efficiency and prediction accuracy of the model. The preprocessing step can highlight the key information in the speech signal, reduce the complexity of the model, and make it easier to learn effective speech feature representations.

[0036] In a specific embodiment of the present invention, a high-performance microphone array is designed, including factors such as the number, layout, sensitivity, and frequency response of the microphones. The microphone array is installed in an appropriate position, the hardware and software parameters of the microphone array are configured, the microphone array is started, and it is in a standby state, ready to capture voice signals. When voice signals appear, the microphone array will capture these signals and convert them into analog electrical signals, use an analog-to-digital converter to sample the analog voice signals, and quantize the sampled discrete signals, that is, map them to a limited numerical range. The digital audio data is divided into short-time frames, and a window function (such as a Hamming window) is applied to each frame of data, and a fast Fourier transform is performed on each frame of data, which is converted from the time domain to the frequency domain to obtain a complex spectrum, and the power spectrum estimation of the clean voice signal and the noise is calculated based on the complex spectrum. According to actual conditions and experience, the weight coefficient, offset, and small positive number parameters in the noise reduction formula are set, and the complex spectrum of each frame of data is calculated using the provided noise reduction formula. After the formula calculation, the complex spectrum of each frame of data after noise reduction is obtained. The denoised complex spectrum is inverse fast Fourier transformed (IFFT) to convert it from the frequency domain back to the time domain to obtain the denoised time domain audio data. According to the time domain audio data, select a parametric equalizer or a graphic equalizer, adjust the parameters of the equalizer, such as gain and frequency points, according to the spectral characteristics of the audio signal and the target effect; and apply the equalizer to the denoised audio data. Select a dynamic range compressor, set the threshold, ratio, attack time, release time and other parameters of the compressor, and apply the compressor to the equalized audio data to reduce its dynamic difference and increase the overall volume level. Scan the audio data after equalization and compression to find its maximum amplitude value, set the scaling factor according to the maximum amplitude value and the predefined upper limit value (such as 1.0), and apply the scaling factor to the entire audio data to normalize its amplitude range to the predefined upper limit.

[0037] In a preferred embodiment of the present invention, the denoised audio data is subjected to equalization processing and compression processing to obtain processed audio data, including:

[0038] Analyze the denoised audio data using the FFT algorithm to obtain spectrum analysis results;

[0039] Adjust the frequency response of the audio through the equalizer to adjust the equalizer settings based on the spectrum analysis results;

[0040] Applying the equalizer settings to the entire audio data and playing the equalized audio to achieve equalization of the audio data;

[0041] Import the equalized audio file into the compression tool and adjust the compression parameters, including bit rate, sampling rate and compression level;

[0042] A compression operation is performed according to the adjusted compression parameters to generate a compressed audio file.

[0043] In an embodiment of the present invention, the FFT algorithm can analyze the spectral characteristics of the audio data in detail. By performing FFT analysis on the audio after noise reduction, the various frequency components in the audio can be more accurately identified. Spectral analysis helps to find possible problem areas in the audio; by fine-tuning the frequency response of the audio through an equalizer, the sound quality can be effectively improved, making the audio sound more natural and balanced. Applying the equalizer setting to the entire audio data can ensure that the audio maintains consistent sound quality and listening experience from beginning to end. By adjusting compression parameters such as bit rate, sampling rate, etc., the size of the audio file can be effectively controlled for easy storage and transmission. Compressed audio files take up less storage space, making it easier to save more audio content in limited devices or cloud storage.

[0044] In a specific embodiment of the present invention, an appropriate audio processing library (such as Python's librosa, scipy or numpy) is used to read the denoised audio file, and a fast Fourier transform (FFT) algorithm is applied to the audio sample array, wherein FFT is an algorithm for converting a time domain signal into a frequency domain signal, which can decompose a complex waveform into sine waves of different frequencies. Through FFT, a complex array can be obtained, and the modulus of each complex number represents the amplitude of the corresponding frequency, and the argument represents the phase. From the result of FFT, the amplitude of each frequency component can be calculated to obtain the spectrum of the audio. Observe the spectrum graph, identify which frequency regions have too high or too low energy, and enhance the energy of certain frequency regions or reduce the energy of other regions according to the different audio content to achieve a more balanced auditory effect, and use the equalizer function of the audio processing software or library to adjust the gain or attenuation of each frequency band according to the results of the spectrum analysis, wherein the equalizer usually provides control of multiple frequency bands, and the volume of different frequency bands can be adjusted separately. By performing point-by-point multiplication operations on the audio samples, the value of each sample is adjusted according to the gain or attenuation in the equalizer settings, and the equalizer settings adjusted in the above steps are applied to the entire audio data. Play the equalized audio through an audio playback library or software, import the equalized audio file into the audio compression tool, and adjust the compression parameters, including the bit rate (which determines the amount of audio data and the sound quality), the sampling rate (which determines the fidelity of the audio and the file size), and the compression level (which controls the strength of the compression algorithm and the size of the generated file). Apply the adjusted parameters in the compression tool and perform the compression operation. During the compression process, the tool will use a specific audio encoding algorithm (such as MP3, AAC, etc.) to reduce the file size while trying to maintain the sound quality. After the compression is completed, the tool will generate a new, smaller audio file.

[0045] In a preferred embodiment of the present invention, the spectral features and the speech prosody features are extracted from the preprocessed speech data, and the spectral features and the speech prosody features are processed to obtain deep feature data, including:

[0046] The preprocessed speech signal is divided into frames of fixed size, each frame signal is windowed, each frame signal after windowing is subjected to fast Fourier transform, and the time domain signal is converted into a frequency domain signal to obtain the spectrum of each frame.

[0047] According to the spectrum of each frame, a variety of spectral features are extracted, including Mel frequency cepstral coefficients, linear prediction coefficients and linear prediction cepstral coefficients;

[0048] The preprocessed speech signal is Calculate to obtain the speech prosodic features, where E i is the energy of the i-th frame, x[n] is the n-th sample of the speech signal, w[ni] is the window function centered on the i-th sample, and N is the number of samples in each frame;

[0049] Perform dimensionality reduction processing on the extracted spectral features and rhythmic features to remove redundant information to obtain features after dimensionality reduction;

[0050] The reduced-dimensional features are learned through a deep belief network to extract more abstract and deeper features to obtain deep feature data.

[0051] In an embodiment of the present invention, by dividing the speech signal into frames of fixed size and performing windowing processing on each frame, the refinement and localization of signal processing can be ensured, which helps to capture the short-time characteristics of the speech signal. The fast Fourier transform can quickly convert the time domain signal into the frequency domain signal, thereby obtaining the spectrum of each frame, which helps to more intuitively analyze the frequency components of the speech signal; by extracting a variety of spectral features such as Mel frequency cepstral coefficients, linear prediction coefficients and linear prediction cepstral coefficients, and calculating speech prosodic features, the characteristics of the speech signal can be described comprehensively and from multiple angles. Dimensionality reduction processing is performed on the extracted spectral features and prosodic features, which can remove redundant information, retain key features, help reduce the computational complexity of subsequent models, improve processing efficiency, and avoid overfitting problems. By learning the reduced-dimensional features through a deep belief network, more abstract and deep features can be extracted, which can capture the essential information and potential structure in speech data.

[0052] In a specific embodiment of the present invention, according to the pre-processed speech signal, the signal is divided into frames of fixed size (for example, 256 or 512 sample points), and the overlap (frame shift) between adjacent frames is determined. According to the determined frame length and frame shift, the pre-processed speech signal is cut into multiple short-time frames. A suitable window function is selected, such as a Hamming window, a Hanning window, etc. The window function is used to reduce the discontinuity between frames and reduce spectrum leakage. The parameters are set according to the type of the selected window function. For each frame of signal, the signal frame is weighted and the window function is multiplied with it; FFT is an efficient algorithm for calculating discrete Fourier transform (DFT), which converts the time domain signal into the frequency domain signal. In the frequency domain, the signal is represented as a collection of different frequency components. Each frame of the windowed signal is subjected to FFT transformation to generate the spectrum of each frame, that is, the amplitude and phase information of the signal at different frequencies. The amplitude spectrum is obtained by taking the absolute value of the FFT result and performing logarithmic transformation (such as logarithmic amplitude spectrum). Mel scale is a nonlinear frequency scale based on the human auditory system. It has a higher resolution in the low frequency region and a lower resolution in the high frequency region. Therefore, the linear spectrum obtained by FFT is converted into Mel spectrum, and then the discrete cosine transform (DCT) is applied to the Mel spectrum to obtain MFCC, where MFCC is a commonly used feature in speech recognition and speech synthesis. Linear prediction is a method for estimating speech signals. It assumes that the current sample can be predicted by a linear combination of past samples. The coefficients of the linear prediction model are solved by minimizing the prediction error. These coefficients are LPC. LPC reflects the vocal tract characteristics of the speech signal. The cepstrum is the inverse Fourier transform of the logarithm of the spectrum of the signal. In speech processing, cepstrum analysis helps to separate the sound source and vocal tract characteristics. The cepstrum is calculated based on LPC, and then the cepstrum is further processed by DCT or other transformations to obtain LPCC, where LPCC is another effective feature for speech recognition. In order to reduce the influence of noise and inter-frame changes, the extracted features can be smoothed, such as using a moving average filter, and the features can be normalized, such as using mean and variance normalization. The extracted spectral features (such as MFCC, LPC, LPCC) can be applied to various speech processing tasks, such as speech recognition, speaker recognition, sentiment analysis, etc.

[0053] In another specific embodiment of the present invention, for the preprocessed speech signal, the speech prosodic features are obtained by calculating the energy of each frame. By calculating the energy of each frame, the prosodic features of the speech signal, such as sound intensity and sound length, can be obtained. The extracted spectral features and prosodic features are subjected to dimensionality reduction processing by principal component analysis to remove redundant information. After dimensionality reduction processing, a more compact and effective feature set can be obtained. A deep belief network model is constructed, which is composed of a plurality of restricted Boltzmann machines stacked together and can learn deep representations in the data. The reduced-dimensional features are used as the input of the deep belief network model, and more abstract and deep features are extracted by training the restricted Boltzmann machines layer by layer. Each layer of restricted Boltzmann machines will learn a representation of the data and pass it to the next layer. After learning the deep belief network model, more abstract and deep feature data can be obtained.

[0054] In a preferred embodiment of the present invention, based on deep feature data, multiple deep learning algorithms are integrated to capture the temporal dependency and semantic information in speech data, including:

[0055] The deep feature data is fused by concatenation and weighted summation to obtain fused features, which include short-term information, long-term dependency and context information of speech data;

[0056] Based on the fused features, a Transformer deep learning model is constructed;

[0057] Perform model training on the Transformer deep learning model to obtain a trained model;

[0058] Based on the trained model, new speech data is predicted and inferred to capture the temporal dependencies and semantic information.

[0059] In an embodiment of the present invention, by fusing deep feature data through concatenation, weighted summation, etc., it is possible to combine the advantages of different features to form a richer and more comprehensive feature representation, which includes both the short-term information of speech data and the long-term dependency and contextual information; the Transformer deep learning model is well-known for its excellent self-attention mechanism, which can effectively capture the temporal dependency in sequence data. In speech processing, the model can better understand the continuity and dynamic changes of speech signals, thereby improving the accuracy of speech recognition and reasoning. By training the Transformer deep learning model, the information in a large amount of speech data can be fully utilized, so that the model can learn the complex mapping relationship between speech signals and semantic content, which helps to improve the generalization ability of the model and enable it to maintain good performance when facing new speech data. The trained Transformer model can accurately predict and reason about new speech data. By capturing the temporal dependency and semantic information in speech data, prediction results closely related to speech content can be generated.

[0060] In a specific embodiment of the present invention, deep feature data is extracted through a deep belief network, and deep feature data of different time steps or different levels are spliced. For example, the features of adjacent frames can be spliced ​​together to capture short-term information; or the output features of DBNs at different levels can be spliced ​​to fuse information at different abstract levels. According to the importance or contribution of each feature, a weight is assigned to it, and then all features are weighted and summed to obtain a fused feature vector. Through splicing and weighted summation, a feature vector that fuses short-term information, long-term dependency and contextual information is finally obtained. The architecture of the Transformer model is designed, including an input layer, multiple Transformer encoder layers, and an output layer. Each Transformer layer includes a self-attention mechanism and a feedforward neural network, and the parameters of the Transformer model, including weights and biases, are initialized. The fused feature vector is paired with the corresponding label to form a training data set. According to the task type, a suitable loss function is defined, and the Adam optimizer is selected to adjust the parameters of the model during the training process to minimize the loss function. Input the training data into the Transformer model in batches, perform forward propagation to calculate the loss, and then update the model parameters through backpropagation and optimizer. Repeat this process until the model performance reaches a satisfactory level or no longer improves significantly. Evaluate the performance of the model on the validation set. If the model performance improves, save the parameters of the current model for subsequent use. Load the trained Transformer model and perform the same preprocessing steps on the new speech data as the training data, including framing, windowing, and FFT; to extract the corresponding feature vectors. Fuse the feature vectors of the new data, and then input the fused features into the trained Transformer model. Perform forward propagation through the Transformer model to capture the temporal dependencies and semantic information in the input sequence.

[0061] In a preferred embodiment of the present invention, the speech data is input into the Transformer deep learning model for model training through forward propagation calculation to obtain a trained model, including:

[0062] Input the speech data into the Transformer model and perform forward propagation to obtain the prediction results of the deep learning model;

[0063] According to the prediction results and actual labels of the deep learning model, During the training process, the parameters of the deep learning model are optimized, where L is the value of the loss function, C is the total number of categories, and y c is the target output, p c is the probability that the model predicts class C;

[0064] According to the parameters of the optimized deep learning model, the gradient is propagated from the output layer to the input layer layer by layer through the chain rule, and the weights and biases of the model are updated to obtain the gradient of the deep learning model parameters;

[0065] According to the obtained gradient, the deep learning model parameters are updated through gradient descent to minimize the loss function value to achieve parameter optimization;

[0066] The model training is completed by continuously iterating and optimizing the parameters until the preset number of training rounds is reached.

[0067] In an embodiment of the present invention, through forward propagation, the Transformer model can generate prediction results based on the input speech data. Due to its powerful time series processing capability and self-attention mechanism, it can capture the key information in the speech, thereby generating accurate predictions. The loss function value is calculated according to the prediction results of the model and the actual label, and the accuracy of the model prediction can be quantified. By optimizing the model parameters during the training process to minimize the loss function value, it can be ensured that the model gradually learns a more accurate mapping relationship and improves the prediction performance. Using the chain rule to propagate the gradient from the output layer to the input layer layer by layer, the gradient of the model parameters can be efficiently calculated, allowing the model to quickly adjust its internal parameters during the training process to adapt to the characteristics of the input data and accelerate the convergence speed of the model. Updating the model parameters by gradient descent can ensure that the model moves in the direction of reducing the loss function value in each iteration. The update strategy helps to avoid the model from falling into a local optimal solution and improve the generalization ability and stability of the model. By setting a preset number of training rounds, the training process of the model can be controlled, which helps to complete the training of the model in a reasonable time and avoid waste of resources or degradation of model performance due to excessive training time.

[0068] In a specific embodiment of the present invention, necessary preprocessing is performed on the speech data, including framing, windowing, and feature extraction steps; a feature vector sequence that can be directly input into the Transformer model is obtained. The preprocessed speech data is input into the Transformer model, and the Transformer model is processed by the self-attention mechanism to capture the dependencies in the sequence, and the feedforward neural network is used for nonlinear transformation, and the prediction result is output. The prediction result of the model and the actual label are substituted into the loss function to calculate the loss value of the current batch of data. Before starting back propagation, the gradients of all trainable parameters in the model need to be initialized to zero. Back propagation is performed layer by layer from the output layer to the input layer through the chain rule. During the back propagation process, the learning rate is set before updating the model parameters according to the derivative of the loss function with respect to the model output and the local derivatives of each layer of the model. The gradient of each parameter is obtained through the formula. Before updating the model parameters, the learning rate is set. The calculated gradient value and learning rate are used to update the model parameters (weights and biases) through the gradient descent algorithm. In each iteration, the model parameters are updated according to the calculated gradient to optimize the parameters. The above steps are repeated for multiple rounds of training. Each round will traverse the entire training data set once, reaching the preset number of training rounds, and the model training is terminated.

[0069] In a preferred embodiment of the present invention, according to optimizing the parameters of the deep learning model, the gradient is propagated from the output layer to the input layer layer by layer through the chain rule, and the weights and biases of the model are updated to obtain the gradients of the deep learning model parameters, including:

[0070] According to the parameters of the optimization model, the gradient is propagated from the output layer to the input layer layer by layer through the chain rule to update the weights and biases of the model;

[0071] According to the updated model weights and biases, by To get the gradient of the model parameters, where Wl is the weight matrix of the lth layer, η is the learning rate, δ (l+1) is the error gradient of the l+1th layer, W (l+1) is the weight matrix of the l+1th layer, f′(z (l) ) is the derivative of the activation function of the lth layer, ⊙ is the element-wise multiplication, is the weighted input z of the lth layer (l) Weight W (l) The partial derivative of .

[0072] In an embodiment of the present invention, the chain rule allows the neural network to efficiently calculate the gradient of each parameter. By using the chain rule in a layer-by-layer transmission mode, the gradient of the loss function to each parameter can be accurately calculated, so that the update of the model parameters is more accurate, which helps to improve the performance of the model. Each layer only focuses on the information of the previous and next layers that are directly related to itself, and realizes the modularization of gradient calculation. Although back propagation requires more memory to store intermediate values ​​for reuse when calculating gradients, this strategy avoids repeated calculations and improves the efficiency of the training process as a whole. Through the continuous iteration of the optimization process, the weights and biases of the model are dynamically adjusted, so that the model can gradually adapt to the training data and improve the prediction accuracy. The chain rule is not only applicable to neural networks with a single output, but can also be extended to the partial derivative calculation of multivariate functions, and is applicable to various complex neural network structures. The gradient calculated by the chain rule can be used in combination with various optimization algorithms to achieve rapid convergence and optimization of model parameters.

[0073] In a specific embodiment of the present invention, the structure of the neural network is determined, including the number of layers, the number of neurons in each layer, and the activation function, the weight matrix of each layer is randomly initialized, and an appropriate learning rate is set. The learning rate is a positive number used to control the step size of parameter update; for each layer l, the weighted input z is calculated (l) =W (l) a (l-1) +b (l) , where a (l-1) is the activation output of the previous layer (i.e., layer l-1), b (l) Is the bias of this layer. The activation output a of this layer is calculated by the activation function f (l) =f(z (l) ). Define a loss function to measure the gap between the network prediction and the true value, and calculate the error gradient δ of the output layer according to the loss function (L) , where L is the total number of layers in the network. Starting from the output layer, the error gradient δ of each layer is calculated layer by layer towards the input layer (L) , by the formula δ(L)=((W (l+1) ) T δ (l+1) )⊙f(z (l) ) is completed. For each layer l, the weight matrix W is calculated according to the formula (l) The gradient of Complete the calculation of the gradient of the bias vector. Update the weights and biases according to the gradient descent algorithm, and repeat the above steps of forward propagation, error gradient calculation, back propagation, and parameter update until the stop condition is met (the preset number of iterations is reached).

[0074] In a preferred embodiment of the present invention, the temporal dependency and semantic information are accelerated to train the process, and the intention prediction is performed on the data set to obtain the prediction result, including:

[0075] Based on the trained model, fine-tune it on the temporal dependency and semantic information dataset to obtain a fine-tuned training model;

[0076] According to the fine-tuned training model, adjust the learning rate and train the model performance through the test set to obtain the prediction model;

[0077] According to the prediction model, intent prediction is performed on the temporal dependency and semantic information datasets to obtain multiple prediction results.

[0078] In an embodiment of the present invention, by fine-tuning on a specific data set, the model can better adapt to the characteristics of temporal dependency and semantic information, thereby improving performance on tasks in this field. Fine-tuning allows the model to learn the specialized knowledge required for a specific task while retaining a large amount of general knowledge learned in the pre-training phase. Compared with training a model from scratch, fine-tuning usually requires less training data and computing resources, thereby reducing training costs. Adjusting the learning rate can help the model learn more effectively during the training process, avoiding convergence problems caused by learning too fast or inefficient training caused by learning too slowly. Evaluating the performance of the model through the test set can ensure that the model not only performs well on the training data, but also maintains good generalization capabilities on unseen data. The performance feedback of the test set can be used as a basis for adjusting the model structure, parameters or training strategies. Accurate intent prediction can enhance the user's interactive experience with the system. In automated tasks or decision-making support systems, accurate intent prediction can significantly improve the success rate and efficiency of tasks.

[0079] In a specific embodiment of the present invention, a data set containing temporal dependency and semantic information is collected. The data set may include text sequences and time series data, which contain rich temporal relationships and semantic contexts. The data is cleaned, standardized and formatted to meet the input requirements of the model. For example, for text data, operations such as word segmentation, stop word removal, and word vectorization may be required. A suitable pre-trained model is selected as a starting point. The pre-trained model is a model trained on a large amount of general data, such as BERT and GPT; the weights and configuration of the pre-trained model are loaded. According to the requirements of a specific task, the structure of the pre-trained model is fine-tuned, such as adding or removing certain layers. In order to retain the general knowledge in the pre-trained model, the weights of some layers can be frozen so that they are not updated during the fine-tuning process. The model is fine-tuned with the prepared temporal dependency and semantic information data set, and multiple training iterations are performed. In each iteration, the model attempts to minimize the loss function to learn specific patterns in the data set. An initial learning rate is set, and as the training progresses, a learning rate scheduling strategy is used to dynamically adjust the learning rate, such as learning rate decay and periodic adjustment. A portion of the original dataset is divided as a validation set or test set to evaluate the performance of the model. According to the task type, appropriate performance indicators such as accuracy, recall, F1 score, mean square error are selected. After each training iteration, the validation set is used to evaluate the performance of the model. The learning rate or other hyperparameters are adjusted according to the changes in the performance indicators. Based on the performance on the validation set, the best performing model is selected as the final prediction model. According to the temporal dependency and semantic information dataset of the previous prediction, the dataset has similar structure and characteristics as the training set, and the prediction dataset is preprocessed in the same way as the training set. Load the best model weights and configuration selected in the fine-tuning phase, input the preprocessed prediction dataset into the model, and obtain the output of the model, which may represent the probability distribution of different categories, specific prediction values, etc. According to the output of the model and the nature of the task, the final prediction result is parsed. For example, in a classification task, the category with the highest probability is selected as the prediction result.

[0080] In a preferred embodiment of the present invention, the learning rate is adjusted according to the fine-tuned training model, and the performance of the model is trained through a test set to obtain a prediction model, including:

[0081] Initialize the learning rate according to the fine-tuned training model to obtain a training model with adjusted parameters;

[0082] Fine-tune the parameter-adjusted training model using the training set, and regularly evaluate the performance of the model on the validation set to obtain evaluation results;

[0083] The hyperparameters are adjusted according to the evaluation results to obtain a prediction model, where the hyperparameters include a learning rate, a regularization strength, or an optimizer parameter.

[0084] In an embodiment of the present invention, by fine-tuning the training model and adjusting the hyperparameters for performance evaluation on the validation set, the accuracy, efficiency and generalization ability of the model can be significantly improved. Adjusting the regularization strength hyperparameter helps to control the complexity of the model, thereby reducing the overfitting phenomenon on the training data. The model can be better generalized to unseen data and improve the prediction performance in practical applications. The learning rate and optimizer parameter settings can significantly speed up the training speed of the model. By adjusting these hyperparameters, computing resources can be used more effectively and the time required for the model to achieve ideal performance can be shortened. Through the careful adjustment of the hyperparameters, a model that is more robust to changes in input data can be constructed. Reasonable hyperparameter settings can also optimize the resource consumption of the model. By adjusting the model hyperparameters, a more concise and easier to explain model structure or output can sometimes be obtained. The carefully fine-tuned and evaluated model can be used as a benchmark to provide a valuable reference and starting point for subsequent research and development.

[0085] In a specific embodiment of the present invention, a preliminary fine-tuned model is obtained, an initial learning rate is set for model training, and it is ensured that there are already divided training sets and validation sets, and the model is fine-tuned using the training set. After each iteration, the validation set is used to evaluate the performance of the model. According to the validation set, the evaluation results of the accuracy and loss value indicators of the model are obtained. According to the evaluation results on the validation set, the changes in model performance are analyzed. If the performance does not meet expectations, or problems such as overfitting and underfitting occur, hyperparameter adjustment is required. According to the analysis results, hyperparameters such as learning rate, regularization strength, and optimizer parameters are adjusted. For example, if the model performs well on the training set but the performance on the validation set decreases, it may be necessary to reduce the learning rate or reduce the model complexity or increase the regularization strength. The adjusted hyperparameters are applied to the model, and the fine-tuning process on the training set is restarted. After each adjustment of the hyperparameters, the performance evaluation on the fine-tuning and validation set is repeated until the model performance reaches a satisfactory level or no longer has a significant improvement. In the iterative process, the best performing model parameters and configurations are saved for subsequent use or further analysis. After the model performance reaches a stable or optimal state, a final comprehensive evaluation is performed and the final fine-tuned model is exported.

[0086] like Figure 2 As shown, a method 20 for constructing a voice robot intention prediction model based on deep learning includes:

[0087] Step 21, capturing a voice signal in an environment, and converting the captured voice signal into a digital audio data format to obtain digital audio data;

[0088] Step 22, preprocessing the digital audio data to obtain preprocessed digital audio data;

[0089] Step 23, extracting spectral features and speech prosodic features from the preprocessed digital audio data, and processing the extracted spectral features and speech prosodic features to obtain deep feature data;

[0090] Step 24, constructing a deep learning model based on the deep feature data to capture the temporal dependency and semantic information in the speech data;

[0091] Step 25, training the deep learning model according to the temporal dependency and semantic information in the speech data to obtain a trained deep learning model;

[0092] Step 26, based on the trained deep learning model, the test set tests the intent prediction capability of the model to obtain a prediction model;

[0093] Step 27, based on the prediction model, to achieve the voice robot intention prediction.

[0094] In an embodiment of the present invention, the model can capture the voice signal in the environment in real time and quickly convert it into digital audio data. By extracting the spectral features and speech rhythm features and performing feature fusion, the model can generate deep feature data containing multiple aspects of speech information, which helps to fully capture the intrinsic information in the speech data; by using the fusion of multiple deep learning algorithms, the model can effectively capture the temporal dependency and semantic information in the speech data, which helps the model to understand the speech content more deeply and provide strong support for intention prediction. By accelerating the training process, the model can complete model training in a shorter time and quickly predict the intention of the data set, which improves the response speed and efficiency of the model. The weighted average method is used to fuse multiple prediction results, which can comprehensively consider the opinions of different prediction results and improve the accuracy and stability of the final prediction. Post-processing the fused prediction results can further optimize the prediction results and enhance the scalability and adaptability of the model. Through data storage and adaptive learning, the model can continuously accumulate new knowledge and optimize its own performance.

[0095] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A voice robot intention prediction model based on deep learning, characterized in that: The model includes: A data acquisition module, which captures speech through a high-resolution microphone array, converts it into digital audio data, and pre-processes the digital audio data; The feature extraction module extracts the spectral features and the speech prosody features from the preprocessed speech data, and processes the spectral features and the speech prosody features to obtain deep feature data; The deep learning module captures the temporal dependency and semantic information in speech data through the fusion of multiple deep learning algorithms based on deep feature data; The prediction module accelerates the training process of temporal dependency and semantic information and performs intent prediction on the dataset to build an intent prediction model.

2. The deep learning-based voice robot intention prediction model according to claim 1, characterized in that: Capture speech through a high-resolution microphone array and convert it into digital audio data. Pre-process the digital audio data, including: Capturing speech through a microphone array, and converting the speech signal captured through the microphone array into digital audio data; Performing noise reduction processing on the digital audio data to obtain noise-reduced audio data; Performing equalization and compression processing on the denoised audio data to obtain processed audio data; The amplitude of the data-augmented audio data is adjusted to a uniform range to obtain preprocessed data.

3. The deep learning-based voice robot intention prediction model according to claim 2, characterized in that: The denoised audio data is equalized and compressed to obtain processed audio data, including: Analyze the denoised audio data using the FFT algorithm to obtain spectrum analysis results; Adjust the frequency response of the audio through the equalizer to adjust the equalizer settings based on the spectrum analysis results; Applying the equalizer settings to the entire audio data and playing the equalized audio to achieve equalization of the audio data; Import the equalized audio file into the compression tool and adjust the compression parameters, including bit rate, sampling rate and compression level; A compression operation is performed according to the adjusted compression parameters to generate a compressed audio file.

4. The deep learning-based voice robot intention prediction model according to claim 3, characterized in that: Extract spectral features and speech prosody features from the preprocessed speech data, and process the spectral features and speech prosody features to obtain deep feature data, including: The preprocessed speech signal is divided into frames of fixed size, each frame signal is subjected to windowing processing, each frame signal after windowing is subjected to fast Fourier transform, and the time domain signal is converted into a frequency domain signal to obtain the spectrum of each frame; According to the spectrum of each frame, a variety of spectral features are extracted, including Mel frequency cepstral coefficients, linear prediction coefficients and linear prediction cepstral coefficients; The preprocessed speech signal is calculated to obtain speech prosodic features; Perform dimensionality reduction processing on the extracted spectral features and rhythmic features to remove redundant information to obtain features after dimensionality reduction; The reduced-dimensional features are learned through a deep belief network to extract deep features.

5. The deep learning-based voice robot intention prediction model according to claim 4, characterized in that: Based on deep feature data, the temporal dependencies and semantic information in speech data are captured, including: The deep feature data is fused by concatenation and weighted summation to obtain fused features, which include short-term information, long-term dependency and context information of speech data; Based on the fused features, a Transformer deep learning model is constructed; Perform model training on the Transformer deep learning model to obtain a trained model; Based on the trained model, new speech data is predicted and inferred to capture the temporal dependencies and semantic information.

6. The deep learning-based voice robot intention prediction model according to claim 5, characterized in that: Input the speech data into the Transformer deep learning model and perform model training through forward propagation calculation to obtain the trained model, including: Input the speech data into the Transformer model and perform forward propagation to obtain the prediction results of the deep learning model; By optimizing the parameters of the deep learning model during training, based on the prediction results and actual labels of the deep learning model; According to the parameters of the optimized deep learning model, the gradient is propagated from the output layer to the input layer layer by layer through the chain rule, and the weights and biases of the model are updated to obtain the gradient of the deep learning model parameters; According to the obtained gradient, the deep learning model parameters are updated through gradient descent to minimize the loss function value to achieve parameter optimization; The model training is completed by continuously iterating and optimizing the parameters until the preset number of training rounds is reached.

7. The deep learning-based voice robot intention prediction model according to claim 6, characterized in that: According to the parameters of the optimized deep learning model, the gradient is propagated from the output layer to the input layer layer by layer through the chain rule, and the weights and biases of the model are updated to obtain the gradient of the deep learning model parameters, including: According to the parameters of the optimization model, the gradient is propagated from the output layer to the input layer layer by layer through the chain rule to update the weights and biases of the model; Based on the updated model weights and biases, the gradient of the model parameters is calculated.

8. The deep learning-based voice robot intention prediction model according to claim 7, characterized in that: Accelerate the training process of temporal dependency and semantic information, and perform intent prediction on the data set to obtain prediction results, including: Based on the trained model, fine-tune it on the temporal dependency and semantic information dataset to obtain a fine-tuned training model; According to the fine-tuned training model, adjust the learning rate and train the model performance through the test set to obtain the prediction model; According to the prediction model, intent prediction is performed on the temporal dependency and semantic information datasets to obtain multiple prediction results.

9. The deep learning-based voice robot intention prediction model according to claim 8, characterized in that: According to the fine-tuned training model, the learning rate is adjusted, and the performance of the model is trained through the test set to obtain a prediction model, including: Initialize the learning rate according to the fine-tuned training model to obtain a training model with adjusted parameters; Fine-tune the parameter-adjusted training model using the training set, and regularly evaluate the performance of the model on the validation set to obtain evaluation results; The hyperparameters are adjusted according to the evaluation results to obtain a prediction model, where the hyperparameters include a learning rate, a regularization strength, or an optimizer parameter.

10. A method for constructing a voice robot intention prediction model based on deep learning, characterized in that: Applied to the model as claimed in any one of claims 1 to 9, the method comprises: Capturing a voice signal in an environment, and converting the captured voice signal into a digital audio data format to obtain digital audio data; Preprocessing the digital audio data to obtain preprocessed digital audio data; Extracting spectral features and speech prosody features from the preprocessed digital audio data, and processing the extracted spectral features and speech prosody features to obtain deep feature data; Based on the deep feature data, a deep learning model is constructed to capture the temporal dependencies and semantic information in the speech data; According to the temporal dependency and semantic information in the speech data, the deep learning model is trained to obtain a trained deep learning model; Based on the trained deep learning model, the test set tests the model's intent prediction ability to obtain a prediction model; Based on the prediction model, the voice robot intention prediction can be realized.

Images