Dictaphone keyword sound recognition method, device and equipment

Abstract

The application relates to the technical field of sound recognition, and discloses a sound recognition method, device and equipment for keywords of a sound recorder, which comprises the following steps: performing Fourier transformation and time domain randomization processing on original sound signals collected by a sound recorder to obtain target time domain sound signals; extracting pitch features, tone features and loudness features in the target time domain sound signals to generate target sound feature vectors; establishing a standard baseline feature library based on a plurality of standard voice samples; performing weighted k-means clustering analysis on the target sound feature vectors to generate a dynamic feature weight matrix; inputting the target sound feature vectors and the dynamic feature weight matrix into a graph structure multi-task learning model to perform keyword recognition and output initial keyword recognition results; and performing weighted optimization on the initial keyword recognition results according to the standard baseline feature library to generate target keyword recognition results. The application improves the accuracy of keyword sound recognition results of the sound recorder.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sound recognition, in particular to a voice recognition method, device and equipment for a voice recorder. BACKGROUND

[0002] The conventional voice recognition method for a voice recorder mainly relies on frequency domain analysis and feature matching, which is easily disturbed by noise in a complex environment, resulting in a decrease in recognition accuracy.

[0003] The existing voice recognition technology for a voice recorder often processes sound feature extraction and keyword recognition as independent tasks, ignoring the internal relationship between the two tasks, and does not fully consider the difference in sound features under different environmental conditions in the feature extraction process, resulting in insufficient environmental adaptability of the recognition model. This fragmented processing method limits the overall performance of the system. In addition, the traditional method performs poorly when dealing with non-stationary noise, lacks effective noise suppression mechanisms and adaptive feature weight adjustment strategies, and is difficult to accurately capture the acoustic features of keywords. At the same time, the existing method generally lacks a reliable baseline feature library and model evaluation mechanism, and cannot effectively optimize and correct the recognition results. SUMMARY

[0004] The present application provides a voice recognition method, device and equipment for a voice recorder, which improves the accuracy of keyword voice recognition results for a voice recorder.

[0005] In a first aspect, the present application provides a voice recognition method for a voice recorder, which comprises:

[0006] Performing Fourier transform and time domain randomization processing on the original sound signal collected by the voice recorder to obtain a target time domain sound signal;

[0007] Extracting pitch features, timbre features and loudness features in the target time domain sound signal to generate a target sound feature vector;

[0008] Establishing a standard baseline feature library based on a plurality of standard speech samples;

[0009] Performing weighted k-means clustering analysis on the target sound feature vector to generate a dynamic feature weight matrix;

[0010] Inputting the target sound feature vector and the dynamic feature weight matrix into a graph structure multi-task learning model for keyword recognition to output an initial keyword recognition result;

[0011] According to the standard baseline feature library, performing weighted optimization on the initial keyword recognition result to generate a target keyword recognition result.

[0012] In a second aspect, the present application provides a voice keyword recognition device of a voice recorder, which comprises:

[0013] a transformation module, configured to perform Fourier transform and time domain randomization on an original sound signal collected by the voice recorder to obtain a target time domain sound signal;

[0014] an extraction module, configured to extract pitch features, timbre features and loudness features in the target time domain sound signal to generate a target sound feature vector;

[0015] a building module, configured to build a standard baseline feature library based on a plurality of standard voice samples;

[0016] an analysis module, configured to perform weighted k-means clustering analysis on the target sound feature vector to generate a dynamic feature weight matrix;

[0017] a recognition module, configured to input the target sound feature vector and the dynamic feature weight matrix into a graph structure multi-task learning model to perform keyword recognition and output an initial keyword recognition result;

[0018] a generation module, configured to perform weighted optimization on the initial keyword recognition result according to the standard baseline feature library to generate a target keyword recognition result.

[0019] In a third aspect, the present application provides a computer device, which comprises a memory and at least one processor, wherein the memory stores instructions; and the at least one processor invokes the instructions in the memory to enable the computer device to perform the voice keyword recognition method of the voice recorder.

[0020] In a fourth aspect, the present application provides a computer readable storage medium, which stores instructions, and when the instructions are run on a computer, the computer is enabled to perform the voice keyword recognition method of the voice recorder.

[0021] The technical scheme provided by the application effectively suppresses non-stationary noise in the recording signal by introducing time domain randomization processing and adaptive filtering technology, improves the quality of the sound signal, realizes efficient extraction and screening of sound features by adopting a double feature extraction mechanism of a full convolutional neural network combined with a full connection layer and a sparse connection layer, and improves the accuracy of feature representation; a standard baseline feature library is established, a reliable feature evaluation benchmark is constructed by combining random forest regression and a probability density distribution model, and the credibility of the recognition result is enhanced; a dynamic feature weight adjustment mechanism based on weighted k-means clustering is designed, so that the system can adaptively adjust the feature weight according to different environmental conditions, and the environmental adaptability of the model is improved; a graph structure multi-task learning model is adopted to jointly optimize the sound feature extraction and keyword recognition tasks, and the generalization ability of the model is improved through parameter sharing; a feature enhancement network based on an attention mechanism is introduced to improve the capture ability of key sound features and improve the accuracy of keyword recognition; and a recognition result optimization mechanism based on multi-level evaluation is designed to improve the accuracy of the final recognition result through feature similarity analysis and consistency evaluation. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0023] Figure 1 The step schematic diagram of the voice keyword recognition method of the recording pen in the embodiment of the present application is shown in the figure.

[0024] Figure 2 The structure schematic diagram of the voice keyword recognition device of the recording pen in the embodiment of the present application is shown in the figure.

[0025] Figure 3 The structure schematic block diagram of the computer device in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0026] The embodiments of the present application provide a voice recognition method, device and equipment for a voice recorder. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be exchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" or "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0027] For ease of understanding, the specific flow of the embodiments of the present application is described below. Please refer to Figure 1 One embodiment of the voice recognition method for a voice recorder in the embodiments of the present application includes the following steps.

[0028] Step S1, performing Fourier transform and time domain randomization processing on the original sound signal collected by the voice recorder to obtain a target time domain sound signal;

[0029] It can be understood that the execution subject of the present application can be a voice recognition device for a voice recorder, and can also be a terminal or a server, which is not limited here. The embodiments of the present application take the server as an example for illustration.

[0030] Specifically, the original sound signal collected by the voice recorder is subjected to sampling frequency normalization processing to adjust the original signal into a standardized sampling signal with consistent frequency distribution, so as to adapt to subsequent signal decomposition and feature extraction steps. The standardized sampling signal is subjected to frame processing to divide the continuous time domain signal into a plurality of time window frames with fixed length, thereby generating a frame signal sequence. The frame signal sequence is subjected to short-time Fourier transform to be converted into a frequency domain signal spectrum. The time domain signal is mapped to the frequency domain to extract the frequency characteristics of the sound signal and generate a frequency domain signal representation capable of describing the sound spectrum distribution. The frequency domain signal spectrum is subjected to amplitude normalization processing to eliminate the influence of signal amplitude variation on the frequency domain characteristics, so that the amplitude of the frequency domain signal is within a unified scale range, and the generated normalized frequency domain signal spectrum has a consistent dynamic range, thereby avoiding the feature deviation problem caused by excessively large or small amplitude. Based on the probability density function of Gaussian distribution, the normalized frequency domain signal spectrum is subjected to randomization transformation. This transformation changes the fixed mode of the frequency domain signal spectrum by adding Gaussian random components, so that the frequency domain signal spectrum is more random and robust. Randomization transformation can improve the immunity of the signal to noise and interference, thereby enhancing the reliability of the system. After randomization, a randomized frequency domain signal satisfying Gaussian distribution is generated. The randomized frequency domain signal is subjected to inverse Fourier transform to map the randomized frequency domain signal back to the time domain, thereby generating an initial time domain signal. The initial time domain signal is subjected to root mean square value calculation to obtain a characteristic root mean square value, which reflects the overall intensity characteristics of the signal and serves as a measurement index of signal quality and stability. Based on the characteristic root mean square value, a signal-to-noise ratio parameter is calculated to evaluate the ratio of useful components to noise components in the signal. The initial time domain signal is subjected to adaptive filtering processing according to the signal-to-noise ratio parameter. The parameters of the filter are dynamically adjusted according to the noise level in the signal to remove noise components while retaining useful signal characteristics, thereby generating a target time domain sound signal.

[0031] Step S2, extract the pitch feature, timbre feature and loudness feature in the target time domain sound signal to generate a target sound feature vector;

[0032] Specifically, the target time-domain sound signal is subjected to cepstrum analysis to extract fundamental frequency feature data. The cepstrum analysis separates the periodicity characteristics of the signal from the frequency distribution characteristics by taking the logarithm of the frequency spectrum of the signal and performing inverse Fourier transform, thereby accurately extracting the fundamental frequency of the speech and effectively capturing the pitch characteristics of the signal. The fundamental frequency feature data is subjected to Mel frequency transform, which can map the frequency distribution to a non-linear Mel scale based on the human ear perception model, so that the extracted pitch characteristics are more in line with the characteristics of the human auditory system, and high-quality data capable of representing the pitch characteristics are obtained. At the same time, the target time-domain sound signal is subjected to short-time energy analysis to calculate the energy distribution of each time frame and obtain loudness feature data. The short-time energy analysis calculates the energy of each frame of signal through frame processing to extract the intensity information of the signal changing with time, thereby reflecting the loudness characteristics of the signal and obtaining the energy feature value of each time window to describe the loudness change and dynamic range of the sound. Linear predictive coding analysis is applied to the target time-domain sound signal. The linear predictive coding extracts feature data describing the harmonic structure and timbre of the sound by calculating the prediction residual and vocal tract parameters of the signal, reflects the physical characteristics of the speech generator, and can accurately capture the timbre characteristics, so that the timbre feature data can completely represent the timbre details of the signal. The pitch feature data, loudness feature data and timbre feature data are combined to construct a feature matrix. The feature matrix is input into the full connection layer of the full convolutional neural network. The full connection layer processes the feature matrix through convolution operation to capture the correlation and local patterns between different features. The convolution operation extracts the spatial distribution characteristics in the feature data, and then the convolution result is subjected to non-linear transformation through a non-linear activation function to generate primary combined feature data. The primary combined feature data is set with a threshold to score the importance of the features, and an importance score matrix is generated by calculating the importance score of each feature. The importance score matrix quantifies the contribution of different features in generating the target sound feature vector and provides a basis for feature optimization. According to the importance score matrix, the primary combined feature data is subjected to sparse processing to retain high-importance features and filter out redundant or low-contribution feature data, thereby obtaining filtered feature data. The filtered feature data is subjected to dimension reduction processing. Through principal component analysis or linear discriminant analysis, as much key information as possible is retained while reducing the dimension, thereby generating the final target sound feature vector.

[0033] Step S3, establishing a standard baseline feature library based on a plurality of standard speech samples;

[0034] Specifically, for multiple standard voice samples, Fourier transform and time domain randomization processing are adopted to extract frequency domain features from the original signal while eliminating noise and redundant interference in the signal. Fourier transform converts the voice signal from the time domain to the frequency domain, providing spectral information for subsequent feature extraction, while time domain randomization processing enhances the robustness and generalization ability of the features by introducing randomness. Through this step, a set of standard time domain sound signals is obtained. Sound feature extraction is performed on the standard time domain sound signals, and the extracted features include key information such as pitch, timbre, and loudness, and these features are integrated into a standard sound feature vector. Through this process, multiple standard sound feature vectors are generated. The standard sound feature vectors are segmented into data. According to the 8:2 ratio, the training feature vector and the verification feature vector are divided, where the training feature vector is used for model construction, and the verification feature vector is used to evaluate the performance and stability of the model. This data division method can effectively reduce the overfitting phenomenon and ensure the prediction ability of the model on unknown data. In the model construction stage, based on the training feature vector, N decision trees are generated, each decision tree is split based on the random selection of M feature nodes, and the mapping relationship is obtained. The splitting process of the decision tree is to find the best split point to gradually divide the feature space into different sub-regions, so as to capture the correlation and conditional probability distribution between features. The random forest type decision tree ensemble method improves the robustness of the model through the voting mechanism of multiple trees, while reducing the overfitting problem of a single decision tree. At the same time, the probability density function value of each dimension of the training feature vector is calculated to capture the distribution of the feature in different dimensions. By fitting K Gaussian distribution models, a feature dimension probability distribution matrix is obtained. The Gaussian distribution model describes the probability density characteristics of the data, making the distribution structure of the feature dimension more explicit. On this basis, the expectation maximization (EM) algorithm is applied to iteratively calculate the feature dimension probability distribution matrix, and the optimal parameters are gradually approached by alternating the expectation step and the maximization step to obtain a high-precision model that describes the feature distribution. In order to verify the performance of the model, the verification feature vector is input into the mapping relationship and the feature distribution model for prediction calculation to obtain the corresponding prediction results. Model fusion technology is used to calculate the weight coefficient of model fusion by comparing the prediction performance of the verification feature vector under different models. The calculation of the weight coefficient is based on the prediction accuracy of each model on the verification data, aiming to ensure that the final combined model can maximize the advantages of the two models. According to the model fusion weight coefficient, the mapping relationship and the feature distribution model are linearly combined to generate a standard baseline feature library. This feature library integrates the partitioning ability of the decision tree model for the feature space and the precise description of the feature probability distribution by the Gaussian distribution model, and has strong adaptability and generalization.

[0035] Step S4, weighted k-means clustering analysis is performed on the target sound feature vector to generate a dynamic feature weight matrix;

[0036] Specifically, based on the target sound feature vector, a plurality of initial clustering centers are set, and an initial framework of clustering analysis is established by assigning an initial position to each clustering center. Based on the initial clustering centers, feature distance calculation is performed on the target sound feature vector to determine the distance between each feature vector and the clustering center. The calculation adopts Euclidean distance or other distance measurement methods suitable for the feature space, and the feature vector is assigned to the clustering center with the shortest distance to obtain initial feature grouping data. The centroid coordinates of each grouping of the initial feature grouping data are calculated. The centroid coordinates reflect the central position of all feature vectors in the grouping and are the aggregated representation of the grouping features. By calculating the centroid coordinates of each grouping and taking these centroids as new clustering center positions, the update of the clustering centers is completed. Based on the updated clustering centers, the distances between the target sound feature vector and the clustering centers are calculated, and the feature vectors are re-grouped according to the new distances. Through iterative optimization, the clustering result is gradually optimized and converges to a stable feature grouping result. The intra-group variance of each group of features in the feature grouping result is calculated. The intra-group variance reflects the distribution range of the features in the grouping, and the smaller the variance, the higher the similarity of the features in the group, while the larger the variance, the stronger the dispersion of the features in the group. The intra-group variance is regarded as a noise measurement parameter, and by calculating the intra-group variance of all groupings, a feature noise matrix is generated to describe the noise level of each grouping. Based on the feature noise matrix, the signal-to-noise ratio data of each group of features is calculated. The signal-to-noise ratio is an important indicator for measuring feature quality, and the higher the value, the greater the importance of the feature and the smaller the noise interference. By calculating the signal-to-noise ratio data, the importance of each grouping is quantitatively sorted to obtain a feature importance sequence. The feature importance sequence is normalized to ensure that the importance values of different features can be compared within the same scale range, and a feature weight vector is generated. The feature weight vector is grouped according to the environmental conditions to generate an environmental feature correspondence table. Based on the environmental feature correspondence table, conditional probability calculation is performed on the feature weight vector to quantify the dynamic distribution of feature weights in different environments and generate a conditional weight matrix. Singular value decomposition is performed on the conditional weight matrix. Singular value decomposition can effectively reduce the dimensionality of the matrix while preserving the main information, removing redundant features, and improving the representation ability of the matrix. Through singular value decomposition of the conditional weight matrix, a dynamic feature weight matrix is generated. The dynamic feature weight matrix can reflect the importance distribution of the target sound features in different environments and has the ability to flexibly adjust the weights.

[0037] Step S5, inputting the target sound feature vector and the dynamic feature weight matrix into a graph structure multi-task learning model to perform keyword recognition and output an initial keyword recognition result;

[0038] Specifically, the target sound feature vector and the dynamic feature weight matrix are spliced according to the feature dimension to obtain a fusion feature matrix. The fusion feature matrix is input into a bidirectional long short-term memory (BiLSTM) module in the graph structure multi-task learning model. The module extracts time sequence information, utilizes its forward and backward bidirectional information processing capability, generates a time sequence feature representation vector, and thus captures the dynamic changes of the target sound features in the time dimension. The time sequence feature representation vector is input into a graph convolution feature extraction subnetwork of the graph structure multi-task learning model. The subnetwork is composed of 8 consecutive graph convolution layers, each of which contains 64 convolution kernels for extracting graph structure features through graph convolution operations. In the graph convolution operation process, feature information is not only transmitted within the nodes but also diffused between the nodes through the edge weights of the graph, capturing the complex correlation between the features. At the same time, the ReLU activation function and the residual connection between each graph convolution layer realize the nonlinear transformation and efficient transmission of the features, which not only enhances the expression ability of the model but also alleviates the gradient vanishing problem, and finally outputs the graph structure features. The graph structure features are input into a feature enhancement network of the graph structure multi-task learning model for optimization. The feature enhancement network is composed of 3 graph attention layers connected in series, each of which contains 8 parallel attention heads, and the feature dimension of each attention head is 64. Through attention mechanism calculation, the network can identify and amplify important features while suppressing irrelevant or redundant features. Feature weighting operations are independently completed in each attention head, and are combined into enhanced graph features. The enhanced graph features are input into a shared encoding layer. The shared encoding layer is composed of 3 fully connected layers connected in series, and the number of neurons is 512, 256 and 128 respectively. Each fully connected layer is followed by a batch normalization layer and a ReLU activation function. Through the feature transformation of this module, the model extracts high-level shared feature representations. The shared feature representations are input into a sound feature extraction branch and a keyword recognition branch of the graph structure multi-task learning model. The sound feature extraction branch contains 2 fully connected layers connected in series, and the number of neurons is 128 and 64 respectively, outputting a feature vector of sound features; while the keyword recognition branch also contains 2 fully connected layers connected in series, and the number of neurons is 128 and the number of keyword categories respectively, used to output a recognition vector matched with the candidate keywords. In these two branches, the network models the feature distribution of the sound signal and the keyword categories respectively, ensuring mutual promotion and independent optimization between tasks. After obtaining the feature vector output by the sound feature extraction branch, a sound feature probability distribution sequence is generated through Softmax normalization operation, which represents the importance of each feature dimension and provides a probabilistic weight representation for feature analysis. Similarly, the recognition vector output by the keyword recognition branch is subjected to Softmax normalization operation to obtain a keyword probability distribution sequence, which reflects the matching probability of each candidate keyword and clearly identifies the most likely keyword category.According to the sequence of keyword probability distribution, the category label with the highest matching probability is selected as the final keyword recognition result, and this category label is output as the initial keyword recognition result.

[0039] Step S6, according to the standard baseline feature library, the initial keyword recognition result is weighted and optimized to generate the target keyword recognition result.

[0040] Specifically, the standard feature data reflecting the typical features of various keywords under different environmental conditions is extracted from the standard baseline feature library. The standard feature data is normalized to generate a standardized feature vector, so that the feature values of each dimension are within the same scale range. At the same time, the initial keyword recognition result is converted into a numerical feature vector form to generate an identification feature vector. The feature similarity data between the identification feature vector and the standardized feature vector is obtained by calculating the Euclidean distance between them. The Euclidean distance calculation can quantify the closeness of the two groups of feature vectors in the multi-dimensional space, and the smaller the distance, the higher the similarity. The feature similarity data is normalized to generate an initial similarity matrix, which reflects the similarity of the identification feature vector and the standard feature vector in each dimension. The average similarity of each keyword is calculated by averaging the initial similarity matrix, which is used as a direct quantitative indicator of recognition confidence. The dynamic feature weight matrix is used to weight each feature dimension in the initial similarity matrix. The dynamic feature weight matrix adjusts the contribution value of each dimension in the similarity matrix according to the weight importance of different features to calculate the feature weighting coefficient. According to the feature weighting coefficient, the initial similarity matrix is re-weighted to generate a weighted similarity matrix. The recognition confidence and the weighted similarity matrix are spliced according to the feature dimension to construct a fusion feature vector. This fusion feature vector contains both the confidence and the similarity features after weighted optimization, providing higher-dimensional input for subsequent mapping and optimization. The fusion feature vector is linearly transformed and processed by the ReLU activation function to capture non-linear relationships and optimize feature representation, resulting in a keyword mapping result. Based on the keyword mapping result, a candidate keyword set is constructed, which contains the keyword matching result. To determine the final optimized keyword, each candidate keyword in the candidate keyword set is scored and sorted. The scoring is based on multi-dimensional information such as recognition confidence and weighted similarity, and the candidate keyword with the highest score is selected as the optimized keyword. By integrating feature data and model output, the selected keyword has the highest credibility and matching degree. The consistency score between the optimized keyword and the initial keyword recognition result is calculated. The consistency score is used to quantify the matching degree of the optimized keyword and the initial result in terms of semantics and features, and based on this, the target keyword recognition result is finally determined in combination with the previous optimization result.

[0041] In the embodiment of the present application, by introducing time domain randomization processing and adaptive filtering technology, non-stationary noise in the recorded signal is effectively suppressed, the quality of the sound signal is improved, the efficient extraction and screening of sound features are realized by adopting the double feature extraction mechanism of full convolution neural network combined with full connection layer and sparse connection layer, and the accuracy of feature representation is improved; a standard baseline feature library is established, a reliable feature evaluation benchmark is constructed by combining random forest regression and probability density distribution model, and the credibility of the recognition result is enhanced; a dynamic feature weight adjustment mechanism based on weighted k-means clustering is designed, so that the system can adaptively adjust the feature weight according to different environmental conditions, and the environmental adaptability of the model is improved; a graph structure multi-task learning model is adopted to jointly optimize the sound feature extraction and keyword recognition tasks, and the generalization ability of the model is improved through parameter sharing; an attention mechanism based feature enhancement network is introduced to strengthen the capture ability of key sound features and improve the accuracy of keyword recognition; a recognition result optimization mechanism based on multi-level evaluation is designed, and the accuracy of the final recognition result is improved through feature similarity analysis and consistency evaluation.

[0042] In a specific embodiment, the process of step S1 can specifically include the following steps:

[0043] The original sound signal collected by the sound recorder is subjected to sampling frequency normalization processing to obtain a standardized sampling signal;

[0044] The standardized sampling signal is subjected to frame processing to divide the standardized sampling signal into a plurality of time window frames to obtain a frame signal sequence, and the frame signal sequence is subjected to short-time Fourier transform processing to obtain a frequency domain signal spectrum;

[0045] The frequency domain signal spectrum is subjected to amplitude normalization processing to obtain a normalized frequency domain signal spectrum;

[0046] The normalized frequency domain signal spectrum is subjected to randomization transformation according to a Gaussian distribution probability density function to obtain a randomization frequency domain signal satisfying Gaussian distribution, and the randomization frequency domain signal is subjected to inverse Fourier transform to obtain an initial time domain signal;

[0047] The initial time domain signal is subjected to root mean square value calculation to obtain a feature root mean square value, and a signal-to-noise ratio parameter is calculated according to the feature root mean square value;

[0048] The initial time domain signal is subjected to adaptive filtering processing according to the signal-to-noise ratio parameter to obtain a target time domain sound signal.

[0049] Specifically, the original sound signal collected is subjected to sampling frequency normalization processing to ensure that the input signal has a consistent sampling frequency, so that the subsequent processing steps can be performed on a unified frequency benchmark. For example, assuming that the sampling frequency of the original sound signal is , and the target sampling frequency is , the normalization process adjusts the signal to . If the input signal is , the result of the normalization process is represented as:

[0050] ;

[0051] where represents the normalized signal, and are the target sampling frequency and the original sampling frequency, respectively. The normalized sampling signal is processed by frame segmentation, which divides the signal into multiple time window frames. The signal is segmented by setting the frame length and the frame shift . The mathematical expression of the frame segmentation process is:

[0052] ;

[0053] where represents the th sample point of the th frame. The short-time Fourier transform is applied to the frame signal sequence, which converts the time-domain signal into the frequency-domain signal spectrum. The formula of the Fourier transform is:

[0054] ;

[0055] where represents the spectrum value of the th frame at frequency , is the length of each frame. The result of the short-time Fourier transform is a two-dimensional matrix, with the spectrum values spread in the time and frequency dimensions, providing the time-frequency features of the sound signal. The amplitude normalization process is applied to the frequency-domain signal spectrum, which enhances the contrast of signal features by normalizing the amplitude of each frequency component to a uniform range. The formula of the amplitude normalization is:

[0056] ;

[0057] where is the normalized spectrum value. The randomized transform is applied to the normalized frequency-domain signal spectrum according to the Gaussian distribution probability density function, which introduces random noise to the signal to make it more consistent with the Gaussian distribution characteristics. Assuming that the randomized frequency-domain signal is , the formula of the Gaussian distribution is:

[0058] ;

[0059] where represents the mean , and the variance Gaussian noise. After randomization, the frequency-domain signal is inverse Fourier transformed to restore the original time-domain signal:

[0060] ;

[0061] The root mean square (RMS) value of the original time-domain signal is calculated to quantify the energy characteristics of the signal. The formula for calculating the RMS value is:

[0062] ;

[0063] wherein is the total number of signal sampling points. The signal-to-noise ratio (SNR) parameter is calculated based on the RMS value, defined as the ratio of signal power to noise power, and the formula is as follows:

[0064] ;

[0065] wherein represents the signal power, which is calculated by noise modeling. The initial time-domain signal is adaptively filtered based on the signal-to-noise ratio parameter, with the goal of suppressing noise interference and preserving valid signal components. The gain of the filter is set as a function of SNR, for example:

[0066] ;

[0067] wherein is the gain at frequency . The target time-domain sound signal after adaptive filtering is obtained by the following formula:

[0068] ;

[0069] wherein IFFT represents the inverse Fourier transform.

[0070] In a specific embodiment, the process of performing step S2 can specifically include the following steps:

[0071] Performing cepstrum analysis on the target time-domain sound signal, extracting fundamental frequency feature data, and performing Mel frequency transform on the fundamental frequency feature data to obtain pitch feature data;

[0072] Performing short-time energy analysis on the target time-domain sound signal, calculating the energy distribution of each time frame to obtain loudness feature data, and performing linear predictive coding analysis on the target time-domain sound signal to extract vocal tract parameters to obtain timbre feature data;

[0073] Combining the pitch feature data, loudness feature data, and timbre feature data to construct a feature matrix, and inputting the feature matrix into the fully connected layer of the fully convolutional neural network;

[0074] ​In the full connection layer, the feature matrix is subjected to convolution operation and nonlinear activation function processing to obtain primary combined feature data, and the primary combined feature data is subjected to threshold setting to perform feature importance scoring to generate an importance score matrix;

[0075] According to the importance score matrix, the primary combined feature data is subjected to sparse processing to retain high importance features to obtain screened feature data, and the screened feature data is subjected to dimension reduction processing to generate a target sound feature vector.

[0076] Specifically, cepstrum analysis is performed on the target time-domain sound signal to extract fundamental frequency feature data. The core of the cepstrum analysis lies in separating the acoustic characteristics and source characteristics of the signal, and the periodic information is reflected by the log spectrum of the signal. Given a target sound signal , the short-time Fourier transform (STFT) thereof is , and the calculation formula of the cepstrum is:

[0077] ;

[0078] Among them, represents the cepstrum coefficient, is the spectrum value of the STFT, is the amplitude thereof, and IFFT is the inverse Fourier transform operation. Through the cepstrum, the energy is concentrated in the low-order components, and the fundamental frequency is determined by finding the cepstrum peak position. The extracted fundamental frequency feature data is subjected to Mel frequency transformation, and the frequency is mapped to the Mel frequency on a logarithmic scale based on a human ear perception model, and the formula is:

[0079] ;

[0080] Among them, is the linear frequency, is the corresponding Mel frequency. Through the transformation, the pitch feature data is obtained, which represents the fundamental frequency and its perceptual characteristics of the sound. At the same time, the short-time energy analysis is performed on the target time-domain sound signal , the energy distribution of each time frame is calculated, and the loudness characteristics of the sound are quantified. Assuming that each frame contains sampling points, the calculation formula of the intra-frame energy is:

[0081] ;

[0082] Among them, represents the th sampling point of the th frame, is the ​Short-time energy of frame. The output of this step is loudness feature data, which describes the energy distribution of the sound signal in the time domain. In order to extract the timbre characteristics of the sound, linear predictive coding analysis is performed on the target time-domain sound signal, and the current sampling point is represented as a weighted sum of several previous sampling points, and the mathematical model is:

[0083] ;

[0084] wherein is the linear prediction coefficient, is the prediction order, is the prediction error. By characterizing the prediction coefficient, the timbre feature data describing the sound channel characteristics is obtained. After completing the above feature extraction, the pitch feature data, the loudness feature data and the timbre feature data are combined into a feature matrix. Assuming that the dimensions of the pitch, loudness and timbre features are , , , the combined feature matrix is:

[0085] ;

[0086] wherein , , represent the pitch, loudness and timbre feature vectors respectively. The feature matrix is input into the fully connected layer of the fully convolutional neural network, and convolution operation and nonlinear activation function processing are performed thereon to extract primary combined feature data. The mathematical expression of the convolution operation is:

[0087] ;

[0088] wherein is the convolution output, is the convolution kernel, is the bias term, is a nonlinear activation function (such as ReLU). After the primary combined feature data is processed by the activation function, the nonlinear relationship between the high-dimensional features is reflected. The primary combined feature data is set with a threshold value, and the feature importance score is generated. Assuming that the score of a certain feature is calculated according to the following formula:

[0089] ;

[0090] wherein is the score of the feature . By performing sparse processing on the importance score matrix, high-score features are retained and low-score redundant features are removed, and the screened feature data is obtained. The screened feature data is subjected to dimension reduction processing. Principal component analysis method is adopted, and the formula is:

[0091] ;

[0092] wherein is the principal component basis vector of the feature matrix, the matrix after dimension reduction is the target sound feature vector.

[0093] In a specific embodiment, the process of performing step S3 can specifically include the following steps:

[0094] performing Fourier transform and time domain randomization processing on the plurality of standard voice samples respectively to obtain a plurality of standard time domain sound signals, and performing sound feature extraction on the plurality of standard time domain sound signals respectively to obtain a plurality of standard sound feature vectors;

[0095] performing data segmentation on the plurality of standard sound feature vectors, and dividing according to a training set and a validation set in a ratio of 8:2 to obtain training feature vectors and validation feature vectors;

[0096] constructing N decision trees based on the training feature vectors, and performing split calculation on M feature nodes of each decision tree to obtain a mapping relationship, wherein N is the number of decision trees, and M is the number of feature nodes;

[0097] calculating a probability density function value for each dimension of the training feature vectors to construct K Gaussian distribution models to obtain a feature dimension probability distribution matrix, wherein K is the number of Gaussian distribution models;

[0098] performing expectation maximization iterative calculation according to the feature dimension probability distribution matrix to obtain a feature distribution model;

[0099] inputting the validation feature vectors into the mapping relationship and the feature distribution model respectively to perform prediction calculation to obtain model fusion weight coefficients, and performing linear combination on the mapping relationship and the feature distribution model according to the model fusion weight coefficients to obtain a standard baseline feature library.

[0100] Specifically, for each standard voice sample , Fourier transform is used to convert it from a time domain signal to a frequency domain signal to extract its spectral features. The formula of Fourier transform is:

[0101] ;

[0102] wherein is the th frequency component of the frequency domain signal, is the total number of sampling points of the signal. In order to enhance the robustness of the signal to noise, time domain randomization processing is performed on it by introducing a random component into the original signal to obtain a randomized time domain signal:

[0103] ;

[0104] in is the mean of the Gaussian distribution, This process generates multiple standard time domain sound signals . Perform sound feature extraction on the standard time domain sound signal to generate multiple standard sound feature vectors. Each feature vector Include Feature dimensions include pitch, timbre, and loudness. and cepstral features For example, the calculation formula for short-time energy is:

[0105] ;

[0106] The cepstrum feature is obtained by performing an inverse Fourier transform on the logarithmic spectrum of the signal:

[0107] ;

[0108] After integrating these features, a set of standard sound feature vectors for each sample is formed. Data segmentation is performed on the standard sound feature vectors. Assuming that there are feature vectors, divided into training set and validation set in the ratio of 8:2. The training set contains feature vectors, the validation set contains The training feature vector is used for model construction, and the validation feature vector is used for model evaluation and weight optimization. decision trees, where each tree is randomly selected The splitting of decision tree is based on information gain or Gini coefficient calculation. The gain is expressed as:

[0109] ;

[0110] in is the entropy before the split, is the entropy of the child node after splitting, is the sample set of the current node, It is A sample set of child nodes is randomly selected. feature nodes, generate multiple subsets and construct A decision tree is built to form a mapping relationship between features and categories. At the same time, the probability density function value is calculated for each dimension of the training feature vector, and the Gaussian mixture model (GMM) is used to build Gaussian distribution models. The probability density function of each Gaussian distribution is expressed as:

[0111] ;

[0112] wherein is the mean of the th Gaussian model, is its variance. Combining the weights of all Gaussian models forms the feature dimension probability distribution matrix . The Gaussian distribution model is optimized by the Expectation Maximization (EM) algorithm, iteratively updating the parameters , and finally obtaining the feature distribution model. The specific steps include calculating the expectation value and maximizing as follows:

[0113] ;

[0114] ;

[0115] After training, the verification feature vector is input into the mapping relationship and the feature distribution model for prediction, and the accuracy of the verification result and the distribution fitting degree are calculated respectively. By linearly combining the prediction results of the two models, the model fusion weight coefficient is calculated:

[0116] ;

[0117] wherein and represent the accuracy of the mapping relationship model and the feature distribution model respectively. According to the weight coefficient , the mapping relationship and the feature distribution model are linearly combined to construct the standard baseline feature library:

[0118] ;

[0119] wherein is the output of the mapping relationship model, is the output of the feature distribution model.

[0120] In a specific embodiment, the process of performing step S4 can specifically include the following steps:

[0121] Based on the target sound feature vector, a plurality of initial clustering centers are set, the feature distance of the target sound feature vector is calculated, and initial feature grouping data is obtained;

[0122] The centroid coordinates of each grouping of the initial feature grouping data are calculated, and the clustering center position is updated according to the centroid coordinates to obtain an updated clustering center;

[0123] According to the updated cluster center, the feature distance is recalculated, the target sound feature vector is regrouped, and a feature grouping result is obtained;

[0124] The within-group variance of each group of features in the feature grouping result is calculated, and the within-group variance is taken as the noise measurement parameter of the group of features, and a feature noise matrix is obtained;

[0125] The signal-to-noise ratio data of each group of features is calculated according to the feature noise matrix, and the feature group importance is sorted based on the signal-to-noise ratio data, and a feature importance sequence is obtained;

[0126] The feature importance sequence is normalized to obtain a feature weight vector, and the feature weight vector is grouped according to different environmental conditions to obtain an environmental feature correspondence table;

[0127] According to the environmental feature correspondence table, the conditional probability of the feature weight vector is calculated to obtain a conditional weight matrix, and the singular value decomposition of the conditional weight matrix is performed to generate a dynamic feature weight matrix.

[0128] Specifically, based on the target sound feature vector, an initial cluster center is set. Assuming that the target sound feature vector set is , where each feature vector is a -dimensional vector, is the number of feature vectors. Set initial cluster centers , which are completed by random selection or uniform initialization. The feature distance between each target feature vector and the cluster center is calculated, and the Euclidean distance is used as the measurement standard, and the calculation formula is.

[0129] ;

[0130] , where is the distance between the target feature vector and the cluster center , and and represent the components of vectors and in the first dimension. By calculating the distance of all feature vectors to all cluster centers, each feature vector is assigned to the nearest cluster center to form initial feature grouping data. The centroid coordinates of each group in the initial feature grouping data are calculated. Assuming that the feature vector set in the first group is , containing vectors, the calculation formula of the centroid coordinates is:

[0131] ;

[0132] wherein is the centroid of the th group. The centroid represents the central position of the feature vectors in the current group, which is used to update the cluster center position. The initial cluster center is replaced by the updated centroid , and the feature distance calculation and grouping process are repeated until the change of the cluster center is less than the preset convergence threshold, obtaining the final feature grouping result. After obtaining the stable feature grouping result, the intra-group variance of each group is calculated, and the intra-group variance is taken as the noise measurement parameter of the feature in the group. For the th group, the formula of the intra-group variance is:

[0133] ;

[0134] wherein represents the intra-group variance of the th group, reflecting the dispersion degree of the feature vectors in the group. The variance values of all groups constitute the feature noise matrix . Based on the feature noise matrix, the signal-to-noise ratio (SNR) data of each group of features is calculated, which is used to quantify the degree of advantage of the signal compared to the noise. The signal-to-noise ratio is defined as the ratio of the signal power to the noise power , and its formula is:

[0135] ;

[0136] wherein represents the signal power of the th group of features, which is estimated by the mean square value of the feature vectors in the group. According to the SNR values of all groups, the importance of the feature groups is sorted, obtaining the feature importance sequence. The feature importance sequence is normalized to generate the feature weight vector. Assuming that the feature importance value is , the normalization formula is:

[0137] ;

[0138] wherein is the normalized weight of the th group, . The feature weight vector is grouped according to different environmental conditions, such as noise level, recording device characteristics, etc., to generate an environmental feature correspondence table. According to the environmental feature correspondence table, conditional probability calculation is performed on the feature weight vector to generate a conditional weight matrix. The conditional probability calculates the importance of the feature group in a specific environment , and the conditional weight matrix is represented as:

[0139] ;

[0140] wherein is the total number of environmental conditions. Singular value decomposition is performed on the condition weight matrix, decomposed into:

[0141] ;

[0142] wherein and are orthogonal matrices, is a diagonal matrix containing the singular values of the condition weight matrix. Through decomposition, the matrix dimension is effectively reduced, and redundant features are eliminated to generate a dynamic feature weight matrix.

[0143] In a specific embodiment, the process of performing step S5 can specifically include the following steps:

[0144] The target sound feature vector and the dynamic feature weight matrix are spliced according to the feature dimension to obtain a fusion feature matrix, and the fusion feature matrix is input into a bidirectional long short-term memory network in the graph structure multi-task learning model to extract time sequence information, and obtain a time sequence feature representation vector;

[0145] The time sequence feature representation vector is input into a graph convolution feature extraction subnetwork of the graph structure multi-task learning model, and the graph convolution feature extraction subnetwork is composed of 8 consecutive graph convolution layers, each graph convolution layer contains 64 convolution kernels, and the graph convolution layers adopt ReLU activation function and residual connection for feature transmission, and the graph structure features are obtained through graph convolution operation;

[0146] The graph structure features are input into a feature enhancement network of the graph structure multi-task learning model, and the feature enhancement network is composed of 3 graph attention layers connected in series, each graph attention layer contains 8 parallel attention heads, and the feature dimension of each attention head is 64. Enhanced graph features are obtained through attention mechanism calculation and feature weighting;

[0147] The enhanced graph features are input into a shared encoding layer of the graph structure multi-task learning model, and the shared encoding layer is composed of 3 fully connected layers connected in series, and the number of neurons of the 3 fully connected layers is 512, 256 and 128 respectively. Each fully connected layer is followed by a batch normalization layer and a ReLU activation function, and shared feature representation is obtained through feature transformation;

[0148] The shared feature representation is respectively input into a sound feature extraction branch and a keyword recognition branch of the graph structure multi-task learning model, the sound feature extraction branch comprises two full connection layers connected in series, the neuron numbers of the full connection layers are 128 and 64 respectively, the keyword recognition branch comprises two full connection layers connected in series, the neuron numbers of the full connection layers are 128 and the number of keyword categories respectively, a feature vector output by the sound feature extraction branch and a recognition vector output by the keyword recognition branch are obtained;

[0149] The feature vector output by the sound feature extraction branch is subjected to a Softmax normalization operation to obtain a sound feature probability distribution sequence, the sound feature probability distribution sequence represents the importance of each feature dimension;

[0150] The recognition vector output by the keyword recognition branch is subjected to a Softmax normalization operation to obtain a keyword probability distribution sequence, the keyword probability distribution sequence represents the matching probability of each candidate keyword;

[0151] According to the keyword probability distribution sequence, the class label with the highest matching probability is selected as the recognition result, and the class label with the highest matching probability is output as the initial keyword recognition result.

[0152] Specifically, the target sound feature vector and the dynamic feature weight matrix are spliced according to the feature dimension. Assuming that the target sound feature vector is , the dynamic feature weight matrix is , wherein is the number of samples, and are the dimensions of the feature vector and the weight matrix respectively. The splicing processing generates a fusion feature matrix:

[0153] ;

[0154] contains sound features and weight information. The fusion feature matrix is input into a bidirectional long short-term memory network (BiLSTM) in the graph structure multi-task learning model to extract time sequence features. Assuming that the first sample of the fusion feature matrix is , the output formula of the BiLSTM is as follows:

[0155] ;

[0156] wherein is a time sequence feature representation vector, is the first The feature input of a time step. After BiLSTM processing, the time sequence feature representation matrix is obtained wherein is the number of hidden units of BiLSTM. The time sequence feature representation matrix is input into the graph convolution feature extraction subnetwork to extract graph structure features. The subnetwork is composed of 8 consecutive graph convolution layers, each containing 64 convolution kernels. Assuming that the input of the graph convolution layer is , and the output is , the operation formula is:

[0157] ;

[0158] wherein is the adjacency matrix of the graph, used to describe the connection relationship between nodes; is the convolution weight of the layer; is a nonlinear activation function (such as ReLU). In order to enhance feature transmission, residual connection is used between graph convolution layers, in the form of:

[0159] ;

[0160] After 8 layers of graph convolution processing, the graph structure feature matrix is obtained. The graph structure features are input into the feature enhancement network, which is composed of 3 graph attention layers in series. Each attention layer contains 8 parallel attention heads, and the feature dimension of each attention head is 64. The graph attention mechanism assigns weights by calculating the correlation between nodes, and the formula is as follows:

[0161] ;

[0162] ;

[0163] wherein and are the feature vectors of nodes and , is the attention weight vector, represents vector splicing, is the neighbor set of node . The feature after attention weighting is:

[0164] ;

[0165] After the graph attention layer processing, the enhanced graph feature matrix is generated. The enhanced graph feature matrix is input into the shared encoding layer, and the shared feature representation is obtained after feature transformation by 3 fully connected layers. Assuming that the input of the layer is , the output is , the formula is:

[0166] ;

[0167] wherein and are weights and biases. The output of the shared encoding layer is . The shared feature representation is input to the sound feature extraction branch and the keyword recognition branch, respectively. The sound feature extraction branch contains two fully connected layers, and finally outputs a feature vector ; the keyword recognition branch contains two fully connected layers, and outputs a recognition vector . Softmax normalization is performed on and , respectively:

[0168] ;

[0169] wherein represents the importance of the th feature dimension, represents the matching probability of the th candidate keyword. According to the keyword probability distribution sequence , the class label with the highest matching probability is selected as the initial keyword recognition result. Assuming that the keyword class set is , the initial recognition result is:

[0170] ;

[0171] After the above process, the model can output the matching probability of each keyword and accurately recognize the most likely keyword class.

[0172] In a specific embodiment, the process of performing step S6 can specifically include the following steps:

[0173] Extracting standard feature data from the standard baseline feature library, normalizing the standard feature data to obtain a standardized feature vector, and converting the initial keyword recognition result to a feature vector to obtain a recognition feature vector;

[0174] Calculating the Euclidean distance between the recognition feature vector and the standardized feature vector to obtain feature similarity data, and performing normalization operation on the feature similarity data to obtain an initial similarity matrix;

[0175] Calculating the average similarity of the initial similarity matrix to obtain an average similarity, and taking the average similarity as the recognition confidence;

[0176] The initial similarity matrix is weighted based on a dynamic feature weight matrix to obtain a feature weighting coefficient, and the initial similarity matrix is re-weighted based on the feature weighting coefficient to obtain a weighted similarity matrix.

[0177] The recognition confidence and the weighted similarity matrix are spliced according to the feature dimension to obtain a fusion feature vector, and the fusion feature vector is subjected to linear transformation and ReLU activation function processing to obtain a keyword mapping result.

[0178] A candidate keyword set is constructed based on the keyword mapping result, and candidate keywords in the candidate keyword set are scored and sorted according to the recognition confidence to select an optimized keyword with the highest score.

[0179] The consistency score of the optimized keyword with the initial keyword recognition result is calculated, and the target keyword recognition result is determined according to the consistency score.

[0180] Specifically, the standard feature data related to the current task is extracted from the standard baseline feature library. It is assumed that the standard baseline feature library contains feature vectors, each with a dimension of , denoted as , where . In order to eliminate the difference in feature scale, the standard feature data is normalized, and the normalized feature vector is denoted as:

[0181] ;

[0182] where is the feature mean, is the feature standard deviation, and the normalized feature vector set is denoted as . The initial keyword recognition result is converted into a feature vector to obtain a recognition feature vector. It is assumed that the initial keyword recognition result contains keywords, each with a feature vector dimension of , denoted as . For each pair of normalized feature vectors and recognition feature vectors, the Euclidean distance between them is calculated to quantify the similarity of the features. The formula for calculating the Euclidean distance is:

[0183] ;

[0184] where is the distance between the recognition feature vector and the standard feature vector , and represent and in the first Components on dimensions. All Euclidean distances are combined into a feature similarity matrix . For the feature similarity matrix Perform normalization operation to generate the initial similarity matrix S. The normalization formula is:

[0185] ;

[0186] in Is the normalized similarity value, reflecting the normalized result of feature similarity. The initial similarity matrix is ​​averaged to obtain the average similarity of each identified feature vector. :

[0187] ;

[0188] The average similarity is used as the confidence indicator of keyword recognition and is expressed as Based on the dynamic feature weight matrix, each feature dimension in the initial similarity matrix is ​​weighted. Assume that the dynamic feature weight matrix is , the weighted feature similarity matrix The element calculation formula is:

[0189] ;

[0190] in It is The weight of each feature dimension indicates the importance of the feature dimension. Through this weighting operation, a weighted similarity matrix is ​​generated. . The confidence vector and weighted similarity matrix Splice by feature dimension to generate fused feature vector In order to optimize the features, the fused feature vector is processed by linear transformation and ReLU activation function, and the formula is:

[0191] ;

[0192] in is the transformation matrix, is the bias vector, Is the hidden dimension. On this basis, construct the candidate keyword set , the candidate keywords are scored and ranked by recognition confidence, and the scoring formula is:

[0193] ;

[0194] Select the keyword with the highest score as the optimization keyword To verify the reliability of the optimization result, a consistency score between the optimized keywords and the initial keyword recognition result is calculated, and the formula is:

[0195] ;

[0196] The lower the consistency score is, the closer the optimized keywords are to the initial recognition result. The target keyword recognition result is determined according to the consistency score.

[0197] The above describes the voice keyword recognition method of the recording pen in the embodiment of the application, and the following describes the voice keyword recognition device of the recording pen in the embodiment of the application. Please refer to Figure 2 An embodiment of the voice keyword recognition device of the recording pen in the embodiment of the application includes:

[0198] The transformation module is configured to perform Fourier transform and time domain randomization processing on the original sound signal collected by the recording pen to obtain a target time domain sound signal.

[0199] The extraction module is configured to extract pitch features, timbre features, and loudness features in the target time domain sound signal to generate a target sound feature vector.

[0200] The establishment module is configured to establish a standard baseline feature library based on a plurality of standard speech samples.

[0201] The analysis module is configured to perform weighted k-means clustering analysis on the target sound feature vector to generate a dynamic feature weight matrix.

[0202] The recognition module is configured to input the target sound feature vector and the dynamic feature weight matrix into a graph structure multi-task learning model to perform keyword recognition, and output an initial keyword recognition result.

[0203] The generation module is configured to perform weighted optimization on the initial keyword recognition result according to the standard baseline feature library to generate a target keyword recognition result.

[0204] Through the synergistic cooperation of the above various components, by introducing time domain randomization processing and adaptive filtering technology, the non-stationary noise in the recording signal is effectively suppressed, and the quality of the sound signal is improved. The dual feature extraction mechanism of the full convolutional neural network combined with the full connection layer and the sparse connection layer is adopted to realize efficient extraction and screening of sound features, and the accuracy of feature representation is improved. A standard baseline feature library is established, combined with random forest regression and probability density distribution model, a reliable feature evaluation benchmark is constructed, and the credibility of the recognition result is enhanced. A dynamic feature weight adjustment mechanism based on weighted k-means clustering is designed, so that the system can adaptively adjust the feature weight according to different environmental conditions, and the environmental adaptability of the model is improved. The graph structure multi-task learning model is used to jointly optimize the sound feature extraction and keyword recognition tasks, and the generalization ability of the model is improved through parameter sharing. The feature enhancement network based on attention mechanism is introduced to strengthen the capture ability of the key sound features and improve the accuracy of keyword recognition. A recognition result optimization mechanism based on multi-level evaluation is designed, and the accuracy of the final recognition result is improved through feature similarity analysis and consistency evaluation.

[0205] Reference Figure 3 In the embodiment of the present application, a computer device is also provided, which can be a server, and the internal structure thereof can be as shown in Figure 3 The computer device comprises a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in the embodiment. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement the above method.

[0206] Those skilled in the art can understand Figure 3 that the structure shown in the embodiment is only a block diagram of part of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied.

[0207] The embodiment of the present application also provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by the processor to implement the above method. It can be understood that the computer readable storage medium in the embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0208] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium provided by the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0209] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-mentioned system, system and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0210] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0211] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for recognizing keywords using a voice recorder, characterized in that: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The standard baseline feature library is established based on a plurality of standard voice samples, and specifically includes: Fourier transform and time domain randomization processing are respectively performed on the plurality of standard voice samples to obtain a plurality of standard time domain sound signals, and sound feature extraction is respectively performed on the plurality of standard time domain sound signals to obtain a plurality of standard sound feature vectors; data segmentation is performed on the plurality of standard sound feature vectors, and the training feature vector and the verification feature vector are obtained by dividing according to the proportion of 8:2 of the training set and the verification set; N decision trees are constructed based on the training feature vector, M feature nodes are randomly selected for each decision tree to perform split calculation, and a mapping relationship is obtained, wherein N is the number of decision trees, and M is the number of feature nodes; a probability density function value is calculated for each dimension of the training feature vector to construct K Gaussian distribution models to obtain a feature dimension probability distribution matrix, wherein K is the number of Gaussian distribution models; expectation maximization iterative calculation is performed according to the feature dimension probability distribution matrix to obtain a feature distribution model; the verification feature vector is input into the mapping relationship and the feature distribution model respectively for prediction calculation to obtain a model fusion weight coefficient, and the mapping relationship and the feature distribution model are linearly combined according to the model fusion weight coefficient to obtain the standard baseline feature library; The target sound feature vector is subjected to weighted k-means clustering analysis to generate a dynamic feature weight matrix; The target sound feature vector and the dynamic feature weight matrix are input into a graph structure multi-task learning model for keyword recognition to output an initial keyword recognition result; The initial keyword recognition result is subjected to weighted optimization according to the standard baseline feature library to generate a target keyword recognition result.

2. The voice recognition method of claim 1, wherein, The target sound feature vector is subjected to weighted k-means clustering analysis to generate a dynamic feature weight matrix, and specifically includes: A plurality of initial clustering centers are set based on the target sound feature vector, feature distance calculation is performed on the target sound feature vector to obtain initial feature grouping data; Centroid coordinate calculation is performed on each group of the initial feature grouping data, and the clustering center position is updated according to the centroid coordinate to obtain an updated clustering center; Feature distance is recalculated according to the updated clustering center, and the target sound feature vector is regrouped to obtain a feature grouping result; In-group variance calculation is performed on each group of features in the feature grouping result, the in-group variance is taken as a noise measurement parameter of the group of features, and a feature noise matrix is obtained; Signal-to-noise ratio data of each group of features is calculated according to the feature noise matrix, and feature group importance sorting is performed based on the signal-to-noise ratio data to obtain a feature importance sequence; Normalization processing is performed on the feature importance sequence to obtain a feature weight vector, and the feature weight vector is grouped according to different environmental conditions to obtain an environmental feature correspondence table; Conditional probability calculation is performed on the feature weight vector according to the environmental feature correspondence table to obtain a conditional weight matrix, and singular value decomposition is performed on the conditional weight matrix to generate a dynamic feature weight matrix.

3. The voice recognition method of claim 2, wherein, The target sound feature vector and the dynamic feature weight matrix are input into a graph structure multi-task learning model for keyword recognition, and an initial keyword recognition result is output, which comprises: The target sound feature vector and the dynamic feature weight matrix are spliced according to the feature dimension to obtain a fusion feature matrix, and the fusion feature matrix is input into a bidirectional long short-term memory network in the graph structure multi-task learning model for time sequence information extraction to obtain a time sequence feature representation vector; The time sequence feature representation vector is input into a graph convolution feature extraction subnetwork of the graph structure multi-task learning model, the graph convolution feature extraction subnetwork is composed of 8 consecutive graph convolution layers, each graph convolution layer contains 64 convolution kernels, and the graph convolution layers are connected by ReLU activation functions and residual connections for feature transmission, and the graph structure feature is obtained by graph convolution operation; The graph structure feature is input into a feature enhancement network of the graph structure multi-task learning model, the feature enhancement network is composed of 3 serial graph attention layers, each graph attention layer contains 8 parallel attention heads, the feature dimension of each attention head is 64, and the enhanced graph feature is obtained by attention mechanism calculation and feature weighting; The enhanced graph feature is input into a shared encoding layer of the graph structure multi-task learning model, the shared encoding layer is composed of 3 serial fully connected layers, the number of neurons of the 3 fully connected layers is 512, 256 and 128 respectively, each fully connected layer is connected with a batch normalization layer and a ReLU activation function, and the shared feature representation is obtained by feature transformation; The shared feature representation is input into a sound feature extraction branch and a keyword recognition branch of the graph structure multi-task learning model respectively, the sound feature extraction branch contains 2 serial fully connected layers, the number of neurons of the fully connected layers is 128 and 64 respectively, the keyword recognition branch contains 2 serial fully connected layers, the number of neurons of the fully connected layers is 128 and the number of keyword categories respectively, and a feature vector output by the sound feature extraction branch and a recognition vector output by the keyword recognition branch are obtained; The feature vector output by the sound feature extraction branch is subjected to Softmax normalization operation to obtain a sound feature probability distribution sequence, and the sound feature probability distribution sequence represents the importance of each feature dimension; The recognition vector output by the keyword recognition branch is subjected to Softmax normalization operation to obtain a keyword probability distribution sequence, and the keyword probability distribution sequence represents the matching probability of each candidate keyword; According to the keyword probability distribution sequence, the class label with the highest matching probability is selected as the recognition result, and the class label with the highest matching probability is output as the initial keyword recognition result.

4. The voice recognition method of claim 3, wherein, The initial keyword recognition result is weighted and optimized according to the standard baseline feature library to generate a target keyword recognition result, which comprises: Standard feature data is extracted from the standard baseline feature library, the standard feature data is normalized to obtain a standardized feature vector, and the initial keyword recognition result is converted into a recognition feature vector; Euclidean distance calculation is performed on the identification feature vector and the standardized feature vector to obtain feature similarity data, and normalization operation is performed on the feature similarity data to obtain an initial similarity matrix; Mean value calculation is performed on the initial similarity matrix to obtain an average similarity, and the average similarity is taken as an identification confidence; Based on the dynamic feature weight matrix, each feature dimension in the initial similarity matrix is weighted to obtain a feature weighting coefficient, and the initial similarity matrix is re-weighted according to the feature weighting coefficient to obtain a weighted similarity matrix; The identification confidence and the weighted similarity matrix are spliced according to the feature dimension to obtain a fusion feature vector, and linear transformation and ReLU activation function processing are performed on the fusion feature vector to obtain a keyword mapping result; Based on the keyword mapping result, a candidate keyword set is constructed, and the candidate keywords in the candidate keyword set are scored and sorted according to the identification confidence, and an optimized keyword with the highest score is selected; The consistency score of the optimized keyword and the initial keyword recognition result is calculated, and the target keyword recognition result is determined according to the consistency score.

5. A voice recognition apparatus for a dictation device, comprising: The voice recognition device for the voice recognition method of the voice recorder comprises: A transformation module is configured to perform Fourier transform and time domain randomization processing on the original sound signal collected by the voice recorder to obtain a target time domain sound signal; An extraction module is configured to extract pitch features, timbre features and loudness features from the target time domain sound signal to generate a target sound feature vector; An establishment module is configured to establish a standard baseline feature library based on a plurality of standard voice samples; An analysis module is configured to perform weighted k-means clustering analysis on the target sound feature vector to generate a dynamic feature weight matrix; An identification module is configured to input the target sound feature vector and the dynamic feature weight matrix into a graph structure multi-task learning model to perform keyword recognition and output an initial keyword recognition result; A generation module is configured to perform weighted optimization on the initial keyword recognition result based on the standard baseline feature library to generate a target keyword recognition result.

6. A computer device, comprising: The voice recognition device for the voice recognition method of the voice recorder comprises:

7. A computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, causes the processor to perform the voice recognition method of the voice recorder according to any one of claims 1 to 4.

Images