Accompanying robot voice recognition method and system

Abstract

The invention provides a voice recognition method and system for an accompanying robot. The method comprises the following steps: acquiring voice data; processing the voice data based on the time domain feature and obtaining a time domain feature data set of the first voice feature vector; acquiring an awakening rate corresponding to each first voice feature vector in the time domain feature data set; screening the first voice feature vectors of which the wake-up rates are higher than a preset first wake-up rate threshold; processing the screened first voice feature vector based on the frequency domain feature and obtaining a frequency domain feature data set of the second voice feature vector; obtaining the accuracy corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; screening second voice feature vectors with the accuracy higher than a preset second accuracy threshold; and determining voice preference features of the accompanying robot according to the second voice feature vector. According to the invention, the technical effects of effectively improving the voice recognition accuracy of the accompanying robot and improving the user interaction experience can be realized.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and more specifically, to a companion robot speech recognition method and system. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, machine learning, and the Internet of Things, the performance of intelligent companion robots has been continuously improved, and their functions have become more diverse. Intelligent companion robots can communicate fluently with users through natural language processing technology and provide customized services based on the personalized needs of users, becoming an important auxiliary tool in modern families, educational institutions, and medical services. In the application of intelligent companion robots, the accuracy of speech recognition is crucial to the user experience. The current speech recognition framework often faces challenges when processing speech content with local accents, unclear speech of young children or the elderly, which directly affects the recognition accuracy and the user experience during the interaction process.

[0003] Therefore, there is an urgent need for a high-accuracy speech recognition method. Summary of the invention

[0004] In view of the above problems, an object of the present invention is to provide a companion robot speech recognition method and system to solve at least one problem existing in the prior art.

[0005] According to one aspect of the present invention, there is provided a companion robot speech recognition method, which is applied to an electronic device, comprising:

[0006] Get voice data;

[0007] The voice data is processed based on the time domain feature to obtain a time domain feature data set of a first voice feature vector; the time domain feature data set of the first voice feature vector is input into the companion robot, and the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector is obtained; and the first voice feature vector whose wake-up rate is higher than a preset first wake-up rate threshold is screened;

[0008] The first voice feature vector selected is processed based on the frequency domain feature to obtain a frequency domain feature data set of a second voice feature vector; the frequency domain feature data set of the second voice feature vector is input into the companion robot to obtain the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; and the second voice feature vector having an accuracy rate higher than a preset second accuracy rate threshold is selected;

[0009] The voice preference feature of the companion robot is determined according to the second voice feature vector.

[0010] In addition, an optional technical solution is that the time domain feature data set of the first speech feature vector includes: the first speech feature vector after the short-time energy feature parameter is adjusted and the first speech feature vector after the zero-crossing rate feature parameter is adjusted.

[0011] In addition, an optional technical solution is that the frequency domain feature data set of the second speech feature vector includes the second speech feature vector after the Mel-frequency cepstral coefficient feature parameters are adjusted and the second speech feature vector after the spectrum centroid feature parameters are adjusted.

[0012] In addition, an optional technical solution is that the first speech feature vector after the short-time energy feature parameter is adjusted includes the first speech feature vector after the frame size parameter is adjusted, the first speech feature vector after the frame shift parameter is adjusted, the first speech feature vector after the smoothing window size parameter is adjusted, and the first speech feature vector after the pre-emphasis coefficient parameter is adjusted;

[0013] The first speech feature vector after the zero crossing rate characteristic parameter is adjusted includes the first speech feature vector after the frame size characteristic parameter is adjusted, the first speech feature vector after the frame shift characteristic parameter is adjusted, the first speech feature vector after the window function characteristic parameter is adjusted, and the first speech feature vector after the high pass filter cutoff frequency characteristic parameter is adjusted;

[0014] Among them, the first speech feature vector after the short-time energy feature parameter is adjusted and the first speech feature vector after the zero-crossing rate feature parameter is adjusted are the same as the first speech feature vector after the frame size feature parameter is adjusted and the first speech feature vector after the frame shift feature parameter is adjusted.

[0015] In addition, an optional technical solution is that the second speech feature vector after the Mel-frequency cepstral coefficient feature parameters are adjusted includes the second speech feature vector after the Mel-frequency range feature parameters are adjusted, the second speech feature vector after the DCT coefficient number feature parameters are adjusted, the second speech feature vector after the pre-emphasis feature parameters are adjusted, and the second speech feature vector after the logarithmic operation feature parameters are adjusted.

[0016] In addition, an optional technical solution is that the second speech feature vector after the spectral centroid characteristic parameters are adjusted includes the second speech feature vector after the spectral resolution characteristic parameters are adjusted, the second speech feature vector after the spectral smoothing characteristic parameters are adjusted, the second speech feature vector after the window function characteristic parameters are adjusted, and the second speech feature vector after the custom weighted characteristic parameters are adjusted.

[0017] In addition, an optional technical solution is that both the first wake-up threshold and the second wake-up threshold are 95%.

[0018] In addition, the optional technical solution also includes:

[0019] Performing feature alignment processing on the voice data to be recognized according to the voice preference features of the companion robot;

[0020] Speech recognition is performed on the speech data after the feature alignment processing.

[0021] In addition, an optional technical solution is that before the step of processing the speech data based on the short-time energy and the zero crossing rate and obtaining the time domain feature data set, it also includes:

[0022] The speech data is subjected to noise reduction preprocessing using a Wiener filter model.

[0023] On the other hand, the present invention also provides a companion robot speech recognition system, which uses the companion robot speech recognition method as described above to perform speech recognition; comprising:

[0024] An acquisition unit, used for acquiring voice data;

[0025] A first screening unit is used to process the voice data based on the time domain feature and obtain a time domain feature data set of a first voice feature vector; input the time domain feature data set of the first voice feature vector into the companion robot, and obtain the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector; and screen the first voice feature vector whose wake-up rate is higher than a preset first wake-up rate threshold;

[0026] A second screening unit is used to process the screened first voice feature vector based on the frequency domain feature and obtain a frequency domain feature data set of a second voice feature vector; input the frequency domain feature data set of the second voice feature vector into the companion robot to obtain the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; and screen the second voice feature vector whose accuracy rate is higher than a preset second accuracy rate threshold;

[0027] A feature determination unit is used to determine the voice preference feature of the companion robot according to the second voice feature vector.

[0028] The above companion robot speech recognition method and system processes speech data based on time domain features and obtains a time domain feature data set of a first speech feature vector; obtains the awakening rate corresponding to each first speech feature vector in the time domain feature data set of the first speech feature vector; screens the first speech feature vector with an awakening rate higher than a preset first awakening rate threshold; processes the screened first speech feature vector based on frequency domain features and obtains a frequency domain feature data set of a second speech feature vector; obtains the accuracy rate corresponding to each speech feature vector in the frequency domain feature data set of the second speech feature vector; screens the second speech feature vector with an accuracy rate higher than a preset second accuracy rate threshold; and determines the speech preference feature of the companion robot according to the second speech feature vector. Through the screening mechanism based on the awakening rate and accuracy rate, the most effective speech feature for the companion robot can be identified, thereby improving the response speed and accuracy of speech recognition according to the most effective speech feature, and finally achieving the technical effect of effectively improving the accuracy rate of speech recognition of the companion robot and enhancing the user interaction experience.

[0029] In order to achieve the above and related purposes, one or more aspects of the present invention include the features that will be described in detail later. The following description and the accompanying drawings describe some exemplary aspects of the present invention in detail. However, these aspects indicate only some of the various ways in which the principles of the present invention can be used. In addition, the present invention is intended to include all these aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings:

[0031] Figure 1 A schematic diagram of a flow chart of a companion robot speech recognition method according to an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of the modules of a companion robot speech recognition system provided in one embodiment of the present invention.

[0033] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a companion robot voice recognition method provided by an embodiment of the present invention.

[0034] The same reference numerals throughout the drawings indicate similar or corresponding features or functions. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, "and / or" in the text is only a description of the association relationship of the associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone.

[0037] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In addition, in the description of the embodiments of the present application, "plurality" means two or more than two.

[0038] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0039] In order to describe the companion robot speech recognition method and system of the present invention in detail, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0040] AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0041] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0042] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can enable effective communication between people and computers using natural language. Natural language processing involves natural language, which is the language people use in daily life, and is closely related to linguistic research; it also involves computer science and mathematics, and is an important technology for model training in the field of artificial intelligence. The pre-trained model is developed from the large language model (LLM) in the field of NLP. After fine-tuning, the large language model can be widely used in downstream tasks. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graphs and other technologies.

[0043] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0044] Figure 1 A flow chart of a companion robot speech recognition method according to an embodiment of the present invention is shown.

[0045] like Figure 1 As shown, the companion robot voice recognition method provided in this embodiment is applied to electronic equipment and mainly includes the following steps:

[0046] S110: Acquire voice data.

[0047] The companion robot in this embodiment can be any terminal device capable of voice interaction data. The terminal device can be a smart phone, tablet computer, laptop computer, PDA, personal computer, smart TV, smart watch, vehicle-mounted device, wearable device, etc., but is not limited to this. The interactive information between the terminal device and the user may include voice information and video information. It should be noted that the voice data is the voice data selected from the user-robot interactive language database. The companion robot allows the user to communicate with the system in natural language through its voice interaction function, easily obtain information and complete various tasks according to user instructions. Exemplarily, the companion robot can also actively provide emotional support services based on the user's emotions captured by emotion recognition and analysis technology.

[0048] The companion robot involved in this embodiment may include a speaker, a microphone and / or a camera; as an example, it may also have a communication function, and may establish a wired or wireless communication connection with electronic devices such as mobile phones and computers. Among them, for example, the wireless connection may be a short-distance transmission technology such as a wireless fidelity (Wi-Fi) connection and a Bluetooth connection. The wired connection may be a universal serial bus (USB) connection, a high-definition multimedia interface (HDMI) connection, etc. This embodiment does not limit the type of communication connection. The device can share call functions, such as answering and making calls, with other communication devices through the established communication connection. The device may have a built-in SIM card to realize answering and making calls, so that it has the function of a mobile communication device. When interacting with the companion robot, the user can conveniently make calls without holding other communication devices.

[0049] In this embodiment, the audio data transmission interface can adopt an I2S interface, a PCM interface or a PDM interface. Among them, I2S stands for Inter-IC Sound, Integrated Interchip Sound, or IIS for short, which is a digital audio transmission standard used for transmitting digital audio data between internal components of the system, such as codec CODEC, DSP, digital input / output interface, ADC, DAC and digital filter. The PCM (pulse code modulation) interface consists of a clock pulse (BCLK), a frame synchronization signal (FS), received data (DR) and sent data (DX), and correspondingly includes four signal lines, which are respectively used to transmit data clock signal, frame synchronization clock signal, received data signal and sent data signal. The transmission content of the PDM interface is the data after PDM encoding, and there are only two signal lines, which are respectively used to transmit clock signal and data signal.

[0050] Accordingly, the signal processing module in this embodiment can adopt a DSP (Digital Signal Process) module, or other similar signal processing modules. The signal processing module can be an external signal processing module independent of the electronic device, that is, a signal processing module arranged outside the electronic device, so that on the basis of the existing electronic device, the hardware setting of the electronic device itself is not changed, and the dynamic music modulation function is realized through an additional device; it can also be a signal processing module built into the electronic device, so that the user can realize the dynamic music modulation function based on one device.

[0051] Specifically, as an example, the SOC (System On Chip) of the electronic device sends the audio stream data played by the electronic device to its own or external DSP module through its I2S interface, so that the DSP module modulates the audio stream data.

[0052] Before the step of processing the speech data based on short-time energy and zero crossing rate and obtaining a time domain feature data set, the method also includes pre-processing the speech data with noise reduction using a Wiener filter model. Wiener filtering is a linear minimum mean square error (LMMSE) estimator, which is mainly used to recover the original signal from a noise-contaminated signal. Its goal is to minimize the mean square error (MSE) between the restored signal and the original signal. In speech enhancement, when the speech signal is interfered by environmental noise to form noisy speech, the Wiener filter can be used to estimate the pure speech signal. For example, in mobile phone calls or speech recognition systems, background noise is reduced and speech quality and recognition rate are improved.

[0053] Noise reduction processing may also include, for example, removing background sounds from voice data, thereby eliminating noise interference and improving the accuracy of subsequent voice data recognition. In a specific implementation, preprocessing may include, after the voice signal to be recognized is divided into multiple audio frames, feature vectors that can characterize the voice signal may be extracted frame by frame. Fourier transform may be performed on each audio frame, and then frequency domain features may be extracted as feature vectors of the audio frame. Preprocessing may also include effective sound detection, speaker separation, voice enhancement, etc.; no specific limitation is made here.

[0054] In the specific implementation process, the voice data can be feature extracted by principal component analysis, short-time Fourier transform, time-frequency analysis, mel frequency cepstral coefficients (MFCCs), deep learning feature extraction methods, etc. A machine learning model is a mathematical structure or algorithm that can learn and infer useful information from input data, and then predict or make decisions on new data. In the field of machine learning, models are established by training on existing data. This training process involves optimizing model parameters so that it can fit the training data to the greatest extent, and it is expected to show good generalization ability on unknown data. In this embodiment, the preset machine learning model can be a machine learning model suitable for classification and recognition tasks, such as a logistic regression model, a decision tree, a random forest, a support vector machine, or a deep learning model.

[0055] Take the logistic regression model as an example to explain. The logistic regression model is a generalized linear model used to deal with binary classification problems (of course, it can also be extended to multi-classification problems). Its basic structure is based on linear regression. Assume that the input speech feature vector is x = [x1, x2, ..., x n ](here x i is the speech feature obtained by the feature extraction method mentioned above, n is the feature dimension), the model will first calculate the linear combination

[0056] z=θ0+θ1x1+θ2x2+…+θ n x n , where θ[θ0, θ1,…,θ n ] are the parameters that the model needs to learn (θ0 is the intercept term).

[0057] Then the result of the linear combination is converted into a probability value through a logical function (such as the sigmoid function). The form of the Sigmoid function is The output y represents the probability that the sample belongs to a certain category (for example, the probability of belonging to the positive category). During the training process, the parameter θ is optimized by methods such as maximum likelihood estimation to minimize the difference between the predicted probability and the true label, thereby constructing a logistic regression model structure suitable for speech classification and recognition tasks.

[0058] S120: Process the voice data based on the time domain features and obtain a time domain feature data set of the first voice feature vector; input the time domain feature data set of the first voice feature vector into the companion robot, and obtain the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector; filter the first voice feature vector whose wake-up rate is higher than a preset first wake-up rate threshold.

[0059] It should be noted that the time domain features are directly extracted from the waveform of the speech signal without the need for spectral analysis. They reflect the instantaneous characteristics or changes of the signal. The present invention adjusts the time domain features of the speech data from the two aspects of short-time energy (Short-Time Energy) and zero-crossing rate (Zero-Crossing Rate, ZCR). In other words, the time domain feature data set formed after the time domain feature adjustment of the speech data includes the first speech feature vector after the short-time energy feature parameter is adjusted and the first speech feature vector after the zero-crossing rate feature parameter is adjusted.

[0060] Specifically, the time domain feature data set of the first speech feature vector includes the first speech feature vector after the short-time energy feature parameter is adjusted and the first speech feature vector after the zero-crossing rate feature parameter is adjusted.

[0061] In a specific implementation process, the first speech feature vector after the short-time energy feature parameter is adjusted includes the first speech feature vector after the frame size parameter is adjusted, the first speech feature vector after the frame shift parameter is adjusted, the first speech feature vector after the smoothing window size parameter is adjusted, and the first speech feature vector after the pre-emphasis coefficient parameter is adjusted. The first speech feature vector after the zero-crossing rate feature parameter is adjusted includes the first speech feature vector after the frame size feature parameter is adjusted, the first speech feature vector after the frame shift feature parameter is adjusted, the first speech feature vector after the window function feature parameter is adjusted, and the first speech feature vector after the high-pass filter cutoff frequency feature parameter is adjusted. Among them, the first speech feature vector after the short-time energy feature parameter is adjusted and the first speech feature vector after the frame size feature parameter is adjusted and the first speech feature vector after the frame shift feature parameter is adjusted in the first speech feature vector after the zero-crossing rate feature parameter is adjusted are the same.

[0062] In a specific embodiment, for the short-time energy characteristic parameter adjustment process, the frame size parameters of the voice data are adjusted according to 10ms, 15ms, and 20ms respectively to obtain the first voice feature vector after the frame size parameters are adjusted; the frame shift parameters of the voice data are adjusted according to 60%, 50%, and 40% respectively to obtain the first voice feature vector after the frame shift parameters are adjusted; the smoothing window size parameters of the voice data are adjusted according to 20ms, 25ms, and 30ms respectively to obtain the first voice feature vector after the smoothing window size parameters are adjusted; the pre-emphasis coefficient parameters of the voice data are adjusted according to 0.95, 0.96, and 0.97 respectively to obtain the first voice feature vector after the pre-emphasis coefficient parameters are adjusted.

[0063] For the zero crossing rate feature parameter adjustment process, the frame size parameters of the speech data are adjusted according to 10ms, 15ms, and 20ms to obtain the first speech feature vector after the frame size parameters are adjusted; the frame shift parameters of the speech data are adjusted according to 60%, 50%, and 40% to obtain the first speech feature vector after the frame shift parameters are adjusted; the window function feature parameters of the speech data are adjusted according to the Hamming window, Hanning window, and rectangular window to obtain the first speech feature vector after the window function feature parameters are adjusted; the high-pass filter cutoff frequency feature parameters of the speech data are adjusted according to 80Hz, 90Hz, and 100Hz to obtain the first speech feature vector after the high-pass filter cutoff frequency feature parameters are adjusted. After the time domain parameters of a speech data are adjusted, the first speech feature vector after the short-time energy feature parameters are adjusted and the first speech feature vector after the zero crossing rate feature parameters are adjusted are obtained. When forming the time domain feature data set, the frame size parameters and frame shift parameters in the short-time energy feature parameters and the zero crossing rate feature parameters are kept consistent, and an orthogonal experiment is designed to obtain 729 feature vectors.

[0064] The time domain feature data set of the first voice feature vector is input into the companion robot, and the wake-up rate corresponding to each first voice feature vector in the time domain feature data set is obtained; and the first voice feature vector having a wake-up rate higher than a preset first wake-up rate threshold is selected. In the specific implementation process, the first wake-up rate threshold is 95%.

[0065] In the specific implementation process, the accuracy can be obtained through the classification algorithm of the support vector machine. Specifically, support vector machine (SVM): SVM is a supervised learning classification algorithm. Its basic idea is to find an optimal hyperplane in the feature space to separate data points of different categories. For the speech domain feature data set, the SVM model is trained by using the speech feature vector with known accuracy as the training sample. For example, in a two-class classification problem (such as accurate and inaccurate), the SVM can find a hyperplane based on the frequency domain features of the sample so that the interval between the two types of samples is the largest. In the prediction stage, the speech feature vector to be tested is input into the trained SVM model, and the model will output the category corresponding to the feature vector (and then the accuracy-related information can be obtained). The implementation steps of the classification algorithm of the support vector machine include collecting labeled data: collecting speech domain feature vectors with known accuracy as training data, including the feature vector itself and the corresponding accuracy label (for example, the accuracy is 80% marked as 1, and the accuracy is less than 80% marked as 0). At the same time, the training set, validation set and test set are divided, usually according to a certain ratio (such as 70% training set, 10% validation set, 20% test set). Data normalization: Normalize the speech domain feature vector so that the features of different dimensions have the same scale. Common normalization methods include minimum-maximum normalization, which maps the feature value to the [0,1] interval. The formula is Among them, x is the original eigenvalue, x min and x max is the minimum and maximum value of the feature dimension. Select the kernel function: SVM has a variety of kernel functions to choose from, such as linear kernel, polynomial kernel, Gaussian kernel (RBF kernel), etc. For speech domain feature data, select a suitable kernel function according to the distribution characteristics of the data. If the data may be linearly separable in the feature space, the linear kernel may be a good choice; if the data distribution is more complex, the Gaussian kernel is usually more suitable. Training model: Using the training set data, the parameters of the SVM model, including the parameters of the support vector and hyperplane, are determined through an optimization algorithm (such as the sequential minimum optimization algorithm, SMO). During the training process, the model is adjusted by minimizing the loss function (such as the hinge loss function) so that the model can accurately classify the training data. At the same time, the validation set is used to adjust the model's hyperparameters (such as kernel function parameters, penalty coefficients, etc.) to prevent overfitting. Model evaluation: Use the test set to evaluate the trained SVM model. Common evaluation indicators include accuracy, recall, F1-score, etc. The accuracy calculation formula is Among them, TP is true positive (correctly classified as positive example), TN is true negative (correctly classified as negative example), FP is false positive (incorrectly classified as positive example), and FN is false negative (incorrectly classified as negative example).

[0066] Application model: Input the speech domain feature dataset to be tested into the trained SVM model, and the model will output the category corresponding to each speech feature vector (accuracy related information can be inferred based on the category). For example, if the output category is 1, it means that the accuracy corresponding to the speech feature vector may be high; if the output category is 0, it means that the accuracy may be low.

[0067] S130: Process the screened first voice feature vector based on the frequency domain features and obtain a frequency domain feature data set of the second voice feature vector; input the frequency domain feature data set of the second voice feature vector into the companion robot to obtain the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; and screen the second voice feature vector whose accuracy rate is higher than a preset second accuracy rate threshold.

[0068] In the specific implementation process, suitable time domain features can ensure the wake-up rate of large models, while frequency domain features can ensure the accuracy of large model recognition. Frequency domain features describe the spectrum information of the signal, which is usually extracted by short-time Fourier transform (STFT) or other spectrum analysis methods. The present invention will make adjustments from two directions: Mel Frequency Cepstral Coefficient (MFCC) and Spectral Centroid.

[0069] Specifically, the frequency domain feature data set of the second speech feature vector includes the second speech feature vector after the Mel-frequency cepstral coefficient feature parameters are adjusted and the second speech feature vector after the spectrum centroid feature parameters are adjusted.

[0070] The second speech feature vector after the Mel frequency cepstral coefficient feature parameter is adjusted includes the second speech feature vector after the Mel frequency range feature parameter is adjusted, the second speech feature vector after the DCT coefficient number feature parameter is adjusted, the second speech feature vector after the pre-emphasis feature parameter is adjusted, and the second speech feature vector after the logarithmic operation feature parameter is adjusted. The second speech feature vector after the spectrum centroid feature parameter is adjusted includes the second speech feature vector after the spectrum resolution feature parameter is adjusted, the second speech feature vector after the spectrum smoothing feature parameter is adjusted, the second speech feature vector after the window function feature parameter is adjusted, and the second speech feature vector after the custom weighting feature parameter is adjusted.

[0071] In a specific embodiment, for the Mel-frequency cepstral coefficient feature parameter adjustment process, the first speech feature vector is adjusted according to 20Hz-8000Hz, 0Hz-4000Hz, and 50Hz-4000Hz respectively to obtain a second speech feature vector after the Mel-frequency range feature parameter is adjusted; the first speech feature vector is adjusted according to 12 coefficients, 13 coefficients, and 20 coefficients respectively to obtain a second speech feature vector after the DCT coefficient number feature parameter is adjusted; the first speech feature vector is adjusted according to 0.97, 0.9, and 1.0 respectively to obtain a second speech feature vector after the pre-emphasis feature parameter is adjusted; the first speech feature vector is adjusted according to logarithmic operation, logarithmic smoothing (log+eps), and no logarithmic operation to obtain a second speech feature vector after the logarithmic operation feature parameter is adjusted.

[0072] For the process of adjusting the spectrum centroid characteristic parameters, the spectrum resolution characteristic parameters of the first speech feature vector are adjusted according to 512-point FFT, 1024-point FFT, and 2048-point FFT to obtain the second speech feature vector after the spectrum resolution characteristic parameters are adjusted; the frame shift parameters of the first speech feature vector are adjusted according to 60%, 50%, and 40% to obtain the first speech feature vector after the frame shift parameters are adjusted; the spectrum smoothing characteristic parameters of the first speech feature vector are adjusted according to moving average, Gaussian smoothing, and low-pass filtering to obtain the second speech feature vector after the spectrum smoothing characteristic parameters are adjusted; the first speech feature vector is adjusted according to the custom weighted characteristic parameters of increasing the weight from 100Hz to 500Hz, increasing the weight from 500Hz to 1500Hz, and increasing the weight above 2kHz to obtain the second speech feature vector after the custom weighted characteristic parameters are adjusted. The window function characteristic parameters of the first speech feature vector are adjusted according to the Hamming Window, Hanning Window, and Rectangular Window to obtain the second speech feature vector after the window function characteristic parameters are adjusted.

[0073] A second speech feature vector after adjusting the Mel-frequency cepstral coefficient feature parameters obtained after frequency domain parameter adjustment and a second speech feature vector after adjusting the spectrum centroid feature parameters are obtained. When forming a frequency domain feature data set, the frame size parameter and frame shift parameter in the short-time energy feature parameter and the zero crossing rate feature parameter are kept consistent, an orthogonal experiment is designed, and 6561 feature vectors are obtained.

[0074] The frequency domain feature data set of the second voice feature vector is input into the companion robot, and the accuracy rate corresponding to each second voice feature vector in the frequency domain feature data set is obtained; and the second voice feature vector whose accuracy rate is higher than the preset second accuracy rate threshold is selected. In the specific implementation process, the second accuracy rate threshold is 95%. In the specific implementation process, the accuracy rate can be obtained by the classification algorithm of the support vector machine.

[0075] S140: Determine the voice preference feature of the companion robot according to the second voice feature vector.

[0076] In a specific embodiment, the following processing is also performed on the acquired voice preference features of the companion robot in order to improve the voice recognition accuracy of the companion robot.

[0077] S150: Performing feature alignment processing on the voice data to be recognized according to the voice preference features of the companion robot.

[0078] In the specific implementation process, it can be implemented by using the dynamic time warping (DTW) algorithm. DTW is an algorithm used to measure the similarity of two time series. Its basic principle is to construct a distance matrix, and the elements in the matrix represent the distance between speech features at two time points. Then a path with the minimum cumulative distance is found through a dynamic programming algorithm. This path represents the optimal alignment of the two speech sequences. The specific implementation process is illustrated as follows: Distance matrix calculation: Calculate the distance matrix between two speech feature sequences (the speech feature sequence to be recognized and the robot voice preference feature sequence). Assume that the speech feature sequence to be recognized is X=[x1, x2,…, m ], the robot voice preference feature sequence is Y = [y1, y2, ..., y m ], then the element d in the distance matrix D ij It can be calculated using methods such as Euclidean distance, such as Where l is the dimension of the feature vector. Next, dynamic programming is used to find the optimal path. Initialize a cumulative distance matrix C of size (m+1)×(n+1). Set the boundary conditions C[0,0]=0, C[i,0]=∞(i>0), C[0,j]=∞(j>0). Then, through the dynamic programming formula C[i,j=d ij+min(C[i-1, j-1], C[i-1, j], C[i, j-1]) to fill the cumulative distance matrix. Finally, backtrack from the lower right corner of the matrix to find the path with the smallest cumulative distance. This path is the alignment path of the two speech feature sequences. Align the speech features according to the alignment path found. For example, if the alignment path indicates that the i-th feature in the speech feature sequence to be identified should be aligned with the j-th feature in the robot speech preference feature sequence, then the two features are matched or further processed, such as by weighted averaging to obtain the aligned speech features.

[0079] S160: Performing speech recognition on the speech data after the feature alignment processing.

[0080] like Figure 2 As shown, the present invention provides a companion robot speech recognition system, which performs speech recognition using the companion robot speech recognition method as described above. According to the functions implemented, the companion robot speech recognition system 200 may include an acquisition unit 210, a first screening unit 220, a second screening unit 230, and a feature determination unit 240. The unit of the present invention may also be referred to as a module, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0081] In this embodiment, the functions of each module / unit are as follows:

[0082] The acquiring unit 210 is configured to acquire voice data.

[0083] The first screening unit 220 is used to process the voice data based on the time domain features and obtain a time domain feature data set of the first voice feature vector; input the time domain feature data set of the first voice feature vector into the companion robot, and obtain the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector; and screen the first voice feature vector whose wake-up rate is higher than a preset first wake-up rate threshold.

[0084] The second screening unit 230 is used to process the screened first voice feature vector based on the frequency domain features and obtain a frequency domain feature data set of the second voice feature vector; input the frequency domain feature data set of the second voice feature vector into the companion robot to obtain the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; and screen the second voice feature vector whose accuracy rate is higher than a preset second accuracy rate threshold.

[0085] The feature determination unit 240 is used to determine the voice preference feature of the companion robot according to the second voice feature vector.

[0086] The companion robot speech recognition system of the present invention processes speech data based on short-time energy and zero-crossing rate and obtains a time-domain feature data set; obtains the wake-up rate corresponding to each first speech feature vector in the time-domain feature data set; screens the first speech feature vector with a wake-up rate higher than a preset first wake-up rate threshold; processes the screened first speech feature vector based on Mel-frequency cepstral coefficients and spectrum centroid and obtains a frequency-domain feature data set; obtains the accuracy rate corresponding to each speech feature vector in the frequency-domain feature data set; screens the second speech feature vector with an accuracy rate higher than a preset second accuracy rate threshold; and determines the speech preference feature of the companion robot according to the second speech feature vector. Through the screening mechanism based on the wake-up rate and accuracy rate, the most effective speech feature for the companion robot can be identified, thereby improving the response speed and accuracy of speech recognition according to the most effective speech feature, and finally achieving the technical effect of effectively improving the accuracy rate of speech recognition of the companion robot and enhancing the user interaction experience.

[0087] For more specific implementation methods of the above companion robot voice recognition system, please refer to the aforementioned description of the embodiment of the companion robot voice recognition method, which will not be described in detail here.

[0088] like Figure 3 As shown, the present invention also provides an electronic device 1 for a wearing detection method.

[0089] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a companion robot voice recognition program 12. The memory 11 may also include both an internal storage unit of the companion robot voice recognition system and an external storage device. The memory 11 may be used not only to store application software and various data installed therein, such as the code of the companion robot voice recognition program, but also to temporarily store data that has been output or is to be output.

[0090] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a companion robot voice recognition program 12. The memory 11 may also include both an internal storage unit of the companion robot voice recognition system and an external storage device. The memory 11 may be used not only to store application software and various data installed therein, such as the code of the companion robot voice recognition program, but also to temporarily store data that has been output or is to be output.

[0091] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the companion robot voice recognition program, etc., but also be used to temporarily store data that has been output or is to be output.

[0092] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules (such as companion robot voice recognition programs, etc.) stored in the memory 11, and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0093] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0094] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0095] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management system, so that the power management system can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0096] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0097] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0098] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0099] The companion robot voice recognition program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve: obtaining voice data; processing the voice data based on time domain features and obtaining a time domain feature data set of a first voice feature vector; inputting the time domain feature data set of the first voice feature vector into the companion robot, and obtaining the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector; screening the first voice feature vector whose wake-up rate is higher than a preset first wake-up rate threshold; processing the screened first voice feature vector based on frequency domain features and obtaining a frequency domain feature data set of a second voice feature vector; inputting the frequency domain feature data set of the second voice feature vector into the companion robot, and obtaining the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; screening the second voice feature vector whose accuracy rate is higher than a preset second accuracy rate threshold; and determining the voice preference characteristics of the companion robot according to the second voice feature vector.

[0100] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiment is not repeated here. Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0101] An embodiment of the present invention also provides a computer-readable storage medium, which may be non-volatile or volatile, and stores a computer program, which, when executed by a processor, implements the following: acquiring voice data; acquiring voice data; processing the voice data based on time domain features and obtaining a time domain feature data set of a first voice feature vector; inputting the time domain feature data set of the first voice feature vector into a companion robot, and obtaining the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector; screening first voice feature vectors whose wake-up rate is higher than a preset first wake-up rate threshold; processing the screened first voice feature vector based on frequency domain features and obtaining a frequency domain feature data set of a second voice feature vector; inputting the frequency domain feature data set of the second voice feature vector into the companion robot, and obtaining the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; screening second voice feature vectors whose accuracy rate is higher than a preset second accuracy rate threshold; and determining the voice preference characteristics of the companion robot according to the second voice feature vector.

[0102] Specifically, the specific implementation method when the computer program is executed by the processor can refer to the description of the relevant steps in the wearing detection method in the embodiment, which will not be repeated here.

[0103] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0104] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0106] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0107] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0108] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or systems stated in the system claim can also be implemented by one unit or system through software or hardware.

[0109] As described above, the companion robot speech recognition method and the companion robot speech recognition system according to the present invention are described by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various improvements can be made to the companion robot speech recognition method and the companion robot speech recognition system proposed by the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. A companion robot voice recognition method, applied to electronic equipment, characterized in that: include: Get voice data; The voice data is processed based on the time domain feature to obtain a time domain feature data set of a first voice feature vector; the time domain feature data set of the first voice feature vector is input into the companion robot, and the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector is obtained; and the first voice feature vector whose wake-up rate is higher than a preset first wake-up rate threshold is screened; The first voice feature vector selected is processed based on the frequency domain feature to obtain a frequency domain feature data set of a second voice feature vector; the frequency domain feature data set of the second voice feature vector is input into the companion robot to obtain the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; and the second voice feature vector having an accuracy rate higher than a preset second accuracy rate threshold is selected; The voice preference feature of the companion robot is determined according to the second voice feature vector.

2. The companion robot speech recognition method according to claim 1, characterized in that: The time domain feature data set of the first speech feature vector includes: the first speech feature vector after the short-time energy feature parameter is adjusted and the first speech feature vector after the zero-crossing rate feature parameter is adjusted.

3. The companion robot speech recognition method according to claim 1, characterized in that: The frequency domain feature data set of the second speech feature vector includes the second speech feature vector after the Mel-frequency cepstral coefficient feature parameters are adjusted and the second speech feature vector after the spectrum centroid feature parameters are adjusted.

4. The companion robot speech recognition method according to claim 2, characterized in that: The first speech feature vector after the short-time energy feature parameter is adjusted includes the first speech feature vector after the frame size parameter is adjusted, the first speech feature vector after the frame shift parameter is adjusted, the first speech feature vector after the smoothing window size parameter is adjusted, and the first speech feature vector after the pre-emphasis coefficient parameter is adjusted; The first speech feature vector after the zero crossing rate characteristic parameter is adjusted includes the first speech feature vector after the frame size characteristic parameter is adjusted, the first speech feature vector after the frame shift characteristic parameter is adjusted, the first speech feature vector after the window function characteristic parameter is adjusted, and the first speech feature vector after the high pass filter cutoff frequency characteristic parameter is adjusted; Among them, the first speech feature vector after the short-time energy feature parameter is adjusted and the first speech feature vector after the zero-crossing rate feature parameter is adjusted are the same as the first speech feature vector after the frame size feature parameter is adjusted and the first speech feature vector after the frame shift feature parameter is adjusted.

5. The companion robot speech recognition method according to claim 3, characterized in that: The second speech feature vector after the Mel frequency cepstral coefficient feature parameters are adjusted includes the second speech feature vector after the Mel frequency range feature parameters are adjusted, the second speech feature vector after the DCT coefficient number feature parameters are adjusted, the second speech feature vector after the pre-emphasis feature parameters are adjusted, and the second speech feature vector after the logarithmic operation feature parameters are adjusted.

6. The companion robot speech recognition method according to claim 3, characterized in that: 。 The second speech feature vector after the spectrum centroid feature parameters are adjusted includes a second speech feature vector after the spectrum resolution feature parameters are adjusted, a second speech feature vector after the spectrum smoothing feature parameters are adjusted, a second speech feature vector after the window function feature parameters are adjusted, and a second speech feature vector after the custom weighted feature parameters are adjusted.

7. The companion robot speech recognition method according to claim 1, characterized in that: The first awakening threshold and the second awakening threshold are both 95%.

8. The companion robot speech recognition method according to claim 1, characterized in that: The method further includes performing feature alignment processing on the voice data to be recognized according to the voice preference features of the companion robot; Speech recognition is performed on the speech data after the feature alignment processing.

9. The companion robot speech recognition method according to claim 1, characterized in that: Before the step of processing the speech data based on the short-time energy and the zero crossing rate and obtaining a time domain feature data set, the method further includes: The speech data is subjected to noise reduction preprocessing using a Wiener filter model.

10. A companion robot speech recognition system, which performs speech recognition using the companion robot speech recognition method according to any one of claims 1 to 9; comprising: An acquisition unit, used for acquiring voice data; A first screening unit is used to process the voice data based on the time domain feature and obtain a time domain feature data set of a first voice feature vector; input the time domain feature data set of the first voice feature vector into the companion robot, and obtain the wake-up rate corresponding to each first voice feature vector in the time domain feature data set of the first voice feature vector; and screen the first voice feature vector whose wake-up rate is higher than a preset first wake-up rate threshold; A second screening unit is used to process the screened first voice feature vector based on the frequency domain feature and obtain a frequency domain feature data set of a second voice feature vector; input the frequency domain feature data set of the second voice feature vector into the companion robot to obtain the accuracy rate corresponding to each voice feature vector in the frequency domain feature data set of the second voice feature vector; and screen the second voice feature vector whose accuracy rate is higher than a preset second accuracy rate threshold; A feature determination unit is used to determine the voice preference feature of the companion robot according to the second voice feature vector.

11. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a companion robot voice recognition program stored in the memory and executable on the processor. When the companion robot voice recognition program is executed by the processor, the steps of the wearing detection method as described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the wearing detection method according to any one of claims 1 to 9 is implemented.

Images