Keyword identification method and device, storage medium and electronic equipment

Abstract

The invention discloses a keyword recognition method and device, a storage medium and electronic equipment, and the method comprises the steps: extracting a bone conduction signal and an air conduction signal corresponding to a voice signal when the voice signal is received; respectively carrying out feature extraction on the bone conduction signal and the air conduction signal to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; performing feature fusion on the first MFCC feature and the second MFCC feature to generate a target feature; inputting the target features into a keyword recognition model for recognition to obtain a preliminary recognition result of each frame; and performing multi-frame logic post-processing on all the preliminary identification results to obtain a final identification result. According to the scheme, the keyword recognition accuracy can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio processing technology, and specifically to a keyword recognition method, device, storage medium, and electronic device. Background Art

[0002] Keyword Spotting (KWS) is a technology that identifies predefined keywords in real time by analyzing audio streams. It is widely used in scenarios such as smart voice assistants, wearable devices, smart homes, and mobile terminals. This technology needs to accurately detect keywords with limited device resources while maintaining low latency and low power consumption.

[0003] However, traditional keyword detection methods usually rely on speech signals acquired by air conduction microphones. However, air conduction signals are easily interfered by environmental noise (such as background conversations, traffic noise, etc.), resulting in a significant decrease in the accuracy of keyword recognition. Summary of the invention

[0004] The embodiments of the present application provide a keyword recognition method, device, storage medium and electronic device, which can improve the accuracy of keyword recognition.

[0005] In a first aspect, an embodiment of the present application provides a keyword recognition method, comprising: When a speech signal is received, a bone conduction signal and an air conduction signal corresponding to the speech signal are extracted; Performing feature extraction on the bone conduction signal and the air conduction signal respectively to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; Fusing the first MFCC feature and the second MFCC feature to generate a target feature; Input the target features into a keyword recognition model for recognition, and obtain a preliminary recognition result for each frame; All the preliminary recognition results are subjected to multi-frame logic post-processing to obtain the final recognition result.

[0006] In the keyword recognition method provided in the embodiment of the present application, the multi-frame logic post-processing of all the preliminary recognition results to obtain the final recognition result includes: Performing a sliding window analysis on all the preliminary recognition results to determine the number of frames that meet a probability threshold in a plurality of consecutive frames; When the number of frames that meet the probability threshold reaches a preset value, the corresponding keyword category is determined as the final recognition result.

[0007] In the keyword recognition method provided in the embodiment of the present application, the first MFCC feature and the second MFCC feature are fused to generate a target feature, including: The first MFCC feature and the second MFCC feature are concatenated or weighted to generate a target feature.

[0008] In the keyword identification method provided in the embodiment of the present application, the first preset dimension is the second dimension to the eleventh dimension, and the second preset dimension is the first dimension to the twentieth dimension.

[0009] In the keyword recognition method provided in the embodiment of the present application, when a voice signal is received, after extracting the bone conduction signal and the air conduction signal corresponding to the voice signal, and before performing feature extraction on the bone conduction signal and the air conduction signal respectively to obtain the first MFCC feature of the first preset dimension and the second MFCC feature of the second preset dimension, it also includes: Performing energy calculation on the bone conduction signal, and judging whether the speech signal is a silent segment according to the calculation result; If not, the step of respectively extracting features from the bone conduction signal and the air conduction signal to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension is performed.

[0010] In the keyword recognition method provided in the embodiment of the present application, before extracting the bone conduction signal and the air conduction signal corresponding to the voice signal when receiving the voice signal, it also includes: Build a keyword recognition model.

[0011] In the keyword recognition method provided in the embodiment of the present application, the step of constructing a keyword recognition model includes: Acquire a speech data set, where the speech data set includes a plurality of bone conduction signals and their corresponding air conduction signals; Performing feature extraction on the corresponding bone conduction signal and the air conduction signal respectively to obtain a third MFCC feature of a first preset dimension and a fourth MFCC feature of a second preset dimension; Perform feature fusion on the third MFCC feature and the fourth MFCC feature to generate training features; The CRN neural network is trained based on the training features to obtain a keyword recognition model.

[0012] In a second aspect, an embodiment of the present application provides a keyword recognition device, including: A signal extraction unit, configured to extract a bone conduction signal and an air conduction signal corresponding to the speech signal when a speech signal is received; A feature extraction unit, configured to perform feature extraction on the bone conduction signal and the air conduction signal respectively, to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; A feature fusion unit, used for fusing the first MFCC feature and the second MFCC feature to generate a target feature; A feature recognition unit, used to input the target feature into a keyword recognition model for recognition, and obtain a preliminary recognition result for each frame; The result judgment unit is used to perform multi-frame logic post-processing on all the preliminary recognition results to obtain the final recognition result.

[0013] In a third aspect, the present application provides a storage medium, wherein the storage medium stores a plurality of instructions, wherein the instructions are suitable for loading by a processor to execute any of the keyword recognition methods described above.

[0014] In a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described keyword recognition methods when executing the computer program.

[0015] In summary, the keyword recognition method provided in the embodiment of the present application adopts the method of extracting the bone conduction signal and the air conduction signal corresponding to the voice signal when receiving the voice signal; performing feature extraction on the bone conduction signal and the air conduction signal respectively to obtain the first MFCC feature of the first preset dimension and the second MFCC feature of the second preset dimension; performing feature fusion on the first MFCC feature and the second MFCC feature to generate a target feature; inputting the target feature into the keyword recognition model for recognition to obtain a preliminary recognition result of each frame; performing multi-frame logic post-processing on all the preliminary recognition results to obtain a final recognition result. This scheme effectively improves the accuracy of keyword recognition by extracting the bone conduction signal and the air conduction signal corresponding to the voice signal, then performing feature fusion on the bone conduction signal and the air conduction signal, and then identifying the target feature through the keyword recognition model, and further improves the accuracy of keyword recognition by combining the multi-frame logic post-processing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1It is a schematic diagram of an application scenario of the keyword recognition method provided in an embodiment of the present application.

[0018] Figure 2 It is a flowchart of the keyword recognition method provided in the embodiment of the present application.

[0019] Figure 3 It is a structural diagram of a keyword recognition device provided in an embodiment of the present application.

[0020] Figure 4 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0022] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0023] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0025] In the description of the present application, it should be noted that the terms "upper", "lower", "left", "right", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, terms such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0026] Traditional keyword detection methods usually rely on speech signals acquired by air conduction microphones. However, air conduction signals are easily interfered by environmental noise (such as background conversations, traffic noise, etc.), resulting in a significant decrease in the accuracy of keyword recognition.

[0027] Based on this, the embodiments of the present application provide a keyword recognition method, device, storage medium and electronic device. Specifically, the keyword recognition device can be integrated in an electronic device, in which a bone conduction sensor and an air conduction microphone are integrated. The electronic device can be a server or a terminal or other device; wherein the terminal can include a mobile phone, a wearable smart device, a tablet computer, a laptop computer, and a personal computer (PC), etc.; the server can be a single server or a server cluster composed of multiple servers, and can be a physical server or a virtual server.

[0028] For example, Figure 1 As shown, after receiving the voice signal, the electronic device can extract the bone conduction signal and the air conduction signal corresponding to the voice signal, and then perform feature extraction on the bone conduction signal and the air conduction signal respectively to obtain the first Mel Frequency Cepstrum Coefficient (MFCC) feature of the first preset dimension and the second MFCC feature of the second preset dimension, and then perform feature fusion on the first MFCC feature and the second MFCC feature to generate the target feature, and then input the target feature into the keyword recognition model for recognition to obtain the preliminary recognition result of each frame, and finally perform multi-frame logic post-processing on all the preliminary recognition results to obtain the final recognition result. That is, the electronic device can effectively improve the accuracy of keyword recognition by extracting the bone conduction signal and the air conduction signal corresponding to the voice signal, and then perform feature fusion on the bone conduction signal and the air conduction signal, and then recognize the target feature through the keyword recognition model, and further improve the accuracy of keyword recognition by combining the multi-frame logic post-processing technology.

[0029] The technical solutions shown in the present application will be described in detail below through specific embodiments. It should be noted that the description order of the following embodiments is not intended to limit the priority order of the embodiments.

[0030] See also Figure 2 , Figure 2 : is a flowchart of a keyword recognition method provided in an embodiment of the present application. The specific process of the keyword recognition method can be as follows: 101. When a speech signal is received, a bone conduction signal and an air conduction signal corresponding to the speech signal are extracted.

[0031] It can be understood that the voice signal can be acquired simultaneously by a microphone and a bone vibration sensor.

[0032] Among them, the air conduction signal is a conventional environmental speech signal, which includes the speech content and the environmental noise around it (such as background conversation, traffic noise, etc.). The air conduction signal is more susceptible to interference from environmental noise. The air conduction signal can be obtained by using an ordinary microphone (such as a condenser microphone, a dynamic microphone, etc.) to capture the sound fluctuations in the air.

[0033] Bone conduction signals are speech signals transmitted through the vibration of human bones and are not affected by environmental noise. Bone conduction signals only capture the speaker's voice and are less affected by external background noise. The bone conduction signal can capture the vibration signal of the human skull (such as ear bones or cheekbones) through a bone vibration sensor (such as a piezoelectric sensor, a piezoelectric ceramic sensor, etc.). Bone conduction sensors are usually worn near the ears or on the head.

[0034] In practical applications, in order to ensure that the bone conduction signal and the air conduction signal can be correctly paired and fused, it is usually necessary to synchronize the bone conduction signal and the air conduction signal. Specifically, during the signal acquisition process, the sampling frequency of the bone conduction signal and the air conduction signal should be consistent. The common sampling rate is 16kHz or higher to ensure that enough detailed information is collected. In addition, the collected bone conduction signal and air conduction signal need to be time synchronized to ensure that their time axes are aligned.

[0035] In some embodiments, the bone conduction signal and the air conduction signal may also be preprocessed. Specifically, the air conduction signal may be subjected to noise suppression or noise reduction processing to reduce the impact of external noise on the signal. The bone conduction signal and the air conduction signal are high-pass filtered to remove low-frequency noise and non-speech interference (such as power supply noise). Afterwards, the signal gain of the bone conduction signal and the air conduction signal is adjusted to ensure that the amplitude of the bone conduction signal and the air conduction signal is suitable for subsequent processing.

[0036] Commonly used noise reduction methods include spectral subtraction and Wiener filtering, etc. These methods reduce the impact of background noise on speech signals by estimating the noise spectrum and subtracting the noise component from the original signal.

[0037] In addition, high-pass filtering is a method commonly used to remove low-frequency noise (such as power noise or mechanical vibration). Especially in the processing of bone conduction signals, low-frequency noise may affect the clarity of the signal, so high-pass filtering can enhance the quality of the signal. Among them, since bone conduction signals mainly reflect the vibration of the human skull, low-frequency noise (such as equipment power supply noise, mechanical vibration, etc.) may affect the effectiveness of the signal. Usually, a high-pass filter of 80Hz to 150Hz is used to remove low-frequency noise. High-pass filtering of air conduction signals can remove low-frequency noise, such as wind noise, electromagnetic interference, etc. The commonly used high-pass filter frequency is between 50Hz and 150Hz.

[0038] Signal gain adjustment is to adjust the signal amplitude so that it is within the appropriate range for subsequent processing and feature extraction. Gain adjustment can prevent the signal from being too weak or too strong, resulting in information loss or distortion. In some embodiments, automatic gain control (AGC) can be used to adjust the signal gain of air conduction signals and bone conduction signals. Automatic gain control is a common technology. Especially in the process of air conduction signal processing, AGC can dynamically adjust the gain according to the strength of the input signal, maintain the appropriate amplitude of the signal, and avoid excessive or low signal interference. Due to the different sensors, bone conduction signals may cause signal strength fluctuations due to unstable wearing or poor contact. Therefore, appropriate gain adjustment helps to improve signal quality.

[0039] In speech signal processing, a silent segment is a signal segment without valid speech content. Processing these segments may waste computing resources and interfere with model performance. Therefore, in order to quickly and accurately filter silent segments, thereby reducing invalid calculations. In some embodiments, after obtaining the bone conduction signal, the energy of the bone conduction signal can also be calculated, and it is determined whether the speech signal is a silent segment based on the calculation result; if not, step 102 is executed, and if so, the subsequent process is interrupted.

[0040] Specifically, the bone conduction signal can be divided into short-time frames (for example, 25ms per frame, frame shift 10ms), and the short-time energy of each frame is calculated. If the energy of several consecutive frames is lower than the threshold, the signal is considered to be a silent segment. For example, setting 5 consecutive silent frames as the judgment condition reduces the possibility of misjudgment of a single frame.

[0041] 102. Perform feature extraction on the bone conduction signal and the air conduction signal respectively to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension.

[0042] Specifically, the bone conduction signal can be divided into multiple frame signals, each frame is usually 20ms to 25ms long, and the frame shift is 10ms to 15ms. A window function (such as a Hamming window) can be applied to each frame signal to reduce the boundary effect. Then, a Fourier transform is performed on each frame signal to obtain the spectrum of the frame signal. After that, the spectrum is mapped to the Mel frequency axis, and the logarithmic energy of each Mel frequency band is calculated. Mel frequency is a frequency scale close to the auditory perception of the human ear, and this process can be achieved through a Mel filter bank. Finally, the MFCC features of the frame signal are extracted from the logarithmic energy through discrete cosine transform (DCT). Then, the MFCC features of the bone conduction signal can be obtained.

[0043] In an embodiment of the present application, MFCC features of the second dimension to the eleventh dimension are extracted from the MFCC features of the bone conduction signal as the first MFCC features. Experiments have shown that the final model effect is better when the MFCC features of the first dimension are removed from the first MFCC features of the bone conduction signal. Similarly, the air conduction signal can adopt the same feature extraction process as the bone conduction signal to obtain the MFCC features of the air conduction signal. Afterwards, the MFCC features of the first dimension to the twentieth dimension can be extracted from the MFCC features of the air conduction signal as the second MFCC features. That is, the first preset dimension is the second dimension to the eleventh dimension, and the second preset dimension is the first dimension to the twentieth dimension.

[0044] 103. Fusing the first MFCC feature and the second MFCC feature to generate a target feature.

[0045] Specifically, the first MFCC feature and the second MFCC feature may be concatenated or weighted to generate a target feature.

[0046] The splicing process is to directly connect the first MFCC feature and the second MFCC feature into a larger target feature. Specifically, the time axis of the first MFCC feature and the second MFCC feature can be aligned first (that is, the number of frames is consistent), and then the first MFCC feature (10 dimensions) and the second MFCC feature (20 dimensions) can be directly spliced ​​in the feature dimension.

[0047] For example, if the first MFCC feature is [f1,f2,…,f 10 ], the second MFCC feature is [g1,g2,…,g 20 ], then the target features after concatenation are: [f1,f2,…,f 10 ,g1,g2,…,g 20 ].

[0048] The weighted processing is to assign different weights to the first MFCC feature and the second MFCC feature according to the importance or signal-to-noise ratio of the bone conduction signal and the air conduction signal, and merge them. Specifically, according to the signal-to-noise ratio or reliability of the bone conduction signal and the air conduction signal, the weight coefficients α and β can be set so that α+β=1. Then, the first MFCC feature is multiplied by the weight α, and the second MFCC feature is multiplied by the weight β. After that, the two weighted sets of features are added to generate the target feature.

[0049] For example, if the first MFCC feature is [f1,f2,…,f 10 ], the second MFCC feature is [g1,g2,…,g 20 ], the weighting coefficients are α and β respectively, then the target feature is: α·f i +β·g i (i=1,2,…,10).

[0050] In some embodiments, after the splicing process or weighting process is completed, in order to balance the influence of the bone conduction signal and the air conduction signal and ensure that they have the same weight in the model input, the target feature may be normalized. The normalization process may adopt a linear normalization method or a standardization method.

[0051] 104. Input the target features into the keyword recognition model for recognition, and obtain the preliminary recognition results of each frame.

[0052] It should be noted that the keyword recognition model is trained based on the Convolutional Recurrent Neural Network (CRN). The CRN neural network is a neural network that combines Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN).

[0053] The process of inputting the target features into the keyword spotting model for recognition can be divided into three main stages: feature extraction, time series modeling, and classification output. First, the input target features are processed by CNN to extract their spatial patterns and local features. Through multiple convolutional layers, the model is able to capture the speech spectrum characteristics within the time frame while retaining the time series structure of the target features.

[0054] Next, the keyword recognition model uses the gated recurrent unit (GRU) in the CRN to process the time series structure. GRU can capture the short-term and long-term dependencies of speech signals and extract the hidden temporal patterns in the sequence. The dual GRU layer structure further enhances the ability to express complex time series features and provides more accurate high-dimensional feature representation for keyword classification.

[0055] Finally, the extracted high-dimensional features are nonlinearly mapped through the fully connected layer, and the category probability distribution of each frame is generated through the Softmax function. The keyword recognition model takes the category with the highest probability as the preliminary recognition result of the frame and outputs the predicted label and corresponding probability distribution of each frame.

[0056] For example, suppose the input target feature is a 99×30 two-dimensional matrix, representing a 99-frame time series, and each frame contains a 30-dimensional fusion feature of the bone conduction signal and the air conduction signal.

[0057] The keyword recognition model processing process can be as follows: 1. Convolutional layer processing (spatial feature extraction): 1.1, Conv1D (k=1, s=1): The first layer of convolution processes each frame separately, extracts local features through a 1x1 convolution kernel, and keeps the output size at 99×30.

[0058] 1.2, Conv1D (k=5, s=3): The second layer of convolution extracts temporal context information across frames, with a kernel size of 5 and a stride of 3. The output time dimension is reduced to 33 frames, and the feature dimension remains 30.

[0059] 2. GRU layer processing (temporal feature extraction): 2.1. The first GRU layer (h=32): uses 32 hidden units to extract temporal dependency information, and the output is 33×32.

[0060] 2.2. Second GRU layer (h=32): extracts higher-level temporal features again, and the output dimension remains 33×32.

[0061] 3. Fully connected layer processing (feature transformation and classification): 3.1. Flatten layer: Expand the GRU output into a one-dimensional vector with a shape of 1×(33×32).

[0062] 3.2. Dense layer (64): maps the features to a 64-dimensional space and extracts higher-level nonlinear features through the ReLU activation function.

[0063] 3.3. Dense layer (numtypes): The last layer outputs the category distribution of each frame, for example, it contains 10 keyword categories such as "previous song" and "next song", plus "silence" and "others", a total of 12 categories.

[0064] 4. Softmax output (probability distribution): The Softmax function normalizes the unnormalized scores (logits) output by the Dense layer to generate a probability distribution for each frame. For example, for the 5th frame, the Softmax output is [0.05, 0.10, 0.80, …, 0.02], with the highest probability of 0.80, corresponding to the category "next song".

[0065] Therefore, before step 101, a keyword recognition model may also be constructed. Specifically, a speech data set may be obtained, the speech data set including a number of bone conduction signals and their corresponding air conduction signals; then, feature extraction is performed on the corresponding bone conduction signals and air conduction signals respectively to obtain a third MFCC feature of a first preset dimension and a fourth MFCC feature of a second preset dimension; then, the third MFCC feature and the fourth MFCC feature are feature fused to generate training features; finally, the CRN neural network is trained based on the training features to obtain a keyword recognition model.

[0066] In the embodiment of the present application, a Bluetooth headset with a built-in bone vibration sensor and an air conduction microphone can be used to collect data of the bone conduction signal and its corresponding air conduction signal in a quiet laboratory to ensure high-quality recording of the bone conduction signal and the air conduction signal. It should be noted that in the data collection process, different speech speeds (normal, faster, slower) and different volumes (high volume and low volume) need to be covered to enhance the adaptability of the model to diversified voice input.

[0067] The embodiment of the present application presets 10 keywords, including "previous song", "next song", "turn up the volume", "turn down the volume", "turn on noise reduction", "turn off noise reduction", "play music", "stop playing", "answer call", and "hang up call". In addition, two additional tags, "others" and "silence", are added. "Others" contains non-keyword speech, and the data volume is 10 times that of each keyword, which is used to reduce the misrecognition rate. "Silence" records quiet clips, which may contain slight background noise, but there is no human voice (such as bone conduction vibration caused by touching the ears and cheeks). The amount of data for the "silence" category is comparable to the number of samples for other keywords.

[0068] During the implementation, the speaker can wear a Bluetooth headset with a bone conduction sensor and a microphone to record bone conduction signals and air conduction signals. During the recording process, each keyword is recorded 10 times at three speech speeds (normal, faster, slower) and two volumes (high volume, low volume). There is a 2-second interval between each recording for subsequent segmentation. The audio length of each keyword is 1.5 seconds to ensure sufficient voice information for feature extraction.

[0069] Afterwards, voice activity detection (VAD) can be used to segment the air conduction signal, extract valid voice segments for storage, and simultaneously segment the corresponding bone conduction signal for storage. In some embodiments, whisper technology can be used to perform speech recognition on the segmented air conduction signal, and the air conduction signal and the corresponding bone conduction signal can be stored in folders according to the names corresponding to the labels. In addition, recordings with too fast a speech speed, too low a volume, or a keyword pronunciation duration exceeding 1.5 seconds can also be manually removed.

[0070] In some embodiments, noise can also be added to the air conduction signal to simulate various scenarios, including indoor noise, subway station noise, noise of multiple people talking, etc. The signal-to-noise ratio can range from -10 to 25 dB.

[0071] After obtaining the speech dataset, the speech dataset can be divided into training set, validation set and test set in a ratio of 8:1:1. It is worth noting that it is necessary to ensure that each category (10 keywords, others, silence) is evenly distributed in each subset to avoid category bias during model training.

[0072] Then, the first MFCC feature of the bone conduction signal and the second MFCC feature of the corresponding air conduction signal can be extracted, and then the first MFCC feature and the corresponding second MFCC feature are fused to generate a 30-dimensional target feature.

[0073] After that, the target features corresponding to the training set can be used to train the CRN neural network to obtain a primary keyword recognition model. Then, the primary keyword recognition model is verified by the target features corresponding to the validation set. Finally, the verified primary keyword recognition model is tested by the target features corresponding to the test set, and the keyword recognition model can be obtained after the test is completed.

[0074] 105. Perform multi-frame logic post-processing on all preliminary recognition results to obtain the final recognition results.

[0075] Specifically, a sliding window analysis may be performed on all preliminary recognition results to determine the number of frames that meet the probability threshold in a plurality of consecutive frames; when the number of frames that meet the probability threshold reaches a preset value, the corresponding keyword category is determined as the final recognition result.

[0076] Among them, sliding window analysis refers to setting a sliding window of a fixed size (for example, 7 frames) on the time series, sliding frame by frame, and counting the number of frames in the sliding window that meet the probability threshold. The size of the sliding window is usually related to the pronunciation duration of the keyword. Parameters such as the size of the sliding window, the sliding step, the probability threshold, and the number of consecutive frames (number of frames) can be adjusted according to the actual scenario to adapt to different speech characteristics and environmental complexity. For example, if the pronunciation of the keyword is short, the sliding window can be reduced; if the pronunciation is long, the sliding window can be appropriately increased. In a noisy environment, the probability threshold can be appropriately lowered to tolerate noise interference; in a quiet environment, the probability threshold can be increased to reduce misjudgment.

[0077] For example, when the probability threshold is 0.9 and the preset value is 4. If within the sliding window, the number of frames in which the probability value of the "next song" category is greater than 0.9 is 4 frames, it can be determined that the current keyword is "next song".

[0078] This embodiment utilizes the time continuity feature of the speech signal and, by judging the stability of multiple frames, can avoid the influence of single-frame prediction misjudgment or short-term noise interference on the recognition results, especially words that are relatively close to the keywords.

[0079] In summary, the keyword recognition method provided in the embodiment of the present application adopts the method of extracting the bone conduction signal and the air conduction signal corresponding to the voice signal when the voice signal is received; performing feature extraction on the bone conduction signal and the air conduction signal respectively to obtain the first MFCC feature of the first preset dimension and the second MFCC feature of the second preset dimension; performing feature fusion on the first MFCC feature and the second MFCC feature to generate the target feature; inputting the target feature into the keyword recognition model for recognition to obtain the preliminary recognition result of each frame; performing multi-frame logic post-processing on all the preliminary recognition results to obtain the final recognition result. This scheme effectively improves the accuracy of keyword recognition by extracting the bone conduction signal and the air conduction signal corresponding to the voice signal, then performing feature fusion on the bone conduction signal and the air conduction signal, and then identifying the target feature through the keyword recognition model, and further improves the accuracy of keyword recognition by combining the multi-frame logic post-processing technology.

[0080] In order to better implement the keyword recognition method provided in the embodiment of the present application, the embodiment of the present application also provides a keyword recognition device, wherein the meanings of the nouns are the same as those in the above keyword recognition method, and the specific implementation details can refer to the description in the method embodiment.

[0081] See also Figure 3 , Figure 3201 is a schematic diagram of the structure of a keyword recognition device provided in an embodiment of the present application. The keyword recognition device may include a signal extraction unit 201, a feature extraction unit 202, a feature fusion unit 203, a feature recognition unit 204 and a result judgment unit 205. The signal extraction unit 201 is used to extract the bone conduction signal and the air conduction signal corresponding to the speech signal when receiving the speech signal; A feature extraction unit 202 is used to extract features from the bone conduction signal and the air conduction signal respectively to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; A feature fusion unit 203 is used to fuse the first MFCC feature and the second MFCC feature to generate a target feature; The feature recognition unit 204 is used to input the target feature into the keyword recognition model for recognition, and obtain a preliminary recognition result for each frame; The result judgment unit 205 is used to perform multi-frame logic post-processing on all the preliminary recognition results to obtain a final recognition result.

[0082] The specific implementation of each of the above units can refer to the above-mentioned embodiment of the keyword recognition method, which will not be described one by one here.

[0083] In summary, the keyword recognition device provided by the embodiment of the present application can extract the bone conduction signal and the air conduction signal corresponding to the speech signal through the signal extraction unit 201 when receiving the speech signal; the feature extraction unit 202 performs feature extraction on the bone conduction signal and the air conduction signal respectively to obtain the first MFCC feature of the first preset dimension and the second MFCC feature of the second preset dimension; the feature fusion unit 203 performs feature fusion on the first MFCC feature and the second MFCC feature to generate the target feature; the feature recognition unit 204 inputs the target feature into the keyword recognition model for recognition to obtain the preliminary recognition result of each frame; the result judgment unit 205 performs multi-frame logic post-processing on all the preliminary recognition results to obtain the final recognition result. This scheme effectively improves the accuracy of keyword recognition by extracting the bone conduction signal and the air conduction signal corresponding to the speech signal, then performs feature fusion on the bone conduction signal and the air conduction signal, and then recognizes the target feature through the keyword recognition model, and further improves the accuracy of keyword recognition by combining the multi-frame logic post-processing technology.

[0084] The present application also provides an electronic device, in which the keyword recognition device of the present application can be integrated, such as Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically: The electronic device may include one or more processors 301 of processing cores and one or more computer-readable storage media memories 302 and other components. Those skilled in the art will appreciate that Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. The processor 301 is the control center of the electronic device, which uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing the software program and / or the present application stored in the memory 302, and calling the data stored in the memory 302, so as to monitor the electronic device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operation storage medium, user interface and application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0085] The memory 302 can be used to store software programs and the present application. The processor 301 executes various functional applications and data processing by running the software programs and the present application stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating storage medium, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0086] Although not shown, the electronic device may further include a display unit, an input unit, a power supply, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby realizing various functions, as follows: When a speech signal is received, a bone conduction signal and an air conduction signal corresponding to the speech signal are extracted; Feature extraction is performed on the bone conduction signal and the air conduction signal respectively to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; The first MFCC feature and the second MFCC feature are fused to generate the target feature; Input the target features into the keyword recognition model for recognition, and obtain the preliminary recognition results of each frame; All preliminary recognition results are post-processed with multi-frame logic to obtain the final recognition results.

[0087] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0088] To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the generation methods provided in the embodiment of the present application. For example, the instructions can execute the following steps: When a speech signal is received, a bone conduction signal and an air conduction signal corresponding to the speech signal are extracted; Feature extraction is performed on the bone conduction signal and the air conduction signal respectively to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; The first MFCC feature and the second MFCC feature are fused to generate the target feature; Input the target features into the keyword recognition model for recognition, and obtain the preliminary recognition results of each frame; All preliminary recognition results are post-processed with multi-frame logic to obtain the final recognition results.

[0089] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0090] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0091] Since the instructions stored in the storage medium can execute the steps in any method provided in the embodiments of the present application, the beneficial effects that can be achieved by any method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0092] The keyword recognition method, device, storage medium and electronic device provided by the present application are respectively introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the core idea of ​​the present application; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A keyword recognition method, characterized in that: include: When a speech signal is received, a bone conduction signal and an air conduction signal corresponding to the speech signal are extracted; Performing feature extraction on the bone conduction signal and the air conduction signal respectively to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; Fusing the first MFCC feature and the second MFCC feature to generate a target feature; Input the target features into a keyword recognition model for recognition, and obtain a preliminary recognition result for each frame; All the preliminary recognition results are subjected to multi-frame logic post-processing to obtain the final recognition result.

2. The keyword recognition method according to claim 1, characterized in that: The multi-frame logic post-processing of all the preliminary recognition results to obtain the final recognition result includes: Performing a sliding window analysis on all the preliminary recognition results to determine the number of frames that meet a probability threshold in a plurality of consecutive frames; When the number of frames that meet the probability threshold reaches a preset value, the corresponding keyword category is determined as the final recognition result.

3. The keyword recognition method according to claim 2, characterized in that: The step of fusing the first MFCC feature and the second MFCC feature to generate a target feature includes: The first MFCC feature and the second MFCC feature are concatenated or weighted to generate a target feature.

4. The keyword recognition method according to claim 1, characterized in that: The first preset dimension is the second dimension to the eleventh dimension, and the second preset dimension is the first dimension to the twentieth dimension.

5. The keyword recognition method according to claim 1, characterized in that: When a speech signal is received, after extracting the bone conduction signal and the air conduction signal corresponding to the speech signal, and before respectively extracting features from the bone conduction signal and the air conduction signal to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension, the method further includes: Performing energy calculation on the bone conduction signal, and judging whether the speech signal is a silent segment according to the calculation result; If not, the step of respectively extracting features from the bone conduction signal and the air conduction signal to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension is performed.

6. The keyword recognition method according to claim 1, characterized in that: Before extracting the bone conduction signal and the air conduction signal corresponding to the speech signal when the speech signal is received, the method further includes: Build a keyword recognition model.

7. The keyword recognition method according to claim 6, characterized in that: The step of constructing a keyword recognition model includes: Acquire a speech data set, where the speech data set includes a plurality of bone conduction signals and their corresponding air conduction signals; Performing feature extraction on the corresponding bone conduction signal and the air conduction signal respectively to obtain a third MFCC feature of a first preset dimension and a fourth MFCC feature of a second preset dimension; Perform feature fusion on the third MFCC feature and the fourth MFCC feature to generate training features; The CRN neural network is trained based on the training features to obtain a keyword recognition model.

8. A keyword recognition device, characterized in that: include: A signal extraction unit, configured to extract a bone conduction signal and an air conduction signal corresponding to the speech signal when a speech signal is received; A feature extraction unit, configured to perform feature extraction on the bone conduction signal and the air conduction signal respectively, to obtain a first MFCC feature of a first preset dimension and a second MFCC feature of a second preset dimension; A feature fusion unit, used for fusing the first MFCC feature and the second MFCC feature to generate a target feature; A feature recognition unit, used to input the target feature into a keyword recognition model for recognition, and obtain a preliminary recognition result for each frame; The result judgment unit is used to perform multi-frame logic post-processing on all the preliminary recognition results to obtain the final recognition result.

9. A storage medium, characterized in that: The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the keyword recognition method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the keyword recognition method according to any one of claims 1 to 7 when executing the computer program.

Images