A speech recognition method and apparatus

Abstract

The embodiment of the application relates to a speech recognition method, comprising: acquiring audio data collected by a microphone. Performing speech endpoint detection on the acquired audio data to determine a human voice segment. Extracting an embedded feature vector for the human voice segment. Comparing the embedded feature vector with data in a feature vector database in sequence for similarity. If the similarity of the embedded feature vector with each item of data in the feature vector database is less than a pre-set similarity threshold, the embedded feature vector is stored in the feature vector database. The human voice segment corresponding to the embedded feature vector is subjected to automatic speech recognition to execute a corresponding instruction. The application can match multiple different speakers through data stored in the feature vector database, thereby effectively dealing with the situation of speech recognition when different speakers exist.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronics, and in particular, to a voice recognition method and a terminal device. BACKGROUND

[0002] Voice activity detection (VAD), which can also be referred to as voice boundary detection, is mainly used to identify which part of an audio does not have speech and which part has speech. The part without speech can also be referred to as a silence period, and the part can only contain noise and the like. When the detection is performed through VAD, the silence period in the audio can be effectively eliminated. In, for example, internet protocol (IP) calls or automatic speech recognition (ASR) technology, a long silence period can be identified and eliminated through VAD to save resources without reducing service quality. This approach can save valuable computing resources and is conducive to reducing the end-to-end latency experienced by users.

[0003] Current VAD technology is usually applied to a voice recognition system and is responsible for accurately locating the start and end points of speech from audio data with noise. Therefore, the quality of VAD will directly affect the effect of the detection result. As shown in the prior art, current VAD can mainly be performed in two different ways, such as threshold comparison and statistical classification. Figure 1 For VAD of the threshold comparison type, specific implementation can include single-feature threshold comparison and multi-feature threshold comparison. However, in current VAD schemes, detection is performed based on a threshold. If the noise in the audio is relatively complex, for example, contains broadcast music, broadcast e-books, multiple people speaking, vehicle wind noise, and the like, it is difficult to identify human voice and noise in the audio, which obviously leads to a very inaccurate detection result.

[0004] For VAD of the statistical classification type, specific implementation can include supervised classification models and unsupervised classification models. In some schemes, the supervised classification models can include support vector machines and neural networks. In other schemes, the unsupervised classification models can include a mixture Gaussian model and a hidden Markov model. For VAD of the statistical classification type, current schemes usually perform binary classification, that is, only human voice segments can be divided, and it is impossible to identify which person the human voice segments belong to. In some schemes, only the human voice segments of a certain person can be identified, and different voice recognition of different people cannot be performed.

[0005] Therefore, there is an urgent need for a scheme that can effectively identify different people speaking in an audio. SUMMARY

[0006] The embodiment of the present application provides a speech recognition method. The speech recognition method can divide a human voice segment from acquired audio data through VAD. Then, the human voice segment is subjected to feature extraction, and the extracted feature is compared with pre-stored data in a feature vector database. If the comparison fails, the feature of the human voice segment is stored in the feature vector database. In this way, when subsequent speech recognition is performed, multiple different speakers can be matched, so that the speech recognition can be effectively performed in the presence of different speakers.

[0007] In a first aspect, a speech recognition method is provided. The method can be applied to a terminal device. The method can include: acquiring audio data collected by a microphone. In some examples, the audio data can also be collected by other devices and sent to the terminal device. Then, speech endpoint detection is performed on the acquired audio data to determine at least one human voice segment. Then, the human voice segment is subjected to feature extraction to determine an embedding feature vector of the human voice segment. In some examples, feature extraction can be performed on each human voice segment. Then, the embedding feature vector of the human voice segment is compared with data in a feature vector database in sequence. It can be understood that the feature vector database stores embedding feature vectors corresponding to one or more speakers. Of course, when the feature vector database is used for the first time, there can be no data in the feature vector database, and the similarity comparison result can be 0. If the similarity of the embedding feature vector to each item of data in the feature vector database is less than a pre-set similarity threshold, the embedding feature vector is stored in the feature vector database. Then, automatic speech recognition is performed on the human voice segment corresponding to the embedding feature vector to execute a corresponding instruction. The speech recognition method provided by the present application can divide a human voice segment from acquired audio data through VAD, and then compare the extracted feature with pre-stored data in a feature vector database. If the comparison fails, the feature of the human voice segment is stored in the feature vector database. In this way, when subsequent speech recognition is performed, multiple different speakers can be matched through the data stored in the feature vector database, so that the speech recognition can be effectively performed in the presence of different speakers.

[0008] In one possible implementation, the speech endpoint detection on the audio data to determine at least one human voice segment can include: inputting the audio data into a speech endpoint detection model containing at least one long short-term memory artificial neural network (LSTM) for detection, so as to determine at least one human voice segment. The speech endpoint detection model containing at least one LSTM can divide a human voice segment more accurately than existing VAD, so as to ensure that more effective embedding feature vectors can be extracted during subsequent feature extraction.

[0009] In a possible implementation, the speech endpoint detection model comprising at least one LSTM can comprise: at least 6 one-dimensional convolutional neural network layers and at least 3 one-way 128-node LSTMs.

[0010] In a possible implementation, the feature extraction on the voice segment can comprise: for the voice segment, a signal-to-noise ratio of the voice segment can be determined by combining the voice segment and a silent segment adjacent to the voice segment. The silent segment can be obtained through speech endpoint detection on the audio data or preconfigured. If the signal-to-noise ratio is greater than a preset signal-to-noise ratio threshold, the feature extraction can be performed on the voice segment. In this application, the signal-to-noise ratio detection on the voice segment can filter out the voice segment with less interference for subsequent feature extraction, thereby ensuring that more effective embedding feature vectors can be extracted during feature extraction.

[0011] In a possible implementation, the determination of the signal-to-noise ratio of the voice segment in combination with the silent segment adjacent to the voice segment can comprise: determining a segment with a preset time length from the voice segment as a signal segment, and determining a segment with a preset time length from the silent segment adjacent to the voice segment as a noise segment. Then, the signal-to-noise ratio of the voice segment can be determined according to the signal segment and the noise segment.

[0012] In a possible implementation, the preset time length can be 320 milliseconds.

[0013] In a possible implementation, the determination of the embedding feature vector of the voice segment can comprise: inputting the voice segment into an embedding feature extraction model comprising multiple hidden layers for feature extraction, thereby determining the embedding feature vector of the voice segment. In this application, the embedding feature extraction model comprising multiple hidden layers is used to extract features from the voice segment, so that the extracted features can more effectively identify the corresponding speaker.

[0014] In a possible implementation, the embedding feature extraction model can comprise at least 4 hidden layers. The embedding feature extraction model in this application comprises at least 4 hidden layers, which can ensure that the extracted embedding feature vector can better represent the speaker.

[0015] In a possible implementation, before storing the embedding feature vector into the feature vector database, the method can further comprise: inputting the voice segment corresponding to the embedding feature vector into a detection instruction model for detection. If it is detected that the voice segment corresponding to the embedding feature vector contains an instruction, the embedding feature vector corresponding to the voice segment is stored into the feature vector database. In this application, the embedding feature vector is detected to avoid saving the embedding feature vector without instructions into the feature vector database, thereby reducing resource waste.

[0016] In a possible implementation, the method can further include: if the similarity between the embedding feature vector and any data in the feature vector database is greater than or equal to the similarity threshold, performing automatic speech recognition on the human voice segment corresponding to the embedding feature vector, so that corresponding instructions can be executed according to the speech recognition result. In this application, when the embedding feature vector of the human voice segment has a high similarity with a certain data in the feature vector database, it can be determined that the matching is successful, so that the speaker is identified, and then the speech of the speaker can be ASR and corresponding instructions can be executed.

[0017] In a possible implementation, each data in the feature vector database corresponds to a number of successful matches, and each data in the feature vector database can be sequentially sorted from large to small according to the number of successful matches. The method can further include: if the similarity between the embedding feature vector and any data in the feature vector database is greater than or equal to the similarity threshold, it can be further determined whether the data is located in the top n of the above-mentioned sorting. Wherein, n is a positive integer. When the data belongs to the top n of the above-mentioned sorting, the automatic speech recognition can be performed on the human voice segment corresponding to the embedding feature vector, so that corresponding instructions can be executed according to the speech recognition result. In this application, whether the human voice segment matched successfully this time is ASR can also be determined according to the number of successful matches of each data in the feature vector database. Avoid the situation that the user experience is reduced due to ASR in the case of less matching times.

[0018] In a possible implementation, the method can further include: if the similarity between the embedding feature vector and any data in the feature vector database is greater than or equal to the similarity threshold, the number of successful matches corresponding to the data can be incremented by one. And each data in the feature vector database can be sequentially sorted again from large to small. In this application, the number of successful matches of the corresponding data can be incremented by one after the matching is successful, so that the data in the feature vector database that allows ASR can be dynamically updated.

[0019] In a possible implementation, the method can further include: if the signal-to-noise ratio of the human voice segment is less than or equal to the signal-to-noise ratio threshold, the human voice segment can be automatically speech recognized, and corresponding instructions can be executed according to the speech recognition result. In this application, when the human voice segment signal interference is strong, ASR can be directly performed and corresponding instructions can be executed, without feature extraction, so as to avoid extracting speaker features from human voice segments with poor signal quality, and to ensure that the data in the feature vector database are good embedding feature vectors.

[0020] In a second aspect, a terminal device for speech recognition is provided. The terminal device includes a processor coupled with a memory and configured to read and execute instructions stored in the memory. When the processor executes the instructions, the processor is configured to obtain audio data collected by a microphone. The audio data is subjected to voice activity detection to determine at least one voice segment. Feature extraction is performed on the voice segment to determine an embedding feature vector of the voice segment. In some examples, the feature extraction can be performed on each voice segment. The embedding feature vector is then compared with data in a feature vector database in sequence. If the similarity between the embedding feature vector and each item of data in the feature vector database is less than a similarity threshold, the embedding feature vector is stored in the feature vector database, and automatic speech recognition is performed on the voice segment corresponding to the embedding feature vector to execute corresponding instructions. In the present application, the obtained audio data is divided into voice segments by VAD, and then compared with pre-stored data in the feature vector database after feature extraction. If all the comparisons fail, the feature of the voice segment can be stored in the feature vector database. In this way, when speech recognition is performed subsequently, a plurality of different speakers can be matched by the data stored in the feature vector database, so that the situation of speech recognition in the presence of different speakers can be effectively coped with.

[0021] In one possible implementation, the processor is further configured to input the audio data into a voice activity detection model comprising at least one LSTM to determine at least one voice segment. In the present application, the VAD comprising at least one LSTM is used for detection, which can divide the voice segment more accurately than the existing VAD, so as to ensure that more effective embedding feature vectors can be extracted during subsequent feature extraction.

[0022] In one possible implementation, the voice activity detection model comprising at least one LSTM comprises at least 6 one-dimensional convolutional neural networks and at least 3 one-way LSTMs with 128 nodes.

[0023] In one possible implementation, the processor is further configured to determine a signal-to-noise ratio of the voice segment in combination with a silent segment adjacent to the voice segment, wherein the silent segment is obtained by voice activity detection of the audio data or pre-configured. If the signal-to-noise ratio is greater than a preset signal-to-noise ratio threshold, feature extraction is performed on the voice segment. In the present application, the signal-to-noise ratio of the voice segment is detected, so that the voice segment with less interference can be screened for subsequent feature extraction, and more effective embedding feature vectors can be extracted during feature extraction.

[0024] In a possible implementation, the processor is further configured to: determine a segment with a preset time length from the human voice segment as a signal segment, and determine a segment with the preset time length from a silent segment adjacent to the human voice segment as a noise segment; and determine a signal-to-noise ratio of the human voice segment according to the signal segment and the noise segment.

[0025] In a possible implementation, the preset time length is 320 milliseconds.

[0026] In a possible implementation, the processor is further configured to: input the human voice segment into an embedding feature extraction model including multiple hidden layers to perform feature extraction, so as to determine an embedding feature vector of the human voice segment. The embedding feature extraction model including multiple hidden layers is used to perform feature extraction on the human voice segment, so that the extracted features can be more effectively identified to the corresponding speaker.

[0027] In a possible implementation, the embedding feature extraction model includes at least 4 hidden layers. The embedding feature extraction model including at least 4 hidden layers can guarantee that the extracted embedding feature vector can better represent the speaker.

[0028] In a possible implementation, the processor is further configured to: if the similarity between the embedding feature vector and each item of data in the feature vector database is less than the similarity threshold, input the human voice segment corresponding to the embedding feature vector into a detection instruction model to perform detection; and if the human voice segment corresponding to the embedding feature vector is detected to include an instruction, store the embedding feature vector into the feature vector database. The embedding feature vector is detected, so that the embedding feature vector without the instruction is avoided to be stored into the feature vector database, and resource waste is reduced.

[0029] In a possible implementation, the processor is further configured to: if the similarity between the embedding feature vector and any item of data in the feature vector database is greater than or equal to the similarity threshold, perform automatic speech recognition on the human voice segment corresponding to the embedding feature vector, and execute a corresponding instruction according to a speech recognition result. In the application, when the embedding feature vector of the human voice segment has a high similarity with a certain data in the feature vector database, it can be determined that the matching is successful, so that the speaker is determined to be recognized, and then the speech of the speaker can be subjected to ASR and a corresponding instruction can be executed.

[0030] In a possible implementation, each item of data in the feature vector database corresponds to a number of successful matches, and each item of data in the feature vector database is sequentially sorted in descending order according to the number of successful matches; the processor is further configured to: if the similarity between the embedded feature vector and any item of data in the feature vector database is greater than or equal to the similarity threshold, determine whether the data is in the top n, where n is a positive integer; and when the data is in the top n, perform automatic speech recognition on the vocal segment corresponding to the embedded feature vector, and execute a corresponding instruction according to the speech recognition result. In this application, whether to perform ASR on the vocal segment of the current successful match can be determined according to the number of successful matches of each data in the feature vector database. This avoids the situation that ASR is performed in the case of a small number of matches, thereby reducing the user experience.

[0031] In a possible implementation, the processor is further configured to: if the similarity between the embedded feature vector and any item of data in the feature vector database is greater than or equal to the similarity threshold, increase the number of successful matches corresponding to the data by one, and sequentially sort each item of data in the feature vector database again in descending order. In this application, the number of successful matches of the corresponding data is increased by one after a successful match, thereby ensuring that the data in the feature vector database that is allowed to perform ASR is dynamically updated.

[0032] In a possible implementation, the processor is further configured to: if the signal-to-noise ratio of the vocal segment is less than or equal to a signal-to-noise ratio threshold, perform automatic speech recognition on the vocal segment, and execute a corresponding instruction according to the speech recognition result. In this application, when the vocal segment signal is strongly interfered, ASR can be directly performed and a corresponding instruction can be executed, without feature extraction, thereby avoiding extracting speaker features from a vocal segment with poor signal quality, and ensuring that the data in the feature vector database is a good embedded feature vector.

[0033] In a third aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores instructions. When the instructions run on a terminal device, the terminal device performs any of the methods in the first aspect.

[0034] In a fourth aspect, a computer containing instructions is provided, and when the computer runs, the computer performs any of the methods in the first aspect.

[0035] In a fifth aspect, a computer program product containing instructions is provided, and when the computer program product runs on a computer, the computer performs any of the methods in the first aspect.

[0036] The application discloses a voice recognition method, which comprises the following steps: dividing at least one vocal fragment from acquired audio data through VAD; extracting features of the vocal fragment; and comparing the features with pre-stored data in a feature vector database. When the vocal fragment fails to match all data in the feature vector database, the embedding feature vector of the vocal fragment is stored in the feature vector database, so that the speaker information is dynamically registered, and multiple different speakers can be matched in subsequent voice recognition, and the speaker's instruction is executed correspondingly. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 It is a VAD classification schematic diagram;

[0038] Figure 2 It is a voice recognition scene schematic diagram provided by the embodiment of the application;

[0039] Figure 3 It is a specific person voice recognition flowchart;

[0040] Figure 4 It is a voice recognition system architecture schematic diagram provided by the embodiment of the application;

[0041] Figure 5 It is a voice recognition method flowchart provided by the embodiment of the application;

[0042] Figure 6 It is a VAD model structure schematic diagram provided by the embodiment of the application;

[0043] Figure 7 It is an embedding model structure schematic diagram provided by the embodiment of the application;

[0044] Figure 8 It is an audio signal schematic diagram provided by the embodiment of the application;

[0045] Figure 9 It is another voice recognition method flowchart provided by the embodiment of the application;

[0046] Figure 10 It is a terminal device schematic diagram provided by the embodiment of the application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the application will be described below with reference to the drawings in the embodiments of the application.

[0048] The application is mainly applied to a voice recognition scene. The scene can be, for example, a home scene, a vehicle-mounted scene or a scene in which multiple people perform voice recognition. As Figure 2As shown, the scenario in this case includes a terminal device 100, which is used to cope with voice recognition in the presence of multiple speakers. As can be seen, the scenario can also include multiple speakers, such as speaker A, speaker B, speaker C, and speaker D. Of course, more or fewer speakers can also be included in the actual scenario, which is not limited by the present application. The scenario can be, for example, a home scenario, such as multiple family members present in the home, who can all need to perform some voice operation on the terminal device.

[0049] However, in some existing solutions, although the terminal device can match the voice command of any person and then execute the related command, the speaker usually needs to say some pre-set keywords first, such as "Xiao X" or "Hello, Xiao X", etc. When the terminal device recognizes such pre-set keywords, it triggers ASR for voice recognition, thereby executing the corresponding instruction expressed by the speaker. As can be seen, when the speaker does not say the corresponding keywords in some cases, the existing terminal device usually cannot perform ASR, and thus will not be able to execute the corresponding instruction expressed by the speaker. This is very inconvenient for the user, causing a serious decline in the use experience.

[0050] In some solutions, some terminal devices use VAD combined with embedding to detect the audio data of the speaker. For example, some deep networks based on statistical models are used for detection. Such deep networks can include multiple layers, such as the front part of the layers designed using convolutional neural networks (CNN), and the back part of the layers designed using LSTM. The features can be extracted through the front part of the CNN layers and connected with the time domain features, so that the LSTM layer can better recognize and identify the voice of a specific person. The feature of such a solution is that it has good anti-noise performance. For VAD that does not combine embedding, the recognition performance is usually poor in the scene with vehicle noise and human voice features. For VAD that combines embedding, although it can recognize the voice of a specific speaker, it needs to be registered for the voice of a specific person before recognition.

[0051] As Figure 3 As shown, the current voice recognition method for a specific speaker can include:

[0052] S301, input the audio of the registered person.

[0053] S302, feature extraction is performed on the audio of the registered person to obtain the voiceprint features of the registered person.

[0054] S303, an embedding model is obtained.

[0055] Among them, S301, S302 and S303 are executed without strict order.

[0056] S304, according to the registered voiceprint features obtained after the characteristics extraction and the embedding model, combined with the initial VAD model for training, to obtain the personalized VAD model.

[0057] Obviously, the personalized VAD model can only be used for voice recognition of the registered person.

[0058] Then, S305, the personalized VAD model can be used for corresponding speech recognition.

[0059] That is, the above method needs to extract the audio features of a certain speaker in advance, that is, S301 and S302. Then the exclusive deep features of the speaker can be extracted by combining the audio features with the embedding model. Then, the exclusive deep features are used to train the initial VAD model to obtain the exclusive VAD model of the speaker, that is, S304. Obviously, the exclusive VAD model can accurately identify the audio of the speaker, thereby facilitating the subsequent execution of the instructions spoken by the speaker. However, the exclusive VAD model cannot effectively identify other speakers. Therefore, in the execution process of S305, for people other than the registered person, speech recognition cannot be effectively performed, and it is impossible to execute corresponding voice instructions for non-registered persons.

[0060] At the same time, for Figure 3 The scheme shown in the figure can only register a certain speaker, that is, the number of registered persons is 1, and multiple speaker registration cannot be realized. Obviously, if different speaker voice recognition is to be completed, multiple exclusive VAD models need to be generated to correspond to different registered persons (i.e. different speakers). Obviously, the experience of the user will be greatly reduced, and the complexity of the system will be increased, and the resource overhead will be increased.

[0061] Therefore, the present application provides a speech recognition method, which divides at least one voice segment from the acquired audio data by VAD, then extracts features from the voice segment, and compares with the pre-stored data in the feature vector database. When the voice segment fails to match all the data in the feature vector database, the embedding feature vector of the voice segment can be stored in the feature vector database. So that the feature vector database can store embedding feature vectors of multiple different speakers, realizing dynamic registration of speaker information. In order to effectively identify the voice segment in the future, it is ensured that multiple different speakers can be matched and corresponding instructions of the speakers can be executed.

[0062] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application.

[0063] Figure 4 A schematic diagram of a voice recognition system architecture provided in the embodiments of the present application is provided.

[0064] As shown in Figure 4 , the present application provides a voice recognition system architecture, which can be applied to a terminal device. The terminal device can be the terminal device 100 mentioned in Figure 2 , and can include but is not limited to any terminal device or portable terminal device such as a mobile phone, a smart TV, a smart sound, a wearable device, a tablet computer, a desktop computer, an all-in-one computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a laptop, a mobile computer, an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, and / or a vehicle-mounted device, etc. The vehicle-mounted device can be, for example, a smart car, etc.

[0065] It can be seen that the architecture includes an application (APP) layer, a framework layer, and a kernel layer. The voice recognition APP can be included in the APP layer. It can be understood that the user can implement the voice recognition method involved in the present application through the voice recognition APP. The voice recognition APP can be executed after receiving the user's opening instruction, or can be continuously executed in the background by default after the terminal device is turned on, which is not limited in the present application. The framework layer can include a plurality of services, such as an ASR service and a speaker recognition service, etc. The speaker recognition service can include VAD and speaker recognition, etc. to effectively identify different speakers and facilitate ASR service for the recognized speaker's voice. The kernel layer can include a microphone driver, which is mainly used to drive the microphone to obtain the audio signal collected by the microphone. Then the audio signal is converted into audio data.

[0066] It can be understood that Figure 4The system architecture shown can obtain the current audio data through the microphone driver of the kernel layer when implementing voice recognition. The microphone collection process can refer to the existing mode, and details are not described herein. Then, the kernel layer transmits the obtained audio data to the speaker recognition service in the framework layer. For example, the audio data can be detected by VAD to obtain at least one voice segment. Then, speaker recognition can be performed on the at least one voice segment to identify one or more registered speakers. Then, the voice segment of the identified registered speaker can be subjected to ASR service, thereby facilitating the terminal device to execute corresponding instructions. Finally, the voice recognition APP in the application layer can execute corresponding operations on the identified instructions.

[0067] Figure 5 A flowchart of a voice recognition method provided by an embodiment of the present application is shown.

[0068] As shown in Figure 5 The present application provides a voice recognition method. The method can be applied to one terminal device or multiple terminal devices working together. One or more terminal devices can run Figure 4 The system shown. When the terminal device starts the voice recognition APP, the method shown in Figure 5 In some examples, the method can run on one terminal device. In other examples, the method can run on multiple terminal devices and be completed cooperatively. The multiple terminal devices may, for example, include a vehicle-mounted device and a terminal device that is not vehicle-mounted. Next, one terminal device will be described as an example.

[0069] The method can include the following steps:

[0070] S501, obtaining audio data collected by a microphone.

[0071] The terminal device can obtain the audio data collected by the microphone on the device itself.

[0072] In some examples, the terminal device can also be connected to other devices in a wired or wireless manner to obtain corresponding audio data from the other devices. For example, the audio data can be collected in advance by a microphone and saved on the other device, or collected in real time by a microphone on the other device, or collected in advance by a microphone on the other device and stored on the other device.

[0073] The wireless mode can include a solution of 2G / 3G / 4G / 5G / 6G wireless communication, or a solution of wireless local area network (WLAN), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), zigbee, infrared (IR) and other wireless communication technologies. The WLAN can be a wireless fidelity (Wi-Fi) network, for example.

[0074] In one example, the user can start the voice recognition process and obtain the audio data by starting the voice recognition APP 411 shown in FIG. 11, so as to start the voice recognition process and obtain the audio data. If the scenario is a vehicle driving scenario, it can be manifested as starting the vehicle voice and starting the voice recognition process. Figure 4

[0075] It can be understood that the present application does not limit how the audio data is collected.

[0076] In one example, the audio data obtained by the terminal device can be a mel frequency cepstrum coefficient (MFCC) feature obtained by performing MFCC feature extraction on the audio signal collected by the microphone on the terminal device. For example, the terminal device can perform preprocessing on the audio signal collected by the microphone, such as pre-emphasis, framing, windowing, etc. Then, the preprocessed audio signal is subjected to fast fourier transform (FFT) to obtain the frequency spectrum corresponding to each frame in the frequency domain. Then, the frequency spectrum is squared to obtain the spectral line energy of the audio signal. Then, the spectral line energy of the audio signal is filtered by a mel filter, and the log energy output by the mel filter is subjected to discrete cosine transform (DCT) to obtain the MFCC feature. It can be understood that the MFCC feature can also be referred to as MFCC feature parameter or MFCC coefficient, etc. Of course, the specific way of calculating the MFCC feature can refer to the existing way, which will not be described here.

[0077] S502, performing VAD on the collected audio data to determine at least one vocal segment.

[0078] ​After the terminal device acquires the audio data, it can input the audio data acquired in S501 into the VAD model for recognition. The VAD model is a pre-trained VAD model.

[0079] In one example, such as Figure 6 The diagram illustrates a VAD model provided in this application. In this VAD model, the audio data obtained in step S501 is used as input and undergoes convolution operations through at least one CNN layer 601 to extract deep features from the audio data. It is understood that the number of CNN layers in 601 can be one or more; for example, at least one CNN layer 601 can have six CNN layers. Of course, this application does not specifically limit the exact number of CNN layers. When the number of CNN layers is determined to be six, it can extract deeper features while ensuring that the VAD model is not overly complex, so that the human voice segments in the audio data can be more accurately identified subsequently. Each CNN layer can be a one-dimensional CNN layer, that is, each CNN layer uses a one-dimensional convolution kernel for convolution operations. Of course, the specific operators for each CNN layer can refer to existing VAD models, and the corresponding parameters in the operators can be set and adjusted according to the actual situation, which will not be elaborated upon in this application.

[0080] After the convolution operation of the at least one CNN layer 601, the extracted deep features can be sequentially input into the first LSTM layer 602, the second LSTM layer 603 and the third LSTM layer 604. For example, each LSTM layer can be a unidirectional LSTM layer containing 128 nodes. Of course, the parameter settings of each node in each LSTM layer can be set according to actual conditions. In other examples, the number of nodes in the first LSTM layer 602, the second LSTM layer 603 and the third LSTM layer 604 can also be sequentially decreasing, for example, the first LSTM layer 602 is a unidirectional LSTM layer containing 768 nodes, the second LSTM layer 603 is a unidirectional LSTM layer containing 512 nodes, and the third LSTM layer 604 is a unidirectional LSTM layer containing 256 nodes. It can be understood that the number of nodes in each LSTM layer is not specifically limited in the present application. The calculation process in each LSTM layer is similar, so the first LSTM layer 602 is described. In one example, the first LSTM layer receives the deep features output by the at least one CNN layer 601 for calculation. In combination with the time dimension of the deep features, for example, the input deep features are the deep features corresponding to 160 millisecond (ms) of audio data, then the LSTM calculation can be performed in units of 10 ms, for example, the first 10 ms is recorded as t0, each time the LSTM calculation considers the calculation result and state information of the last time, and then outputs the calculation result and state information of the current LSTM. It can be understood that the state information of the LSTM layer changes relatively slowly, so it can be considered that long-time memory is performed, and the calculation result changes greatly each time, so it can be considered that short-time memory is performed. Then, the first LSTM layer 602 inputs the output result into the second LSTM layer 603 for calculation.

[0081] After the calculation of the three layers, i.e., the first LSTM layer 602, the second LSTM layer 603 and the third LSTM layer 604, the output result thereof is subjected to the full connection layer 605. It can be understood that the full connection layer 605 can include 2 neurons for representing whether there is a person speaking in the audio data. Obviously, the number of neurons in the full connection layer 605 is related to the number of results, i.e., the number of results is the same as the number of neurons in the full connection layer 605. Then, the normalization function (softmax) layer 606 normalizes the results output by each neuron in the full connection layer 605, which can be used to represent the probability of the result appearing. Then the probability of each result is output, thereby completing the process of identifying the speaker by the VAD model. For example, when the probability of the speaker result is greater than or equal to a preset probability threshold, it can be considered that the audio data identifies the speaker; otherwise, if the probability of the speaker result is less than the preset probability threshold, it can be considered that the audio data does not identify the speaker. It can be understood that the sum of the probabilities corresponding to the two results (i.e., the existence of a speaker and the non-existence of a speaker) in the VAD model is 1.

[0082] Of course, in some examples, Figure 6 The VAD model shown can also not include at least one CNN layer 601, but directly input the audio data acquired by the terminal device to the first LSTM layer 602. In still other examples, the VAD model can include more or fewer LSTM layers. It should be understood that when including 6 CNN layers and 3 LSTM layers, the VAD model can guarantee a higher accuracy of the output result while maintaining a certain complexity, and also can guarantee a faster calculation speed, facilitating the terminal device to perform calculation.

[0083] In one example, the VAD can be performed on the audio data in a manner of 160 ms per frame, that is, the audio data is framed in 160 ms. Then, the VAD is performed on each frame of audio data one by one, so as to more accurately determine which part of the audio data has human voice, thereby determining at least one human voice segment.

[0084] It should be understood that, Figure 6 The specific parameter values included in each layer of the VAD model shown can be set according to actual conditions, and the present application does not limit it. Meanwhile, in some examples, the existing VAD model can be directly used to determine the human voice segment, and the specific implementation process can refer to the existing manner, and the present application will not be repeated.

[0085] S503, performing feature extraction on the human voice segment to determine an embedding feature vector of the human voice segment.

[0086] When the terminal device determines at least one vocal fragment through VAD on the audio data at S502, feature extraction can be performed on the vocal fragment to determine the embedding feature vector corresponding to the vocal fragment. In some examples, feature extraction can be performed on each vocal fragment to determine the embedding feature vector corresponding to each vocal fragment.

[0087] In one example, when the terminal device determines which parts of the audio data contain vocals through VAD at S502, the audio data containing vocals can be input into the embedding model for feature extraction, i.e., the embedding model is used to perform feature extraction on the vocal fragment. Figure 7 An embedding model is shown, which includes multiple hidden layers. In one example, the number of hidden layers can be 4. In some examples, the 4 hidden layers in the embedding model can be LSTM layers. Of course, in other examples, CNN layers or the like can be selected, and the specific settings of each hidden layer can be adjusted according to actual conditions, or existing embedding models can be used. The parameters in each hidden layer can also be set according to actual conditions, which are not limited by the present application.

[0088] The terminal device extracts the embedding feature vector corresponding to the vocal fragment through the embedding model. In one example, the terminal device takes the result of the last hidden layer of the embedding model as the embedding feature vector, which can be a d-vector, for example. It can be understood that the embedding model usually includes a fully connected layer, and some embedding models also include a softmax layer. The embedding feature vector involved in the present application does not need the output of the last fully connected layer or the softmax layer, but uses the calculation result of the last hidden layer before the fully connected layer or the softmax layer as the output, i.e., the calculation result of the last hidden layer is the d-vector.

[0089] Of course, in some examples, the terminal device can also include step S507 before performing feature extraction on the vocal fragment.

[0090] S507, for the vocal fragment, determining whether the signal-to-noise ratio of the vocal fragment is greater than a signal-to-noise ratio threshold.

[0091] After the terminal device determines at least one voice segment through the VAD at S502, the terminal device can further determine a signal-to-noise ratio of the voice segment. Then, the terminal device compares the signal-to-noise ratio of the voice segment with a preset signal-to-noise ratio threshold. When the signal-to-noise ratio of the voice segment is greater than the preset signal-to-noise ratio threshold, the terminal device performs S503. Otherwise, when the signal-to-noise ratio of the voice segment is less than or equal to the preset signal-to-noise ratio threshold, the terminal device performs S506. In some examples, the terminal device can determine the signal-to-noise ratio of each voice segment.

[0092] In one example, for a voice segment, the terminal device can select a segment of a preset time length in the voice segment as a signal segment. Meanwhile, the terminal device can determine a segment of silence adjacent to the voice segment and select a segment of a preset time length in the segment of silence as a noise segment. Then, the terminal device determines a signal-to-noise ratio (SNR) of the voice segment in combination with the signal segment and the noise segment. Since the terminal device can accurately identify the voice segment with voice through the VAD, the part of the audio data without voice can be referred to as a segment of silence, and the segment of silence can be considered to contain only noise. The specific calculation process of the SNR can refer to the existing manner, and details are not described herein.

[0093] In some examples, the signal segment can be a segment of a preset time length at the beginning of the voice segment. For example, if the preset time length is set to 320 ms, the terminal device can select the first 320 ms of the voice segment as the signal segment. Of course, in other examples, the terminal device can select any continuous segment of a preset time length in the voice segment, which is not limited herein. Generally, the segment of silence can be a segment of silence adjacent to the voice segment before the voice segment. For example, Figure 8 An audio signal diagram is shown. As can be seen, the upper half of the signal is the original audio signal, i.e., ① in the figure, which can be an audio signal collected by a microphone. The black-and-white alternating area in the lower half represents the voice segment identified by the terminal device through the VAD, i.e., ② in the figure. The first 320 ms of the voice segment can be selected as the signal segment, and 320 ms can be selected from the segment of silence adjacent to the signal segment as the noise segment. Figure 8 The noise segment in the above example is selected as the 320 ms segment adjacent to the voice segment before the voice segment. Of course, in other examples, the terminal device can also select the 320 ms segment adjacent to the voice segment after the voice segment. The specific selection of the noise segment can be selected according to the preset rules, which are not limited herein. In some cases, if the noise segment cannot be obtained according to the preset rules, the terminal device can not perform S507, i.e., skip S507 and directly perform S503.

[0094] It can be understood that the purpose of S507 is to determine whether the human voice in the human voice segment is clear. When the signal-to-noise ratio is greater than the signal-to-noise ratio threshold, it can be considered that the signal in the human voice segment is strong and the interference is weak, and therefore the human voice segment can be subjected to subsequent operations, so as to ensure that the registered information is more accurate when the subsequent speaker is registered. When the signal-to-noise ratio is less than or equal to the signal-to-noise ratio threshold, it can be considered that the signal in the human voice segment is weak and the interference is strong, and therefore the human voice segment can be directly subjected to ASR for command recognition. This step ensures that the registered information is more accurate when the subsequent speaker is registered, which facilitates accurate identification of multiple different speakers in later use.

[0095] After the terminal device performs S503, S504 can be continued.

[0096] S504, the embedded feature vector is compared with the data in the feature vector database in sequence.

[0097] The terminal device obtains the embedded feature vector corresponding to the human voice segment, i.e., the d-vector, in S503. The d-vector can be compared with all data in the feature vector database in sequence. The feature vector database stores d-vectors corresponding to different speakers. It can be understood that each piece of data in the feature vector database corresponds to a d-vector for representing a speaker.

[0098] In one example, the similarity can be calculated in the manner of formula 1.

[0099]

[0100] Wherein n' represents the length of the feature vector, i.e., the length of the d-vector. i represents the i-th feature in the feature vector. S(cosθ) represents the cosine distance between two feature vectors, i.e., the similarity. x and y represent different d-vectors.

[0101] Of course, in other examples, the similarity can also be calculated in the manner of Euclidean distance, Manhattan distance, etc., which is not limited in the present application.

[0102] S505, if the similarity between the embedded feature vector and each piece of data in the feature vector database is less than the similarity threshold, the embedded feature vector is stored in the feature vector database.

[0103] ​When the terminal device calculates the similarity between the embedding feature vector corresponding to the human voice segment and each data in the feature vector database through formula 1 in S504 respectively. When the similarity between the embedding feature vector corresponding to the human voice segment and each data in the feature vector database is less than the pre-set similarity threshold, it can be considered that the embedding feature vector corresponding to the human voice segment is not matched successfully with any data in the feature vector database. Then, the embedding feature vector corresponding to the human voice segment can be stored in the feature vector database. That is, the d-vector corresponding to the human voice segment is stored in the feature vector database, thereby completing the registration of a speaker. Then, S506 can be continuously executed.

[0104] Of course, in some examples, if the similarity between the embedding feature vector corresponding to the human voice segment and a certain data in the feature vector database is greater than or equal to the similarity threshold, it can be considered that the human voice segment is matched successfully with the data in the feature vector database. At this time, the embedding feature vector corresponding to the human voice segment does not need to be stored, and S506 can be directly executed.

[0105] In yet some examples, for each data in the feature vector database, a parameter can be correspondingly set, which is used to represent the number of times that the data is matched successfully. The parameter can be referred to as the number of successful matches. Whenever the embedding feature vector of the human voice segment is greater than or equal to the similarity threshold, it is considered that the data is matched successfully, and then the number of successful matches of the data can be incremented by one. The terminal device can sort all data in the feature vector database from large to small according to the number of successful matches. The terminal device can be pre-configured to execute S506 only when the number of successful matches of the data is the top n in the above arrangement. Otherwise, the process can be directly ended. Wherein, n is a positive integer. The process is mainly used to execute the corresponding voice instruction only when part of the speakers are identified.

[0106] For example, in a vehicle-mounted scene, the number of n can be set to 2, which is used to represent two commonly used speakers. The terminal device only executes the corresponding instruction operation when the voice of the top two commonly used speakers is identified. Of course, the process can be dynamically changed according to the process used by the user, for example, when the number of successful matches of a certain speaker rises to the top 2, it is equivalent to completing the dynamic update of the commonly used speaker. Of course, the number of n can be arbitrarily set according to the actual situation. For example, it can be pre-configured or set by the user, which is not limited in the present application.

[0107] S506, performing ASR on the human voice segment.

[0108] The terminal device can perform ASR on the corresponding voice segment, so as to execute the corresponding instruction in the voice segment. For example, the voice segment is recognized and converted into text, matched with a preset instruction keyword, and the corresponding instruction is executed. Of course, the specific process of performing ASR can refer to the existing mode, and the present application will not be repeated here.

[0109] In some examples, the VAD model and the embedding model involved in the above can be pre-trained. For example, before S501, the VAD model and the embedding model can be trained. In one example, the VAD model and the embedding model can be trained using a training data set containing more than 10,000 audio data. Of course, each data in the training data set is marked with the start and end time of the real voice, so as to supervise the training of the VAD model. For the embedding model, which is mainly used to more effectively divide the characteristics of the speaker, an unsupervised training method can be used for training by clustering features. In one example, the loss function for training the VAD model can use a cross-entropy loss function, and for training the embedding model, a Sinkhorn distance matrix can be used as a loss function, until the VAD model and the embedding model converge.

[0110] Figure 9 Another speech recognition method flowchart provided by the embodiment of the present application.

[0111] As Figure 9 shown, another implementation of S505 is provided. In some examples, after S504, the following steps can be further included:

[0112] S901, if the similarity between the embedding feature vector and each item of data in the feature vector database is less than the similarity threshold, the voice segment corresponding to the embedding feature vector is input into the instruction detection model for detection.

[0113] The terminal device determines that if the similarity between the embedding feature vector and each item of data in the feature vector database is less than the similarity threshold, the voice segment corresponding to the embedding feature vector can be input into the instruction detection model for detection. In some examples, the instruction detection model can be a spoken language understanding (SLU) model.

[0114] It can be understood that the SLU is performed on the human voice segment to further determine whether the human voice segment contains instructions. Thus, it is avoided to store the human voice segment not containing instructions, and resource waste is reduced. Of course, the role of the SLU is only to identify whether the human voice segment contains instructions, and thus the output result of the SLU is only yes or no, for example, 1 (indicating that there is an instruction) or 0 (indicating that there is no instruction), and the specific instructions contained cannot be determined.

[0115] S902, determining whether the human voice segment contains instructions.

[0116] The terminal device determines whether the human voice segment detected by the SLU contains instructions. If it is determined that the human voice segment contains instructions, S903 is performed; if it is determined that the human voice segment does not contain instructions, S501 is performed or the process is directly ended.

[0117] S903, storing the embedding feature vector into the feature vector database.

[0118] When the terminal device determines that the human voice segment contains instructions, the embedding feature vector corresponding to the human voice segment can be stored into the feature vector database.

[0119] It can be understood that in some examples, the above-mentioned Figures 4 to 9 If the scheme in the above-mentioned is implemented by multiple terminal devices, for example, two terminal devices, such as a vehicle-mounted device and a terminal device that is not vehicle-mounted. The vehicle-mounted device and the terminal device that is not vehicle-mounted can be connected through a wired or wireless manner, and then one of the devices can obtain the audio data collected by the microphone and transmit the audio data to the other device for subsequent operations. Of course, the specific number of the multiple terminal devices is not limited in the present application. Meanwhile, how to allocate the multiple devices to perform operations is also not limited in the present application.

[0120] Figure 10 A schematic diagram of a terminal device provided by an embodiment of the present application.

[0121] As shown in Figure 10 , the present application further provides a terminal device 1000. The terminal device 1000 can be the terminal device 100 in the above-mentioned Figure 2 , and the terminal device mentioned in Figures 4 to 9 . The terminal device 1000 can include a processor 1010, an external memory interface 1020, an internal memory 1021, a universal serial bus (USB) interface 1030, a charging management module 1040, a power management module 1041, a battery 1042, an antenna 1, an antenna 2, a mobile communication module 1050, a wireless communication module 1060, a microphone 1070, and a display screen 1080, etc.

[0122] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the terminal device 1000. The terminal device 1000 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0123] The processor 1010 can be a processor of an architecture such as advanced reduced instruction set computing machines (ARM), X86, microprocessor without interlocked piped stages (MIPS), etc. The processor 1010 can include one or more processing units, such as an application processor (AP), a modem processor, a GPU, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0124] The controller can generate operation control signals according to instruction operation codes and timing signals to complete the control of fetching and executing instructions.

[0125] The processor 1010 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 1010 is a cache memory. The memory can store instructions or data that have just been used or are repeatedly used by the processor 1010. If the processor 1010 needs to use the instructions or data again, it can directly call them from the memory. This avoids repeated access and reduces the waiting time of the processor 1010, thereby improving the efficiency of the system.

[0126] In some embodiments, the processor 1010 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0127] It can be understood that the operating system of the terminal device of the present application is stored in the memory, and the processor 1010 realizes the above Figures 4 to 9 described method by calling the operating system stored in the memory.

[0128] The charging management module 1040 is configured to receive a charging input from a charger. The charger can be a wireless charger or a wired charger.

[0129] In some wired charging embodiments, the charging management module 1040 can receive a charging input of a wired charger through the USB interface 1030. In some wireless charging embodiments, the charging management module 1040 can receive a wireless charging input through a wireless charging coil of the terminal device 1000. The charging management module 1040 can charge the battery 1042 while also supplying power to the terminal device 1000 through the power management module 1041.

[0130] The wireless communication function of the terminal device 1000 can be realized through the antenna 1, the antenna 2, the mobile communication module 1050, the wireless communication module 1060, the modem processor, and the baseband processor, etc.

[0131] The mobile communication module 1050 can provide a solution including 2G / 3G / 4G / 5G / 6G wireless communication applied to the terminal device 1000. The wireless communication module 1060 can provide a solution including WLAN, BT, GNSS, FM, NFC, zigbee, and IR wireless communication applied to the terminal device 1000. The WLAN may, for example, be a Wi-Fi network.

[0132] The microphone 1070, also known as a "microphone", "microphone", is used to convert a sound signal into an electrical signal. When acquiring an audio signal, a user can speak by approaching the microphone 1070 with his mouth, and input a sound signal into the microphone 1070. The terminal device 1000 can be provided with at least one microphone 1070. In other embodiments, the terminal device 1000 can be provided with two microphones 1070, in addition to collecting sound signals, it can also realize the function of noise reduction. In other embodiments, the terminal device 1000 can also be provided with three, four or more microphones 1070, to realize the functions of collecting sound signals, noise reduction, and identifying sound sources, and realizing directional recording functions, etc.

[0133] The terminal device 1000 realizes the display function through the GPU, the display screen 1080, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 1080 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 1010 can include one or more GPUs that execute program instructions to generate or change display information.

[0134] The display screen 1080 is used to display images, videos, etc. The display screen 1080 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, and a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 1000 can include 1 or M display screens 1080, where M is a positive integer greater than or equal to 2.

[0135] The external memory interface 1020 can be used to connect an external memory card, such as a Micro SD card, to realize the function of expanding the storage capacity of the terminal device 1000. The external memory card communicates with the processor 1010 through the external memory interface 1020 to realize the function of data storage.

[0136] The internal memory 1021 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 1021 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required by a function, and the like. The data storage area can store data created during use of the terminal device 1000, and the like. In addition, the internal memory 1021 can include a high-speed random access memory, and can further include a nonvolatile memory such as at least one of a magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like. The processor 1010 executes various function applications and data processing of the terminal device 1000 by running instructions stored in the internal memory 1021 and / or instructions stored in a memory disposed in the processor.

[0137] The terminal device 1000 provided in the present application can implement any one of the methods described in the above Figures 4 to 9 The specific implementation manner can refer to the corresponding description of the Figures 4 to 9 The specific implementation manner can refer to the corresponding description of the

[0138] The present application divides at least one vocal fragment by performing VAD on the acquired audio data. Then the vocal fragment is feature extracted and compared one by one with the pre-stored data in the feature vector database. When the vocal fragment fails to match all the data in the feature vector database, the embedding feature vector of the vocal fragment is stored, realizing dynamic registration of speaker information. The present application determines the signal-to-noise ratio of the audio data, and only extracts features from the audio data with high signal-to-noise ratio, so that the dynamically registered speaker information can significantly improve the recognition effect of the vehicle-mounted ASR in a high-noise scene. The present application can solve the problem that in vehicle-mounted voice recognition, without pre-recording a voiceprint, multiple voiceprints can be automatically recognized and registered to the vehicle-mounted voice system, so that in the recognition scene, the multiple registered speakers can be accurately recognized, effectively improving the voice recognition accuracy.

[0139] Further, the present application evaluates the SNR of the recording, and designs a VAD model and an embedding model structure. Through the above structure, the real vehicle-mounted speaker (i.e. speaker) can be dynamically voiceprint detected and recognized, and the voiceprint thereof can be saved. Thus, in subsequent voice endpoint detection, the saved real vehicle-mounted speaker (i.e. speaker) can be accurately recognized, and corresponding ASR can be performed, avoiding the disturbance caused by the voice recognition of part of the un-saved speakers, and improving the accuracy and performance of voice recognition.

[0140] The present application provides a voice endpoint detection method for dynamically registering user voiceprints, so that the user can automatically register the voiceprint without pre-recording the voiceprint, improving the user experience of voice recognition.

[0141] It can be understood that the VAD model and the embedding model involved in the present application are suitable for end-to-end use, which ensures the end-to-end calculation speed while having good recognition effect.

[0142] Those skilled in the art should further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0143] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by a program instructing a processor, and the program can be stored in a computer readable storage medium. The storage medium is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid state disk, magnetic tape, floppy disk, optical disc and any combination thereof.

[0144] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A voice recognition method, characterized by, The method is applied to a terminal device, and the method comprises: acquiring audio data collected by a microphone; performing voice endpoint detection on the audio data to determine at least one vocal segment; determining a signal-to-noise ratio of the vocal segment in combination with a mute segment adjacent to the vocal segment, wherein the mute segment is obtained through the voice endpoint detection on the audio data or is pre-configured; if the signal-to-noise ratio is greater than a preset signal-to-noise ratio threshold, performing feature extraction on the vocal segment to determine an embedding feature vector of the vocal segment; sequentially comparing the embedding feature vector with data in a feature vector database in terms of similarity, and if the similarity between the embedding feature vector and each item of data in the feature vector database is less than a similarity threshold, storing the embedding feature vector into the feature vector database, and performing automatic speech recognition on the vocal segment corresponding to the embedding feature vector to execute a corresponding instruction.

2. The method of claim 1, wherein, The voice endpoint detection on the audio data to determine at least one vocal segment comprises: inputting the audio data into a voice endpoint detection model comprising at least one long short-term memory artificial neural network (LSTM) for detection to determine at least one vocal segment.

3. The method of claim 2, wherein, The voice endpoint detection model comprising at least one long short-term memory artificial neural network (LSTM) comprises at least 6 one-dimensional convolutional neural networks and at least 3 one-way LSTMs with 128 nodes.

4. The method of claim 1, wherein, The determination of the embedding feature vector of the vocal segment comprises: inputting the vocal segment into an embedding feature extraction model comprising multiple hidden layers for feature extraction to determine the embedding feature vector of the vocal segment.

5. The method of claim 4, wherein, The embedding feature extraction model comprises at least 4 hidden layers.

6. The method of claim 5, wherein, Before storing the embedding feature vector into the feature vector database, the method further comprises: inputting the vocal segment corresponding to the embedding feature vector into a detection instruction model for detection; if the vocal segment corresponding to the embedding feature vector is detected to contain an instruction, storing the embedding feature vector into the feature vector database.

7. The method of claim 6, wherein, The method further comprises: if the similarity between the embedding feature vector and any item of data in the feature vector database is greater than or equal to a similarity threshold, performing automatic speech recognition on the vocal segment corresponding to the embedding feature vector and executing a corresponding instruction according to the speech recognition result.

8. The method according to any one of claims 1 to 7, characterized in that, Each item of data in the feature vector database corresponds to a matching success number, and each item of data in the feature vector database is sequentially sorted from large to small according to the matching success number. The method further comprises: if the similarity between the embedding feature vector and any item of data in the feature vector database is greater than or equal to a similarity threshold, determining whether the data is located in the top n of the sorting, wherein n is a positive integer; when the data belongs to the top n of the sorting, performing automatic speech recognition on the vocal segment corresponding to the embedding feature vector and executing a corresponding instruction according to the speech recognition result.

9. The method of claim 8, wherein, The method further comprises: If the similarity between the embedded feature vector and any data in the feature vector database is greater than or equal to a similarity threshold, the number of successful matches corresponding to the data is incremented by one, and each data in the feature vector database is again sorted in descending order.

10. The method of claim 9, wherein, The method further comprises: If the signal-to-noise ratio of the vocal segment is less than or equal to the signal-to-noise ratio threshold, the automatic speech recognition is performed on the vocal segment, and corresponding instructions are executed according to the speech recognition result.

11. A terminal device for speech recognition, characterized by The terminal device comprises: The processor is coupled with the memory, and reads and executes instructions stored in the memory; When the processor executes the instructions, the processor is configured to execute the method of any one of claims 1-10.

12. A computer-readable storage medium having stored therein instructions, the computer-readable storage medium comprising: When the instructions are executed on the terminal device, the terminal device is configured to execute the method of any one of claims 1-10.

13. A computer comprising instructions which, when executed, cause the computer to perform the method of any one of claims 1-10.

Images