Voice wake-up method, device, electronic device and computer-readable storage medium

Through a neural network model, the problem of redundant model parameters and slow wake-up speed is solved in the prior art, and a more efficient voice wake-up process is achieved.

CN115312040BActive Publication Date: 2024-11-19CHINA TELECOM CORP LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202210946385.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-08
Publication Date
2024-11-19
Estimated Expiration
2042-08-08

AI Technical Summary

Technical Problem

In the prior art, the voice wake-up model needs to train two tasks: wake-up word matching and voiceprint recognition, resulting in redundant model parameters, excessive memory usage, and slow wake-up speed.

Method used

The pre-trained neural network model is used to process the spectral diagram of the speech recognition, extract the voiceprint feature vector and wake up words, and query the registered feature vector through the preset correspondence table, calculate the vector similarity to determine whether to wake up speech. A neural network model is used to simultaneously realize voiceprint recognition and speech wake word matching.

Benefits of technology

The model training process is simplified, parameter redundancy and memory usage are reduced, and wake-up speed and accuracy are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115312040B_ABST
    Figure CN115312040B_ABST
Patent Text Reader

Abstract

The present disclosure provides a voice wake-up method, device, electronic device and computer-readable storage medium, which relate to the field of speech processing technology. The method comprises: processing the spectrogram of the speech to be recognized according to a pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized; querying the registered wake-up word label corresponding to the wake-up word according to the preset correspondence table, and obtaining the registered feature vector according to the registered wake-up word label; calculating the vector similarity according to the voiceprint feature vector and the registered feature vector; and performing voice wake-up when the vector similarity is greater than a preset threshold. Using a neural network model to simultaneously realize the two tasks of voiceprint recognition and speech wake-up word matching solves the problem of parameter redundancy and excessive memory usage caused by constructing models for the two tasks separately. The disclosed process is more concise and the wake-up speed is faster.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] Voice wake-up means that the user wakes up the electronic device by saying a wake-up word, so that the electronic device enters a state of waiting for voice commands or executing voice commands.

[0003] In the related art, the model training and testing of fixed wake-up word matching is performed on the input voice, and the model training and certification of voiceprint recognition is performed on the input voice at the same time. The intersection of the two model test results is logically taken to output the result of whether to wake up. The method of training models for the two subtasks of wake-up word matching and voiceprint recognition separately leads to redundant model parameters, excessive memory usage, and slow wake-up speed.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a voice wake-up method, device, electronic device and computer-readable storage medium, which at least to some extent overcome the problems of model parameter redundancy, excessive memory usage and slow wake-up speed.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a voice wake-up method is provided, the method comprising:

[0008] Processing the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized;

[0009] According to a preset correspondence table, query the registered wake-up word label corresponding to the wake-up word, and obtain a registered feature vector according to the registered wake-up word label;

[0010] Calculating vector similarity based on the voiceprint feature vector and the registration feature vector;

[0011] When the vector similarity is greater than a preset threshold, voice wake-up is performed.

[0012] In one embodiment of the present disclosure, before the spectrogram of the speech to be recognized is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes:

[0013] Get the spectrogram of the registered speech;

[0014] Processing the spectrogram of the registration speech according to a pre-trained neural network model to obtain a registration feature vector and a registration wake-up word label;

[0015] The corresponding relationship table is constructed according to the registered feature vector and the registered wake-up word label.

[0016] In one embodiment of the present disclosure, before the spectrogram of the speech to be recognized is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes:

[0017] Remove the silent sound in the speech to be recognized to obtain the effective speech to be recognized;

[0018] When the effective audio duration of the effective speech to be recognized is greater than or equal to a preset duration threshold, the effective speech to be recognized is processed to obtain a spectrogram of the speech to be recognized.

[0019] In one embodiment of the present disclosure, the method further includes:

[0020] When the effective audio duration of the effective voice to be recognized is less than a preset duration threshold, voice wake-up fails.

[0021] In one embodiment of the present disclosure, before the spectrogram of the speech to be recognized is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes:

[0022] Segment the speech to be recognized into a plurality of speech files of a first duration, wherein the frame length of the speech files is a second duration and the frame shift is a third duration;

[0023] The speech file is converted into a spectrogram.

[0024] In one embodiment of the present disclosure, before the spectrogram of the speech to be recognized is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes:

[0025] Processing the acquired speech data set to obtain a spectrogram of the speech data set;

[0026] An initial neural network model is trained according to the spectrogram and corresponding labels of the speech data set, wherein the initial neural network model includes a first target network structure and a second target network structure, and if a preset first training stop condition is met, the initial neural network model is used as the target neural network model.

[0027] In one embodiment of the present disclosure, the spectrogram of the speech data set includes a training data set for model training and a test data set for model verification, and the training data set and the test data set are divided according to a preset ratio.

[0028] In one embodiment of the present disclosure, the spectrogram of the speech dataset also includes a verification dataset;

[0029] The first target network structure and the second target network structure are obtained in the following manner:

[0030] Construct training loss function based on multi-gradient descent algorithm MGDA algorithm;

[0031] According to the training data set, the initial network structure with network structure parameters and the training loss function, the network structure parameters of the network structure are fixed, and the network operation weights are obtained through training;

[0032] Construct verification loss function based on MGDA algorithm;

[0033] The network operation weights are fixed, and new network structure parameters are obtained through training according to the verification data set and the verification loss function to obtain a process network structure;

[0034] Calculate the speech wake-up word recognition and voiceprint recognition gradients, obtain updated shared parameters in the common gradient descent direction, retrain until a preset second training stop condition is met, and obtain a target network structure, which includes one of the first target network structure and the second target network structure.

[0035] In one embodiment of the present disclosure, the first training stop condition includes that the training loss is less than a first threshold or the number of training rounds is greater than a second threshold.

[0036] In one embodiment of the present disclosure, the neural network model includes an input node, multiple intermediate nodes and two output nodes, the intermediate nodes include at least two first target network structures and at least one second target network structure, the structures of the at least two first target network structures are the same, and the input of an intermediate node is the output of the first two adjacent intermediate nodes.

[0037] According to another aspect of the present disclosure, a voice wake-up device is provided, the device comprising:

[0038] A processing module, used to process the spectrogram of the speech to be recognized according to a pre-trained neural network model to obtain a voiceprint feature vector and a wake-up word of the speech to be recognized;

[0039] A query module, configured to query a registered wake-up word label corresponding to the wake-up word according to a preset correspondence table, and obtain a registered feature vector according to the registered wake-up word label;

[0040] A vector similarity calculation module, used to calculate vector similarity based on the voiceprint feature vector and the registration feature vector;

[0041] The voice wake-up module is used to perform voice wake-up when the vector similarity is greater than a preset threshold.

[0042] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0043] Processor; and

[0044] A memory, configured to store executable instructions of the processor;

[0045] Wherein, the processor is configured to perform the above-mentioned voice wake-up method by executing the executable instructions.

[0046] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned voice wake-up method is implemented.

[0047] According to another aspect of the present disclosure, a computer program product is provided, the computer program product comprising a computer program or computer instructions, the computer program or the computer instructions being loaded and executed by a processor so that a computer implements any of the above-mentioned voice wake-up methods.

[0048] The voice wake-up method, device, electronic device and computer-readable storage medium provided in the embodiments of the present disclosure input the spectrogram of the speech to be recognized into a pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized, and use one neural network model to simultaneously realize the two tasks of voiceprint recognition and voice wake-up word matching, thereby solving the problems of parameter redundancy and excessive memory usage caused by constructing models separately for the two tasks. The process disclosed in the present disclosure is more concise and the wake-up speed is faster.

[0049] Furthermore, according to the preset correspondence table, the registered wake-up word label corresponding to the wake-up word is queried, and the registered feature vector is obtained according to the registered wake-up word label; the vector similarity is calculated according to the voiceprint feature vector and the registered feature vector; when the vector similarity is greater than the preset threshold, voice wake-up is performed, and whether to wake up is determined by the calculated vector similarity and the preset threshold. The calculation result is accurate, which improves the accuracy of wake-up.

[0050] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0052] Figure 1 A schematic diagram showing a system architecture in an embodiment of the present disclosure;

[0053] Figure 2 A flow chart of a voice wake-up method in an embodiment of the present disclosure is shown;

[0054] Figure 3 A user registration flow chart of a voice wake-up method according to an embodiment of the present disclosure is shown;

[0055] Figure 4 A neural network model training flow chart of a voice wake-up method in an embodiment of the present disclosure is shown;

[0056] Figure 5 A flow chart of generating the first target network structure and the second target network structure in an embodiment of the present disclosure is shown;

[0057] Figure 6 A structural diagram of a neural network model in an embodiment of the present disclosure is shown;

[0058] Figure 7 A schematic diagram of a voice wake-up device in an embodiment of the present disclosure is shown

[0059] Figure 8 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0060] Fig. 9 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0061] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0062] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0063] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0064] Figure 1 A schematic diagram showing an exemplary system architecture that can be applied to a voice wake-up method or a voice wake-up device according to an embodiment of the present disclosure is shown.

[0065] like Figure 1 As shown, the system architecture includes a terminal 101, a network 102 and a server 103; the network 102 is used to provide a medium for a communication link between the terminal 101 and the server 103, and can be a wired network or a wireless network.

[0066] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is typically the Internet, but may also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. may be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.

[0067] Figure 1Any terminal 101 shown can be used as a remote terminal or a relay terminal. Communication based on proximity service can be established between the remote terminal and the relay terminal. The remote terminal can access the network directly or through the relay terminal.

[0068] Optionally, the terminal 101 in the embodiment of the present disclosure may also be referred to as UE (User Equipment). In a specific implementation, the terminal 101 may be a terminal-side device such as a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer), a personal digital assistant (Personal Digital Assistant, PDA), a mobile Internet device (Mobile Internet Device, MID), a wearable device (Wearable Device) or a vehicle-mounted device.

[0069] Optionally, the client of the application installed in different terminals 101 is the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client, etc.

[0070] The server 103 may be a server that provides various services, such as a background management server that provides support for the device operated by the user using the terminal 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal.

[0071] Optionally, the server 103 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 103 and the terminal 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0072] In the related art, the model training and testing of fixed wake-up word matching for the input voice are performed. For the model training process, the voiceprint recognition and voice wake-up tasks are trained separately, resulting in clumsy model training and poor generalization ability. In addition, when performing multi-task training such as voiceprint recognition and voice wake-up word matching, considering that the voice wake-up word matching task is to find common factors from the word signals of different people, it blurs the sound features and emphasizes the text wake-up words during training, while voiceprint recognition is a voice parameter that reflects the physiological and behavioral characteristics of the speaker in the voice waveform, and it does not pay attention to the semantics of the voice signal. This task extracts personal voiceprint features from the voice signal and mines out the personality factors contained in the voice signal. During training, the sound features are emphasized to extract features. Therefore, there is competition in the optimization process of these two tasks. It is difficult to use an artificially designed neural network model to consider the characteristics of two different tasks at the same time. In addition, according to the two tasks of wake-up word matching and voiceprint recognition, separate model training is performed, and then logical judgment is performed to decide whether to wake up, resulting in more memory usage and more resource consumption.

[0073] In the embodiment of the present disclosure, a voice wake-up method is provided. The terminal 101 obtains the voice to be recognized and sends the obtained voice to be recognized to the server 103. The server 103 processes the obtained voice to be recognized to obtain the spectrogram of the voice to be recognized; the spectrogram is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the voice to be recognized; according to the preset correspondence table, the registered wake-up word label corresponding to the wake-up word is queried, and the registered feature vector is obtained according to the registered wake-up word label; the vector similarity is calculated according to the voiceprint feature vector and the registered feature vector; when the vector similarity is greater than the preset threshold, voice wake-up is performed. This method can be applied to the voiceprint recognition voice wake-up system scenario, for example, through voice wake-up, the electronic device can place an order for shopping, through voice wake-up, the electronic device can control the switch of home appliances, etc. The present disclosure uses a neural network model to simultaneously complete the two tasks of wake-up word matching and voiceprint recognition, and the wake-up speed is faster, which solves the problem of parameter redundancy and excessive memory occupation caused by building models for the two tasks separately, and the process of the present disclosure is more concise.

[0074] Those skilled in the art will know that Figure 1 The number of terminals 101, networks 102 and servers 103 is only illustrative, and any number of terminals 101, networks 102 and servers 103 may be provided according to actual needs. The embodiments of the present disclosure are not limited to this.

[0075] Under the above system architecture, the present disclosure provides a voice wake-up method, which can be executed by any electronic device with computing and processing capabilities. In some embodiments, the voice wake-up method provided in the present disclosure can be performed by Figure 1 Executed in the server 103 shown.

[0076] Figure 2 A flow chart of a voice wake-up method in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the voice wake-up method provided in the embodiment of the present disclosure includes the following S201 to S204.

[0077] S201. Process the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized.

[0078] Before outputting the spectrogram of the speech to be recognized to the pre-trained neural network model, the acquired speech to be recognized needs to be processed to obtain the spectrogram of the speech to be recognized.

[0079] Among them, the spectrogram is a spectrum analysis view, the terminal collects the speech to be recognized, processes the speech to be recognized after collection or sends the speech to be recognized to the server, the server processes the speech to be recognized, performs voice liveness detection on the speech to be recognized, removes the silent sound that is meaningless to the recognition in the speech to be recognized, sets the frame length of the audio of the speech to be recognized to 25 milliseconds and the frame shift to 10 milliseconds, and processes it into a spectrogram. The disclosed embodiment does not limit the method of how to process the speech to be recognized to obtain a spectrogram, and any method that can realize the processing of the speech to be recognized to obtain a spectrogram can be applied here. For example, the speech to be recognized is converted into a spectrogram through the spectrogram function in MATLAB (matrix & laboratory, mathematical software).

[0080] The voiceprint feature vector is a feature description vector of the speech to be recognized, which is used to distinguish different users. For example, registered users and unregistered users can be distinguished based on the voiceprint feature vector. The wake-up word is a voice command that triggers the wake-up function of the interactive device. Voice wake-up refers to the process in which the user restores the electronic device from the standby state to the normal working state by saying the wake-up word with the same label as the registered wake-up word. The feature extraction of the spectrogram is performed through the pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized.

[0081] S202: According to a preset correspondence table, query the registered wake-up word label corresponding to the wake-up word, and obtain a registered feature vector according to the registered wake-up word label.

[0082] The registered user inputs the spectrogram of the registered speech into the pre-trained neural network model to obtain the registered wake-up word label and the registered feature vector, and stores the registered wake-up word label and the registered feature vector in a correspondence table pre-set on the server. The registered wake-up word label corresponding to the wake-up word is queried through the correspondence table. If there is a registered wake-up word label corresponding to the wake-up word, the corresponding registered feature vector is found through the registered wake-up word label.

[0083] S203. Calculate vector similarity based on the voiceprint feature vector and the registration feature vector.

[0084] In this embodiment, the vector similarity is calculated by cosine similarity. The cosine similarity between the voiceprint feature vector of the speech to be recognized and the registered feature vector is calculated to obtain the vector similarity. The similarity between the voiceprint feature vector and the registered feature vector is measured by cosine similarity. The value range of vector similarity is from -1 to 1. When the vector similarity is 1, it means that the voiceprint feature vector and the registered feature vector are the same. When the vector similarity is -1, it means that the voiceprint feature vector and the registered feature vector are completely different. The closer the vector similarity is to 1, the higher the similarity between the voiceprint feature vector and the registered feature vector. In addition, other methods that can obtain the similarity of two vectors are also applicable.

[0085] S204: When the vector similarity is greater than a preset threshold, voice wake-up is performed.

[0086] Among them, the threshold is a preset value, for example, the threshold is 0.8. When the value of the vector similarity is greater than 0.8, it means that the similarity between the voiceprint feature vector and the registered feature vector is very high, and it is determined that the user who makes the voice to be recognized and the registered user are the same person. The electronic device executes the wake-up command to achieve voice wake-up.

[0087] The present invention utilizes a neural network model to simultaneously implement the two tasks of voiceprint recognition and voice wake-up word matching, solving the problems of parameter redundancy and excessive memory usage caused by constructing models for the two tasks separately. The process of the present invention is simpler and the wake-up speed is faster.

[0088] In one embodiment, before processing the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes:

[0089] Remove the silent sound in the speech to be recognized to obtain the effective speech to be recognized;

[0090] When the effective audio duration of the effective speech to be recognized is greater than or equal to a preset duration threshold, the effective speech to be recognized is processed to obtain a spectrogram of the speech to be recognized.

[0091] The embodiment of the present disclosure does not limit the value range of the time threshold. For example, the value range of the duration threshold is 1s-4s.

[0092] Exemplarily, voice liveness detection is performed on the speech to be recognized, and silent sounds that are meaningless to recognition in the speech to be recognized are removed to obtain valid speech to be recognized. When the effective audio duration of the valid speech to be recognized is greater than or equal to 2s, the valid speech to be recognized is processed to obtain a spectrogram of the speech to be recognized.

[0093] In one embodiment, the method further comprises:

[0094] When the effective audio duration of the effective speech to be recognized is less than the preset duration threshold, the voice wake-up fails.

[0095] For example, the duration threshold value ranges from 1s to 4s.

[0096] Exemplarily, when the effective audio duration of the effective voice to be recognized is less than 2 seconds, the voice wake-up fails.

[0097] It should be noted that the value range of the above-mentioned duration threshold is only an example provided to illustrate the embodiments of the present disclosure and should not be regarded as limiting the protection scope of the present disclosure.

[0098] In one embodiment, before processing the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes:

[0099] The speech to be recognized is divided into a plurality of speech files with a first duration, wherein the frame length of the speech files is a second duration, and the frame shift is a third duration;

[0100] Convert speech files into spectrograms.

[0101] Exemplarily, the speech to be recognized is divided into multiple speech files with a duration of 3 seconds, the speech files are converted into 360dpi*360dpi spectrograms, and the frame length and frame shift are set to 25 milliseconds and 10 milliseconds.

[0102] It should be noted that the values ​​of the first duration, the second duration and the third duration are merely examples provided to illustrate the embodiments of the present disclosure and should not be regarded as limiting the scope of protection of the present disclosure.

[0103] Figure 3 A user registration flow chart of a voice wake-up method according to an embodiment of the present disclosure is shown as follows: Figure 3As shown, before the spectrogram of the speech to be recognized is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized, the user registers, and the user registration steps of the voice wake-up method provided in the embodiment of the present disclosure include the following S301 to S303.

[0104] S301, obtaining a spectrogram of the registration speech.

[0105] In one embodiment, before obtaining the spectrogram of the registration speech, the method further includes:

[0106] Remove the silent sound in the registered voice to obtain the effective registered voice;

[0107] When the effective audio duration of the effective registration voice is greater than or equal to 2s, an operation of processing the acquired registration voice to obtain a spectrogram of the registration voice is performed.

[0108] The registration voice is divided into multiple registration voice files of 3s in length, and the registration voice files are converted into 360dpi*360dpi spectrograms. The frame length is set to 25 milliseconds and the frame shift is set to 10 milliseconds. The method for generating the spectrogram of the registration voice is the same as the method for generating the spectrogram of the speech to be recognized mentioned above. The specific generation method is shown in S201 and will not be repeated here.

[0109] S302: Process the spectrogram of the registration speech according to the pre-trained neural network model to obtain a registration feature vector and a registration wake-up word label.

[0110] The registered feature vector of the registered voice is used to distinguish registered users from unregistered users, and the registered wake-up word label is the voice command that triggers the wake-up function of the interactive device. The registered feature vector and the registered wake-up word label are obtained by extracting features from the spectrogram of the registered voice through a pre-trained neural network model.

[0111] S303: Construct a correspondence table according to the registered feature vectors and the registered wake-up word labels.

[0112] Exemplarily, the registered feature vectors and the registered wake-up word labels are stored in a registration library, and a correspondence table between the registered feature vectors and the registered wake-up word labels is constructed to facilitate querying the corresponding registered feature vectors through the registered wake-up word labels.

[0113] Figure 4 A neural network model training flow chart of a voice wake-up method in an embodiment of the present disclosure is shown as follows: Figure 4As shown, before the spectrogram of the speech to be recognized is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized, the initial neural network model needs to be trained. The neural network model training steps of a speech wake-up method include the following S401 to S402.

[0114] S401: Process the acquired speech data set to obtain a spectrogram of the speech data set.

[0115] It should be noted that the speech data set of the initial neural network model to be trained is first determined. For example, the speech data set is determined to be MobvoiHotwords (a speech data set). MobvoiHotwords is a corpus of wake-up words collected from commercial smart speakers of Mobvoi (an artificial intelligence voice platform), which consists of keyword and non-keyword speech.

[0116] In one embodiment, the spectrogram of the speech data set includes a training data set for model training and a test data set for model verification, and the training data set and the test data set are divided according to a preset ratio.

[0117] Exemplarily, there are 1,000 spectrograms in the speech data set, of which 400 spectrograms are training data sets and 300 spectrograms are testing data sets.

[0118] S402. Train an initial neural network model according to the spectrogram and corresponding labels of the speech data set, wherein the initial neural network model includes a first target network structure and a second target network structure. If a preset first training stop condition is met, the initial neural network model is used as the target neural network model.

[0119] Exemplarily, the initial neural network model is trained based on a multi-gradient descent algorithm, and a new initial neural network model is obtained according to the training data set;

[0120] When the new initial neural network model satisfies the preset first training stop condition, a process neural network model is obtained;

[0121] The process neural network model is verified by using a test data set, and if a preset third training stop condition is met, the process neural network model is determined to be a target neural network model;

[0122] If not, the process neural network model is determined as the initial neural network model and training continues.

[0123] It should be noted that the target neural network model can be a convolutional neural network model, and the first training stop condition includes that the training loss is less than the first threshold or the number of training rounds is greater than the second threshold, and the first threshold and the second threshold are preset in advance on the server. The third training stop condition includes that the training loss is less than the third threshold or the number of training rounds is greater than the fourth threshold, and the third threshold and the fourth threshold are preset in advance on the server. The label corresponds to the spectrogram of the speech dataset, which is the identity label of the user corresponding to the spectrogram of the speech dataset, and is used to distinguish the spectrogram of the speech dataset.

[0124] In one embodiment, the spectrogram of the speech data set includes a training data set for model training and a test data set and a verification data set for verifying the model. The training data set, the verification data set and the test data set are divided according to a preset ratio, and the spectrogram of the speech data set is shuffled before division. The training data set and the test data set are used for training the initial neural network model, and the training data set and the verification data set are used for training the initial network structure.

[0125] The training data set, the validation data set and the test data set can be divided according to a preset ratio of 4:4:1. For example, there are 900 spectrograms of the speech data set, of which 400 spectrograms of the speech data set are the training data set, 400 spectrograms of the speech data set are the validation data set, and the remaining 100 spectrograms of the speech data set are the test data set.

[0126] Figure 5 The flowchart of generating the first target network structure and the second target network structure in the embodiment of the present disclosure is shown as follows. Figure 5 As shown, the first target network structure and the second target network structure are obtained through the following methods S501 to S505.

[0127] S501. Construct a training loss function based on the MGDA algorithm.

[0128] S502, according to the training data set, the initial network structure with network structure parameters and the training loss function, the network structure parameters of the network structure are fixed, and the network operation weights are obtained through training.

[0129] S503. Construct a verification loss function based on the MGDA algorithm.

[0130] S504, fixing the network operation weights, and obtaining new network structure parameters through training according to the verification data set and the verification loss function to obtain the process network structure.

[0131] S505. Calculate the gradients of speech wake-up word recognition and voiceprint recognition, obtain updated shared parameters in the common gradient descent direction, and retrain until the preset second training stop condition is met to obtain a target network structure, which includes one of a first target network structure and a second target network structure.

[0132] The target network structure is obtained through DARTS (Differentiable Architecture Search). DARTS uses a Cell-Based network architecture search method, which is also divided into normal cell and reduction cell. After the search is completed, a complete network will be formed by splicing, that is, the initial neural network model.

[0133] The process of constructing a training loss function based on the MGDA algorithm is as follows:

[0134] The training data set is input into the initial network structure, and the training loss function is constructed using the MGDA algorithm. After calculating the gradient of the wake-up word recognition and voiceprint recognition tasks each time, the common gradient descent direction is obtained to update the shared parameters to obtain the optimal network operation weight. The optimal network operation weight is expressed as follows:

[0135] ω * (α) = argmin ω L train (ω, α) (Formula 1)

[0136] Among them, α is the network structure parameter, ω is the network operation weight, and ω * (α) represents the optimal network operation weight obtained by training under the current network structure parameter α, L train is the training loss function.

[0137] The process of constructing the verification loss function based on the MGDA algorithm is as follows:

[0138] When the network operation weights are determined, the verification data set is output to the initial network structure with the optimal network operation weights, and the verification loss function is constructed using the MGDA algorithm to obtain the optimal target network structure.

[0139] The optimal network structure parameters are expressed as follows:

[0140]

[0141] Among them, L val is the validation loss function.

[0142] The network operation weight ω and the network structure parameter α are optimized by alternately training through Formula 1 and Formula 2 until the preset second training stop condition is met to obtain the target network structure.

[0143] It should be noted that in the training data set, under the condition of fixed network structure parameter α, the optimal network operation parameter ω is obtained through training; in the validation data set, under the condition of fixed specific network operation parameter ω, the optimal network structure parameter α is obtained through training. The goal of DARTS is to solve a two-layer optimization problem: find the network structure parameters so that the optimal results are obtained in the validation data set, and find the optimal network operation parameters so that the target network structure obtains the optimal performance. Both the first target network structure and the second target network structure are obtained by the DARTS method.

[0144] The second training stop condition includes that the training loss is less than a fifth threshold or the number of training rounds is greater than a sixth threshold. Exemplarily, when the number of training rounds is greater than 3, the training ends.

[0145] The present invention uses the MGDA algorithm to construct a loss function in the search target network structure. After each calculation of the gradients of the wake-up word recognition and voiceprint recognition tasks, a common gradient descent direction is obtained to update the shared parameters, ensuring that the shared parameters will not appear in conflicting gradients, making the loss convergence of the voiceprint recognition and wake-up word recognition tasks smoother and improving the wake-up success rate.

[0146] In addition, the present invention fully considers the characteristics of voiceprint recognition tasks and wake-up word tasks, searches for a suitable network structure through automatic search architecture, and designs a loss function to optimize the process of training the model, further improving the wake-up success rate.

[0147] In one embodiment, the neural network model includes an input node, multiple intermediate nodes and two output nodes, the intermediate nodes include at least two first target network structures and at least one second target network structure, the structures of at least two first target network structures are the same, and the input of an intermediate node is the output of the first two adjacent intermediate nodes.

[0148] Exemplarily, N trained target network structures are stacked to construct the entire neural network model, where N is a positive integer greater than 5. The neural network model is a convolutional neural network model, and its convolutional neural network structure is as follows: Figure 6As shown in the figure, the input node input is used for the input of the spectrogram, the implementation function of the convolutional neural network model is Conv2d, and the BatchNorm2d function is used to normalize the data, so that the data will not cause instability in network performance due to excessive data. The first target network structure is normal cell, and the second target network structure is reduction cell. Normal cell and reduction cell share network structure parameters. Reduction cell is used at 1 / 3*N and 2 / 3*N to reduce the spatial resolution. AdaptiveAvgPool2d function is used for global pooling. After the global pooling process, the fully connected operation FC is performed, and then the voiceprint feature vector and wake-up word are output simultaneously.

[0149] For example, before processing the acquired speech to be recognized, it is necessary to build a neural network model and train the neural network model;

[0150] First, determine the speech dataset that needs to train the model. For example, determine that the speech dataset is MobvoiHotwords, select the speech data of M speakers with the most concentrated speaking time distribution in the speech dataset, where M is a positive integer, remove the silence in the speech, and cut the speech of each speaker into a speech file every 3 seconds. Then convert these 3-second speech files into 360dpi×360dpi spectrograms, set the frame length to 25 milliseconds, and the frame shift to 10 milliseconds.

[0151] The MGDA algorithm is used to construct the training loss function and the verification loss function to obtain the optimal network operation weight and the optimal network structure parameters. The target unit structure, network structure parameters and network operation weight are finally determined by iterative search through minimizing the loss function.

[0152] like Figure 6 As shown in the figure, N trained target unit architectures are stacked, reduction cells are used at 1 / 3*N and 2 / 3*N, and normal cells are used to stack the rest to build the entire convolutional neural network model. The output of the convolutional neural network model is two labels, namely the voiceprint feature vector and the wake-up word. The MGDA algorithm is used to construct the training loss function and the verification loss function. After calculating the gradients of the wake-up word recognition and voiceprint recognition tasks each time, the common gradient descent direction is obtained to update the shared parameters. The processed speech data set is input into the convolutional neural network model, and the training is iterated by minimizing the loss function to finally determine the weight parameters of the convolutional neural network model.

[0153] Then, voice liveness detection is performed on the registered voice, and the silent sound that is meaningless for recognition is removed from the registered voice. If the effective audio duration is less than 2000 milliseconds, the registration failure is directly output. The frame length of the audio of the registered voice without the silent sound is set to 25 milliseconds and the frame shift is set to 10 milliseconds. It is processed and converted into a spectrogram as the input of the convolutional neural network model, and the spectrogram of the registered voice is input into the convolutional neural network model for voiceprint feature extraction and speech-to-text transcription to obtain the registered wake-up word label and registered feature vector, which are then stored in the registration library.

[0154] Perform voice liveness detection on the speech to be recognized, remove silent sounds that are meaningless to recognition, if the effective audio duration is less than 2000 milliseconds, direct wake-up fails, set the frame length of the audio of the speech to be recognized without silent sounds to 25 milliseconds and the frame shift to 10 milliseconds, process it and convert it into a spectrogram as the input of the convolutional neural network model, input the spectrogram of the speech to be recognized into the convolutional neural network model for voiceprint feature extraction and voice transcription wake-up words, obtain the voiceprint feature vector and wake-up word, match the wake-up word of the speech to be recognized with the corresponding registered wake-up word label according to the correspondence table, if there is a wake-up word that matches the registered wake-up word label, obtain the registered feature vector according to the registered wake-up word label, calculate the cosine similarity between the voiceprint feature vector of the speech to be recognized and the corresponding registered feature vector, obtain the vector similarity, and perform speech wake-up when the vector similarity is greater than the preset threshold.

[0155] The wake-up word task and voiceprint recognition task disclosed in the present invention share a neural network model so that the training model occupies less memory and the wake-up speed is accelerated.

[0156] Based on the same inventive concept, the present disclosure also provides a voice wake-up device, as described in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0157] Figure 7 A schematic diagram of a voice wake-up device in an embodiment of the present disclosure is shown. Figure 7 As shown, the device includes a processing module 71, a query module 72, a vector similarity calculation module 73 and a voice wake-up module 74, wherein:

[0158] A processing module 71 is used to process the spectrogram of the speech to be recognized according to a pre-trained neural network model to obtain a voiceprint feature vector and a wake-up word of the speech to be recognized;

[0159] A query module 72, configured to query a registered wake-up word label corresponding to the wake-up word according to a preset correspondence table, and obtain a registered feature vector according to the registered wake-up word label;

[0160] A vector similarity calculation module 73, used to calculate vector similarity based on the voiceprint feature vector and the registration feature vector;

[0161] The voice wake-up module 74 is used to perform voice wake-up when the vector similarity is greater than a preset threshold.

[0162] In one embodiment, the processing module 71 is further used to obtain the spectrogram of the registered voice before processing the spectrogram of the voice to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the voice to be recognized;

[0163] The spectrogram of the registered speech is processed according to the pre-trained neural network model to obtain the registered feature vector and the registered wake-up word label;

[0164] Construct a correspondence table based on the registered feature vectors and registered wake-up word labels.

[0165] In one embodiment, the processing module 71 is further configured to process the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized,

[0166] Remove the silent sound in the speech to be recognized to obtain the effective speech to be recognized;

[0167] When the effective audio duration of the effective speech to be recognized is greater than or equal to a preset duration threshold, the effective speech to be recognized is processed to obtain a spectrogram of the speech to be recognized.

[0168] In one embodiment, the processing module 71 is further configured to determine that voice wake-up fails when the effective audio duration of the effective voice to be recognized is less than a preset duration threshold.

[0169] In one embodiment, the processing module 71 is further used to divide the speech to be recognized into a plurality of speech files with a first duration, wherein the frame length of the speech files is a second duration, and the frame shift is a third duration;

[0170] Convert speech files into spectrograms.

[0171] In one embodiment, the processing module 71 is further used to process the acquired speech data set to obtain a spectrogram of the speech data set;

[0172] The processing module 71 is also used to train an initial neural network model based on the spectrogram and corresponding labels of the speech data set, wherein the initial neural network model includes a first target network structure and a second target network structure. If a preset first training stop condition is met, the initial neural network model is used as the target neural network model.

[0173] In one embodiment, the spectrogram of the speech data set includes a training data set for model training and a test data set for model verification, and the training data set and the test data set are divided according to a preset ratio.

[0174] In one embodiment, the first training stop condition includes that the training loss is less than a first threshold or the number of training rounds is greater than a second threshold.

[0175] In one embodiment, the spectrogram of the speech dataset also includes a validation dataset;

[0176] The first target network structure and the second target network structure are obtained in the following way:

[0177] The processing module 71 is also used to construct a training loss function based on the MGDA algorithm;

[0178] According to the training data set, the initial network structure with network structure parameters and the training loss function, the network structure parameters of the network structure are fixed, and the network operation weights are obtained through training;

[0179] The processing module 71 is also used to construct a verification loss function based on the MGDA algorithm;

[0180] The network operation weights are fixed, and new network structure parameters are obtained through training according to the validation data set and validation loss function to obtain the process network structure;

[0181] Calculate the speech wake-up word recognition and voiceprint recognition gradients, obtain updated shared parameters in the common gradient descent direction, retrain until the preset second training stop condition is met, and obtain the target network structure, which includes one of the first target network structure and the second target network structure.

[0182] In one embodiment, the neural network model includes an input node, multiple intermediate nodes and two output nodes, the intermediate nodes include at least two first target network structures and at least one second target network structure, the structures of at least two first target network structures are the same, and the input of an intermediate node is the output of the first two adjacent intermediate nodes.

[0183] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0184] Refer to the following Figure 8 The electronic device 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0185] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0186] Among them, the storage unit stores a program code, and the program code can be executed by the processing unit 810, so that the processing unit 810 executes the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 810 can execute the following steps of the above method embodiment: processing the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and wake-up word of the speech to be recognized; according to the preset correspondence table, querying the registered wake-up word label corresponding to the wake-up word, and obtaining the registered feature vector according to the registered wake-up word label; calculating the vector similarity based on the voiceprint feature vector and the registered feature vector; when the vector similarity is greater than the preset threshold, voice wake-up is performed.

[0187] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0188] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0189] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0190] The electronic device 800 may also communicate with one or more external devices 840 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0191] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0192] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, which may be a readable signal medium or a readable storage medium. Fig. 9 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present disclosure is shown. Fig. 9 As shown, a program product capable of implementing the above method of the present disclosure is stored on the computer-readable storage medium 900. In an exemplary embodiment of the present disclosure, a computer program product is also provided, the computer program product includes a computer program or a computer instruction, and the computer program or the computer instruction is loaded and executed by a processor to enable the computer to implement any of the above voice wake-up methods.

[0193] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0194] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0195] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0196] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0197] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0198] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0199] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0200] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope of the present disclosure is indicated by the appended claims.

Claims

1. A voice wake-up method, It is characterized in that include: Processing the spectrogram of the speech to be recognized according to a pre-trained neural network model to obtain a voiceprint feature vector and a wake-up word of the speech to be recognized, wherein the neural network model includes a first target network structure and a second target network structure; According to a preset correspondence table, query the registered wake-up word label corresponding to the wake-up word, and obtain a registered feature vector according to the registered wake-up word label; Calculating vector similarity based on the voiceprint feature vector and the registration feature vector; When the vector similarity is greater than a preset threshold, voice wake-up is performed; The first target network structure and the second target network structure are obtained in the following manner: Constructing a training loss function based on a multi-gradient descent algorithm MGDA algorithm; fixing the network structure parameters of the network structure according to a training data set, an initial network structure with network structure parameters and the training loss function, and obtaining a network operation weight through training; Constructing a verification loss function based on the MGDA algorithm; fixing the network operation weights, and obtaining new network structure parameters through training according to the verification data set and the verification loss function to obtain a process network structure; calculating the speech wake-up word recognition and voiceprint recognition gradients to obtain updated shared parameters in the common gradient descent direction, and retraining until the preset second training stop condition is met to obtain a target network structure, wherein the target network structure includes one of the first target network structure and the second target network structure.

2. The voice wake-up method according to claim 1, It is characterized in that Before processing the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes: Get the spectrogram of the registered speech; Processing the spectrogram of the registration speech according to a pre-trained neural network model to obtain a registration feature vector and a registration wake-up word label; The corresponding relationship table is constructed according to the registered feature vector and the registered wake-up word label.

3. The voice wake-up method according to claim 1, It is characterized in that Before processing the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes: Remove the silent sound in the speech to be recognized to obtain the effective speech to be recognized; When the effective audio duration of the effective speech to be recognized is greater than or equal to a preset duration threshold, the effective speech to be recognized is processed to obtain a spectrogram of the speech to be recognized.

4. The voice wake-up method according to claim 3, It is characterized in that The method further comprises: When the effective audio duration of the effective voice to be recognized is less than a preset duration threshold, voice wake-up fails.

5. The voice wake-up method according to claim 1, It is characterized in that Before the spectrogram of the speech to be recognized is processed according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes: Segment the speech to be recognized into a plurality of speech files of a first duration, wherein the frame length of the speech files is a second duration and the frame shift is a third duration; The speech file is converted into a spectrogram.

6. The voice wake-up method according to claim 2, It is characterized in that Before processing the spectrogram of the speech to be recognized according to the pre-trained neural network model to obtain the voiceprint feature vector and the wake-up word of the speech to be recognized, the method further includes: Processing the acquired speech data set to obtain a spectrogram of the speech data set; An initial neural network model is trained according to the spectrogram and corresponding labels of the speech data set, wherein the initial neural network model includes a first target network structure and a second target network structure, and if a preset first training stop condition is met, the initial neural network model is used as the target neural network model.

7. The voice wake-up method according to claim 6, It is characterized in that The spectrogram of the speech data set includes a training data set for model training and a test data set for model verification, and the training data set and the test data set are divided according to a preset ratio.

8. The voice wake-up method according to claim 6, It is characterized in that The first training stop condition includes that the training loss is less than a first threshold or the number of training rounds is greater than a second threshold.

9. The voice wake-up method according to claim 1, It is characterized in that The neural network model includes an input node, multiple intermediate nodes and two output nodes. The intermediate nodes include at least two first target network structures and at least one second target network structure. The structures of the at least two first target network structures are the same. The input of an intermediate node is the output of the first two adjacent intermediate nodes.

10. A voice wake-up device, It is characterized in that include: a processing module, configured to process a spectrogram of a speech to be recognized according to a pre-trained neural network model to obtain a voiceprint feature vector and a wake-up word of the speech to be recognized, wherein the neural network model includes a first target network structure and a second target network structure; A query module, configured to query a registered wake-up word label corresponding to the wake-up word according to a preset correspondence table, and obtain a registered feature vector according to the registered wake-up word label; A vector similarity calculation module, used to calculate vector similarity based on the voiceprint feature vector and the registration feature vector; A voice wake-up module, used for performing voice wake-up when the vector similarity is greater than a preset threshold; The first target network structure and the second target network structure are obtained in the following manner: The processing module is further used to construct a training loss function based on a multi-gradient descent algorithm MGDA algorithm; according to a training data set, an initial network structure with network structure parameters and the training loss function, the network structure parameters of the network structure are fixed, and a network operation weight is obtained through training; The processing module is also used to construct a verification loss function based on the MGDA algorithm; fix the network operation weights, and obtain new network structure parameters through training according to the verification data set and the verification loss function to obtain a process network structure; calculate the speech wake-up word recognition and voiceprint recognition gradients to obtain updated shared parameters in the common gradient descent direction, and re-train until a preset second training stop condition is met to obtain a target network structure, wherein the target network structure includes one of the first target network structure and the second target network structure.

11. An electronic device, It is characterized in that include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the voice wake-up method described in any one of claims 1 to 9 by executing the executable instructions.

12. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the voice wake-up method according to any one of claims 1 to 9 is implemented.

Citation Information

Patent Citations

  • Voice wakeup method and device based on voiceprint recognition

    CN107886957A

  • Voiceprint-based user identification method and device as well as equipment

    CN108958810A

  • Voice wake-up method and electronic equipment

    CN109979438A

  • Speech recognition method, device, system and equipment and computer readable storage medium

    CN111210829A

  • Voice wake-up method and device, equipment and storage medium

    CN111223488A