Voice wake-up method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a voice wake-up method and apparatus, an electronic device and a computer readable storage medium. The method comprises the steps of obtaining a target registration wake-up audio corresponding to a target device and a target wake-up audio input by a user; respectively obtaining a target wake-up feature corresponding to the target wake-up audio and a target voice feature corresponding to the target registration wake-up audio; comparing the target wake-up feature with the target voice feature to obtain a target wake-up result; wherein the target awakening result is used for indicating whether the target equipment is awakened or not. In this way, the accuracy of voice wake-up can be improved.

Description

Technical Field

[0001] The present application relates to the field of voice wake-up technology, and in particular to a voice wake-up method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the rapid development of artificial intelligence and deep learning, voice wake-up technology has been widely used in smart homes, smart assistants, in-vehicle systems and other fields.

[0003] At present, most voice wake-up technologies rely too much on manual feature extraction and rule setting. Although they perform well in specific scenarios, they have problems such as sensitivity to environmental noise, low recognition accuracy, and poor adaptability. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a voice wake-up method, device, electronic device and computer-readable storage medium, which can improve the accuracy of voice wake-up.

[0005] To solve the above technical problems, a technical solution adopted in the present application is: to provide a voice wake-up method, the method comprising: obtaining a target registered wake-up audio corresponding to the target device and a target wake-up audio input by the user; respectively obtaining a target wake-up feature corresponding to the target wake-up audio and a target voice feature corresponding to the target registered wake-up audio; comparing the target wake-up feature and the target voice feature to obtain a target wake-up result; wherein the target wake-up result is used to indicate whether to wake up the target device.

[0006] Among them, obtaining the target wake-up feature corresponding to the target wake-up audio includes: encoding the target wake-up audio to obtain a first target acoustic feature; decoding the first target acoustic feature to obtain the target wake-up feature.

[0007] The target wake-up audio is encoded to obtain the first target acoustic feature, which is performed by using a first encoding network of a first encoder, and the first encoder is an encoder in a wake-up subnetwork in a voice wake-up network.

[0008] Among them, obtaining the target voice feature corresponding to the target registration wake-up audio includes: encoding the target wake-up audio to obtain a second target acoustic feature; fusing the second target acoustic feature and the target registration wake-up audio to obtain the target voice feature.

[0009] Among them, the second target acoustic feature and the target registration wake-up audio are fused to obtain the target voice feature, including: fusing the second target acoustic feature and the target registration wake-up audio to obtain target fusion data; encoding the target fusion data to obtain the third target acoustic feature; and extracting features from the third target acoustic feature to obtain the target voice feature.

[0010] Among them, encoding the target fusion data to obtain the third target acoustic feature is performed using the second encoder, and the second encoder is an encoder in the voice registration subnetwork in the voice wake-up network; encoding the target wake-up audio to obtain the second target acoustic feature is performed using the first encoder; encoding the target wake-up audio to obtain the second target acoustic feature is performed using the first encoder, and the first encoder is an encoder in the wake-up subnetwork in the voice wake-up network; the first encoder is an encoder in the wake-up subnetwork in the voice wake-up network, and the parameter update speed of the second encoder is lower than that of the first encoder; and / or the second target acoustic feature is obtained by encoding the target wake-up audio using the second encoding network of the first encoder.

[0011] Among them, feature extraction of the third target acoustic feature to obtain the target speech feature is performed by using the first gated recurrent unit network in the speech registration subnetwork in the speech wake-up network, and the speech wake-up network also includes a text registration subnetwork, and the text registration subnetwork includes a trained second gated recurrent unit network; the training step of the first gated recurrent unit network includes: using the first gated recurrent unit network to extract the first sample acoustic feature to obtain the sample speech feature; wherein the first sample acoustic feature is obtained by encoding the first sample fusion data, the first sample fusion data is obtained by fusing the second sample acoustic feature and the sample registration wake-up audio, and the second sample acoustic feature is obtained by encoding the sample wake-up audio using the first encoder; and, using the second gated recurrent unit network, based on the sample registration wake-up text corresponding to the sample registration wake-up audio, generate a sample text feature; using the difference between the sample text feature and the sample speech feature, adjust the network parameters of the first gated recurrent unit network.

[0012] Among them, before using the second gated recurrent unit network to generate sample text features based on the sample registration wake-up text corresponding to the sample registration wake-up audio, the voice wake-up method also includes: obtaining a third sample acoustic feature; wherein the third sample acoustic feature is obtained by encoding the sample wake-up audio using the third encoding network of the first encoder; using the second gated recurrent unit network to generate sample text features based on the sample registration wake-up text corresponding to the sample registration wake-up audio, including: fusing the third sample acoustic feature and the sample registration wake-up text to obtain second sample fusion data; using the second gated recurrent unit network to extract features from the second sample fusion data to obtain sample text features.

[0013] Among them, the voice wake-up method is executed by using a voice wake-up network. In the training stage, the voice wake-up network includes a wake-up subnetwork, a voice registration subnetwork and a text registration subnetwork. The wake-up subnetwork includes a first encoder. The training order of the first encoder precedes the training order of the wake-up subnetwork and the text registration subnetwork. The training order of the wake-up subnetwork and the text registration subnetwork precedes the training order of the voice registration subnetwork.

[0014] Among them, the target wake-up result includes a target wake-up tag; the voice wake-up method also includes: in response to the target wake-up tag being a first tag, determining the target wake-up result is used to indicate that the target device is woken up; in response to the target wake-up tag being a second tag, determining the target wake-up result is used to indicate that the target device is not woken up.

[0015] To solve the above technical problems, another technical solution adopted in the present application is: to provide a voice wake-up device, which includes a first acquisition module, a second acquisition module and a comparison module; the first acquisition module is used to obtain the target registered wake-up audio corresponding to the target device and the target wake-up audio input by the user; the second acquisition module is used to respectively obtain the target wake-up features corresponding to the target wake-up audio, and the target voice features corresponding to the target registered wake-up audio; the comparison module is used to compare the target wake-up features and the target voice features to obtain the target wake-up result; wherein, the target wake-up result is used to indicate whether to wake up the target device.

[0016] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, the electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the above voice wake-up method.

[0017] In order to solve the above technical problem, another technical solution adopted by the present application is: providing a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed to implement the above voice wake-up method.

[0018] In the above technical solution, the target wake-up feature is a valid feature extracted from the target wake-up audio input by the user, and the target voice feature is a valid feature extracted from the target registered wake-up audio corresponding to the target device. By comparing the target wake-up feature of the target wake-up audio input by the user and the target voice feature of the target registered wake-up audio corresponding to the target device, it is determined whether the target wake-up audio input by the user matches the target registered wake-up audio corresponding to the target device, thereby determining whether to indicate to wake up the target device. Therefore, by comparing the target wake-up audio input by the user and the target registered wake-up audio corresponding to the target device, it is determined whether to indicate to wake up the target device, thereby realizing a direct mapping of the wake-up voice input to the wake-up result, avoiding the cumbersome steps of feature extraction and model building in the traditional voice wake-up method, and greatly improving the accuracy and efficiency of voice wake-up. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of an embodiment of a voice wake-up method provided by the present application;

[0020] Figure 2 This is a schematic diagram of the framework structure of an embodiment of a voice wake-up network provided by the present application;

[0021] Figure 3 It is a schematic diagram of an embodiment of a sample provided by the present application;

[0022] Figure 4 yes Figure 1 The flowchart of step S12 is shown as an embodiment;

[0023] Figure 5 yes Figure 1 The flowchart of another embodiment of step S12 is shown;

[0024] Figure 6 yes Figure 5 The flowchart of step S52 is shown as an embodiment;

[0025] Figure 7 It is a flowchart of an embodiment of a training method for a first gated recurrent unit network provided by the present application;

[0026] Figure 8 yes Figure 7 The flowchart of step S72 is shown as an embodiment;

[0027] Fig. 9 It is a schematic diagram of another embodiment of the sample provided by the present application;

[0028] Fig.10 It is a structural diagram of an embodiment of a voice wake-up device provided by the present application;

[0029] Fig.11It is a structural schematic diagram of an embodiment of an electronic device provided by the present application;

[0030] Fig.12 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0031] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0032] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0033] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0034] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of the voice wake-up method provided in the present application. It should be noted that if there are substantially the same results, this embodiment does not take Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0035] Step S11: Obtain the target registered wake-up audio corresponding to the target device and the target wake-up audio input by the user.

[0036] In this implementation, the target registered wake-up audio corresponding to the target device and the target wake-up audio input by the user are obtained.

[0037] The target registered wake-up audio corresponding to the target device is not limited and can be set according to actual usage needs. For example, if the target registered wake-up audio corresponding to the target device is "Xiao Nuan Xiao Nuan", then the subsequent user needs to enter the target wake-up audio "Xiao Nuan Xiao Nuan" or a target wake-up audio containing "Xiao Nuan Xiao Nuan" to wake up the target device.

[0038] Step S12: respectively obtaining target wake-up features corresponding to the target wake-up audio and target voice features corresponding to the target registration wake-up audio.

[0039] In this implementation, the target wake-up features corresponding to the target wake-up audio and the target voice features corresponding to the target registration wake-up audio are obtained respectively.

[0040] Step S13: Compare the target wake-up feature and the target speech feature to obtain the target wake-up result.

[0041] In this embodiment, the target wake-up feature and the target voice feature are compared to obtain the target wake-up result; wherein the target wake-up result is used to indicate whether to wake up the target device. That is, the target wake-up feature of the target wake-up audio input by the user and the target voice feature of the target registered wake-up audio corresponding to the target device are compared to determine whether the target wake-up audio input by the user matches the target registered wake-up audio corresponding to the target device, thereby determining whether to indicate to wake up the target device.

[0042] The target wake-up feature is a valid feature extracted from the target wake-up audio input by the user, and the target voice feature is a valid feature extracted from the target registered wake-up audio corresponding to the target device. By comparing the target wake-up feature of the target wake-up audio input by the user and the target voice feature of the target registered wake-up audio corresponding to the target device, it is determined whether the target wake-up audio input by the user matches the target registered wake-up audio corresponding to the target device, thereby determining whether to indicate to wake up the target device. Therefore, by comparing the target wake-up audio input by the user and the target registered wake-up audio corresponding to the target device, it is determined whether to indicate to wake up the target device, thereby realizing a direct mapping of the wake-up voice input to the wake-up result, avoiding the cumbersome steps of feature extraction and model building in the traditional voice wake-up method, and greatly improving the accuracy and efficiency of voice wake-up.

[0043] In one embodiment, the target wake-up result includes a target wake-up tag. In response to the target wake-up tag being a first tag, the target wake-up result is determined to indicate that the target device is to be woken up; in response to the target wake-up tag being a second tag, the target wake-up result is determined to indicate that the target device is not to be woken up.

[0044] Specifically, the voice wake-up method provided in the present application is executed by a voice wake-up network. When the voice wake-up network outputs a first label, it is determined to indicate that the target device is to be woken up, and when the voice wake-up network outputs a second label, it is determined to indicate that the target device is not to be woken up.

[0045] The label contents of the first label and the second label are not limited and can be set according to actual use needs. For example, the label content of the first label is "wake up yes" and the label content of the second label is "wake up no".

[0046] In one embodiment, if Figure 2 As shown, Figure 2It is a schematic diagram of the framework structure of an embodiment of a voice wake-up network provided in the present application. The voice wake-up network includes a voice registration subnetwork, and the voice registration subnetwork includes a contrastive learning classification network. The contrastive learning classification network compares the target wake-up features and the target voice features to obtain the target wake-up result.

[0047] In a specific embodiment, during the training phase of the contrastive learning classification network, the contrastive learning classification network is based on the CLIP contrastive learning algorithm idea, and aims to train the contrastive learning classification network by generating positive and negative sample pairs, so that the feature distance between the matching sample wake-up feature and the sample voice feature is closer, while the feature distance between the mismatched sample wake-up feature and the sample voice feature is farther, and then in subsequent applications, when the target wake-up feature and the target voice feature match, it is indicated to wake up the target device, otherwise it is suppressed to wake up the target device. Based on the contrastive learning method, the ability of the voice wake-up network to distinguish the target wake-up audio is improved by shrinking the intra-class distance.

[0048] Among them, Figure 3 As shown, Figure 3 is a schematic diagram of an embodiment of a sample pair provided by the present application, wherein the diagonal line represents positive samples and the rest represent negative samples. Figure 3 Here, Ai represents the sample speech feature and Bi represents the sample wake-up feature.

[0049] Specifically, step one: input the sample pair into the contrastive learning classification network to obtain the output result - the sample wake-up result, where the sample pair is marked with the real wake-up result; step two: use the loss function to adjust the parameters of the contrastive learning classification network based on the difference between the sample wake-up result and the real wake-up result.

[0050] Among them, the loss function can be MaxPooling Loss, and the specific loss is as follows:

[0051] loss3=MaxPoolingloss(X 1 ,X 真 )

[0052] Among them, loss3 represents the loss corresponding to the contrastive learning classification network; MaxPooling Loss represents the loss function; X 1 Indicates the sample wake-up result; X 真 Indicates the actual wake-up result.

[0053] See also Figure 4 , Figure 4 yes Figure 1 FIG. 1 is a flow chart of an embodiment of step S12. It should be noted that if there is substantially the same result, this embodiment does not necessarily use Figure 4 The process sequence shown is limited. Figure 4 As shown, obtaining the target wake-up feature corresponding to the target wake-up audio specifically includes the following sub-steps:

[0054] Step S41: Encode the target wake-up audio to obtain a first target acoustic feature.

[0055] In this implementation, the target wake-up audio is encoded to obtain a first target acoustic feature.

[0056] In one embodiment, if Figure 2 As shown, the voice wake-up network includes a wake-up subnetwork, the wake-up subnetwork includes a first encoder, the first encoder includes a first encoding network, and the first encoding network encodes the target wake-up audio to obtain a first target acoustic feature. Specifically, the first encoding network can be regarded as a branch of the first encoder, the first encoding network receives the target wake-up audio input by the user, and performs feature extraction (encoding) on ​​the target wake-up audio input by the user to obtain the first target acoustic feature, so as to convert the target wake-up audio input by the user into a high-dimensional feature representation. Among them, the output of the first encoding network is specifically as follows:

[0057] X 2 =Encoder 1 (X 输1 )

[0058] Among them, X 2 Represents the output result of the first encoding network, that is, the first target acoustic feature; Encoder 1 represents the first coding network; X 输1 Indicates the target wake-up audio.

[0059] In a specific embodiment, before the first encoding network extracts features from the target wake-up audio input by the user, it will first perform preprocessing operations such as scrambling, adding noise, and length splicing on the target wake-up audio input by the user. By performing a scrambling preprocessing operation on the target wake-up audio input by the user, it is possible to prevent the voice wake-up network from learning the specific sequence features of the target wake-up audio, so that the voice wake-up network can be more adaptable to target wake-up audio inputs of different orders, thereby improving the voice wake-up ability of the voice wake-up network. By performing a noise preprocessing operation on the target wake-up audio input by the user, various interferences in the real environment can be simulated, such as electromagnetic interference, environmental noise, etc., which helps the voice wake-up network learn features under different noise conditions and improve the voice wake-up ability of the voice wake-up network in complex environments. By performing a length splicing preprocessing operation on the target wake-up audio input by the user, audio clips of different lengths are spliced ​​to generate more diverse audio clips, which helps the voice wake-up network learn the features of audios of different lengths and improves the adaptability of the voice wake-up network to wake-up audios of different lengths.

[0060] In one specific implementation, the first target acoustic feature is a filter bank energy (FBank) feature. Of course, in other specific implementations, the first target acoustic feature may also be a Mel-frequency cepstral coefficient (MFCC) feature, which is not limited here.

[0061] In a specific embodiment, the first encoding network is a ResGroupCNN network, which can effectively capture the contextual information in the target wake-up audio input by the user, thereby effectively improving the expressiveness of the features. The ResGroupCNN network is a deep learning network that combines a residual network (ResNet) and a convolutional neural network (CNN). The residual network solves the gradient vanishing and gradient exploding problems in deep neural networks by introducing residual blocks, making the network deeper and more complex; while the convolutional neural network automatically extracts features from the original data through structures such as convolutional layers and pooling layers. Therefore, the ResGroupCNN network combines the advantages of residual networks and convolutional neural networks, and can effectively capture the contextual information in the target wake-up audio, that is, it can effectively extract features from the target wake-up audio.

[0062] Of course, in other specific implementations, the first encoding network may also use other networks, which are not specifically limited here.

[0063] Step S42: Decode the first target acoustic feature to obtain a target wake-up feature.

[0064] In this implementation, the first target acoustic feature is decoded to obtain the target wake-up feature.

[0065] In one embodiment, if Figure 2 As shown, the voice wake-up network includes a wake-up subnetwork, the wake-up subnetwork includes a first encoder and a scorer, the first encoder includes a first encoding network, the wake-up subnetwork also includes a scorer (ScoreBoard), and the output of the first encoding network is used as the input of the scorer. Specifically, the scorer decodes the first target acoustic feature to obtain the target wake-up feature, that is, the scorer decodes the first target acoustic feature output by the first encoding network into a recognizable target wake-up feature.

[0066] In a specific implementation, the scorer uses the MDTC neural network architecture. The MDTC neural network architecture is a network architecture for processing time series data. This network architecture can capture feature information of different time scales by introducing convolution kernels with different expansion rates, thereby improving the modeling ability of the voice wake-up network for time series data.

[0067] Of course, in other specific implementations, the scorer may also adopt other neural network architectures, which are not specifically limited here.

[0068] In a specific implementation, during the training phase of the scorer, the scorer is trained in combination with the MaxPoolingLoss loss function to improve the scorer's decoding capability for the first target acoustic feature, thereby improving the speech wake-up capability of the speech wake-up network. The loss relationship of the scorer is as follows:

[0069] X 3 = S coreBoard(X 2 )

[0070] Among them, X 2 represents the output result of the first encoding network, that is, the first target acoustic feature; MaxPoolingLoss represents the loss function; X 3 Represents the output of the scorer, i.e., the target arousal feature.

[0071] It should be noted that the MaxPoolingLoss loss function introduces the MaxPooling mechanism in the loss calculation stage, aiming to strengthen the scorer's attention to difficult-to-classify acoustic features, which helps the scorer learn more complex acoustic features, thereby improving the scorer's decoding accuracy and further improving the speech wake-up capability of the speech wake-up network.

[0072] In other specific implementations, during the training phase of the scorer, the scorer may also be trained in combination with other loss functions, which are not specifically limited here.

[0073] See also Figure 5 , Figure 5 yes Figure 1 FIG. 1 is a flow chart of another embodiment of step S12. It should be noted that if there is substantially the same result, this embodiment does not necessarily use Figure 5 The process sequence shown is limited. Figure 5 As shown, obtaining the target voice features corresponding to the target registration wake-up audio includes the following sub-steps:

[0074] Step S51: Encode the target wake-up audio to obtain a second target acoustic feature.

[0075] In this implementation, the target wake-up audio is encoded to obtain a second target acoustic feature.

[0076] In one embodiment, if Figure 2As shown, the sound wake-up network includes a wake-up subnetwork, the wake-up subnetwork includes a first encoder, the first encoder includes a second encoding network, and the second encoding network encodes the target wake-up audio to obtain a second target acoustic feature. Specifically, the second encoding network can be regarded as another branch of the first encoder. The second encoding network receives the target wake-up audio input by the user, and performs feature extraction (encoding) on ​​the target wake-up audio input by the user to obtain the second target acoustic feature, so as to convert the target wake-up audio input by the user into a high-dimensional feature representation. Among them, the output of the second encoding network is specifically as follows:

[0077] X 4 =Encoder 2 (X 输1 )

[0078] Among them, X 4 Represents the output result of the second encoding network, that is, the second target acoustic feature; Encoder 2 represents the second coding network; X 输1 Indicates the target wake-up audio.

[0079] In a specific embodiment, before the second encoding network extracts features from the target wake-up audio input by the user, it will first perform preprocessing operations such as scrambling, adding noise, and length splicing on the target wake-up audio input by the user. By performing a scrambling preprocessing operation on the target wake-up audio input by the user, it is possible to prevent the voice wake-up network from learning the specific sequence features of the target wake-up audio, so that the voice wake-up network can be more adaptable to target wake-up audio inputs of different orders, thereby improving the voice wake-up ability of the voice wake-up network. By performing a noise preprocessing operation on the target wake-up audio input by the user, various interferences in the real environment can be simulated, such as electromagnetic interference, environmental noise, etc., which helps the voice wake-up network learn features under different noise conditions and improve the voice wake-up ability of the voice wake-up network in complex environments. By performing a length splicing preprocessing operation on the target wake-up audio input by the user, audio clips of different lengths are spliced ​​to generate more diverse audio clips, which helps the voice wake-up network learn the features of audios of different lengths and improves the adaptability of the voice wake-up network to wake-up audios of different lengths.

[0080] In one specific implementation, the second target acoustic feature is a filter bank energy (FBank) feature. Of course, in other specific implementations, the second target acoustic feature may also be a Mel-frequency cepstral coefficient (MFCC) feature, which is not limited here.

[0081] In a specific embodiment, the second encoding network is a ResGroupCNN network, which can effectively capture the contextual information in the target wake-up audio input by the user, thereby effectively improving the expressiveness of the features. The ResGroupCNN network is a deep learning network that combines a residual network (ResNet) and a convolutional neural network (CNN). The residual network solves the gradient vanishing and gradient exploding problems in deep neural networks by introducing residual blocks, making the network deeper and more complex; while the convolutional neural network automatically extracts features from the original data through structures such as convolutional layers and pooling layers. Therefore, the ResGroupCNN network combines the advantages of residual networks and convolutional neural networks, and can effectively capture the contextual information in the target wake-up audio, that is, it can effectively extract features from the target wake-up audio.

[0082] Of course, in other specific implementations, the second encoding network may also use other networks, which are not specifically limited here.

[0083] Step S52: Fusing the second target acoustic feature and the target registration wake-up audio to obtain the target speech feature.

[0084] In this implementation, the second target acoustic feature and the target registration wake-up audio are fused to obtain the target speech feature.

[0085] In one embodiment, if Figure 6 As shown, Figure 6 yes Figure 5 The flowchart of step S52 of an embodiment shown in FIG. 1 is to fuse the second target acoustic feature and the target registration wake-up audio to obtain the target voice feature, and specifically includes the following sub-steps:

[0086] Step S61: Fusing the second target acoustic feature and the target registered wake-up audio to obtain target fusion data.

[0087] In this implementation, the second target acoustic feature and the target registration wake-up audio are fused to obtain target fusion data.

[0088] In one embodiment, if Figure 2 As shown, the voice wake-up network includes a voice registration subnetwork, and the voice registration subnetwork includes a second encoder, and the second encoder fuses the second target acoustic feature and the target registration wake-up audio to obtain target fusion data. Specifically, the second encoder receives the target registration wake-up audio and the second target acoustic feature output by the second encoding network of the first encoder of the wake-up subnetwork, and fuses the two to obtain target fusion data.

[0089] Step S62: Encode the target fusion data to obtain a third target acoustic feature.

[0090] In this implementation, the target fusion data is encoded to obtain the third target acoustic feature.

[0091] In one embodiment, if Figure 2 As shown, the voice wake-up network includes a voice registration sub-network, and the voice registration sub-network includes a second encoder, and the second encoder encodes the target fusion data to obtain the third target acoustic feature. Specifically, the second encoder encodes the target fusion data to obtain the third target acoustic feature.

[0092] In a specific embodiment, encoding the target fusion data to obtain the third target acoustic feature is performed using a second encoder, which is an encoder in the voice registration subnetwork in the voice wake-up network. Encoding the target wake-up audio to obtain the second target acoustic feature is performed using a first encoder, which is an encoder in the wake-up subnetwork in the voice wake-up network. The parameter update speed of the second encoder is lower than that of the first encoder. In other words, the present application combines the fast and slow Encoder strategies. The fast and slow Encoders mainly refer to two different models, one is an ordinary Encoder, and the other is a Momentum Encoder. The ordinary Encoder updates the parameters in each step, while the Momentum Encoder updates the parameters more slowly, but its update is achieved by momentum. On the one hand, the Momentum Encoder uses the momentum update mechanism, and its parameter update speed is slow, which is conducive to the stability of the encoder during the application process. Because the Momentum Encoder updates slowly, it can accumulate knowledge more stably and generate consistent key vectors, which is crucial for the feature representation in subsequent contrastive learning. On the other hand, ordinary Encoders update parameters in each step, while Momentum Encoders update slowly through momentum. This strategy reduces the waste of computing resources and improves training efficiency. By introducing Momentum Encoder, the voice wake-up performance of the voice wake-up network can be improved without adding additional computing burden. On the other hand, the fast and slow Encoder strategy enables the voice wake-up network to maintain a certain stability while still having the ability to adapt to new data, and helps the voice wake-up network learn more discriminative feature representations in subsequent contrastive learning tasks, thereby improving the voice wake-up ability of the voice wake-up network. Among them, the specific representation of Momentum Encoder is as follows:

[0093] Momentum_Encoder=m*Momentum_Encoder+(1-m)*Encoder

[0094] Among them, Encoder represents the first encoder; Momentum Encoder represents the second encoder. It should be noted that when the value of m approaches 1, the parameters of Momentum Encoder are updated more slowly. This slow update allows Momentum Encoder to accumulate knowledge more stably. Even if it is updated slowly, Momentum Encoder will make a small update in each step to maintain its dynamic balance with the ordinary Encoder.

[0095] The output of Momentum Encoder is as follows:

[0096] X 6 = Momentum_Encoder(X 5 )

[0097] Among them, X 6 represents the third target acoustic feature; X 5 Represents the target fusion data.

[0098] Step S63: extracting the third target acoustic feature to obtain the target speech feature.

[0099] In this implementation, feature extraction is performed on the third target acoustic feature to obtain the target speech feature.

[0100] In one embodiment, if Figure 2 As shown, the voice wake-up network includes a voice registration sub-network, and the voice registration sub-network includes a first gated recurrent unit (GRU) network, and the first gated recurrent unit network extracts features from the third target acoustic features to obtain target voice features. Specifically, the first gated recurrent unit network receives the third target acoustic features output by the second encoder, and extracts features from the third target acoustic features to obtain target voice features. The first gated recurrent unit network can effectively capture the temporal features in the third target acoustic features through the gating mechanism, which is conducive to improving the voice wake-up ability of the voice wake-up network; in addition, compared with the traditional RNN, the first gated recurrent unit network can better alleviate the problems of gradient disappearance and gradient explosion, so that the voice wake-up network can learn longer dependencies; furthermore, compared with the long short-term memory network (LSTM), the first gated recurrent unit network has fewer parameters and higher computational efficiency, which makes the first gated recurrent unit network more advantageous in processing large-scale voice data.

[0101] Among them, the output of the first gated recurrent unit network is as follows:

[0102] X 7 =GRU(X 6 )

[0103] Among them, X 7 represents the target speech feature; X 6 Represents the third target acoustic feature.

[0104] In one embodiment, if Figure 7 As shown, Figure 7 It is a flow chart of an embodiment of a training method for a first gated recurrent unit network provided by the present application, wherein feature extraction is performed on a third target acoustic feature to obtain a target speech feature, which is performed by using a first gated recurrent unit network in a speech registration subnetwork in a speech wake-up network, wherein the speech wake-up network also includes a text registration subnetwork, and the text registration subnetwork includes a trained second gated recurrent unit network. The training method for the first gated recurrent unit network specifically includes the following sub-steps:

[0105] Step S71: extracting the acoustic features of the first sample using the first gated recurrent unit network to obtain sample speech features.

[0106] In this embodiment, a first gated recurrent unit network is used to extract features of the first sample acoustic features to obtain sample speech features; wherein the first sample acoustic features are obtained by encoding first sample fusion data, the first sample fusion data is obtained by fusing the second sample acoustic features and the sample registration wake-up audio, and the second sample acoustic features are obtained by encoding the sample wake-up audio using the first encoder.

[0107] In one embodiment, if Figure 2 As shown, the audio wake-up network includes a wake-up subnetwork, the wake-up subnetwork includes a first encoder, the first encoder includes a second encoding network, and the second encoding network encodes the sample wake-up audio to obtain a second sample acoustic feature. Specifically, the second encoding network can be regarded as a branch of the first encoder, the second encoding network receives the sample wake-up audio input by the user, and extracts (encodes) the features of the sample wake-up audio input by the user to obtain the second sample acoustic feature, so as to convert the sample wake-up audio input by the user into a high-dimensional feature representation.

[0108] like Figure 2As shown, the voice wake-up network includes a voice registration sub-network, and the voice registration sub-network includes a second encoder, and the second encoder fuses the second sample acoustic feature and the sample registration wake-up audio to obtain the first sample fusion data. Specifically, the second encoder receives the sample registration wake-up audio and the second sample acoustic feature output by the second encoding network of the first encoder of the wake-up sub-network, and fuses the two to obtain the first sample fusion data.

[0109] like Figure 3 As shown, the voice wake-up network includes a voice registration sub-network, and the voice registration sub-network includes a second encoder, and the second encoder encodes the first sample fusion data to obtain the first sample acoustic feature. Specifically, the second encoder encodes the first sample fusion data to obtain the first sample acoustic feature.

[0110] Step S72: using the second gated recurrent unit network, based on the sample registration wake-up text corresponding to the sample registration wake-up audio, to generate a sample text feature.

[0111] In this implementation, a second gated recurrent unit network is used to generate sample text features based on the sample registration wake-up text corresponding to the sample registration wake-up audio.

[0112] In one embodiment, if Figure 8 As shown, Figure 8 yes Figure 7 The flowchart of step S72 of an embodiment shown in the figure uses a second gated recurrent unit network to generate sample text features based on the sample registration wake-up text corresponding to the sample registration wake-up audio, specifically including the following sub-steps:

[0113] Before using the second gated recurrent unit network to generate the sample text feature based on the sample registration wake-up text corresponding to the sample registration wake-up audio, a third sample acoustic feature is also obtained, wherein the third sample acoustic feature is obtained by encoding the sample wake-up audio using the third encoding network of the first encoder.

[0114] In one embodiment, if Figure 2 As shown, the voice wake-up network includes a wake-up subnetwork, the wake-up subnetwork includes a first encoder, the first encoder includes a third encoding network, and the third encoding network encodes the sample wake-up audio to obtain a third sample acoustic feature. Specifically, the third encoding network can be regarded as another branch of the first encoder. The third encoding network receives the sample wake-up audio input by the user, and extracts (encodes) the features of the sample wake-up audio input by the user to obtain the third sample acoustic feature, so as to convert the sample wake-up audio input by the user into a high-dimensional feature representation. Among them, the output of the third encoding network is specifically as follows:

[0115] X 8=Encoder 3 (X 输2 )

[0116] Among them, X 8 Represents the output result of the third encoding network, that is, the third sample acoustic feature; Encoder 3 represents the third coding network; X 输2 Represents sample wake-up audio.

[0117] In a specific embodiment, before the third encoding network extracts features from the sample wake-up audio input by the user, it will first perform preprocessing operations such as scrambling, adding noise, and length splicing on the sample wake-up audio input by the user. By performing a scrambling preprocessing operation on the sample wake-up audio input by the user, it is possible to prevent the voice wake-up network from learning the specific sequence features of the sample wake-up audio, so that the voice wake-up network can be more adaptable to sample wake-up audio inputs of different orders, thereby improving the voice wake-up ability of the voice wake-up network. By performing a noise preprocessing operation on the sample wake-up audio input by the user, various interferences in the real environment can be simulated, such as electromagnetic interference, environmental noise, etc., which helps the voice wake-up network learn features under different noise conditions and improve the voice wake-up ability of the voice wake-up network in complex environments. By performing a length splicing preprocessing operation on the sample wake-up audio input by the user, audio clips of different lengths are spliced ​​to generate more diverse audio clips, which helps the voice wake-up network learn the features of audios of different lengths and improves the adaptability of the voice wake-up network to wake-up audios of different lengths.

[0118] In one specific implementation, the third sample acoustic feature is a filter bank energy (FBank) feature. Of course, in other specific implementations, the third sample acoustic feature may also be a Mel-frequency cepstral coefficient (MFCC) feature, which is not limited here.

[0119] In a specific embodiment, the third encoding network is a ResGroupCNN network, which can effectively capture the contextual information in the sample wake-up audio input by the user, thereby effectively improving the expression ability of the features. The ResGroupCNN network is a deep learning network that combines the residual network (ResNet) and the convolutional neural network (CNN). The residual network solves the gradient vanishing and gradient exploding problems in the deep neural network by introducing the residual block (Residual Block), making the network deeper and more complex; while the convolutional neural network automatically extracts features from the original data through structures such as convolutional layers and pooling layers. Therefore, the ResGroupCNN network combines the advantages of the residual network and the convolutional neural network, and can effectively capture the contextual information in the sample wake-up audio, that is, it can effectively extract features from the sample wake-up audio.

[0120] Of course, in other specific implementations, the third encoding network may also use other networks, which are not specifically limited here.

[0121] Step S81: Fuse the third sample acoustic feature and the sample registration wake-up text to obtain second sample fusion data.

[0122] In this implementation, the third sample acoustic feature and the sample registration wake-up text are fused to obtain second sample fusion data.

[0123] In one embodiment, if Figure 2 As shown, the voice wake-up network includes a text registration sub-network, and the text registration sub-network includes a second gated recurrent unit network, and the second gated recurrent unit network fuses the third sample acoustic feature and the sample registration wake-up audio to obtain second sample fusion data. Specifically, the second gated recurrent unit network receives the sample registration wake-up audio and the third sample acoustic feature output by the third encoding network of the first encoder of the wake-up sub-network, and fuses the two to obtain second sample fusion data.

[0124] Step S82: Using the second gated recurrent unit network, extract features from the second sample fusion data to obtain sample text features.

[0125] In this implementation, a second gated recurrent unit network is used to perform feature extraction on the second sample fusion data to obtain sample text features.

[0126] Specifically, Figure 2 As shown, the sound wake-up network includes a text registration sub-network, and the text registration sub-network includes a second gated recurrent unit (GRU) network. The second gated recurrent unit network extracts features from the second sample fusion data to obtain sample text features. The second gated recurrent unit network processes the second sample fusion data and converts it into an embedding vector of fixed dimension to capture the semantic relationship within the second sample fusion data. Subsequently, dynamic adjustments are made, and the gating mechanism is used to dynamically adjust the flow of the input second sample fusion data, highlight important information, and improve modeling performance.

[0127] Among them, the output of the second gated recurrent unit network is as follows:

[0128] X 10 =GRU(X 9 )

[0129] Among them, X 9 represents the second sample fusion data; X 10 Represents sample text features.

[0130] In one embodiment, if Figure 2 As shown, the speech registration subnetwork also includes a contrastive learning classification network. During the training phase of the speech registration subnetwork, the contrastive learning classification network of the speech registration subnetwork also needs to be trained. The contrastive learning classification network compares the sample wake-up features and the sample text features to obtain the sample wake-up results.

[0131] In a specific embodiment, during the training phase of the contrastive learning classification network, the contrastive learning classification network is based on the CLIP contrastive learning algorithm idea, and aims to train the contrastive learning classification network by generating positive and negative sample pairs, so that the feature distance between the matching sample wake-up feature and the sample text feature is closer, while the feature distance between the mismatched sample wake-up feature and the sample text feature is farther, and then when the sample wake-up feature and the sample text feature match, it indicates to wake up the target device, otherwise it inhibits the wake-up of the target device. Based on the contrastive learning method, the ability of the speech wake-up network to distinguish the wake-up text is improved by shrinking the intra-class distance.

[0132] Among them, Fig. 9 As shown, Fig. 9 is a schematic diagram of another embodiment of the sample pair provided by the present application, the diagonal line represents positive samples, and the rest represent negative samples. Figure 4 Ti in it represents the sample text feature and Ii represents the sample awakening feature.

[0133] Specifically, step one: input the sample pair into the contrastive learning classification network to obtain the output result - the sample wake-up result, where the sample pair is marked with the real wake-up result; step two: use the loss function to adjust the parameters of the contrastive learning classification network based on the difference between the sample wake-up result and the real wake-up result.

[0134] Among them, the loss function can be MaxPooling Loss, and the specific loss is as follows:

[0135] loss1=MaxPoolingloss(X 11 ,X 真 )

[0136] Among them, loss1 represents the loss corresponding to the contrastive learning classification network; MaxPooling Loss represents the loss function; X 11 Indicates the sample wake-up result; X 真 Indicates the actual wake-up result.

[0137] Step S73: using the difference between the sample text features and the sample speech features, adjusting the network parameters of the first gated recurrent unit network.

[0138] In this embodiment, the network parameters of the first gated recurrent unit network are adjusted by utilizing the difference between the sample text features and the sample speech features. That is, the sample text features output by the second gated recurrent unit network of the text registration subnetwork are used as text anchor information to help the first gated recurrent unit network establish a clearer representation between different audios of the same wake-up word, so that the sample speech features extracted by the first gated recurrent unit network can be accurately mapped to the text information; that is, by introducing text anchor information, the feature extraction capability of the first gated recurrent unit network is optimized, so that the first gated recurrent unit network can extract more accurate speech features, thereby improving the accuracy of the wake-up results determined subsequently.

[0139] In one embodiment, the network parameters of the first gated recurrent unit network are adjusted based on the difference between the sample text features and the sample speech features using mean square error loss (MSE loss). The loss is specifically as follows:

[0140] loss2=MSELOSS(X 10 ,X 12 )

[0141] Among them, loss2 represents the loss; X 10 Indicates the sample text features; X 12 Represents the sample speech features.

[0142] Of course, in other implementations, other losses may also be used to adjust the network parameters of the first gated recurrent unit network based on the difference between the sample text features and the sample speech features, which is not specifically limited here.

[0143] It should be noted that one branch of the voice wake-up network, the text registration sub-network, is only used to train the first gated recurrent unit network of another branch of the voice wake-up network, the voice registration sub-network. The text registration sub-network does not participate in the application of voice wake-up.

[0144] In one embodiment, the voice wake-up method is performed using a voice wake-up network. In the training phase, the voice wake-up network includes a wake-up subnetwork, a voice registration subnetwork, and a text registration subnetwork. The wake-up subnetwork includes a first encoder. The training sequence of the first encoder precedes the training sequence of the wake-up subnetwork and the text registration subnetwork. The training sequence of the wake-up subnetwork and the text registration subnetwork precedes the training sequence of the voice registration subnetwork.

[0145] That is to say, the training of the speech wake-up network is divided into three stages. The first stage is to train the first encoder of the wake-up subnetwork. A large number of labeled sample wake-up audios will be used to train the first encoder. Through supervised learning, high-level features in the sample wake-up audio are extracted. The training target used is the audio classification task to ensure that the first encoder can effectively capture the temporal features and spectral characteristics of the sample wake-up audio. After the training is completed, the first encoder will have the ability to effectively extract features from the sample wake-up audio, laying the foundation for subsequent tasks. The second stage is to train the scorer and text registration subnetwork of the wake-up subnetwork; using the above-mentioned trained first encoder as the initial weight, the text registration subnetwork is constructed. The task of the text registration subnetwork is to align the sample wake-up features with specific wake-up words; using a diverse data set containing rich interactive scenarios to fine-tune the text registration subnetwork, by optimizing the target loss function, the text registration subnetwork can efficiently identify and learn the feature information related to each wake-up word; during the fine-tuning process, the first encoder parameters are still involved in the training, and the parameters of the scorer and text registration subnetwork are updated at the same time to further optimize the feature extraction capability, laying the foundation for subsequent tasks. The third stage is to train the speech wake-up subnetwork. The trained first encoder and text registration subnetwork are used as initialization weights to build and fine-tune the speech registration subnetwork. The goal of the speech registration subnetwork is to further match the sample wake-up features with specific wake-up words; the labeled sample registration wake-up audio is used for fine-tuning, and the feature vector is made more discriminative in similarity measurement by optimizing the corresponding loss function; in this stage, joint training is used, and the parameters of the scorer and text registration subnetwork are frozen. Only the parameters of the first encoder and speech registration subnetwork are updated to maintain the stability of the features learned by the text registration subnetwork, ensure the collaborative work between the first encoder, text registration subnetwork and speech registration subnetwork, and further improve the speech wake-up accuracy of the final speech wake-up network.

[0146] In a specific implementation, an efficient voice wake-up quantization algorithm is used to deploy the algorithm on actual devices so as to achieve fast and accurate wake-up recognition in a variety of environments. After the 32-bit floating-point number training is completed, the voice wake-up network is optimized using quantization technology. The voice wake-up network is initialized with 32-bit floating-point training, and the quantization algorithm is fine-tuned to achieve dynamic fixed-point quantization, compressing the weights and activation values ​​of the voice wake-up network to a low bit width (8 bits) to reduce the size of the voice wake-up network. During the quantization process, quantization-aware training is used to ensure that the performance of the quantized voice wake-up network is as close as possible to the unquantized voice wake-up network.

[0147] See also Fig.10 , Fig.10It is a structural diagram of an embodiment of a voice wake-up device provided by the present application. The voice wake-up device 100 includes a first acquisition module 101, a second acquisition module 102 and a comparison module 103. The first acquisition module 101 is used to obtain the target registered wake-up audio corresponding to the target device and the target wake-up audio input by the user; the second acquisition module 102 is used to respectively obtain the target wake-up features corresponding to the target wake-up audio and the target voice features corresponding to the target registered wake-up audio; the comparison module 103 is used to compare the target wake-up features and the target voice features to obtain the target wake-up result; wherein the target wake-up result is used to indicate whether to wake up the target device.

[0148] Among them, the second acquisition module 102 is used to obtain the target wake-up feature corresponding to the target wake-up audio, including: encoding the target wake-up audio to obtain the first target acoustic feature; decoding the first target acoustic feature to obtain the target wake-up feature.

[0149] The above encoding of the target wake-up audio to obtain the first target acoustic feature is performed using a first encoding network of a first encoder, and the first encoder is an encoder in a wake-up subnetwork in a voice wake-up network.

[0150] Among them, the second acquisition module 102 is used to obtain the target voice feature corresponding to the target registration wake-up audio, including: encoding the target wake-up audio to obtain the second target acoustic feature; fusing the second target acoustic feature and the target registration wake-up audio to obtain the target voice feature.

[0151] Among them, the second acquisition module 102 is used to fuse the second target acoustic features and the target registration wake-up audio to obtain the target voice features, including: fusing the second target acoustic features and the target registration wake-up audio to obtain target fusion data; encoding the target fusion data to obtain the third target acoustic features; and extracting features from the third target acoustic features to obtain the target voice features.

[0152] Among them, the above-mentioned encoding of the target fusion data to obtain the third target acoustic feature is performed using the second encoder, and the second encoder is an encoder in the voice registration subnetwork in the voice wake-up network. The encoding of the target wake-up audio to obtain the second target acoustic feature is performed using the first encoder, and the first encoder is an encoder in the wake-up subnetwork in the voice wake-up network. The parameter update speed of the second encoder is lower than that of the first encoder; and / or the above-mentioned second target acoustic feature is obtained by encoding the target wake-up audio using the second encoding network of the first encoder.

[0153] Among them, the voice wake-up device 100 also includes a training module 104, which performs feature extraction on the third target acoustic feature to obtain the target voice feature, which is performed using the first gated recurrent unit network in the voice registration subnetwork in the voice wake-up network, and the voice wake-up network also includes a text registration subnetwork, and the text registration subnetwork includes a trained second gated recurrent unit network; the training module 104 is used for training the first gated recurrent unit network, including: using the first gated recurrent unit network to extract features from the first sample acoustic feature to obtain a sample voice feature; wherein the first sample acoustic feature is obtained by encoding the first sample fusion data, the first sample fusion data is obtained by fusing the second sample acoustic feature and the sample registration wake-up audio, and the second sample acoustic feature is obtained by encoding the sample wake-up audio using the first encoder; and, using the second gated recurrent unit network, based on the sample registration wake-up text corresponding to the sample registration wake-up audio, a sample text feature is generated; using the difference between the sample text feature and the sample voice feature, the network parameters of the first gated recurrent unit network are adjusted.

[0154] Among them, the training module 104 is used to generate sample text features based on the sample registration wake-up text corresponding to the sample registration wake-up audio using the second gated recurrent unit network, including: obtaining a third sample acoustic feature; wherein the third sample acoustic feature is obtained by encoding the sample wake-up audio using the third encoding network of the first encoder; the training module 104 is used to generate sample text features based on the sample registration wake-up text corresponding to the sample registration wake-up audio using the second gated recurrent unit network, including: fusing the third sample acoustic feature and the sample registration wake-up text to obtain second sample fusion data; using the second gated recurrent unit network to extract features from the second sample fusion data to obtain sample text features.

[0155] The above-mentioned voice wake-up method is performed using a voice wake-up network. In the training stage, the voice wake-up network includes a wake-up subnetwork, a voice registration subnetwork and a text registration subnetwork. The wake-up subnetwork includes a first encoder. The training sequence of the first encoder precedes that of the wake-up subnetwork and the text registration subnetwork. The training sequence of the wake-up subnetwork and the text registration subnetwork precedes that of the voice registration subnetwork.

[0156] Among them, the above-mentioned target wake-up result includes a target wake-up tag; the comparison module 103 is also used to determine that the target wake-up result is used to indicate that the target device is woken up in response to the target wake-up tag being a first tag; in response to the target wake-up tag being a second tag, the target wake-up result is determined to indicate that the target device is not woken up.

[0157] See also Fig.11 , Fig.111 is a schematic diagram of the structure of an embodiment of an electronic device provided by the present application. The electronic device 110 includes a memory 111 and a processor 112 coupled to each other, and the processor 112 is used to execute program instructions stored in the memory 111 to implement the steps of any of the above-mentioned voice wake-up method embodiments. In a specific implementation scenario, the electronic device 110 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 110 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0158] Specifically, the processor 112 is used to control itself and the memory 111 to implement the steps of any of the above-mentioned voice wake-up method embodiments. The processor 112 may also be referred to as a CPU (Central Processing Unit). The processor 112 may be an integrated circuit chip having signal processing capabilities. The processor 112 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 112 may be implemented by an integrated circuit chip.

[0159] See also Fig.12 , Fig.12 It is a structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 120 of the embodiment of the present application stores a program instruction 121, and when the program instruction 121 is executed, it implements the method provided by any embodiment of the voice wake-up method of the present application and any non-conflicting combination. Among them, the program instruction 121 can form a program file and be stored in the above-mentioned computer-readable storage medium 120 in the form of a software product, so that a computer device (which can be a personal computer, server, or network device, etc.) executes all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 120 includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0160] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0161] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A voice wake-up method, characterized in that: The method comprises: Obtain the target registered wake-up audio corresponding to the target device and the target wake-up audio input by the user; Respectively obtaining a target wake-up feature corresponding to the target wake-up audio and a target voice feature corresponding to the target registration wake-up audio; The target wake-up feature and the target voice feature are compared to obtain a target wake-up result; wherein the target wake-up result is used to indicate whether to wake up the target device.

2. The method according to claim 1, characterized in that The acquiring the target wake-up feature corresponding to the target wake-up audio includes: Encoding the target wake-up audio to obtain a first target acoustic feature; The first target acoustic feature is decoded to obtain the target wake-up feature.

3. The method according to claim 2, characterized in that The encoding of the target wake-up audio to obtain the first target acoustic feature is performed using a first encoding network of a first encoder, where the first encoder is an encoder in a wake-up subnetwork in a voice wake-up network.

4. The method according to claim 1, characterized in that: Obtaining the target voice feature corresponding to the target registration wake-up audio includes: Encoding the target wake-up audio to obtain a second target acoustic feature; The second target acoustic feature and the target registration wake-up audio are fused to obtain the target voice feature.

5. The method according to claim 4, characterized in that The fusing the second target acoustic feature and the target registration wake-up audio to obtain the target voice feature includes: fusing the second target acoustic feature and the target registered wake-up audio to obtain target fusion data; Encoding the target fusion data to obtain a third target acoustic feature; Feature extraction is performed on the third target acoustic feature to obtain the target speech feature.

6. The method according to claim 5, characterized in that The encoding of the target fusion data to obtain the third target acoustic feature is performed by using a second encoder, the second encoder is an encoder in a voice registration subnetwork in a voice wake-up network, the encoding of the target wake-up audio to obtain the second target acoustic feature is performed by using a first encoder, the first encoder is an encoder in a wake-up subnetwork in the voice wake-up network, and the parameter update speed of the second encoder is lower than that of the first encoder; And / or, the second target acoustic feature is obtained by encoding the target wake-up audio using a second encoding network of the first encoder.

7. The method according to claim 5, characterized in that The feature extraction of the third target acoustic feature to obtain the target speech feature is performed by using a first gated recurrent unit network in a speech registration subnetwork in a speech wake-up network, wherein the speech wake-up network further includes a text registration subnetwork, and the text registration subnetwork includes a trained second gated recurrent unit network; the training step of the first gated recurrent unit network includes: The first gated recurrent unit network is used to extract the first sample acoustic feature to obtain the sample speech feature; wherein the first sample acoustic feature is obtained by encoding the first sample fusion data, the first sample fusion data is obtained by fusing the second sample acoustic feature and the sample registration wake-up audio, and the second sample acoustic feature is obtained by encoding the sample wake-up audio by using the first encoder; And, using the second gated recurrent unit network, generating a sample text feature based on the sample registration wake-up text corresponding to the sample registration wake-up audio; The network parameters of the first gated recurrent unit network are adjusted by utilizing the difference between the sample text features and the sample speech features.

8. The method according to claim 7, characterized in that Before generating the sample text feature based on the sample registration wake-up text corresponding to the sample registration wake-up audio by using the second gated recurrent unit network, the method further includes: Obtaining a third sample acoustic feature; wherein the third sample acoustic feature is obtained by encoding the sample wake-up audio using a third encoding network of the first encoder; The using the second gated recurrent unit network to generate a sample text feature based on the sample registration wake-up text corresponding to the sample registration wake-up audio includes: fusing the third sample acoustic feature and the sample registration wake-up text to obtain second sample fusion data; The second gated recurrent unit network is used to perform feature extraction on the second sample fusion data to obtain the sample text features.

9. The method according to claim 1, characterized in that: The voice wake-up method is performed using a voice wake-up network. In the training stage, the voice wake-up network includes a wake-up subnetwork, a voice registration subnetwork and a text registration subnetwork. The wake-up subnetwork includes a first encoder. The training sequence of the first encoder precedes that of the wake-up subnetwork and the text registration subnetwork. The training sequence of the wake-up subnetwork and the text registration subnetwork precedes that of the voice registration subnetwork.

10. The method according to claim 1, characterized in that The target awakening result includes a target awakening tag; and the method further includes: In response to the target wake-up tag being a first tag, determining that the target wake-up result is used to indicate waking up the target device; In response to the target wake-up tag being the second tag, determining the target wake-up result is used to indicate not to wake up the target device.

11. A voice wake-up device, characterized in that: The device comprises: A first acquisition module is used to acquire a target registered wake-up audio corresponding to a target device and a target wake-up audio input by a user; A second acquisition module is used to respectively acquire a target wake-up feature corresponding to the target wake-up audio and a target voice feature corresponding to the target registration wake-up audio; A comparison module is used to compare the target wake-up feature and the target voice feature to obtain a target wake-up result; wherein the target wake-up result is used to indicate whether to wake up the target device.

12. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the voice wake-up method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the voice wake-up method according to any one of claims 1 to 10.

Images