Method for detecting keywords in speech and storage medium

Abstract

The present application discloses a method and storage medium for detecting keywords in speech. The method comprises: collecting the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; calling a keyword prediction model, wherein the keyword prediction model is trained and generated based on supervised learning by cross entropy loss between multiple first noise-enhanced samples and contrastive learning by contrast loss between multiple second noise-enhanced samples, the multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and the different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; using the keyword prediction model to extract noise enhancement features from the speech to be detected; removing the noise enhancement features from the speech to be detected to identify the target keyword. The present application solves the technical problem of low robustness in keyword detection.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a method and storage medium for detecting keywords in speech. Background Art

[0002] Currently, keyword detection can be used to detect predetermined words in a given utterance. For example, by issuing a command to a mobile phone in advance, for example, the mobile phone can be activated when "hello" is detected in the utterance.

[0003] In related technologies, the retrieval of words in a given speech is accomplished through voice interaction application functions. However, the above method relies on a clean audio set or an audio set of close-range conversations. In a noisy environment, the accuracy decreases and it is easy to be activated incorrectly. There is also a technical problem of low robustness in keyword detection.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a method and storage medium for detecting keywords in speech, so as to at least solve the technical problem of low robustness in keyword detection.

[0006] According to one aspect of an embodiment of the present application, a method and storage medium for detecting keywords in speech are provided. The method may include: collecting monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; calling a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on cross entropy loss between multiple first noise-enhanced samples and contrastive learning based on contrast loss between multiple second noise-enhanced samples, the multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; using the keyword prediction model to extract noise enhancement features from the speech to be detected; removing the noise enhancement features from the speech to be detected to identify the target keyword.

[0007] According to another aspect of the embodiments of the present application, a method for determining a model is provided. The method may include: obtaining a first speech sample and multiple second speech samples, wherein the multiple second speech samples are of the same type; performing mixed data enhancement on the first speech sample to obtain multiple first noise enhancement samples, and performing mixed data enhancement on the second speech samples of the same type to obtain multiple second noise enhancement samples; determining the cross entropy loss between the multiple first noise enhancement samples, and determining the contrast loss between the multiple second noise enhancement samples; performing self-supervised learning on the first speech sample based on the cross entropy loss, and performing contrast learning on the multiple second speech samples based on the contrast loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected.

[0008] According to another aspect of an embodiment of the present application, another method and storage medium for detecting keywords in speech are provided. The method may include: collecting speech to be detected sent to a client, wherein the speech to be detected contains at least one target keyword to be detected; extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on cross entropy loss between multiple first noise enhancement samples and contrastive learning based on contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; removing noise enhancement features from the speech to be detected to identify the target keyword; activating the client based on the target keyword.

[0009] According to another aspect of the embodiment of the present application, a speech generation method is provided. The method may include: collecting monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; using a keyword prediction model to extract noise enhancement features from the speech to be detected, wherein the keyword prediction model is trained and generated based on supervised learning by cross entropy loss between multiple first noise enhancement samples and contrastive learning by contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; removing the noise enhancement features from the speech to be detected to identify the target keyword; generating a target speech based on the target keyword, and playing the target speech, wherein the target speech includes the reply content to the speech to be detected.

[0010] According to another aspect of the embodiment of the present application, another method and storage medium for detecting keywords in speech are provided. The method may include: inputting a speech to be detected on a virtual reality (VR) device or an augmented reality (AR) device, wherein the speech to be detected contains at least one target keyword to be detected; extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and the different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; removing the noise enhancement features from the speech to be detected to identify the target keyword; and activating the VR device or AR device using the target keyword.

[0011] According to another aspect of the embodiment of the present application, another method and storage medium for detecting keywords in speech are provided. The method may include: obtaining the monitored speech to be detected by calling a first interface, wherein the speech to be detected contains at least one target keyword to be detected, the first interface includes a first parameter, and the parameter value of the first parameter is the speech to be detected; calling a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; using the keyword prediction model to extract noise enhancement features from the speech to be detected; removing the noise enhancement features from the speech to be detected to identify the target keyword; outputting the target keyword by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target keyword.

[0012] According to one aspect of an embodiment of the present application, a device for detecting keywords in speech is provided. The device may include: a first acquisition unit, for acquiring monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; a first calling unit, for calling a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; a first extraction unit, for extracting noise enhancement features from the speech to be detected using the keyword prediction model; a first processing unit, for removing the noise enhancement features from the speech to be detected to identify the target keyword.

[0013] According to another aspect of the embodiment of the present application, a device for determining a model is provided. The device may include: a first acquisition unit for acquiring a first speech sample and multiple second speech samples, wherein the multiple second speech samples are of the same type; a first enhancement unit for performing mixed data enhancement on the first speech sample to obtain multiple first noise enhancement samples, and performing mixed data enhancement on the second speech samples of the same type to obtain multiple second noise enhancement samples; a first determination unit for determining the cross entropy loss between the multiple first noise enhancement samples, and determining the contrast loss between the multiple second noise enhancement samples; a second processing unit for performing self-supervised learning on the first speech sample based on the cross entropy loss, and performing contrast learning on the multiple second speech samples based on the contrast loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected.

[0014] According to another aspect of an embodiment of the present application, another device for detecting keywords in speech is provided. The device may include: a second acquisition unit for collecting speech to be detected sent to a client, wherein the speech to be detected contains at least one target keyword to be detected; a second extraction unit for extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on cross entropy loss between multiple first noise enhancement samples and contrastive learning based on contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; a first recognition unit for removing noise enhancement features from the speech to be detected to identify the target keyword; and a first activation unit for activating the client based on the target keyword.

[0015] According to another aspect of the embodiment of the present application, a speech generation device is provided. The device may include: a third acquisition unit for acquiring the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; a third extraction unit for extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; a third processing unit for removing noise enhancement features from the speech to be detected to identify the target keyword; a fourth processing unit for generating a target speech based on the target keyword and playing the target speech, wherein the target speech includes the reply content to the speech to be detected.

[0016] According to another aspect of an embodiment of the present application, another device for detecting keywords in speech is provided. The device may include: an input unit for inputting a speech to be detected on a virtual reality (VR) device or an augmented reality (AR) device, wherein the speech to be detected contains at least one target keyword to be detected; a fourth extraction unit for extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and the different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; a fifth processing unit for removing noise enhancement features from the speech to be detected to identify the target keyword; and a second activation unit for activating the VR device or AR device using the target keyword.

[0017] According to another aspect of an embodiment of the present application, another device for detecting keywords in speech is provided. The device may include: a second acquisition unit for acquiring the monitored speech to be detected by calling a first interface, wherein the speech to be detected contains at least one target keyword to be detected, the first interface includes a first parameter, and the parameter value of the first parameter is the speech to be detected; a sixth processing unit for calling a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; a fifth extraction unit for extracting noise enhancement features from the speech to be detected using the keyword prediction model; a seventh processing unit for removing the noise enhancement features from the speech to be detected to identify the target keyword; an output unit for outputting the target keyword by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target keyword.

[0018] In an embodiment of the present application, monitored speech to be detected is collected, wherein the speech to be detected contains at least one target keyword to be detected; a keyword prediction model is called, wherein the keyword prediction model is trained and generated by supervised learning based on cross-entropy loss between multiple first noise-enhanced samples and contrastive learning based on contrast loss between multiple second noise-enhanced samples, the multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; noise enhancement features are extracted from the speech to be detected using the keyword prediction model; and the noise enhancement features are removed from the speech to be detected to identify the target keyword. That is to say, in the embodiment of the present application, supervised learning is performed based on the cross-entropy loss between multiple first noise-enhanced samples, and contrastive learning is performed based on the contrast loss between multiple second noise-enhanced samples to generate a keyword prediction model, so as to complete the strict restriction of the keyword prediction model. The embodiment of the present application does not perform noise enhancement from a single instance, which helps to directly optimize the keyword prediction model, and the input speech to be detected is obtained through hybrid enhancement, thereby avoiding over-training fitting of the keyword prediction model, and further achieving the technical effect of improving the robustness of keyword detection, and solving the technical problem of low robustness of keyword detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a structural block diagram of a computing environment according to an embodiment of the present application;

[0021] Figure 2 is a flow chart of a method for detecting keywords in speech according to an embodiment of the present application;

[0022] Figure 3 is a flow chart of a method for determining a model according to an embodiment of the present application;

[0023] Figure 4 is a flowchart of another method for detecting keywords in speech according to an embodiment of the present application;

[0024] Figure 5 is a flowchart of another speech generation method according to an embodiment of the present application;

[0025] Figure 6 is a flowchart of another method for detecting keywords in speech according to an embodiment of the present application;

[0026] Figure 7 is a flowchart of another method for detecting keywords in speech according to an embodiment of the present application;

[0027] Figure 8(a) is a schematic diagram of a comparative learning method in the related art.

[0028] FIG8( b ) is a schematic diagram of a regularization framework according to an embodiment of the present application;

[0029] FIG8( c ) is a schematic diagram of a regularized supervision model according to an embodiment of the present application;

[0030] Figure 9 This is a structural block diagram of a service grid for a method for detecting keywords in speech according to an embodiment of the present application;

[0031] Figure 10 is a schematic diagram of a device for detecting keywords in speech according to an embodiment of the present application;

[0032] Figure 11 is a schematic diagram of another model determination device according to an embodiment of the present application;

[0033] Figure 12 is a schematic diagram of another apparatus for detecting keywords in speech according to an embodiment of the present application;

[0034] Figure 13 is a schematic diagram of a speech generating device according to an embodiment of the present application;

[0035] Figure 14 is a schematic diagram of another apparatus for detecting keywords in speech according to an embodiment of the present application;

[0036] Figure 15 is a schematic diagram of another apparatus for detecting keywords in speech according to an embodiment of the present application;

[0037] Figure 16 This is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0041] Keyword Spotting (KWS), also known as keyword detection, can be used to detect keywords in a discourse.

[0042] Signal-to-noise ratio (SNR) is the ratio of signal to noise in an electronic device or electronic system, and can be used to measure noise levels in decibels.

[0043] Cross-entropy Error (CE) can be used as an optimization function for model training and can be used to measure the difference between two probability distributions.

[0044] Intersection over Union (IoU) can be a metric used in target detection, which can be the overlap rate of the predicted candidate box and the marked box, that is, the ratio of their intersection to their union.

[0045] Example 1

[0046] According to an embodiment of the present application, an embodiment of a method for detecting keywords in speech is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0047] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The block diagram shows the use of the above Figure 1The computer terminal 30 (or mobile device) is shown as an embodiment of a computing node in the computing environment 101 . Figure 1 is a structural block diagram of a computing environment according to an embodiment of the present application, such as Figure 1 As shown, computing environment 101 includes multiple services 120 (illustrated as 120-1, 120-2, ...) running on a distributed network of computing nodes (e.g., servers). Each computing node contains local processing and memory resources, and end users 102 can remotely run applications or store data in computing environment 101. Applications can be provided as multiple services 120-1, 120-2, 120-3, and 120-4 in computing environment 101, representing services "A," "D," "E," and "H," respectively.

[0048] End users 102 can provide and access services through a web browser or other software applications on the user side. In some embodiments, the provisioning and / or requests of end users 102 can be provided to the ingress gateway 130. The ingress gateway 130 can include a corresponding agent to handle the provisioning and / or requests for services 120 (one or more services provided in the computing environment 101).

[0049] The service 120 is provided or deployed based on various virtualization technologies supported by the computing environment 101. In some embodiments, the service 120 can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar approaches. Virtual machine-based virtualization can simulate a real computer by initializing a virtual machine, executing programs and applications without directly accessing any actual hardware resources. While virtual machines virtualize machines, container-based virtualization can launch containers to virtualize an entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0050] In an embodiment based on container virtualization, several containers of service 120 can be assembled into a POD (e.g., Kubernetes POD). Figure 1 As shown, service 120-2 may be equipped with one or more PODs 140-1, 140-2, ..., 140-N (collectively referred to as PODs 140). Each POD 140 may include a proxy 145 and one or more containers 142-1, 142-2, ..., 142-M (collectively referred to as containers 142). One or more containers 142 in POD 140 process requests related to one or more corresponding functions of the service, and proxy 145 generally controls network functions related to the service, such as routing, load balancing, etc. Other services 120 may also be equipped with PODs similar to POD 140.

[0051] During operation, executing a user request from the end user 102 may require calling one or more services 120 in the computing environment 101, and executing one or more functions of a service 120 may require calling one or more functions of another service 120. Figure 1 As shown, service “A” 120 - 1 receives a user request from end user 102 from ingress gateway 130 , service “A” 120 - 1 may call service “D” 120 - 2 , and service “D” 120 - 2 may request service “E” 120 - 3 to perform one or more functions.

[0052] This computing environment can be a cloud computing environment, where resource allocation is managed by the cloud service provider, allowing for feature development without having to worry about implementing, adjusting, or scaling servers. This computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be partitioned to perform a set of functions that can scale independently and automatically, rather than scaling a single hardware device to handle the potential load.

[0053] exist Figure 1 Under the operating environment shown, this application provides an application for Figure 2 It should be noted that the method for detecting keywords in speech of this embodiment can be Figure 1 The illustrated embodiment is executed by a mobile terminal.

[0054] Figure 2 FIG. 1 is a flow chart of a method for detecting keywords in speech according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0055] Step S202 : collecting the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected.

[0056] In the technical solution provided in step S202 above, the speech to be detected is monitored and the monitored speech to be detected is collected, wherein the speech to be detected contains at least one target keyword to be detected. The target keyword can be a pre-set keyword, for example, "Hello, Xiao Ai classmate". The speech to be detected can be speech acquired in a variety of scenarios, for example, it can be speech monitored in scenarios such as on a train or in a subway.

[0057] Step S204: calling a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on the cross entropy loss between multiple first noise-enhanced samples and performing contrastive learning based on the contrast loss between multiple second noise-enhanced samples. The multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0058] In the technical solution provided in the above step S204 of the present application, multiple first voice samples and multiple second voice samples can be obtained. The multiple first voice samples can be subjected to mixed data enhancement to obtain first noise enhancement samples, thereby obtaining multiple first noise enhancement samples. Multiple second voice samples of the same type in the multiple second voice samples can be subjected to mixed data enhancement to obtain second noise enhancement samples, thereby obtaining multiple second noise enhancement samples. Supervised learning can be performed based on the cross entropy loss between the multiple first noise enhancement samples and comparative learning can be performed based on the contrast loss between the multiple second noise enhancement samples to train a keyword prediction model. The keyword prediction model can be called to predict the collected speech to be detected.

[0059] Among them, the keyword prediction model can be a keyword spotting (KWS) model, which can be used to search for keywords in the speech. The first noise enhancement sample can be multiple enhanced views of the same sample (for example, multiple views of enhanced noise), noise enhanced audio, parallel samples of multi-perspective background noise, etc. This is only an example, and there is no specific restriction on the type of the first noise enhancement sample. Cross Entropy (CE) loss can be used to optimize the keyword prediction model. Contrastive loss (L I2CR ) can be used as a regularizer. The second noise enhancement sample can be a noise enhancement sample obtained by performing data enhancement on a speech sample of the same type. The same type can refer to speech samples with the same label or in the same class group.

[0060] Optionally, a first noise-enhanced sample can be obtained by performing mixed data augmentation on the first speech sample. A second noise-enhanced sample can be obtained by performing mixed data augmentation on multiple second speech samples of the same type. For example, a second noise-enhanced sample can be obtained by performing mixed data augmentation on two second speech samples with the same label. It should be noted that the above two examples are merely illustrative and do not impose a specific limit on the number of second speech samples that undergo mixed data augmentation.

[0061] Optionally, data augmentation can be performed by adding randomly sampled background noise to clean speech to artificially simulate a real noisy environment. The noise level of the background noise can be adjusted by calculating the signal-to-noise ratio (SNR) and then adding the noise level to the speech sample, thereby obtaining parallel samples of background noise from multiple perspectives.

[0062] If the noise added at the beginning is too large, it will affect the model's ability to recognize similar features in (positive) multi-view samples, resulting in poor convergence. Therefore, in an embodiment of the present application, fixed noise data can be provided during training, and the SNRs noise can be slowly increased during training. For example, it can be from no noise to 20dB, from no noise to 20dB and finally to 15dB; it can be from no noise to 20dB, then to 15dB, then to 10dB, and all the way to 20dB to -10dB.

[0063] Optionally, while simulating noisy speech, other processing can also be performed on the speech to achieve the purpose of data enhancement. For example, the speech can be pitch-shifted to randomly lower or raise the pitch of the speech; the speech can be codec-enhanced by using a codec vocoder to synthesize audio; the data can be enhanced by time shifting, speed perturbation, video masking, etc. In the embodiments of the present application, multiple data enhancement methods are provided, and different data enhancement methods can be selected according to the generalized scenario of use, thereby improving the adaptability of the model to the usage scenario.

[0064] In this embodiment, data augmentation is performed on different samples belonging to the same class, rather than simply enhancing noise from the same input instance. In other words, this embodiment avoids the related art approach of simply enhancing noise from the same input instance, thereby reducing the possibility of the model being affected by other unfavorable factors and thereby improving the accuracy of the model's predictions. Among these unfavorable factors, such as speaker identity and voice pitch, are examples only and are not specifically limited.

[0065] In the related art, the traditional supervised learning keyword prediction model focuses on identifying association patterns to distinguish data from multiple classes, but the keyword prediction model of this method cannot suppress negative factors. For example, negative factors can be the speaker's speech style and background noise, which leads to a decrease in the accuracy of the keyword prediction model. In the embodiment of the present application, a first noise-enhanced sample can be obtained by performing mixed data enhancement on the first speech sample, thereby achieving the purpose of creating a dual view from one sample, and using contrastive learning constraints to encourage the keyword prediction model to focus more on extracting generalized class features from two different noise-enhanced audios, thereby reducing the impact of negative factors and reducing the amount of noise in the potential features, thereby making the keyword prediction model robust to noise, and achieving the technical effect of improving the robustness of keyword detection.

[0066] Step S206: extracting noise enhancement features from the speech to be detected using the keyword prediction model.

[0067] In the technical solution provided in step S206 of the present application, the keyword prediction model can be called to predict the speech to be detected, so as to extract a noise enhancement feature from the speech to be detected. The noise enhancement feature can be used to characterize the noise in the environment, for example, the noise in the environment can be noise above 0dB.

[0068] Step S208: removing the noise enhancement feature from the speech to be detected to identify the target keyword.

[0069] In the technical solution provided in the above step S208 of the present application, the noise enhancement feature can be removed from the speech to be detected, thereby reducing the impact of noise on the model prediction process, and further accurately identifying the target keyword.

[0070] When searching for target keywords in speech to be detected using a keyword prediction model, the presence of environmental noise can lead to a technical problem of low robustness in keyword detection. However, the present embodiment combines the cross-entropy loss and contrastive loss from supervised learning and applies them to the keyword prediction model, thereby improving the accuracy of the keyword prediction model.

[0071] Optionally, in the process of predicting the target keyword in the speech to be detected, the keyword prediction model can first extract the noise enhancement features from the speech to be detected, and can remove the noise enhancement features from the speech to be detected, thereby identifying the target keyword, achieving the technical effect of improving the robustness of keyword detection, and solving the technical problem of low robustness of keyword detection.

[0072] Through the above-mentioned steps S202 to S208 of the present application, the speech to be detected containing at least one target keyword to be detected is collected and monitored; a keyword prediction model is called to perform supervised learning based on the cross-entropy loss between multiple first noise-enhanced samples, and to perform comparative learning based on the contrast loss between multiple second noise-enhanced samples; the keyword prediction model is used to extract noise enhancement features from the speech to be detected; the noise enhancement features are removed from the speech to be detected to identify the target keyword, thereby achieving the technical effect of improving the robustness of keyword detection and solving the technical problem of low robustness of keyword detection.

[0073] The above method of this embodiment is further introduced below.

[0074] As an optional implementation, the method also includes: performing mixed data enhancement on the first randomly sampled noise data in the first voice sample according to a first mixing coefficient to obtain a first noise enhanced sample, wherein the first randomly sampled noise data increases as the training rounds of the keyword prediction model increase, and the first mixing coefficient conforms to the Beta distribution; performing mixed data enhancement on the second randomly sampled noise data in the second voice sample according to a second mixing coefficient to obtain a second noise sample, wherein the second randomly sampled noise data increases as the training rounds of the keyword prediction model increase, and the second mixing coefficient conforms to the Beta distribution.

[0075] In this embodiment, the first randomly sampled noise data in the first speech sample can be mixed data enhanced according to the first mixing coefficient to obtain a first noise enhanced sample. The first randomly sampled noise data (randomly sampled utterances) can be sampled noise data obtained by random sampling, for example, it can be noise data of 5dB obtained by random sampling in a railway environment, etc. This is only for example, and no specific restrictions are placed on the acquisition scenario of the first randomly sampled noise data and the size of the noise. The first randomly sampled noise data can increase as the number of training rounds of the keyword prediction model increases, and the first mixing coefficient can gradually increase with the Beta distribution. For example, the first mixing coefficient can gradually increase from 0 to 0.5 as the number of training rounds increases. The first mixing coefficient can be a Beta coefficient.

[0076] For example, the first noise-enhanced sample may be obtained by performing mixed data enhancement on the first randomly sampled noise data in the first speech sample according to a first mixing coefficient, wherein the first mixing coefficient may be obtained from a Beta distribution.

[0077] In an embodiment of the present application, a binary cross entropy loss weighted by a beta coefficient can be used to learn mixed inputs, which helps to reduce overfitting during model optimization.

[0078] In this embodiment, the second randomly sampled noise data in the second speech sample can be subjected to mixed data enhancement according to a second mixing coefficient to obtain a second noise sample. The second randomly sampled noise data increases with the number of training rounds of the keyword prediction model, and the second mixing coefficient conforms to a Beta distribution. For example, the second mixing coefficient can be 0, 0.5, etc. This is for illustration only and does not impose a specific limitation on the size of the mixing coefficient.

[0079] For example, the first noise-enhanced sample may be obtained by performing mixed data enhancement on the second randomly sampled noise data in the second speech sample according to a second mixing coefficient, wherein the second mixing coefficient may be obtained from a Beta distribution.

[0080] As an optional implementation, self-supervised learning is performed on the first speech sample based on cross entropy loss, and contrastive learning is performed on multiple second speech samples based on contrastive loss to obtain a keyword prediction model.

[0081] In this embodiment, self-supervised learning can be performed on the first speech sample based on cross entropy loss, and contrastive learning can be performed on multiple second speech samples based on contrastive loss to obtain a keyword prediction model.

[0082] In an embodiment of the present application, during the model training process, a contrast loss is used on the basis of the cross-entropy loss, and a keyword prediction model is obtained by jointly training the cross-entropy loss and the contrast loss, wherein the contrast loss can be an added regularization term. By adding the regularization term, the performance and training speed of the keyword prediction model are improved, and the noise robustness of the keyword prediction model is further improved.

[0083] As an optional implementation, self-supervised learning is performed on the first speech sample based on cross-entropy loss, and comparative learning is performed on multiple second speech samples based on contrastive loss to obtain a keyword prediction model, including: establishing a regularizer based on cross-entropy loss and contrastive loss, wherein the regularizer is used to prohibit the keyword prediction model from encoding noise enhancement features; self-supervised learning is performed on the first speech sample based on the regularizer, and comparative learning is performed on multiple second speech samples to obtain a keyword prediction model.

[0084] In this embodiment, a regularizer can be established based on cross-entropy loss and contrastive loss. By adding a regularization constraint to the standard cross-entropy loss, a regularizer with cross-entropy is obtained. The regularizer can be used to prohibit the keyword prediction model from encoding noise enhancement features, thereby achieving the purpose of constraining the model to encode noise. In this embodiment, self-supervised learning can be performed on a first speech sample based on the obtained regularizer, and contrastive learning can be performed on multiple second speech samples based on the regularizer to obtain a keyword prediction model.

[0085] As an optional implementation, a regularizer is established based on the cross entropy loss and the contrast loss, including: obtaining an initial regularizer corresponding to the contrast loss; and adjusting the initial regularizer based on the cross entropy loss to obtain a regularizer.

[0086] In this embodiment, an initial regularizer corresponding to the contrastive loss may be obtained, and the initial regularizer may be adjusted based on the cross entropy loss to obtain a regularizer. The initial regularizer may be an initially established contrastive regularizer.

[0087] As an optional implementation, a regularizer is established based on cross-entropy loss and contrast loss, including: performing weighted summation of the cross-entropy loss and the contrast loss to obtain a target loss, wherein the weight corresponding to the contrast loss is used to indicate the degree of constraint of the contrast loss on the keyword prediction model, and the weight increases linearly with the increase in the number of training rounds of the keyword prediction model; and establishing a regularizer corresponding to the target loss.

[0088] In this embodiment, the cross entropy loss and the contrast loss can be weighted and summed to obtain the target loss, wherein the weight corresponding to the contrast loss can be used to represent the degree of constraint of the contrast loss on the keyword prediction model, and the weight can be used as the key prediction model. The target loss (L) is a combined loss function, which can be calculated by the following formula:

[0089] L=L CE +αL I2CR

[0090] Among them, L CE is the cross entropy loss for supervised learning, L I2CR is the contrast loss as a regularizer. α is the weight given to the regularizer, which can be used to measure the degree of additional contrast constraint.

[0091] In this embodiment, the contrast loss can be used as a regularizer to achieve the purpose of establishing a regularizer corresponding to the target loss.

[0092] As an optional implementation, self-supervised learning is performed on the first voice sample based on the regularizer, and comparative learning is performed on multiple second voice samples to obtain a keyword prediction model, including: self-supervised learning is performed on the first voice sample based on the regularizer, and comparative learning is performed on multiple second voice samples to obtain model adjustment parameters; parameters of the initial keyword prediction model are adjusted based on the model adjustment parameters to obtain a keyword prediction model.

[0093] In this embodiment, self-supervised learning can be performed on the first speech sample based on the regularizer, and comparative learning can be performed on multiple second speech samples to obtain model adjustment parameters. The initial keyword prediction model can be adjusted based on the model adjustment parameters, and the keyword prediction model can be obtained by continuously fine-tuning the initial keyword prediction model.

[0094] In an embodiment of the present application, a regularizer can be used to continuously fine-tune the initial keyword prediction model to obtain a keyword prediction model, thereby achieving the purpose of improving the accuracy of model prediction.

[0095] As an optional implementation, removing noise enhancement features from the speech to be detected to identify target keywords includes: removing noise enhancement features from the speech to be detected to obtain keyword features; and obtaining the target keyword represented by the keyword features.

[0096] In this embodiment, the noise enhancement feature can be removed from the speech to be detected to obtain a keyword feature, which can be processed to obtain a target keyword represented by the keyword feature. The keyword feature can be an improved feature representation, and the keyword feature can be expressed in the form of a coding feature.

[0097] As an optional implementation manner, the similarity between the first noise-enhanced sample and the second noise-enhanced sample is greater than a first similarity threshold.

[0098] In this embodiment, the similarity between the first noise-enhanced sample and the second noise-enhanced sample may be determined until the similarity between the first noise-enhanced sample and the second noise-enhanced sample is greater than a first similarity threshold, and then the fine-tuning of the initial keyword prediction model is completed.

[0099] Optionally, the similarity between the first noise-enhanced sample and the second noise-enhanced sample can be made greater than a first similarity threshold, thereby maximizing the similarity between noise-enhanced samples of the same type, and then bringing the noise-enhanced samples closer to a broader representation. This representation forms a more prominent cluster and reduces the adverse effects of noise, which can improve the accuracy of the keyword prediction model in predicting the target keyword.

[0100] In an embodiment of the present application, supervised learning is performed based on the cross-entropy loss between multiple first noise-enhanced samples, and contrastive learning is performed based on the contrast loss between multiple second noise-enhanced samples to generate a keyword prediction model, so as to complete the strict restriction of the keyword prediction model. The embodiment of the present application does not perform noise enhancement from a single instance, which helps to directly optimize the keyword prediction model, and the input speech to be detected is obtained through hybrid enhancement, thereby avoiding over-training fitting of the keyword prediction model, and further achieving the technical effect of improving the robustness of keyword detection, and solving the technical problem of low robustness of keyword detection.

[0101] The following further introduces the method for detecting keywords in speech in the embodiment of the present application from the perspective of model training.

[0102] Figure 3 FIG. 1 is a flow chart of a method for determining a model according to an embodiment of the present application. Figure 3 As shown, the method may include the following steps:

[0103] Step S302: Acquire a first voice sample and multiple second voice samples, where the multiple second voice samples are of the same type.

[0104] In the technical solution provided in the above step S302 of the present application, a first voice sample and multiple second voice samples can be obtained, wherein the multiple second voice samples are of the same type.

[0105] Optionally, the first voice sample and the plurality of second voice samples may be acquired by random sampling.

[0106] For example, a first voice sample and multiple second voice samples can be obtained by random sampling based on a voice command dataset, where the voice command dataset can be composed of 10 class subsets, which can correspond to labels of "up", "down", "left", "right", "yes", "no", "on", "off", "go", and "stop" respectively.

[0107] Step S304: performing mixed data enhancement on the first speech sample to obtain a plurality of first noise-enhanced samples, and performing mixed data enhancement on the second speech samples of the same type to obtain a plurality of second noise-enhanced samples.

[0108] In the technical solution provided in step S304 of the present application, mixed data enhancement can be performed on the first speech sample to obtain multiple first noise-enhanced samples, and mixed data enhancement can be performed on second speech samples of the same type to obtain multiple second noise-enhanced data. The second speech samples of the same type can be samples with the same label.

[0109] Optionally, mixed data augmentation (Mixup augmentation) can be performed on multiple first speech samples to obtain multiple first noise-enhanced samples. For example, corresponding scaling coefficients can be set for each of the multiple first speech samples, and mixed data augmentation can be performed on the multiple first speech samples based on the scaling coefficients. Mixup augmentation can also be performed on each of the second speech samples of the same type to obtain multiple second noise-enhanced data.

[0110] Optionally, data augmentation can be performed by adding randomly sampled background noise to the clean speech to artificially simulate a real noisy environment. The noise level of the background noise can be adjusted by calculating the signal-to-noise ratio and then adding the noise level to the speech sample, thereby obtaining parallel samples of background noise from multiple perspectives.

[0111] If the noise added at the beginning is too large, it will affect the model's ability to recognize similar features in (positive) multi-view samples, resulting in poor convergence. Therefore, in an embodiment of the present application, fixed noise data can be provided during training, and the SNRs noise can be slowly increased during training. For example, it can be from no noise to 20dB, from no noise to 20dB and finally to 15dB; it can be from no noise to 20dB, then to 15dB, then to 10dB, and all the way to 20dB to -10dB.

[0112] Optionally, while simulating noisy speech, other processing can also be performed on the speech to achieve the purpose of data enhancement. For example, the speech can be pitch-shifted to randomly lower or raise the pitch of the speech; the speech can be codec-enhanced by using a codec vocoder to synthesize audio; the data can be enhanced by time shifting, speed perturbation, video masking, etc. In the embodiments of the present application, multiple data enhancement methods are provided, and different data enhancement methods can be selected according to the generalized scenario of use, thereby improving the adaptability of the model to the usage scenario.

[0113] In this embodiment, data augmentation is performed on different samples belonging to the same class, rather than simply enhancing noise from the same input instance. In other words, this embodiment avoids the related art approach of simply enhancing noise from the same input instance, thereby reducing the possibility of the model being affected by other unfavorable factors and thereby improving the accuracy of the model's predictions. Among these unfavorable factors, such as speaker identity and voice pitch, are examples only and are not specifically limited.

[0114] Step S306 : determining a cross entropy loss between the plurality of first noise enhanced samples, and determining a contrast loss between the plurality of second noise enhanced samples.

[0115] In the technical solution provided in the above step S306 of the present application, the cross entropy loss between the plurality of first noise-enhanced samples may be determined, and the contrast loss between the plurality of second noise-enhanced samples may be determined.

[0116] Step S308, self-supervised learning is performed on the first speech sample based on cross-entropy loss, and contrastive learning is performed on multiple second speech samples based on contrastive loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected.

[0117] In the technical solution provided in the above step S308 of the present application, self-supervised learning can be performed on the first speech sample with the cross-entropy loss, and comparative learning can be performed on multiple second speech samples based on the contrast loss to obtain a keyword prediction model. The keyword prediction model can be used to predict the speech to be detected and extract noise enhancement features, so that the target keywords in the speech to be detected can be identified.

[0118] Optionally, during supervised learning, self-supervised learning can be performed on the first noise enhancement sample based on cross entropy, the weights during data mixing enhancement can be used when calculating the loss, and comparative learning can be performed on multiple second speech samples based on contrast loss to obtain a keyword prediction model. The keyword prediction model can be used to predict the speech to be detected, and the noise enhancement features can be extracted, so that the target keywords in the speech to be detected can be identified.

[0119] Through the above steps S302 to S308 of the present application, a first speech sample and multiple second speech samples are obtained, wherein the multiple second speech samples are of the same type; mixed data enhancement is performed on the first speech sample to obtain multiple first noise enhancement samples, and mixed data enhancement is performed on the second speech samples of the same type to obtain multiple second noise enhancement samples; the cross entropy loss between the multiple first noise enhancement samples is determined, and the contrast loss between the multiple second noise enhancement samples is determined; self-supervised learning is performed on the first speech sample based on the cross entropy loss, and contrast learning is performed on the multiple second speech samples based on the contrast loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected, thereby achieving the technical effect of improving the robustness of keyword detection and solving the technical problem of low robustness of keyword detection.

[0120] The following further introduces the method for detecting keywords in speech in the embodiment of the present application with respect to the application scenario of application activation.

[0121] Figure 4is a flow chart of another method for detecting keywords in speech according to an embodiment of the present application, such as Figure 4 As shown, the method may include the following steps.

[0122] Step S402: collecting the speech to be detected sent to the client, wherein the speech to be detected contains at least one target keyword to be detected.

[0123] In the technical solution provided in the above step S402 of the present application, the voice to be detected can be collected and sent to the client, which can be a mobile phone, smart device, etc.

[0124] Step S404: Use a keyword prediction model to extract noise enhancement features from the speech to be detected, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples. The multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0125] Step S406: remove the noise enhancement feature from the speech to be detected to identify the target keyword.

[0126] Step S408: activating the client based on the target keyword.

[0127] In the technical solution provided in the above step S408 of the present application, the client can be activated in response to the identified keywords.

[0128] For example, when the target keyword "turn on the alarm clock in the mobile phone" is recognized in the voice to be detected, the client can execute the action of turning on the alarm clock in the mobile phone.

[0129] The above method of this embodiment is further introduced below.

[0130] As an optional implementation, the method further includes: activating the client based on the target keyword, including: activating the client in response to a similarity between the target keyword and a predetermined keyword associated with the client being greater than a second similarity threshold.

[0131] In this embodiment, whether the similarity between the target keyword and a predetermined keyword associated with the client is greater than a second similarity is determined. In response to the similarity between the target keyword and the predetermined keyword associated with the client being greater than the second similarity threshold, the client may be activated. The predetermined keyword may be a pre-set keyword, for example, "Hello, please turn on your phone, Xiao Ai." The second similarity threshold may be a pre-set threshold, which may be set by the client based on actual usage or directly set by the system.

[0132] In an embodiment of the present application, a speech to be detected is collected and sent to a client, wherein the speech to be detected contains at least one target keyword to be detected; a noise enhancement feature is extracted from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; the noise enhancement feature is removed from the speech to be detected to identify the target keyword; the client is activated based on the target keyword, thereby

[0133] The following further introduces the speech generation method in the embodiment of the present application with respect to the application scenario of speech interaction.

[0134] Figure 5 is a flow chart of another speech generation method according to an embodiment of the present application. Figure 5 As shown, the method may include the following steps.

[0135] Step S502 : collecting the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected.

[0136] Step S504: Use a keyword prediction model to extract noise enhancement features from the speech to be detected, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples. The multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0137] Step S506: remove the noise enhancement feature from the speech to be detected to identify the target keyword.

[0138] Step S508: Generate a target voice based on the target keyword and play the target voice, wherein the target voice includes the reply content to the voice to be detected.

[0139] In the technical solution provided in the above step S508 of the present application, a target voice can be generated based on the target keyword, and the target voice can be played, wherein the target voice can include reply content to the voice to be detected.

[0140] For example, if the target keyword is "Xiao Ai, please turn on your phone", the target voice can be "OK", which can be used to reply to the content of the target keyword.

[0141] In an embodiment of the present application, a monitored speech to be detected is collected, wherein the speech to be detected contains at least one target keyword to be detected; a noise enhancement feature is extracted from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on cross-entropy loss between multiple first noise enhancement samples and contrastive learning based on contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; the noise enhancement feature is removed from the speech to be detected to identify the target keyword; a target speech is generated based on the target keyword and the target speech is played, wherein the target speech includes the reply content to the speech to be detected, thereby achieving the technical effect of improving the robustness of keyword detection and solving the technical problem of low robustness of keyword detection.

[0142] According to an embodiment of the present application, a method for detecting keywords in speech in virtual reality scenarios such as virtual reality (VR) devices and augmented reality (AR) devices is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0143] Figure 6 is a flow chart of another method for detecting keywords in speech according to an embodiment of the present application, such as Figure 6 As shown, the method may include the following steps.

[0144] Step S602: inputting a voice to be detected on a virtual reality (VR) device or an augmented reality (AR) device, wherein the voice to be detected contains at least one target keyword to be detected.

[0145] Step S604: Use a keyword prediction model to extract noise enhancement features from the speech to be detected, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples. The multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0146] Step S606: remove the noise enhancement feature from the speech to be detected to identify the target keyword.

[0147] Step S608: Activate the VR device or AR device using the target keyword.

[0148] Through the above steps, by inputting the voice to be detected on the virtual reality VR device or the augmented reality AR device, wherein the voice to be detected contains at least one target keyword that needs to be detected; using the keyword prediction model to extract the noise enhancement feature from the voice to be detected, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first voice samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second voice samples of the same type; removing the noise enhancement feature from the voice to be detected to identify the target keyword; using the target keyword to activate the VR device or AR device, thereby solving the technical problem of low robustness of keyword detection and achieving the technical effect of improving the robustness of keyword detection in voice.

[0149] Optionally, in this embodiment, the above-mentioned method for detecting keywords in speech can be applied to a hardware environment consisting of a server and a virtual reality device. The server can be a server corresponding to a media file operator to control the VR device or AR device to perform human-computer interaction operations corresponding to the positioning information. The above-mentioned network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network. The above-mentioned virtual reality device is not limited to, a virtual reality helmet, virtual reality glasses, or an all-in-one virtual reality device.

[0150] Optionally, the virtual reality device may include: a memory, a processor, and a transmission device. The memory is used to store an application, which can be used to execute: inputting a voice to be detected on a virtual reality VR device or an augmented reality AR device, wherein the voice to be detected contains at least one target keyword to be detected; extracting noise enhancement features from the voice to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on cross-entropy loss between multiple first noise enhancement samples and contrastive learning based on contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first voice samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second voice samples of the same type; removing the noise enhancement features from the voice to be detected to identify the target keyword; and activating the VR device or AR device using the target keyword.

[0151] It should be noted that the above-mentioned method for detecting keywords in speech applied to VR devices or AR devices in this embodiment may include: Figure 6 The method of the illustrated embodiment is used to control a VR device or an AR device to perform human-computer interaction operations corresponding to positioning information.

[0152] Optionally, the processor of this embodiment can call the application stored in the memory to perform the above steps through a transmission device. The transmission device can receive media files sent by the server through the network, and can also be used for data transmission between the processor and the memory.

[0153] Optionally, in a virtual reality device, a head-mounted display with eye tracking is provided, wherein the screen in the HMD is used to display the displayed video images, the eye tracking module in the HMD is used to obtain the real-time movement path of the user's eyes, the tracking system is used to track the user's position information and motion information in the real three-dimensional space, and the computing processing unit is used to obtain the user's real-time position and motion information from the tracking system, and calculate the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space, etc.

[0154] In an embodiment of the present application, a virtual reality device can be connected to a terminal, and the terminal and the server are connected through a network. The above-mentioned virtual reality device is not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines, etc. The above-mentioned terminals are not limited to PCs, mobile phones, tablets, etc. The server can be a server corresponding to a media file operator, and the above-mentioned network includes but is not limited to: wide area network, metropolitan area network or local area network.

[0155] In the above operating environment, one embodiment of the present application also provides another example Figure 7It should be noted that the method for detecting keywords in speech of this embodiment can be Figure 1 The illustrated embodiment is executed by a mobile terminal. Figure 7 FIG. 1 is a flow chart of another method for detecting keywords in speech according to an embodiment of the present application. Figure 7 As shown, the method may include the following steps:

[0156] Step S702: Acquire the monitored speech to be detected by calling the first interface, wherein the speech to be detected contains at least one target keyword to be detected. The first interface includes a first parameter, and the parameter value of the first parameter is the speech to be detected.

[0157] In the technical solution provided in the above step S702 of the present application, the first interface can be an interface for data interaction between the server and the client. The client can use the voice to be detected that contains at least one target keyword to be detected as a first parameter of the first interface to achieve the purpose of obtaining the detected voice to be detected using the first interface.

[0158] Step S704: calling a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on the cross entropy loss between multiple first noise-enhanced samples and performing contrastive learning based on the contrast loss between multiple second noise-enhanced samples. The multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0159] Step S706: extract noise enhancement features from the speech to be detected using the keyword prediction model.

[0160] Step S708: remove the noise enhancement feature from the speech to be detected to identify the target keyword.

[0161] Step S710: Output the target keyword by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target keyword.

[0162] In the technical solution provided in the above step S710 of the present application, the second interface can be an interface for data interaction between the server and the client. The server can pass the target keyword into the second interface as a parameter of the second interface to achieve the purpose of outputting the target keyword.

[0163] Through the above steps, the monitored speech to be detected is obtained by calling the first interface, wherein the speech to be detected contains at least one target keyword to be detected, the first interface includes a first parameter, and the parameter value of the first parameter is the speech to be detected; the keyword prediction model is called, wherein the keyword prediction model is trained and generated based on supervised learning by cross-entropy loss between multiple first noise enhancement samples, and based on contrastive learning by contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; the keyword prediction model is used to extract noise enhancement features from the speech to be detected; the noise enhancement features are removed from the speech to be detected to identify the target keyword; the target keyword is output by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target keyword, thereby achieving the technical effect of improving the robustness of keyword detection and solving the technical problem of low robustness of keyword detection.

[0164] Example 2

[0165] Keyword search can be used to detect predetermined words in a given utterance. For example, if an application allows the phone to be activated by issuing the command "Hello (e.g., Hey Siri)" to the phone, the phone can be activated when Hey Siri is detected in the utterance.

[0166] In the related art, a keyword retrieval model is provided, which can be used to complete voice interaction application functions and retrieve words in a given speech. However, the current construction of this model mainly relies on clean or close-range conversation audio sets. Therefore, under noisy conditions, the trained model has poor robustness and noise robustness, that is, in a noisy environment, the accuracy decreases and there are cases of erroneous activation, resulting in a technical problem of low robustness in keyword detection.

[0167] It is increasingly common to use keyword retrieval models in noisy environments, such as when driving in traffic. However, in such situations, keyword retrieval models are easily affected by traffic noise. Therefore, it is necessary to build noise-robust keyword retrieval models.

[0168] In related art, multi-condition training is used to adapt the model to multiple levels of signal-to-noise ratio noise. However, without any additional noise enhancement calculations, feature representation is still more severely affected by low signal-to-noise ratio (SNR) noise. Therefore, multi-condition training relies heavily on the network to adapt and generalize to the selected SNR noise environment. However, outside the conditioning range, the recognition performance of keywords in different noises with different SNRs is expected to be poor, especially for keyword retrieval models with lower complexity.

[0169] To achieve better representations, keyword prediction models need to be more aware of the key representational features used to predict the class and reduce the impact of noise. This makes contrastive learning a potential means of improving the model's robustness to noise. Figure 8(a) is a schematic diagram of a contrastive learning approach based on related art. As shown in Figure 8(a), contrastive learning aims to extract intrinsic features between two differently enhanced views of a sample (called positive) by bringing the representations of samples closer together in the embedding space while pushing other samples (called negatives) further apart. For example, it pulls closer the distance between sample #12 with car noise and sample #12 with subway noise, and pushes further apart the distances between sample #1 with subway noise, sample #2 with traffic noise, and sample #3 with car noise and sample #4 with traffic noise. By contrasting the positive pair with two differently enhanced noises, the impact of noise can be conceptually reduced, resulting in higher-quality features. However, the model is forced to learn robust speech representations (keyword features) and reject noise information in the input samples.

[0170] To solve the above problems, an embodiment of the present application proposes a method of using internal contrast regularization to improve the noise robustness of keyword retrieval, and by adding a second constraint to help improve feature representation, thereby achieving improved performance without changing the model architecture design, further improving the robustness to invisible noise and reducing the signal-to-noise ratio.

[0171] Figure 8(b) is a schematic diagram of a regularization framework according to an embodiment of the present application. As shown in Figure 8(b), a contrast regularization (Inter-Intra, abbreviated as I2CR) method is proposed in the embodiment of the present application to improve the noise robustness of the KWS model. By using both enhanced samples within a given sample (for example, sample #13 in Figure 8(b)) as positive samples and other samples with the same label (for example, sample #15 and sample #18 in Figure 8(b)) as positive samples, the problems of traditional cross entropy and contrastive learning are alleviated, so that better speech representation and faster training can be achieved. As shown in Figure 8(b), the embodiment of the present application can use additive noise together with other enhancements. Experimental results show that the contrast regularization method proposed in the embodiment of the present application has good robustness to both in-domain noise and out-of-domain noise.

[0172] In the related art, the traditional supervised learning keyword prediction model focuses on identifying association patterns to distinguish data from multiple classes, but the keyword prediction model of this method cannot suppress negative factors, such as the speaker's speaking style and background noise. The inability to suppress negative factors leads to a decrease in the performance of noise keyword retrieval. In other words, the embedding of the potential representation may be severely dominated by background noise. The embodiment of the present application maximizes the information on the positive view and minimizes the information on the negative view by contrastive learning constraints itself. Through noise enhancement, a dual view can be created from the sample, and contrastive learning constraints are used to encourage the model to focus more on extracting generalized class features from two different noise-enhanced audios. This helps to reduce the amount of noise encoded in the potential features, making the model robust to noise.

[0173] The embodiment of the present application proposes a self-supervised contrastive learning framework consisting of training. Figure 8(c) is a schematic diagram of a regularized supervision model according to an embodiment of the present application. As shown in Figure 8(c), we can first perform self-supervised pre-training and then supervise fine-tune the labeled data. However, this usually requires a large amount of training resources (for example, a large pre-training dataset), and optimization is challenging. For example, there may be dimensionality collapse in contrastive self-supervised learning. In this work, we offset the limitations of supervised learning by combining supervised learning of cross entropy with contrastive learning of contrastive loss as a regularizer, using the supervised loss to stabilize the training from dimensionality collapse and improve the robustness of the model from contrastive loss in a noisy environment.

[0174] In this embodiment, anchor pairs can be enhanced by pairing anchor instances with multiple more positive views, allowing the model to better infer the embedding of class representations by adding references from the set of positive pairs. The model encodes basic features that do not contain other attributes besides class information. Therefore, these positive pairs should come from different samples of different speakers from the same class group (i.e., inter-view contrast). Since the labels are provided during supervised training, positive pairs can be formed from the mini-batch by identifying samples with the same target label (i.e., inter-view vs. intra-view). Optionally, the contrastive loss can be calculated by the following formula:

[0175]

[0176] Here, P(i) can be the set of all positive pairs of indices, and sim(zi,zp) can be the dot product (cosine similarity) between the normalized vectors zi and zp.

[0177] Alternatively, in supervised label training, different samples of the same class can be extracted from multiple frontal views, which can be derived from a subset (mini-batch) of the samples sampled during model training. For each iteration, labels can be used to extract samples of the same class, and the obtained samples can be clustered to achieve pairing between internal views. A loss function can be calculated based on the pairing results, and model parameters can be adjusted based on the loss function.

[0178] The regularized supervised model is achieved by using the cross entropy loss function (L CE ) and the loss function of contrast loss (L I2CR ) to train a keyword prediction model. A contrastive regularizer with cross entropy (CE) helps directly optimize the network, reducing complexity and improving noise robustness. Contrastive loss can be used as a regularizer to achieve domain generalization for speech classification, thereby expanding the model's applicability.

[0179] Alternatively, the combined loss function (L) can be obtained by the following formula:

[0180] L=L CE +αL I2CR

[0181] Among them, L CE is the cross entropy loss for supervised learning, L I2CR is the contrast loss as a regularizer. α is the weight given to the regularizer, which can be used to measure the degree of additional contrast constraint.

[0182] As shown in Figure 8(c), through noise enhancement, a dual view can be created from the sample, and contrastive learning is used to constrain the regularized supervision model to pay more attention to the generalized class features extracted from the two noise-enhanced audio (samples), which helps to reduce the amount of noise encoded in the latent features and makes the regularized supervision model robust to noise.

[0183] Alternatively, as shown in FIG8(c), two samples (sample 1 and sample 2) may be randomly sampled and mixed and enhanced to obtain the input data of the model. The mixed input data may be obtained by the following formula:

[0184] μ*sample1+(1-μ)*sample2

[0185] Here, μ can be a mixing coefficient and can be obtained from a beta distribution.

[0186] In this embodiment, by performing mixed data enhancement on the acquired data, overfitting of the model during the optimization process is reduced, thereby improving the accuracy of model prediction.

[0187] Traditional contrastive learning, such as n-pair loss, usually uses a single enhanced view from an anchor sample as a positive sample. However, learning from a single instance is inefficient. Therefore, in the embodiment of the present application, the data is mixed, and a method involving the use of multiple (positive) views with different enhanced noises is proposed to improve feature learning. Multiple views with different enhanced noises can be added to the randomly sampled background noise under clean speech, thereby achieving the purpose of artificially simulating a real noisy environment.

[0188] Since it is generally difficult to have parallel samples of multi-perspective noise in existing real data, in order to obtain parallel samples of multi-perspective background noise, the embodiment of the present application can debug by calculating the signal-to-noise ratio and then adding the noise volume to the speech.

[0189] If the noise added at the beginning is too large, it will affect the model's ability to recognize similar features in (positive) multi-view samples, resulting in poor convergence. Therefore, in an embodiment of the present application, fixed noise data can be provided during training, and the SNRs noise can be slowly increased during training. For example, it can be from no noise to 20dB, from no noise to 20dB and finally to 15dB; it can be from no noise to 20dB, then to 15dB, then to 10dB, and all the way to 20dB to -10dB.

[0190] Optionally, while simulating noisy speech, other processing can also be performed on the speech to achieve the purpose of data enhancement. For example, the speech can be pitch-shifted to randomly lower or raise the pitch of the speech; the speech can be codec-enhanced by using a codec vocoder to synthesize audio; the data can be enhanced by time shifting, speed perturbation, video masking, etc. In the embodiments of the present application, multiple data enhancement methods are provided, and different data enhancement methods can be selected according to the generalized scenario of use, thereby improving the adaptability of the model to the usage scenario.

[0191] In this embodiment, data augmentation is performed on different samples belonging to the same class, rather than simply enhancing noise from the same input instance. In other words, this embodiment avoids the related art approach of simply enhancing noise from the same input instance, thereby reducing the possibility of the model being affected by other unfavorable factors and thereby improving the accuracy of the model's predictions. Among these unfavorable factors, such as speaker identity and voice pitch, are examples only and are not specifically limited.

[0192] As shown in Figure 8(c), the binary cross entropy loss weighted by the beta coefficient can be used to learn the mixed input, that is, the cross entropy can be calculated by adding beta weights (Beta coefficient), thereby improving the accuracy of model prediction.

[0193] In this embodiment, the keyword prediction model can be supervised based on the cross entropy between multiple enhanced samples.

[0194] In this embodiment, contrast loss can be used to suppress interference factors (such as speaker, speaking style and background noise) in the feature encoding module. Modified from the "n-pair loss", multiple front views from the same sample type are added to constrain the encoding features, maximize consistent information (content), and at the same time, minimize other noise to improve the final representation (representation). Similarly, the dot product of the contrast similarity can be weighted according to the beta coefficient.

[0195] Optionally, when training downstream tasks, a weight coefficient can be added to the regularization term, thereby adding a regularization term to the standard cross entropy loss, where the weight coefficient can grow linearly from 0 to a maximum of 0.5.

[0196] As shown in Figure 8(c), the input data (Dataloader) is processed to obtain three returned objects (items). Two speech samples (sample 1 and sample 2) of the same class can be randomly sampled beforehand, and sample 1 and sample 2 can be added based on the beta coefficient. Furthermore, the processed supervised learning labels can include utterance 1 and utterance 2 with the same label as utterance 1, and utterance 3 and utterance 4 with the same label as utterance 3.

[0197] In an embodiment of the present application, a regularizer can be added to the standard cross entropy loss, and the regularizer can use the maximum similarity loss. When the same type of speech has the greatest similarity, the keyword prediction model can find the potential features of the same feature representative sample in the features. When different noises and different samples but the same type of speech are added, the keyword prediction model will learn their common and representative features, and reduce the encoding of other different (interference factors) features to achieve the purpose of constraining the keyword prediction model to encode interference factors (especially noise).

[0198] In this embodiment, the keyword prediction model can be modified from multiple pairs of losses, and multiple views from the same class of samples (different samples of the same class) can be added to impose stricter restrictions to improve the encoding feature representation. The embodiment of the present application uses regularization terms to maximize the same main content while removing noise information, which will help the model predict more accurately in the standard cross entropy loss. And the embodiment of the present application allows the model to be continuously fine-tuned on the new framework, thereby benefiting the existing system.

[0199] The embodiments of the present application improve the accuracy of the keyword prediction model in extremely low signal-to-noise ratio environments and show more stable performance over a wider signal-to-noise ratio range. The embodiments of the present application achieve better representation learning by applying a regularization constraint (contrastive loss) on the standard cross-entropy loss, thereby achieving the technical effect of improving the robustness of keyword detection and solving the technical problem of low robustness of keyword detection.

[0200] In another optional embodiment, Figure 9 The block diagram shows the use of the above Figure 1 The computer terminal (or mobile device) shown is used as an embodiment of the service grid. Figure 9 This is a structural block diagram of a service grid for a method for detecting keywords in speech according to an embodiment of the present application. Figure 9 As shown, the service grid 900 is mainly used to facilitate secure and reliable communication between multiple microservices. Microservices refer to decomposing an application into multiple smaller services or instances and distributing them to run on different clusters / machines.

[0201] like Figure 9 As shown, the microservices may include application service instance A and application service instance B, which form the functional application layer of the service grid 900. In one embodiment, application service instance A runs in the form of container / process 908 on a machine / workload container group 914 (POD), and application service instance B runs in the form of container / process 910 on a machine / workload container group 916 (POD).

[0202] In one implementation, application service instance A may be a product query service, and application service instance B may be a product ordering service.

[0203] like Figure 9As shown, application service instance A and grid proxy (sidecar) 903 coexist in machine workload container group 914, while application service instance B and grid proxy 905 coexist in machine workload container 914. Grid proxy 903 and grid proxy 905 form the data plane layer of service grid 900. Grid proxy 9003 and grid proxy 905 respectively operate as container / process 904, which can receive requests 912 for product query services, and grid proxy 906. Bidirectional communication is possible between grid proxy 903 and application service instance A, and between grid proxy 905 and application service instance B. Furthermore, bidirectional communication is also possible between grid proxy 903 and grid proxy 905.

[0204] In one embodiment, all traffic for application service instance A is routed to the appropriate destination via grid proxy 903, and all network traffic for application service instance B is routed to the appropriate destination via grid proxy 905. It should be noted that network traffic mentioned herein includes, but is not limited to, Hypertext Transfer Protocol (HTTP), Representational State Transfer (REST), the high-performance, general-purpose open source framework (gRPC), and the open source in-memory data structure storage system (Redis).

[0205] In one embodiment, the data plane layer's functionality can be extended by writing custom filters for the proxy (Envoy) in service mesh 900. The service mesh proxy configuration can be designed to enable the service mesh to correctly proxy service traffic, enabling service interoperability and service governance. Mesh proxy 903 and mesh proxy 905 can be configured to perform at least one of the following functions: service discovery, health checking, routing, load balancing, authentication and authorization, and observability.

[0206] like Figure 9 As shown, the service grid 900 also includes a control plane layer. The control plane layer can be a group of services running in a dedicated namespace, and these services are hosted by a hosting control plane component 901 in a machine / workload container group (machine / Pod) 9002. Figure 9As shown, managed control plane component 901 communicates bidirectionally with grid agent 903 and grid agent 905. Managed control plane component 901 is configured to perform certain control and management functions. For example, managed control plane component 901 receives telemetry data transmitted by grid agent 903 and grid agent 905 and can further aggregate this telemetry data. For these services, managed control plane component 901 can also provide a user-oriented application programming interface (API) to easily manipulate network behavior and provide configuration data to grid agent 903 and grid agent 905. It should be noted that for the sake of simplicity, the aforementioned method embodiments are described as a series of actions. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, as certain steps can be performed in a different order or simultaneously according to the present application. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0207] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0208] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0209] Example 3

[0210] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 2 The method for detecting keywords in speech and the device for detecting keywords in speech are shown.

[0211] Figure 10 Schematic diagram of a device for detecting keywords in speech according to an embodiment of the present application. Figure 10As shown, the apparatus 1000 for detecting keywords in speech may include: a first collecting unit 1002 , a first calling unit 1004 , a first extracting unit 1006 and a first processing unit 1008 .

[0212] The first collecting unit 1002 is configured to collect the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected.

[0213] The first calling unit 1004 is used to call the keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on the cross entropy loss between multiple first noise-enhanced samples and performing contrastive learning based on the contrast loss between multiple second noise-enhanced samples, the multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0214] The first extraction unit 1006 is configured to extract noise enhancement features from the speech to be detected using a keyword prediction model.

[0215] The first processing unit 1008 is configured to remove noise enhancement features from the speech to be detected to identify target keywords.

[0216] It should be noted that the first collection unit 1002, the first calling unit 1004, the first extraction unit 1006, and the first processing unit 1008 correspond to steps S202 to S208 in Example 1. The examples and application scenarios implemented by the four units and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above units, as part of the device, can be run in the computer terminal provided in Example 1.

[0217] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 3 The model determination method shown is a model determination device.

[0218] Figure 11 is a schematic diagram of another model determination device according to an embodiment of the present application, such as Figure 11 As shown, the model determination device 1100 may include: a first acquisition unit 1102 , a first enhancement unit 1104 , a first determination unit 1106 and a second processing unit 1108 .

[0219] The first acquiring unit 1102 is configured to acquire a first voice sample and multiple second voice samples, where the multiple second voice samples are of the same type.

[0220] The first enhancement unit 1104 is configured to perform mixed data enhancement on the first speech sample to obtain a plurality of first noise-enhanced samples, and to perform mixed data enhancement on the second speech samples of the same type to obtain a plurality of second noise-enhanced samples.

[0221] The first determining unit 1106 is configured to determine a cross entropy loss between a plurality of first noise-enhanced samples and a contrast loss between a plurality of second noise-enhanced samples.

[0222] The second processing unit 1108 is used to perform self-supervised learning on the first speech sample based on cross-entropy loss, and to perform comparative learning on multiple second speech samples based on contrast loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected.

[0223] It should be noted that the first acquisition unit 1102, the first enhancement unit 1104, the first determination unit 1106, and the second processing unit 1108 described above correspond to steps S302 to S308 in Example 1. The examples and application scenarios implemented by the four units and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above units, as part of the apparatus, can be run in the computer terminal provided in Example 1.

[0224] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 4 The method for detecting keywords in speech and the device for detecting keywords in speech are shown.

[0225] Figure 12 is a schematic diagram of another apparatus for detecting keywords in speech according to an embodiment of the present application, such as Figure 12 As shown, the apparatus 1200 for detecting keywords in speech may include: a second collecting unit 1202 , a second extracting unit 1204 , a first identifying unit 1206 and a first activating unit 1208 .

[0226] The second collecting unit 1202 is configured to collect the speech to be detected sent to the client, wherein the speech to be detected contains at least one target keyword to be detected.

[0227] The second extraction unit 1204 is used to extract noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on the cross-entropy loss between multiple first noise enhancement samples and performing contrastive learning based on the contrast loss between multiple second noise enhancement samples. The multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0228] The first recognition unit 1206 is configured to remove noise enhancement features from the speech to be detected to recognize target keywords.

[0229] The first activation unit 1208 is configured to activate the client based on the target keyword.

[0230] It should be noted that the second acquisition unit 1202, second extraction unit 1204, first identification unit 1206, and first activation unit 1208 described above correspond to steps S402 to S408 in Example 1. The examples and application scenarios implemented by these four units and the corresponding steps are the same, but are not limited to those disclosed in Example 1. It should be noted that the above units, as part of the apparatus, can be run in the computer terminal provided in Example 1.

[0231] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 5 A speech generation device according to the speech generation method shown.

[0232] Figure 13 is a schematic diagram of a speech generating device according to an embodiment of the present application, such as Figure 13 As shown, the speech generating apparatus 1300 may include: a third collecting unit 1302 , a third extracting unit 1304 , a third processing unit 1306 and a fourth processing unit 1308 .

[0233] The third collecting unit 1302 is configured to collect the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected.

[0234] The third extraction unit 1304 is used to extract noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on the cross-entropy loss between multiple first noise enhancement samples and performing contrastive learning based on the contrast loss between multiple second noise enhancement samples. The multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0235] The third processing unit 1306 is configured to remove noise enhancement features from the speech to be detected to identify target keywords.

[0236] The fourth processing unit 1308 is configured to generate a target voice based on the target keyword and play the target voice, wherein the target voice includes a response content to the voice to be detected.

[0237] It should be noted that the third collection unit 1302, the third extraction unit 1304, the third processing unit 1306, and the fourth processing unit 1308 correspond to steps S502 to S508 in Example 1. The examples and application scenarios implemented by the four units and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above units, as part of the device, can be run in the computer terminal provided in Example 1.

[0238] According to an embodiment of the present application, another method for implementing the above Figure 6 The method for detecting keywords in speech and the device for detecting keywords in speech are shown.

[0239] Figure 14 is a schematic diagram of another apparatus for detecting keywords in speech according to an embodiment of the present application, such as Figure 14 As shown, the apparatus 1400 for detecting keywords in speech may include: an input unit 1402 , a fourth extraction unit 1404 , a fifth processing unit 1406 , and a second activation unit 1408 .

[0240] The input unit 1402 is used to input a speech to be detected on a virtual reality (VR) device or an augmented reality (AR) device, wherein the speech to be detected contains at least one target keyword to be detected.

[0241] The fourth extraction unit 1404 is used to extract noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross-entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples. The multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0242] The fifth processing unit 1406 is configured to remove noise enhancement features from the speech to be detected to identify target keywords.

[0243] The second activation unit 1408 is configured to activate the VR device or AR device using the target keyword.

[0244] It should be noted that the input unit 1402, the fourth extraction unit 1404, the fifth processing unit 1406, and the second activation unit 1408 described above correspond to steps S602 to S608 in Example 1. The examples and application scenarios implemented by these four units and the corresponding steps are the same, but are not limited to those disclosed in Example 1. It should be noted that the above units, as part of the apparatus, can be run in the computer terminal provided in Example 1.

[0245] According to an embodiment of the present application, another method for implementing the above Figure 7 The method for detecting keywords in speech and the device for detecting keywords in speech are shown.

[0246] Figure 15 is a schematic diagram of another apparatus for detecting keywords in speech according to an embodiment of the present application, such as Figure 15 As shown, the apparatus 1500 for detecting keywords in speech may include: a second acquiring unit 1502 , a sixth processing unit 1504 , a fifth extracting unit 1506 , a seventh processing unit 1508 and an output unit 1510 .

[0247] The second acquiring unit 1502 is configured to acquire the monitored speech to be detected by calling the first interface, wherein the speech to be detected includes at least one target keyword to be detected, and the first interface includes a first parameter whose parameter value is the speech to be detected.

[0248] The sixth processing unit 1504 is used to call the keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on the cross entropy loss between multiple first noise-enhanced samples and performing contrastive learning based on the contrast loss between multiple second noise-enhanced samples. The multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type.

[0249] The fifth extraction unit 1506 is configured to extract noise enhancement features from the speech to be detected using the keyword prediction model.

[0250] The seventh processing unit 1508 is configured to remove noise enhancement features from the speech to be detected to identify target keywords.

[0251] The output unit 1510 is configured to output the target keyword by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the target keyword.

[0252] It should be noted that the second acquisition unit 1502, the sixth processing unit 1504, the fifth extraction unit 1506, the seventh processing unit 1508, and the output unit 1510 described above correspond to steps S702 to S710 in Example 1. The examples and application scenarios implemented by the five units and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above units, as part of the apparatus, can be run in the computer terminal provided in Example 1.

[0253] In the device for detecting keywords in speech of this embodiment, supervised learning is performed based on the cross-entropy loss between multiple first noise-enhanced samples, and contrastive learning is performed based on the contrast loss between multiple second noise-enhanced samples to generate a keyword prediction model, so as to complete the strict restriction of the keyword prediction model. The embodiment of the present application does not perform noise enhancement from a single instance, which helps to directly optimize the keyword prediction model, and the input speech to be detected is obtained through hybrid enhancement, thereby avoiding over-training fitting of the keyword prediction model, and further achieving the technical effect of improving the robustness of keyword detection, and solving the technical problem of low robustness of keyword detection.

[0254] Example 4

[0255] The embodiment of the present application may provide a processor, which may include a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0256] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0257] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the method for detecting keywords in the speech of an application: collecting the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; calling the keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross-entropy loss between multiple first noise-enhanced samples, and by contrastive learning based on the contrast loss between multiple second noise-enhanced samples, the multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and the different second noise-enhanced samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; using the keyword prediction model to extract noise enhancement features from the speech to be detected; removing the noise enhancement features from the speech to be detected to identify the target keyword.

[0258] Optionally, Figure 16This is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 16 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1602 , a memory 1604 , and a transmission device 1606 .

[0259] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for detecting keywords in speech in the embodiments of the present application. The processor executes various functional applications and predictions by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method for detecting keywords in speech. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0260] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: collect the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; call the keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; use the keyword prediction model to extract noise enhancement features from the speech to be detected; remove the noise enhancement features from the speech to be detected to identify the target keyword.

[0261] Optionally, the processor may also execute the program code of the following steps: performing mixed data enhancement on the first randomly sampled noise data in the first voice sample according to a first mixing coefficient to obtain a first noise enhanced sample, wherein the first randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and the first mixing coefficient conforms to the Beta distribution; performing mixed data enhancement on the second randomly sampled noise data in the second voice sample according to a second mixing coefficient to obtain a second noise sample, wherein the second randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and the second mixing coefficient conforms to the Beta distribution.

[0262] Optionally, the processor may further execute program code of the following steps: performing self-supervised learning on the first speech sample based on cross entropy loss, and performing comparative learning on multiple second speech samples based on contrast loss to obtain a keyword prediction model.

[0263] Optionally, the processor may also execute the program code of the following steps: establishing a regularizer based on cross entropy loss and contrast loss, wherein the regularizer is used to prohibit the keyword prediction model from encoding noise enhancement features; performing self-supervised learning on the first speech sample based on the regularizer, and performing contrastive learning on multiple second speech samples to obtain a keyword prediction model.

[0264] Optionally, the processor may also execute the program code of the following steps: performing weighted summation of the cross entropy loss and the contrast loss to obtain a target loss, wherein the weight corresponding to the contrast loss is used to indicate the degree of constraint of the contrast loss on the keyword prediction model, and the weight increases linearly with the increase in the number of training rounds of the keyword prediction model; and establishing a regularizer corresponding to the target loss.

[0265] Optionally, the processor may also execute the program code of the following steps: performing self-supervised learning on the first speech sample based on the regularizer, and performing comparative learning on multiple second speech samples to obtain model adjustment parameters; adjusting the parameters of the initial keyword prediction model based on the model adjustment parameters to obtain a keyword prediction model.

[0266] Optionally, the processor may further execute program codes of the following steps: removing noise enhancement features from the speech to be detected to obtain keyword features; and obtaining target keywords represented by the keyword features.

[0267] Optionally, the processor may further execute a program code of the following step: the similarity between the first noise-enhanced sample and the second noise-enhanced sample is greater than a first similarity threshold.

[0268] As an optional example, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a first voice sample and multiple second voice samples, wherein the multiple second voice samples are of the same type; perform mixed data enhancement on the first voice sample to obtain multiple first noise enhancement samples, and perform mixed data enhancement on the second voice samples of the same type to obtain multiple second noise enhancement samples; determine the cross-entropy loss between the multiple first noise enhancement samples, and determine the contrast loss between the multiple second noise enhancement samples; perform self-supervised learning on the first voice sample based on the cross-entropy loss, and perform contrast learning on the multiple second voice samples based on the contrast loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected.

[0269] As an optional example, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: collect the voice to be detected sent to the client, wherein the voice to be detected contains at least one target keyword to be detected; use the keyword prediction model to extract the noise enhancement feature from the voice to be detected, wherein the keyword prediction model is trained and generated by supervised learning based on the cross entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first voice samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second voice samples of the same type; remove the noise enhancement feature from the voice to be detected to identify the target keyword; activate the client based on the target keyword.

[0270] Optionally, the processor may further execute program code of the following steps: activating the client in response to a similarity between the target keyword and a predetermined keyword associated with the client being greater than a second similarity threshold.

[0271] As an optional example, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: collect the monitored voice to be detected, wherein the voice to be detected contains at least one target keyword to be detected; use the keyword prediction model to extract noise enhancement features from the voice to be detected, wherein the keyword prediction model is trained and generated based on supervised learning through cross-entropy loss between multiple first noise enhancement samples, and based on contrastive learning through contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first voice samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second voice samples of the same type; remove the noise enhancement features from the voice to be detected to identify the target keyword; generate a target voice based on the target keyword, and play the target voice, wherein the target voice includes the response content to the voice to be detected.

[0272] As an optional example, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: input the voice to be detected on a virtual reality VR device or an augmented reality AR device, wherein the voice to be detected contains at least one target keyword to be detected; use a keyword prediction model to extract noise enhancement features from the voice to be detected, wherein the keyword prediction model is trained and generated by supervised learning based on the cross-entropy loss between multiple first noise enhancement samples and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first voice samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second voice samples of the same type; remove the noise enhancement features from the voice to be detected to identify the target keyword; use the target keyword to activate the VR device or AR device.

[0273] As an optional example, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the monitored voice to be detected by calling the first interface, wherein the voice to be detected contains at least one target keyword to be detected, and the first interface includes a first parameter, and the parameter value of the first parameter is the voice to be detected; call the keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross-entropy loss between multiple first noise enhancement samples, and contrastive learning based on the contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first voice samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second voice samples of the same type; use the keyword prediction model to extract noise enhancement features from the voice to be detected; remove the noise enhancement features from the voice to be detected to identify the target keyword; output the target keyword by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target keyword.

[0274] The embodiment of the present application responds to a computing request from a computing initiating end, obtains sample features and sample labels of the computing initiating end from the network channel of the computing initiating end, and obtains sample features of the computing collaboration end from the network channel of the computing collaboration end; based on the sample features and sample labels of the computing initiating end and the sample features of the computing collaboration end, a data prediction model is trained, wherein the data prediction model is used to characterize the mapping relationship between the sample features of the computing initiating end, the sample features of the computing collaboration end, and the sample labels; the data prediction model is sent to the computing initiating end, thereby achieving the technical effect of improving the efficiency of data security processing and solving the technical problem of low robustness in keyword detection.

[0275] It can be understood by those skilled in the art that Figure 16The structure shown is for illustration only, and the computer terminal A may also be a smart phone (eg, tablet computer, handheld computer, mobile Internet device (MID), PAD and other terminal devices. Figure 16 The structure of the computer terminal A is not limited. For example, the computer terminal A may also include Figure 16 More or fewer components (such as network interfaces, display devices, etc.) shown, or with Figure 16 Different configurations shown.

[0276] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0277] Example 6

[0278] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method for detecting keywords in speech provided in the embodiment 1 above.

[0279] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0280] Optionally, in this embodiment, the above-mentioned computer-readable storage medium is configured to store program code for executing the following steps: performing mixed data enhancement on the first randomly sampled noise data according to a first mixing coefficient to obtain a first noise enhancement sample, wherein the first randomly sampled noise data increases as the training rounds of the keyword prediction model increase, and the first mixing coefficient conforms to the Beta distribution; performing mixed data enhancement on the second randomly sampled noise data in the second speech sample according to the second mixing coefficient to obtain a second noise sample, wherein the second randomly sampled noise data increases as the training rounds of the keyword prediction model increase, and the second mixing coefficient conforms to the Beta distribution.

[0281] Optionally, the computer-readable storage medium may further execute program code of the following steps: performing self-supervised learning on the first speech sample based on cross-entropy loss, and performing comparative learning on multiple second speech samples based on contrast loss to obtain a keyword prediction model.

[0282] Optionally, the computer-readable storage medium can also execute the program code of the following steps: establishing a regularizer based on cross-entropy loss and contrast loss, wherein the regularizer is used to prohibit the keyword prediction model from encoding noise enhancement features; performing self-supervised learning on the first speech sample based on the regularizer, and performing contrastive learning on multiple second speech samples to obtain a keyword prediction model.

[0283] Optionally, the computer-readable storage medium can also execute the program code of the following steps: performing weighted summation of the cross entropy loss and the contrast loss to obtain the target loss, wherein the weight corresponding to the contrast loss is used to indicate the degree of constraint of the contrast loss on the keyword prediction model, and the weight increases linearly with the increase in the number of training rounds of the keyword prediction model; and establishing a regularizer corresponding to the target loss.

[0284] Optionally, the computer-readable storage medium can also execute the program code of the following steps: performing self-supervised learning on the first speech sample based on the regularizer, and performing comparative learning on multiple second speech samples to obtain model adjustment parameters; adjusting the parameters of the initial keyword prediction model based on the model adjustment parameters to obtain a keyword prediction model.

[0285] Optionally, the computer-readable storage medium may further execute program code for the following steps: removing noise enhancement features from the speech to be detected to obtain keyword features; and obtaining target keywords represented by the keyword features.

[0286] Optionally, the computer-readable storage medium may further include program code for executing the following steps: a similarity between the first noise-enhanced sample and the second noise-enhanced sample is greater than a first similarity threshold.

[0287] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: obtaining a first speech sample and multiple second speech samples, wherein the multiple second speech samples are of the same type; performing mixed data enhancement on the first speech sample to obtain multiple first noise enhancement samples, and performing mixed data enhancement on the second speech samples of the same type to obtain multiple second noise enhancement samples; determining the cross-entropy loss between the multiple first noise enhancement samples, and determining the contrast loss between the multiple second noise enhancement samples; performing self-supervised learning on the first speech sample based on the cross-entropy loss, and performing contrast learning on the multiple second speech samples based on the contrast loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected.

[0288] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: collecting speech to be detected sent to a client, wherein the speech to be detected contains at least one target keyword to be detected; extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on cross-entropy loss between multiple first noise enhancement samples and contrastive learning based on contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; removing noise enhancement features from the speech to be detected to identify the target keyword; and activating the client based on the target keyword.

[0289] Optionally, the computer-readable storage medium may further execute program code for the following steps: activating the client in response to a similarity between the target keyword and a predetermined keyword associated with the client being greater than a second similarity threshold.

[0290] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: collecting monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; using a keyword prediction model to extract noise enhancement features from the speech to be detected, wherein the keyword prediction model is trained and generated based on supervised learning by cross-entropy loss between multiple first noise enhancement samples, and based on contrastive learning by contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; removing noise enhancement features from the speech to be detected to identify the target keyword; generating a target speech based on the target keyword, and playing the target speech, wherein the target speech includes the response content to the speech to be detected.

[0291] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: inputting a speech to be detected on a virtual reality (VR) device or an augmented reality (AR) device, wherein the speech to be detected contains at least one target keyword to be detected; extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on cross-entropy loss between multiple first noise enhancement samples and contrastive learning based on contrast loss between multiple second noise enhancement samples, the multiple first noise enhancement samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; removing the noise enhancement features from the speech to be detected to identify the target keyword; and activating the VR device or AR device using the target keyword.

[0292] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: obtaining the monitored speech to be detected by calling a first interface, wherein the speech to be detected contains at least one target keyword to be detected, and the first interface includes a first parameter, and the parameter value of the first parameter is the speech to be detected; calling a keyword prediction model, wherein the keyword prediction model is trained and generated by supervised learning based on the cross-entropy loss between multiple first noise-enhanced samples, and by contrastive learning based on the contrast loss between multiple second noise-enhanced samples, and the multiple first noise-enhanced samples are obtained by performing mixed data enhancement on the first speech samples, and different second noise enhancement samples are obtained by performing mixed data enhancement on multiple second speech samples of the same type; using the keyword prediction model to extract noise enhancement features from the speech to be detected; removing the noise enhancement features from the speech to be detected to identify the target keyword; outputting the target keyword by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target keyword.

[0293] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0294] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0295] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0296] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0297] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0298] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0299] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting keywords in speech, characterized in that: include: Collecting the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; Calling a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on a cross-entropy loss between a plurality of first noise-enhanced samples and performing contrastive learning based on a contrastive loss between a plurality of second noise-enhanced samples, the plurality of first noise-enhanced samples are obtained by performing mixed data enhancement on first randomly sampled noise data in a first speech sample, the first randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and the second noise-enhanced samples are respectively obtained by performing mixed data enhancement on second randomly sampled noise data in a plurality of second speech samples of the same type, the second randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases; Extracting noise enhancement features from the speech to be detected using the keyword prediction model; The noise enhancement feature is removed from the speech to be detected to identify the target keyword.

2. The method according to claim 1, characterized in that The method further comprises: Performing mixed data enhancement on the first randomly sampled noise data according to a first mixing coefficient to obtain a first noise enhancement sample, wherein the first mixing coefficient conforms to a Beta distribution; Performing mixed data enhancement on the second randomly sampled noise data according to a second mixing coefficient to obtain the second noise enhanced sample, wherein the second mixing coefficient conforms to a beta distribution.

3. The method according to claim 1, characterized in that The method further comprises: Self-supervised learning is performed on the first speech sample based on the cross entropy loss, and contrastive learning is performed on the plurality of second speech samples based on the contrastive loss to obtain the keyword prediction model.

4. The method according to claim 3, characterized in that Performing self-supervised learning on the first speech sample based on the cross entropy loss and performing contrastive learning on the plurality of second speech samples based on the contrastive loss to obtain the keyword prediction model includes: Establishing a regularizer based on the cross entropy loss and the contrast loss, wherein the regularizer is used to prohibit the keyword prediction model from encoding the noise enhancement feature; The keyword prediction model is obtained by performing self-supervised learning on the first speech sample based on the regularizer and performing comparative learning on the plurality of second speech samples.

5. The method according to claim 4, characterized in that A regularizer is established based on the cross entropy loss and the contrast loss, comprising: Performing a weighted summation on the cross entropy loss and the contrast loss to obtain a target loss, wherein a weight corresponding to the contrast loss is used to represent the degree of constraint of the contrast loss on the keyword prediction model, and the weight increases linearly with an increase in the number of training rounds of the keyword prediction model; The regularizer corresponding to the target loss is established.

6. The method according to claim 4, characterized in that Performing self-supervised learning on the first speech sample based on the regularizer and performing comparative learning on the plurality of second speech samples to obtain the keyword prediction model includes: Performing self-supervised learning on the first speech sample based on the regularizer, and performing comparative learning on the plurality of second speech samples to obtain model adjustment parameters; The parameters of the initial keyword prediction model are adjusted based on the model adjustment parameters to obtain the keyword prediction model.

7. The method according to claim 1, characterized in that Removing the noise enhancement feature from the speech to be detected to identify the target keyword includes: Removing the noise enhancement feature from the speech to be detected to obtain a keyword feature; The target keyword represented by the keyword feature is obtained.

8. The method according to any one of claims 1 to 7, characterized in that The similarity between the first noise-enhanced sample and the second noise-enhanced sample is greater than a first similarity threshold.

9. A method for determining a model, characterized in that: include: Acquire a first voice sample and a plurality of second voice samples, wherein the plurality of second voice samples are of the same type; Performing mixed data enhancement on first randomly sampled noise data in the first speech sample to obtain a plurality of first noise-enhanced samples, and performing mixed data enhancement on second randomly sampled noise data in the second speech sample of the same type to obtain a plurality of second noise-enhanced samples, wherein the first randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and the second randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases; determining a cross entropy loss between the plurality of first noise-enhanced samples, and determining a contrastive loss between the plurality of second noise-enhanced samples; Self-supervised learning is performed on the first speech sample based on the cross-entropy loss, and contrastive learning is performed on the multiple second speech samples based on the contrastive loss to obtain a keyword prediction model, wherein the keyword prediction model is used to extract noise enhancement features from the speech to be detected to identify target keywords in the speech to be detected.

10. A method for detecting keywords in speech, characterized in that: include: Collecting the speech to be detected sent to the client, wherein the speech to be detected contains at least one target keyword to be detected; Extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on a cross-entropy loss between a plurality of first noise enhancement samples and performing contrastive learning based on a contrast loss between a plurality of second noise enhancement samples, the plurality of first noise enhancement samples are obtained by performing mixed data enhancement on first randomly sampled noise data in a first speech sample, the first randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and different second noise enhancement samples are respectively obtained by performing mixed data enhancement on second randomly sampled noise data in a plurality of second speech samples of the same type, the second randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases; Removing the noise enhancement feature from the speech to be detected to identify the target keyword; The client is activated based on the target keyword.

11. The method according to claim 10, characterized in that Activating the client based on the target keyword includes: In response to a similarity between the target keyword and a predetermined keyword associated with the client being greater than a second similarity threshold, the client is activated.

12. A speech generation method, characterized in that: include: Collecting the monitored speech to be detected, wherein the speech to be detected contains at least one target keyword to be detected; Extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on a cross-entropy loss between a plurality of first noise enhancement samples and performing contrastive learning based on a contrast loss between a plurality of second noise enhancement samples, the plurality of first noise enhancement samples are obtained by performing mixed data enhancement on first randomly sampled noise data in a first speech sample, the first randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and different second noise enhancement samples are respectively obtained by performing mixed data enhancement on second randomly sampled noise data in a plurality of second speech samples of the same type, the second randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases; Removing the noise enhancement feature from the speech to be detected to identify the target keyword; A target voice is generated based on the target keyword and played, wherein the target voice includes reply content to the voice to be detected.

13. A method for detecting keywords in speech, characterized in that: include: Inputting a speech to be detected on a virtual reality (VR) device or an augmented reality (AR) device, wherein the speech to be detected contains at least one target keyword to be detected; Extracting noise enhancement features from the speech to be detected using a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on a cross-entropy loss between a plurality of first noise enhancement samples and performing contrastive learning based on a contrast loss between a plurality of second noise enhancement samples, the plurality of first noise enhancement samples are obtained by performing mixed data enhancement on first randomly sampled noise data in a first speech sample, the first randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and different second noise enhancement samples are respectively obtained by performing mixed data enhancement on second randomly sampled noise data in a plurality of second speech samples of the same type, the second randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases; Removing the noise enhancement feature from the speech to be detected to identify the target keyword; The VR device or the AR device is activated using the target keyword.

14. A method for detecting keywords in speech, characterized in that: include: Acquire a monitored voice to be detected by calling a first interface, wherein the voice to be detected includes at least one target keyword to be detected, and the first interface includes a first parameter, and a parameter value of the first parameter is the voice to be detected; Calling a keyword prediction model, wherein the keyword prediction model is trained and generated by performing supervised learning based on a cross-entropy loss between a plurality of first noise-enhanced samples and performing contrastive learning based on a contrastive loss between a plurality of second noise-enhanced samples, the plurality of first noise-enhanced samples are obtained by performing mixed data enhancement on first randomly sampled noise data in a first speech sample, the first randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases, and the second noise-enhanced samples are respectively obtained by performing mixed data enhancement on second randomly sampled noise data in a plurality of second speech samples of the same type, the second randomly sampled noise data increases as the number of training rounds of the keyword prediction model increases; Extracting noise enhancement features from the speech to be detected using the keyword prediction model; Removing the noise enhancement feature from the speech to be detected to identify the target keyword; The target keyword is output by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target keyword.

Images