Model training method and apparatus, speech detection method and apparatus, and device, medium and product

Abstract

Provided in the present application are a model training method and apparatus, a speech detection method and apparatus, and a device, a medium and a product. The model training method comprises: acquiring at least one positive example speech sample, at least one noise sample, and at least one negative example interference sample comprising speech, wherein a sample label of the positive example speech sample represents a positive example sample, and sample labels of the noise sample and the negative example interference sample represent negative example samples; obtaining sample detection results of a speech detection model performing speech detection on the at least one positive example speech sample, the at least one noise sample and the at least one negative example interference sample, wherein the sample detection result of a sample represents whether the sample belongs to the positive example sample; on the basis of the sample detection results and the sample labels, determining a total training loss, and on the basis of the total training loss, performing iterative training on the speech detection model, so as to obtain a trained speech detection model.

Description

Model training methods, speech detection methods, devices, equipment, media and products

[0001] This application claims priority to Chinese Patent Application No. 202410573134.6, filed on May 9, 2024, entitled “Model Training Method, Speech Detection Method, Apparatus, Device, Medium and Product”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the fields of cloud technology, artificial intelligence and other technical fields, and to a model training method, a speech detection method, a device, an equipment, a medium and a product.

[0003] Background of the Invention

[0004] Voice Activity Detection (VAD) technology detects the presence of valid speech in an audio signal by analyzing its features. VAD typically relies on the energy of the audio signal for detection. For example, it compares the short-time energy of the audio signal with an energy threshold; if the short-time energy exceeds the threshold, the audio is considered speech; otherwise, it is not. However, this method of comparing energy and thresholds becomes less effective for speech detection in noisy environments. Summary of the Invention

[0005] This application provides a model training method, a speech detection method, a device, an equipment, a medium, and a product that can improve the accuracy and practicality of speech detection.

[0006] This application provides a model training method, including:

[0007] Obtain at least one positive speech sample, at least one noise sample, and at least one negative interference sample including speech, wherein the sample label of the positive speech sample represents the positive sample, and the sample labels of the noise sample and the negative interference sample represent the negative sample;

[0008] Obtain the sample detection results of the speech detection model for the at least one positive speech sample, the at least one noise sample and the at least one negative interference sample, wherein the sample detection result of a sample represents whether the sample belongs to the positive sample;

[0009] The total training loss is determined based on the sample detection results and the sample labels, and the speech detection model is iteratively trained based on the total training loss to obtain the trained speech detection model.

[0010] This application provides a speech detection method, including:

[0011] The speech detection model trained using the model training method of any embodiment performs speech detection on at least one audio signal frame, and obtains the detection result for each audio signal frame in the at least one audio signal frame.

[0012] This application embodiment also provides a model training apparatus, the apparatus comprising:

[0013] The sample acquisition module is used to acquire at least one positive speech sample, at least one noise sample, and at least one negative interference sample including speech, wherein the sample label of the positive speech sample represents the positive sample, and the sample labels of the noise sample and the negative interference sample represent the negative sample.

[0014] The first detection module is used to obtain the sample detection results obtained by the speech detection model for the positive speech sample, the noise sample and the negative interference sample, wherein the sample detection result of a sample represents whether the sample belongs to the positive sample;

[0015] The total loss determination module is used to determine the total training loss based on the sample detection results and the sample labels;

[0016] The training module is used to iteratively train the speech detection model based on the total training loss to obtain the trained speech detection model.

[0017] This application also provides a voice detection device, including:

[0018] The second detection module is used to perform speech detection on at least one audio signal frame using a speech detection model trained according to the model training method of any embodiment, and to obtain the detection result of the at least one audio signal frame.

[0019] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the above-described model training method or speech detection method.

[0020] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described model training method or speech detection method.

[0021] This application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described model training method or speech detection method.

[0022] The model training method provided in this application training method trains the speech recognition model using positive speech samples, noise samples, and negative interference samples including speech, which can improve the speech detection model's ability to distinguish between valid human voices and background human voices, thereby improving the accuracy of speech detection.

[0023] Brief description of the attached figures

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0025] Figure 1 is a schematic diagram of the implementation environment of a speech detection method provided in an embodiment of this application;

[0026] Figure 2 is a flowchart illustrating a model training method provided in an embodiment of this application;

[0027] Figure 3 is a schematic diagram of the structure of a speech detection model provided in an embodiment of this application;

[0028] Figure 4 is a schematic diagram of a triplet training process provided in an embodiment of this application;

[0029] Figure 5 is a schematic diagram of a model training process provided in an embodiment of this application;

[0030] Figure 6 is a flowchart illustrating a speech detection method provided in an embodiment of this application;

[0031] Figure 7 is a schematic diagram of an application example of speech detection provided in an embodiment of this application;

[0032] Figure 8 is a schematic diagram of a model training device provided in an embodiment of this application;

[0033] Figure 9 is a schematic diagram of the structure of a voice detection device provided in an embodiment of this application;

[0034] Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0035] Methods of implementing the present invention

[0036] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0037] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. The terms “comprising” and “including” as used in the embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, or operation, but do not exclude implementation as other features, information, data, steps, or operations supported by this art.

[0038] It is understood that, in the specific embodiments of this application, any user-related data, such as user audio signals and the process of collecting audio from the user's room, requires user permission or consent when applied to specific products or technologies. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if any user-related data is involved in the embodiments of this application, such data must be obtained with the user's authorization and consent, and in accordance with the relevant laws, regulations, and standards of the country and region.

[0039] Figure 1 is a schematic diagram of the implementation environment of a speech detection method provided in this application. As shown in Figure 1, the implementation environment includes a server 101 and a terminal 102. The server 101 can be a backend server for an application. The terminal 102 has an application installed, and the terminal 102 and the server 101 can interact with each other based on the application.

[0040] This application features speech detection capabilities. For example, in business scenarios supported by the application, such as online meetings, online video or audio, live streaming, and online teaching, the application can perform speech detection on the collected audio signals to identify valid human voices (which constitute effective speech) and background noise and other ineffective speech. Valid human voices refer to the primary speech signal in the audio signal that conveys identifiable information. They typically possess clear semantic logic and contextual relevance, as well as specific acoustic characteristics (such as spectral stability and fundamental frequency coherence). Background noise and other ineffective speech include interfering human voices and / or environmental noise. Interfering human voices refer to speech from the same speaker (e.g., echoes or reverberation of the primary speech) or speech from one or more different speakers that coexist with the primary speech signal; these are signals with interfering characteristics (e.g., spectral superposition, semantic irrelevance, energy dispersion).

[0041] The server 101 can pre-train a speech detection model using a large number of samples and then distribute the speech detection model to the terminal 102, enabling the terminal 102 to use the speech detection model to detect valid speech, background voices, environmental noise, etc. in an audio segment. Alternatively, the terminal 102 can directly send the collected audio signal to the server 101, which will then use the trained speech detection model for detection.

[0042] It should be noted that server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The aforementioned networks can include, but are not limited to, wired networks and wireless networks. Wired networks include local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). Wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication. Terminal 102 can be an in-vehicle terminal, intelligent voice interaction device, smart home appliance, smartphone (such as Android phones, iOS phones, etc.), tablet computer, laptop computer, digital broadcast receiver, MID (Mobile Internet Devices), PDA (Personal Digital Assistant), desktop computer, in-vehicle terminal (such as in-vehicle navigation terminal, in-vehicle computer, etc.), smart speaker, smartwatch, etc. Terminals and servers can be directly or indirectly connected via wired or wireless communication, but are not limited to these methods. The specific application can be determined based on the actual application scenario requirements, and is not limited here. The embodiments of this application can be applied to the transportation and autonomous driving fields; for example, they can be used in various scenarios such as cloud technology, artificial intelligence, in-vehicle scenarios, assisted driving, and intelligent transportation.

[0043] Figure 2 is a flowchart illustrating a model training method provided in an embodiment of this application. The execution entity of this method can be an electronic device, such as a server, a terminal, or both. The model training methods of each embodiment train the speech recognition model using positive speech samples, noise samples, and negative interference samples including speech, which can improve the speech detection model's ability to distinguish between valid human voices and background human voices, thereby improving the accuracy of speech detection. As shown in Figure 2, the method includes the following steps 201-204.

[0044] Step 201: The electronic device acquires the first sample set and the second sample set.

[0045] The sample labels corresponding to the first sample set and the second sample set respectively represent non-valid speech (also known as negative sample) and valid speech (also known as positive sample); the first sample set includes at least one first reverberation sample (also known as negative interference sample) and at least one noise sample, and the second sample set includes at least one valid speech sample (also known as positive speech sample or positive sample).

[0046] The positive speech samples include valid speech data from human voice sources, i.e., speech data with identifiable semantic information. The negative interference samples are interference data that includes speech. The noise samples include environmental noise data from non-human voice sources.

[0047] In each embodiment, the valid speech refers to speech from a valid human voice source. Invalid speech refers to speech or environmental noise other than a valid human voice source. A valid human voice source can be understood as a valid, active human voice source within the current environment, such as a human voice source that continuously and stably emits speech.

[0048] In one possible scenario example, in a meeting setting, the speaker currently speaking in the meeting room is the valid main voice source, and the speech emitted by the speaker is valid speech. The whispers of other people in the meeting room besides the speaker are considered background noise, while sounds such as doors opening and chairs being moved are considered environmental noise. Both background noise and environmental noise are considered invalid speech.

[0049] In each embodiment, the second sample set includes at least one synthesized sample; that is, the at least one positive example speech sample may include at least one clean speech sample and / or at least one synthesized sample. The clean speech is audio data recorded using a recording device in a specific environment (e.g., a low-noise, echo-free environment, such as a quiet room, a professional anechoic chamber, etc.), containing little or no noise. To facilitate obtaining a larger number of positive example speech samples, each embodiment synthesizes positive example speech samples (hereinafter also referred to as the second reverberation sample) by processing the clean speech samples. To facilitate obtaining a larger number of negative example interference samples, each embodiment synthesizes negative example interference samples (hereinafter also referred to as the first reverberation sample) by processing the clean speech samples. By using pre-collected clean speech to generate positive and negative example samples through audio synthesis, a large number of positive and negative example samples can be easily obtained in a short time, reducing the workload and time required for audio data processing and annotation when acquiring training samples.

[0050] Each embodiment can process preset clean speech data using at least one of preset environmental noise data and preset spatial propagation characteristics to obtain positive speech samples and negative interference samples. The synthesized samples generated in this way can simulate the propagation effect of main speech and background speech in a real environment. Using these synthesized samples to train a speech detection model allows the model to learn the characteristics of main speech and background speech in a real environment, thereby improving the accuracy of speech detection.

[0051] For example, at least one synthesized sample can be obtained by processing the preset clean speech data using the impulse response in the preset spatial propagation characteristics; the sample label of the at least one synthesized sample can be determined based on the reverberation intensity of the impulse response.

[0052] For example, at least one synthetic sample can be obtained by mixing preset environmental noise data with preset clean speech data at a preset signal-to-noise ratio; and the sample label of the at least one synthetic sample can be determined based on the signal-to-noise ratio.

[0053] The determination of at least one positive speech sample and at least one negative interference sample can be carried out in any feasible manner as needed. A few examples are given below.

[0054] Example A1: Reverberate processing is performed on clean speech data based on at least one first impulse response to obtain at least one first reverberation sample; and reverberation processing is performed on clean speech data based on at least one second impulse response to obtain at least one second reverberation sample.

[0055] The first impulse response or the second impulse response characterizes the propagation characteristics of sound in the corresponding sample room, and the reverberation intensity corresponding to the first impulse response is greater than that of the second impulse response.

[0056] The first and second reverberation samples are reverberation speech samples obtained by processing clean speech data based on the room impulse response of at least one sample room. These first and second impulse responses characterize the acoustic properties of sound propagation in different sample rooms. In this embodiment, the room impulse response can be used to process clean speech data to simulate the acoustic properties of clean speech data propagating in a sample room, thereby obtaining reverberation speech samples. For example, when an audio signal propagates in a sample room, reflection and refraction may occur, resulting in reverberation during the audio signal propagation process. The room impulse response can be used to simulate the system response of the sample room to speech propagation.

[0057] Different impulse responses correspond to different acoustic characteristics, resulting in different reverberation intensities for the obtained reverberant speech samples. Specifically, the reverberation intensity of the second impulse response corresponding to the second reverberant sample is less than the reverberation intensity of the first impulse response corresponding to the first reverberant sample.

[0058] In some scenarios, such as in conference rooms, the speaker (the target speaker, a valid voice source) is usually closer to the audio acquisition device, while others (non-valid voice sources, such as background noise) are farther away. Therefore, the speaker's voice captured by the audio acquisition device is clearer and has lower reverberation, while the voices of others are more blurred and have higher reverberation. In this application, by constructing a first set of non-valid voice samples with higher reverberation intensity into a first set of non-valid voice samples, and a second set of valid voice samples with lower reverberation intensity into a second set of valid voice samples, the initial speech detection model is trained based on these first and second set samples. This allows the initial speech detection model to learn the difference between valid voice sources with lower reverberation intensity and background noise with higher reverberation intensity, enhancing its knowledge of reverberation intensity for valid voice sources and background noise. This enables the trained speech detection model to more accurately identify the speaker's valid voice and background noise in real-world scenarios, improving the accuracy of speech recognition.

[0059] In this embodiment, a pre-built RIR (Room Impulse Response) library can be constructed, which includes room impulse responses corresponding to multiple sample rooms. These sample rooms can correspond to different room sizes, different room impulse responses, and different reverberation intensities. In each embodiment, the RIR library characterizes the room system responses of various sample rooms with different room sizes and different reverberation intensities; for example, the RIR library can be generated using the mirror method and simulation tools.

[0060] In the exemplary embodiment, reverberation intensity characterizes the relative strength between reverberant sound and direct sound generated during audio propagation. A greater reverberation intensity indicates a stronger relative reverberant sound and a weaker relative direct sound; conversely, a smaller reverberation intensity indicates a stronger relative direct sound and a weaker relative reverberant sound. Reverberant sound can be sound waves that have undergone reflection, refraction, etc., in audio, while direct sound can be understood as sound waves that arrive directly at the ear from the sound source without reflection. The reverberation intensity corresponding to each first reverberation sample exceeds the target intensity, while the reverberation intensity corresponding to each second reverberation sample does not exceed the target intensity. That is, negative interference samples can be obtained using room impulse responses with reverberation intensity exceeding the target intensity, and positive speech samples can be obtained using room impulse responses with reverberation intensity not exceeding the target intensity.

[0061] Example A2: Based on the target human voice source corresponding to each clean speech sample in the second sample set, at least one valid speech data from the target human voice source is subjected to reverberation processing to obtain at least one positive speech sample, and at least one valid speech data from a non-target human voice source is subjected to reverberation processing to obtain at least one negative interference sample.

[0062] In this step, based on the target human voice source to which the clean speech data in the clean speech sample belongs, positive example speech samples with the same human voice source as the clean speech sample are constructed into the second sample set; negative example interference samples with different human voice sources as the clean speech sample are constructed into the first sample set.

[0063] For example, based on the room impulse response of at least one sample room, reverberation processing can be performed on clean speech data from each target human voice source to obtain at least one positive speech sample. Furthermore, based on the room impulse response of at least one sample room, reverberation processing can be performed on clean speech data from each other human voice source besides the target human voice source to obtain at least one negative interference sample.

[0064] In one possible scenario, a meeting setting, the speaker (the target speaker belonging to the main voice source) is a fixed speaker. The sound source from the speaker is the main speech, while other sound sources are background noise. Therefore, speech from the same speaker as the main speech sample can be reverberated and used as a positive example of the main speech. Speech from different speakers than the main speech sample can be reverberated and used as background noise, i.e., negative interference samples.

[0065] Example A3: Based on at least one preset first signal-to-noise ratio, at least one preset environmental noise data is mixed with preset clean speech data to obtain at least one first synthesized sample as a positive speech sample; and based on at least one preset second signal-to-noise ratio, at least one preset environmental noise data is mixed with preset clean speech data to obtain at least one second synthesized sample as a negative interference sample.

[0066] The first signal-to-noise ratio or the second signal-to-noise ratio represents the ratio of the power of the main speech to the power of noise in the audio signal, and the first signal-to-noise ratio is greater than the second signal-to-noise ratio.

[0067] In this embodiment of the application, environmental noise collected in different environments can be used as noise samples to simulate noise interference to speech in different environments.

[0068] In each embodiment, by constructing the first synthesized sample with a high signal-to-noise ratio into the second sample set belonging to valid speech, and constructing the second synthesized sample with a low signal-to-noise ratio into the first sample set belonging to invalid speech, the initial speech detection model can be trained based on the first and second sample sets. This enables the initial speech detection model to learn the difference between valid human voice sources with a high signal-to-noise ratio and background noise with a low signal-to-noise ratio, thereby improving the accuracy of speech recognition by the speech detection model.

[0069] In each embodiment, counterexample interference samples can be generated in various ways; a few examples are given below.

[0070] Example A4: Mix at least two clean speech data points from at least two pre-defined voice sources to obtain at least one negative interference sample. The resulting negative interference sample simulates the situation where mixed speech from multiple people speaking simultaneously serves as background noise.

[0071] Example A5: Mix at least two clean speech data points from at least two pre-defined voice sources, process the mixing result using a pre-defined impulse response, and obtain at least one counter-example interference sample. The counter-example interference sample obtained here simulates the sound effect of mixed speech from multiple people speaking simultaneously propagating in a room as background noise.

[0072] Example A6: Based on a preset signal-to-noise ratio, preset environmental noise data is mixed with at least two preset clean speech data from at least two human voice sources to obtain at least one counterexample interference sample. The counterexample interference sample obtained here simulates the situation where mixed speech from multiple people speaking simultaneously in a noisy environment is mixed with environmental noise as background noise.

[0073] Example A7: Process at least one set of clean speech data using a preset impulse response. Based on a preset signal-to-noise ratio, mix the processing result with preset environmental noise data to obtain at least one negative interference sample. The negative interference sample obtained here simulates the situation where the speech of one or more people propagates in a room and mixes with environmental noise, and the resulting sound serves as background noise.

[0074] Counterexample interference samples generated through various synthesis methods can simulate the effect of background speech in complex scenarios, increase the speech detection model's knowledge of background speech features, and thus improve the accuracy of speech detection.

[0075] In various embodiments, positive example speech samples and negative example interference samples at different volumes can also be generated to further improve the accuracy of speech recognition. For example, at least one second positive example speech sample can be obtained by adjusting the volume of the first positive example speech sample using at least one preset adjustment coefficient; at least one second negative example interference sample can be obtained by adjusting the volume of the first negative example interference sample using at least one preset adjustment coefficient. In this way, various positive example speech samples and negative example interference samples at different volumes can be provided in the training samples, enabling the initial speech detection model to learn the differences between effective human voice sources at various volumes and background noise at various volumes, thereby improving the accuracy of speech recognition performed by the speech detection model.

[0076] In each embodiment, the method of further adding noise to the reverberation-processed speech data can be implemented as follows. For example, the implementation of example A1 can be modified to include the following steps A11-A12.

[0077] Step A11: Perform convolution processing on each first impulse response and the clean speech data to generate the initial reverberation data corresponding to each first impulse response;

[0078] Step A12: Obtain noise data from the pre-configured noise library, and synthesize the initial reverberation data corresponding to each first impulse response and the obtained noise data according to at least one signal-to-noise ratio to obtain the at least one counterexample interference sample.

[0079] In each embodiment, the room impulse response can be represented by the corresponding system response function, and convolution processing can be performed using the system response function corresponding to the room impulse response and the effective speech data.

[0080] For any sample room, the room impulse response can be represented by the corresponding RIR function, for example, the RIR function is represented as h(n). The expansion of the RIR function h(n) refers to representing the RIR function as a discrete-time sequence. The form of the expansion of h(n) can be expressed as: h(n)=δ(n)+α1δ(n-T1)+α2δ(n-T2)+……+α N δ(nT N )

[0081] Where δ(n) represents the unit impulse response function, which can be represented by a pulse signal at time n. α1, α2, ..., α N The attenuation coefficients corresponding to each pulse signal are T1, T2, ..., T. N This indicates the delay time corresponding to each pulse signal.

[0082] It should be noted that each term in the expansion of h(n) represents a pulse signal, corresponding to the propagation of sound along different paths. The first term δ(n) represents the direct path, that is, the sound propagates directly from the sound source to the receiver (such as the human ear or a voice acquisition device); the subsequent terms represent multiple reflection paths such as reflection, refraction, and scattering.

[0083] In each embodiment, the valid speech data may include the audio features of the valid speech at each sampling point, such as the amplitude of the valid speech sampled at each sampling point.

[0084] In one possible example, the effective speech data is represented by x(n). Accordingly, in step A11, the room impulse response and the effective speech data can be convolved using the following formula to obtain the initial reverberation data: y(n) = x(n) * h(n);

[0085] Where y(n) represents the initial reverberation data, and * indicates the convolution calculation of the effective speech data x(n) and the RIR function h(n) of the room impulse response.

[0086] In various embodiments, the electronic device may further mix environmental noise data with the initial reverberation data to more realistically reproduce the actual speech detection scenario. In one possible example, in step A12, the initial reverberation data and noise data can be synthesized according to the signal-to-noise ratio using the following formula 1 to obtain the counterexample interference sample: Formula 1: n=γ(a*r+βb);

[0087] Where n represents the negative interference sample; a represents the clean speech data, which can be represented by x(n); r represents the first impulse response, which can be represented by the RIR function h(n) of the room impulse response. β represents the signal-to-noise ratio; for example, β ranges from 0 to 1 and can be used to simulate different noise levels. b represents the noise data. γ represents the volume intensity; for example, γ ranges from 0 to 1 and can be used to simulate different volume levels.

[0088] It should be noted that one or more noise data can be added to the initial reverberation data. In Formula 1, noise data b can represent one or more noise data. If it represents multiple noise data, then βb = β1b1 + β2b2 + ... + βmbm. The signal-to-noise ratio corresponding to different noise data can be different.

[0089] In each embodiment, in step A12, for each initial reverberant speech data, at least one noise data is obtained from a pre-built noise library, and based on the timestamp information of the initial reverberant speech data and the timestamp information of the at least one noise data, the initial reverberant speech data and the at least one noise data are synthesized to obtain at least one reverberant speech data. For example, the initial reverberant speech data and the noise data at corresponding sampling points on the two time axes can be synthesized according to each time on the time axis of the initial reverberant speech data and each time on the time axis of the noise data; for example, for a sampling point at a certain time t1, the amplitude of the initial reverberant speech data a at time t1 is a1, and the amplitude of the noise data b at time t1 is b1. After synthesis, the amplitude of the synthesized reverberant speech data at time t1 is a1+b1.

[0090] It should be noted that multiple positive speech samples can also be constructed in the manner described in steps A11-A12. That is, using Formula 1 above, the impulse response function corresponding to the second impulse response and the effective speech data are convolved, and noise is further increased according to at least one signal-to-noise ratio to obtain multiple positive speech samples. The method for determining negative interference samples is the same, and will not be elaborated here. In addition, for example A2, it can also be implemented in the same way as steps A11-A12. That is, using Formula 1 above, the effective speech data from non-target human voice sources and the corresponding impulse response function are convolved, and the effective speech data from target human voice sources and the corresponding impulse response function are convolved; and noise is further increased according to at least one signal-to-noise ratio to obtain each negative interference sample and positive speech sample. The method for determining negative interference samples is the same, and will not be elaborated here.

[0091] In some embodiments, the methods described in Examples A1 and A2 above can be combined to determine negative interference samples and positive speech samples. For example, for valid speech data from the target human voice source, a second impulse response with a smaller reverberation intensity can be used for reverberation processing to obtain each positive speech sample; for valid speech data from non-target human voice sources, a first impulse response with a larger reverberation intensity can be used for reverberation processing to obtain each negative interference sample.

[0092] In this application, by utilizing multiple signal-to-noise ratios and various room impulse responses, negative interference samples with high reverberation intensity and positive speech samples with low reverberation intensity are constructed. These samples simulate background noisy human voices and the speech of the speaker from an effective human voice source in the actual speech detection environment, respectively. This improves the sample richness of background noisy human voices in ineffective speech and makes the second sample set closer to the effective human voice source in the actual speech environment, thereby improving the accuracy of training.

[0093] In each embodiment, the construction process of each synthetic sample also includes a step of determining sample labels.

[0094] In some embodiments, when determining the sample label of a synthesized sample, the sample label can be determined as a negative sample based on the reverberation intensity of the impulse response being greater than a preset intensity threshold or the signal-to-noise ratio being less than a preset intensity; conversely, the sample label can be determined as a positive sample based on the reverberation intensity of the impulse response being less than a preset intensity threshold or the signal-to-noise ratio being greater than a preset intensity. This method of determining sample labels can automatically generate labels for the generated samples, reducing the workload of manual sample annotation.

[0095] In other embodiments, when determining the sample label of a synthetic sample, the confidence level that the synthetic sample belongs to a positive sample can be determined based on reverberation intensity or signal-to-noise ratio, and this confidence level can be used as the sample label of the synthetic sample. This method of using confidence level values ​​as sample labels not only reduces the workload of manually labeling samples, but also provides more information in the loss calculation stage of subsequent training, thereby improving training efficiency.

[0096] For example, Examples B1 and B2 below are several examples of various ways to determine the sample labels of a synthetic sample.

[0097] Example B1: Determine the sample labels of the first reverberation sample and the second reverberation sample based on their respective reverberation intensities.

[0098] Example B2: Determine the sample labels of the first reverberation sample and the second reverberation sample based on their respective signal-to-noise ratios.

[0099] In each embodiment, in Formula 1, the greater the reverberation intensity corresponding to the room impulse response r, the stronger the reverberation sound is relative to the direct sound in the reverberant speech sample. When the reverberation intensity exceeds the target intensity, the sample label representation of the reverberant speech sample can be determined to be invalid speech. Of course, if the reverberation intensity does not exceed the target intensity, the sample label representation of the reverberant speech sample can be determined to be valid speech. For example, a reverberant speech sample with a large reverberation intensity exceeding the target intensity can be used to simulate background noise in a real speech detection environment.

[0100] In each embodiment, in Formula 1, the signal-to-noise ratio (SNR) characterizes the ratio between effective speech and ambient noise. A higher SNR indicates stronger effective speech and lower noise. If the SNR exceeds a target threshold, the sample label of the reverberant speech sample can be determined to be effective speech; if the SNR does not exceed the target threshold, the sample label of the reverberant speech sample can be determined to be ineffective speech. For example, a reverberant speech sample with a higher SNR can be used to simulate effective speech, while a reverberant speech sample with a lower SNR can be used to simulate background noise such as human voices or ambient noise.

[0101] Of course, the methods of Example B1 and Example B2 can be combined to jointly determine the sample label. For example, when either the reverberation intensity exceeds the target intensity or the signal-to-noise ratio exceeds the target threshold, the sample label of the reverberant speech sample can be determined to be ineffective speech.

[0102] In some embodiments, the sample label may be a category label representing a class. In other embodiments, the sample label may be a numerical value representing the confidence that a sample belongs to a positive sample (hereinafter referred to as effective speech confidence). In various embodiments, taking reverberation intensity as an example, the implementation of example B1 may be modified to include at least one of the following methods 1 and 2.

[0103] Method 1: Use the first category label representing non-effective speech as the sample label of the first reverberation sample, and use the second category label representing effective speech as the sample label of the second reverberation sample;

[0104] Method 2: Based on the reverberation intensity of the first reverberation sample and the second reverberation sample respectively, obtain the effective speech confidence of the first reverberation sample and the second reverberation sample respectively, and use the effective speech confidence of the first reverberation sample and the second reverberation sample respectively as the sample label of the first reverberation sample and the second reverberation sample respectively. The reverberation intensity is negatively correlated with the effective speech confidence.

[0105] In Method 1, if the reverberation intensity exceeds the target intensity, the sample label of the reverberated speech sample is the first label representing invalid speech, meaning the reverberated speech sample is a negative interference sample. If the reverberation intensity does not exceed the target intensity, the sample label is the second label representing valid speech, meaning the reverberated speech sample is a positive speech sample. For example, the sample label can be represented by 0 or 1, where 1 represents the second label representing valid speech and 0 represents the first label representing invalid speech.

[0106] In Method 2, reverberation intensity can be expressed numerically. For example, reverberation intensity can be represented as 1, 0.9, 0.2, 0.05, etc. A larger reverberation intensity value indicates a weaker direct sound and a stronger reverberation; correspondingly, the effective speech confidence is lower. Based on the effective speech confidence, different values ​​can be used to numerically represent the sample labels; a larger reverberation intensity results in a lower effective speech confidence and a smaller sample label value, while a smaller reverberation intensity results in a higher effective speech confidence and a larger sample label value. For example, the sample label can be the effective speech confidence, which can range from 0 to 1. A higher confidence value indicates a stronger direct sound and a weaker reverberation in the reverberated speech sample.

[0107] In some embodiments, the electronic device can map reverberation intensity to effective speech confidence according to a pre-configured mapping relationship. For example, the mapping relationship can be a mapping relationship list, which may include multiple reverberation intensity value ranges and the effective speech confidence corresponding to each reverberation intensity value range. Based on the mapping relationship list, the value range of the reverberation intensity of each reverberated speech sample can be found, and the effective speech confidence corresponding to that value range can be obtained. Of course, a specific mapping function, such as a linear function, can also be used. This application does not specifically limit the mapping method from reverberation intensity to effective speech confidence.

[0108] It should be noted that the reverberation intensity of a reverberant speech sample can be determined by at least one of the following: obtaining the energy attenuation corresponding to the reverberant speech sample, and obtaining the reverberation time corresponding to the reverberant speech sample.

[0109] Among them, energy attenuation represents the degree of attenuation of effective speech during propagation. It can be the ratio of the signal energy of the direct sound and the reverberant sound (sound signal after reflection, refraction, etc.) corresponding to the reverberant speech sample. The greater the energy attenuation, the greater the reverberation intensity.

[0110] Reverberation time, denoted as T60 or RT, is the time required for the sound pressure level to decrease by 60 dB after the sound source of the effective speech data corresponding to the reverberant speech sample stops emitting sound; it is measured in seconds. A longer reverberation time indicates a greater reverberation intensity.

[0111] In one possible implementation, a pre-configured valid speech library and noise library can also be used to construct valid speech samples. In various embodiments, the at least one valid speech sample is constructed by performing the following steps:

[0112] Obtain at least one valid speech data from a pre-configured valid speech library, and obtain at least one noise data from a pre-configured noise library;

[0113] According to at least one signal-to-noise ratio, the acquired effective speech data and noise data are synthesized to obtain at least one effective speech sample.

[0114] In various embodiments, the electronic device can further mix environmental noise data with the effective speech data to more realistically reproduce the actual speech detection scenario. In one possible example, the effective speech data and noise data can be synthesized according to the signal-to-noise ratio using the following formula 2 to obtain the effective speech sample: Formula 2: p = γ(a + βb);

[0115] Where p represents a valid speech sample; a represents valid speech data; b represents noise data; β represents the signal-to-noise ratio; for example, β ranges from 0 to 1 and can be used to simulate different noise levels. γ represents volume intensity; for example, γ ranges from 0 to 1 and can be used to simulate different volume levels.

[0116] In various embodiments, a second label representing the valid speech can be used for each valid speech sample. Alternatively, the sample labels can be numerically represented; for example, the signal-to-noise ratio (SNR) can be mapped to the valid speech confidence level, and the valid speech confidence level can be used to represent the sample label of the valid speech sample. For example, the SNR corresponding to a valid speech sample is positively correlated with the valid speech confidence level; that is, the higher the SNR, the stronger the valid speech and the weaker the noise, and correspondingly, the higher the valid speech confidence level. For example, an SNR of 0.9 corresponds to a valid speech confidence level of 0.9; an SNR of 0.7 corresponds to a valid speech confidence level of 0.7, and so on. This application does not limit the specific representation of the sample labels.

[0117] In some embodiments, a pre-configured noise library may also be used to construct noise samples. In various embodiments, each noise sample is constructed by performing the following steps: obtaining at least one noise data from the pre-configured noise library as initial data, and processing the obtained at least one initial data based on at least one volume intensity to obtain each noise sample.

[0118] For example, noise samples can be obtained by processing data obtained from the noise database according to volume intensity using the following formula 3: Formula 3: n = βb;

[0119] Wherein, β represents the volume intensity, and the value of β ranges from 0 to 1, used to simulate noise of different volume levels.

[0120] In this application, formulas 1 to 3 above can be used to construct a large number of diverse samples, including background noise, effective human voices, and environmental noise.

[0121] In each embodiment, the sample label of the noise sample can be a first label representing ineffective speech. Additionally, the sample label of the noise sample can further characterize environmental noise belonging to the ineffective speech.

[0122] Step 202: Obtain the sample detection results of the speech detection model for the above-mentioned at least one positive speech sample, at least one noise sample and at least one negative interference sample.

[0123] In this step, the electronic device uses an initial speech detection model to perform speech detection on each sample based on the sample audio features of each sample in the first and second sample sets, and obtains the sample detection results for each sample.

[0124] The sample detection result indicates whether the corresponding sample belongs to valid speech (i.e., positive sample).

[0125] In this step, the electronic device can extract the audio features of each sample, input the audio features of each sample into the initial speech detection model, and use the initial speech detection model to perform speech detection on the samples to obtain the corresponding speech detection results. The audio features can be FFT (Fast Fourier Transform) features of the samples.

[0126] For example, each sample is a time-domain signal, and the sample data includes the characteristics of each sampling point of the audio in the time domain, such as the amplitude of each sampling point. In this step, each sample can first be segmented and windowed. Then, the FFT transform is performed on each frame signal, and the FFT coefficients of each frame signal in the frequency domain are extracted. The modulus of the FFT coefficients of each frame signal is taken, and the resulting spectral features are used as the FFT features of the sample.

[0127] In various embodiments, the speech detection model may employ an LSTM (Long Short-Term Memory) network. For example, as shown in Figure 3, an effective speech detection model may include a multi-layer LSTM network and a fully connected layer (FC). The input audio FFT features may first undergo convolution operations in the FC layer, and then be further processed by the LSTM network to extract the contextual features of the samples. The FC layer is then used to repeat the operation on the LSTM network. Finally, the features obtained after repeated operations are input into an FC layer for classification to obtain the detection results of the samples. For example, the FC layer can be used to obtain the probability of a sample corresponding to each category, such as the probability of belonging to valid speech and the probability of belonging to invalid speech, and the final detection result is obtained based on the probabilities of each category.

[0128] In one possible implementation, the sample label of the noise sample also represents environmental noise in the invalid speech; the sample label of the counterexample interference sample also represents background reverberant speech in the invalid speech. In this step, the initial speech detection model can also be used to further detect environmental noise and background reverberant speech in the invalid speech samples; that is, it can detect whether the sample belongs to background reverberant speech in the invalid speech or to environmental noise. Correspondingly,

[0129] In each embodiment, the sample detection result includes at least first result indication information, which indicates whether the sample belongs to valid speech. If the first result indication information indicates that the sample does not belong to valid speech, the sample detection result also includes invalid speech reference information, which indicates that the sample belongs to background reverberant speech or environmental noise in invalid speech.

[0130] Step 203: The electronic device determines the total training loss for each sample based on the sample detection results and sample labels of each sample.

[0131] In this step, the electronic device can measure the total training loss corresponding to each iteration of training based on the distance between the sample detection result and the sample label of each sample in each iteration of training.

[0132] In various embodiments, the electronic device may employ the Cross-Entropy Loss (CE Loss) function to measure the degree of difference between the distribution of each sample's label and the sample detection results of each training iteration, thereby obtaining the training loss. For example, the electronic device may use the following formula 4 to calculate the similarity between each sample detection result and the sample label, thereby obtaining the total training loss:

[0133] Formula 4:

[0134] Where Loss1 represents the cross-entropy loss for each sample. i Labels representing real samples, for example, a first label representing valid speech, or a second label representing invalid speech. i This represents the probability that the network output belongs to valid or invalid speech. This probability can be calculated using a normalized exponential function (softmax) corresponding to the i-th category. In this application, iterative processing is performed based on the training loss to make the classification results of the initial speech detection model increasingly closer to the true sample labels.

[0135] In some embodiments, the distance between samples in the feature space can be further used to iteratively optimize the network. In various embodiments, step 203 may be implemented by steps 2031-2035.

[0136] Step 2031: Based on the similarity between the sample detection results and sample labels of each sample, determine the first training loss corresponding to each sample.

[0137] In some embodiments, the first training loss can be determined using Formula 4 above, that is, the first training loss can be the cross-entropy loss corresponding to each sample.

[0138] In other embodiments, the first training loss can also be determined using the MSE LOSS (Mean Squared Error Loss) function. For example, the first training loss for each sample can be determined using the following formula 5:

[0139] Formula 5:

[0140] Where n is the number of samples, y i1 It is the sample label of the i-th sample, for example, y i1 The value of y can be the effective speech confidence level. i2 It is the speech detection result of the i-th sample, such as the probability that it belongs to valid speech.

[0141] Step 2032: Based on the sample labels of each sample, determine at least one triplet, each triplet including a clean speech sample, a negative interference sample and a positive speech sample.

[0142] The first sample set may contain at least one positive speech sample, which may include at least one pre-defined clean speech data and at least one positive speech sample generated using the clean speech data. The second sample set may contain at least one negative interference sample, which may include at least one negative interference sample generated using the clean speech data.

[0143] In this step, the electronic device can construct multiple triples by using clean speech samples as anchor examples, positive speech samples generated using the clean speech samples as positive examples, and negative interference samples generated using the clean speech samples as negative examples. Alternatively, noise samples and negative interference samples can be used as negative examples. That is, a triple can include a clean speech sample, a negative example, and a positive speech sample; the negative example can be a negative interference sample or a noise sample.

[0144] Step 2033: For each triplet, determine the first feature distance between the audio features of the valid speech samples in the triplet and the negative interference samples; and determine the second feature distance between the audio features of the valid speech samples in the triplet and the positive speech samples.

[0145] The audio features of each sample in the triplet can be obtained by further feature extraction from the initial speech detection model. For example, the audio FFT features of each sample can be input into the initial speech detection model, and after further convolution operations on the input audio FFT features by the FC layer and LSTM network in Figure 3, the features obtained by the FC layer and LSTM network in the initial speech detection model are used as the audio features in step 2033. For example, the feature matrix output by the first or second LSTM in Figure 3 can be used as the audio features of each sample in the triplet to calculate the feature distance using the audio features.

[0146] Step 2034: Based on the first feature distance and the second feature distance corresponding to each triplet, determine the second training loss corresponding to each triplet;

[0147] In each embodiment, the first feature distance and the second feature distance corresponding to the triple can be determined by the following formula 6, and the second training loss can be obtained based on the first feature distance and the second feature distance:

[0148] Formula 6: Loss3=max(d(A,P)-d(A,N)+margin,0);

[0149] Where A represents a valid speech sample, also known as an anchor; P represents a positive speech sample, also known as a positive example. In one example, N can represent a negative interference sample, also known as a negative example; in another example, negative examples can also include noise samples, that is, N can represent both negative interference samples and noise samples. d(A,P) represents the second feature distance; d(A,N) represents the first feature distance; that is, d(A,P) and d(A,N) are the distances between the anchor and the positive example, and between the anchor and the negative example in the feature space, respectively.

[0150] Here, margin is a pre-set distance threshold used to control the difference between positive and negative examples.

[0151] Ideally, the distance between an anchor example and a negative example should be at least greater than the distance between an anchor example and a positive example. As shown in Figure 4, in the early stages of training, the distance between an anchor example and a positive example is greater than the distance between an anchor example and a negative example. As the initial speech detection model is iteratively trained, the distance between an anchor example and a positive example narrows in the feature space, while the distance between an anchor example and a negative example widens, making the distance between an anchor example and a positive example smaller than the distance between an anchor example and a negative example.

[0152] Step 2035: Based on the first training loss corresponding to each sample and the second training loss corresponding to each triplet, obtain the total training loss.

[0153] In each embodiment, the sum of the first training loss and the second training loss can be used as the total training loss. For example, the total loss Loss = Loss1 + Loss3; or, the total loss Loss = Loss2 + Loss3.

[0154] Step 204: The electronic device iteratively trains the initial speech detection model based on the total training loss corresponding to each sample to obtain the speech detection model.

[0155] In this step, the electronic device can use the directional propagation algorithm to calculate the gradient of the total training loss with respect to the network parameters in the initial speech detection model, and then use an optimization algorithm to update the network parameters. For example, the optimization algorithm can use stochastic gradient descent. Based on this, the network detection results can be made closer to the sample labels, thereby reducing the value of the training loss. When the initial speech detection model reaches the iteration stopping condition, the iterative training stops, and the speech detection model is obtained. For example, the iteration stopping condition may include, but is not limited to: the number of iterations exceeding the target number, the total training loss being less than the target loss threshold multiple times consecutively, etc.

[0156] As shown in Figure 5, this application mainly includes two processes: audio FFT feature extraction and effective speech detection model training. For audio FFT feature extraction, operations such as framing and windowing of the audio time-domain signal can be performed, and FFT features are extracted from multiple frames. For the network training process, steps 201-204 described above can be used to iteratively train the initial speech detection model to obtain the final speech detection model.

[0157] In related technologies, speech detection is performed using methods such as energy-based detection and statistical model-based detection. Energy-based detection specifically compares the short-time energy of the audio signal with an energy threshold; if the short-time energy exceeds the energy threshold, the audio is considered speech; otherwise, it is considered non-speech. Statistical model-based detection primarily utilizes modeling techniques for speech and non-speech components. For example, a speech GMM (Gaussian Mixture Model) statistical model and a non-speech GMM statistical model can be modeled separately for detection.

[0158] However, none of the methods in the relevant technologies can detect background noise. In some application scenarios, if these unfiltered background noises are sent to a speech recognition device, they will be recognized as meaningless sentences, affecting the accuracy of speech recognition.

[0159] To address the technical problems in related technologies, this application embodiment designs a model training method. The trained speech detection model can effectively detect background noise, thereby effectively filtering out background noise and ensuring the accuracy and precision of valid speech sources. The technical framework adopted in this application embodiment is based on a deep neural network method, rather than a statistical model based on GMM. This allows for the differentiation of background noise beyond just distinguishing between human speech and noise, thus preventing background noise from being mistaken for valid speech. Furthermore, this application embodiment constructs a rich sample containing a large number of background noise samples by simulating valid speech and using reverberant speech to simulate background noise. During the training phase, valid human speech is treated as one category, while background noise and noise are treated as another category to train the deep neural network model. This enables the trained deep neural network to accurately distinguish between valid speech and background noise, improving the accuracy and practicality of speech detection.

[0160] The model training method provided in this application obtains a first sample set of ineffective speech and a second sample set of effective speech. The first sample set includes counterexample interference samples and noise samples. The effective speech samples include effective speech data from human voice sources. The counterexample interference samples are obtained by propagating the effective speech data in sample rooms with different reverberation intensities. The noise samples include environmental noise data from non-human voice sources. Based on this, the speech of background human voice sources can be simulated using counterexample interference samples. The first and second sample sets are used for training, so that the trained speech detection model can not only accurately identify effective human voices and ineffective environmental noise, but also effectively identify background noisy human voices that belong to ineffective speech, thereby improving the accuracy and practicality of speech detection.

[0161] Furthermore, the trained speech detection model can effectively assist in speech recognition or voiceprint recognition tasks. For speech recognition tasks, the speech detection model trained using the method described in this application can further improve the robustness of the speech detection process, accurately filtering out background noise in addition to effectively filtering out noise. For voiceprint recognition tasks, the speech detection model described in this application, in addition to filtering out noise, can also finely distinguish between valid human voices and background noise, thereby improving the accuracy of voiceprint recognition. This further enhances the practicality of the model training method and the speech detection method.

[0162] In various embodiments, the electronic device may also use different sets of training samples to further train the already deployed speech recognition model.

[0163] Figure 6 is a schematic flowchart of a speech detection method provided in an embodiment of this application. This method can be executed by an electronic device. In some embodiments, the electronic device can be an electronic device that trains a speech detection model; for example, after training the speech detection model, the server uses the trained speech detection model to perform speech detection. In other embodiments, the electronic device can be another electronic device different from the electronic device that trained the speech detection model. For example, after training the speech detection model, a first electronic device provides the trained speech detection model to another electronic device, which then uses the speech detection model to perform speech detection. As shown in Figure 6, the method includes steps 601-602.

[0164] Step 601: The electronic device acquires at least one frame of audio signal (also known as at least one audio signal frame) from the target room where the target object is located.

[0165] In some possible scenario examples, for instance, in a meeting setting, during the speaker's presentation in the meeting room, audio acquisition devices can continuously capture audio from the target object to obtain multiple frames of audio signals.

[0166] For example, in a teaching setting, audio can be continuously collected from the teacher during a lecture in the classroom, resulting in multiple frames of audio signals.

[0167] For example, in online multi-user video or voice scenarios, audio can be captured from each user's room or environment to obtain multiple frames of audio signals.

[0168] Step 602: The electronic device detects each frame of audio signal based on the trained speech detection model to obtain the detection result of each frame of audio signal. The detection result indicates whether the corresponding frame of audio signal belongs to valid speech.

[0169] The speech detection model was trained using the model training method described above.

[0170] For example, the electronic device can pre-configure the speech detection model locally and use the local speech detection model to detect each frame of audio signal to determine whether each frame of audio signal belongs to valid speech or invalid speech; it can also detect whether the audio signal of each frame of invalid speech belongs to environmental noise or background reverberation speech.

[0171] In some embodiments, the detection result includes audio indication information and reference information. The audio indication information indicates whether the corresponding frame audio signal belongs to valid speech (positive example), and the reference information indicates whether the corresponding frame audio signal belongs to background reverberant speech or environmental noise in invalid speech (negative example). For example, background reverberant speech may include background noise. Valid and invalid speech are distinguished based on the audio indication information. In addition, the reference information can be used to further distinguish whether invalid speech is environmental noise or background voices from people other than the speaker.

[0172] The detection results in this application can also be further applied to other audio processing tasks. For example, the detection results can be combined to further perform speech recognition tasks, voiceprint recognition tasks, etc. Accordingly, after step 602, any one of the following steps 603 and 604 may also be included.

[0173] Step 603: Based on the audio indication information of each frame of audio signal, remove the first audio signal corresponding to the background reverberation speech or environmental noise that belongs to non-effective speech from each frame of audio signal.

[0174] In this step, after filtering using audio indication information, valid human voices in the audio are accurately preserved. For example, in speech recognition tasks, not only can environmental noise in an audio clip be effectively filtered out, but background noise can also be further filtered out with greater precision.

[0175] Step 604: Based on the audio indication information and reference information of each frame of audio signal, determine the second audio signal that belongs to valid speech and the third audio signal that belongs to background reverberation speech in each frame of audio signal; and perform voiceprint recognition on each frame of audio signal based on the audio features of the second audio signal and the third audio signal to obtain the audio signals from the target object and other objects outside the target object in each frame of audio signal.

[0176] In this step, for invalid audio segments in the audio, the reference information can be used to further distinguish between background voices and noise in the invalid audio segments. For example, in a voiceprint recognition task, the reference information can be used to extract valid speech from valid voice sources and background reverberant speech from background voice sources. Based on this, the audio features of different voice sources can be combined to distinguish the audio of different voices in an audio segment.

[0177] For example, in a teaching scenario, if an audio clip contains questions asked by student A during a lecture in classroom A, and answers given by student B and student C, the speech detection model of this application can be used to obtain reference information for each frame of invalid audio, determine the audio frames corresponding to student B and student C, and further obtain the content of the answers given by student B and student C in the subsequent answering process audio.

[0178] For example, in a meeting scenario, when it is necessary to generate the content spoken by different users during the discussion, the reference information of each frame of invalid audio obtained by the speech detection model can be combined to identify the speech of different users in an audio segment and accurately generate the content spoken by each user.

[0179] As shown in Figure 7, the left channel represents the original speech, while the right channel represents the valid speech retained based on the network detection results. As shown in Figure 7, the speech detection model provided in this application can effectively detect background noise and human voices as invalid speech. Table 1 below shows the metrics for testing the detection results of different technologies:

[0180] Table 1

[0181] Precision is an indicator used to determine the accuracy of effective speech classification; a higher value indicates higher accuracy in effective speech detection. As shown in Table 1, the speech detection method of this application has higher accuracy than related technologies. The method of this application significantly improves speech detection accuracy.

[0182] The model training method provided in this application obtains a first sample set of ineffective speech and a second sample set of effective speech. The first sample set includes counterexample interference samples and noise samples. The effective speech samples include effective speech data from human voice sources. The counterexample interference samples are obtained by propagating the effective speech data in sample rooms with different reverberation intensities. The noise samples include environmental noise data from non-human voice sources. Based on this, the speech of background human voice sources can be simulated using counterexample interference samples. The first and second sample sets are used for training, so that the trained speech detection model can not only accurately identify effective human voices and ineffective environmental noise, but also effectively identify background noisy human voices that belong to ineffective speech, thereby improving the accuracy and practicality of speech detection.

[0183] Furthermore, the trained speech detection model can effectively assist in speech recognition or voiceprint recognition tasks. For speech recognition tasks, the speech detection model trained using the method described in this application can further improve the robustness of the speech detection process, accurately filtering out background noise in addition to effectively filtering out noise. For voiceprint recognition tasks, the speech detection model described in this application, in addition to filtering out noise, can also finely distinguish between valid human voices and background noise, thereby improving the accuracy of voiceprint recognition. This further enhances the practicality of the model training method and the speech detection method.

[0184] The model training method and speech detection method provided in this application involve the aforementioned artificial intelligence technology, machine learning, speech technology, and other technologies, and can be applied to various technical fields such as online conferencing, intelligent transportation, and autonomous driving.

[0185] Figure 8 is a schematic diagram of a model training device provided in an embodiment of this application. As shown in Figure 8, the device includes:

[0186] The sample acquisition module 801 is used to acquire a first sample set and a second sample set, wherein the sample labels corresponding to the first sample set and the second sample set respectively represent invalid speech and valid speech; the first sample set includes at least one negative interference sample and a noise sample including speech, and the second sample set includes at least one positive speech sample.

[0187] For example, the positive speech sample may include valid speech data from a human voice source; the negative interference samples may be obtained by propagating valid speech data in sample rooms with various reverberation intensities, and the noise sample may include ambient noise data from a non-human voice source.

[0188] The first detection module 802 is used to perform speech detection on each sample based on the audio features of each sample in the first sample set and the second sample set, using a speech detection model to obtain the sample detection result of each sample. The sample detection result represents whether the corresponding sample belongs to valid speech.

[0189] The total loss determination module 803 is used to determine the total training loss based on the sample detection results and sample labels of each sample.

[0190] Training module 804 is used to iteratively train the speech detection model based on the total training loss to obtain the trained speech detection model.

[0191] In some embodiments, the second sample set includes at least one positive speech sample; the sample acquisition module, when determining at least one negative interference sample and a positive speech sample, may include any of the following:

[0192] The first determining unit is configured to perform reverberation processing on the clean speech data based on at least one first impulse response to obtain at least one first reverberation sample as a negative interference sample, and to perform reverberation processing on the clean speech data based on at least one second impulse response to obtain at least one second reverberation sample as a positive speech sample.

[0193] The first impulse response or the second impulse response characterizes the propagation characteristics of sound in the sample room, and the reverberation intensity corresponding to the first impulse response is greater than that of the second impulse response.

[0194] The second determining unit is configured to perform reverberation processing on at least one clean speech data from the target human voice source based on the target human voice source corresponding to each clean speech sample in the second sample set, to obtain at least one second reverberation sample, and to perform reverberation processing on at least one clean speech data from a non-target human voice source, to obtain at least one first reverberation sample.

[0195] In some embodiments, the first determining unit is configured to:

[0196] Convolution processing is performed on each first impulse response and the clean speech data to generate the initial reverberation data corresponding to each first impulse response;

[0197] Noise data is obtained from a pre-configured noise library, and the initial reverberation data corresponding to each first impulse response and the obtained noise data are synthesized according to at least one signal-to-noise ratio to obtain the at least one first reverberation sample.

[0198] In some embodiments, the sample acquisition module, when determining at least one first reverberation sample and a second reverberation sample, further includes at least one of the following:

[0199] The third determining unit is used to determine the sample label of the first reverberation sample and the sample label of the second reverberation sample based on the reverberation intensity corresponding to the first reverberation sample and the second reverberation sample, respectively.

[0200] The fourth determining unit is used to determine the sample label of the first reverberation sample and the sample label of the second reverberation sample based on the signal-to-noise ratio of the first reverberation sample and the second reverberation sample, respectively.

[0201] In some embodiments, the third determining unit is used for at least one of the following:

[0202] The first category label representing non-effective speech is used as the sample label for the first reverberation sample, and the second category label representing effective speech is used as the sample label for the second reverberation sample.

[0203] Based on the reverberation intensity of the first and second reverberation samples respectively, the effective speech confidence scores of the first and second reverberation samples are obtained respectively, and the effective speech confidence scores of the first and second reverberation samples are used as the sample labels of the first and second reverberation samples respectively. The reverberation intensity is negatively correlated with the effective speech confidence score.

[0204] In some embodiments, the second sample set includes at least one pre-defined clean speech data and at least one positive example speech sample synthesized using the clean speech data, and the first sample set includes at least one negative example interference sample synthesized using the clean speech data; the total loss determination module is used to:

[0205] Based on the similarity between the sample detection results and sample labels of each sample, the first training loss corresponding to each sample is determined.

[0206] Based on the sample labels of each sample, at least one triplet is determined. Each triplet includes a clean speech sample, a negative interference sample generated using the clean speech sample, and a positive speech sample generated using the clean speech sample.

[0207] For each triplet, determine a first feature distance between the audio features of the clean speech sample in the triplet and the audio features of the negative interference sample; and determine a second feature distance between the audio features of the clean speech sample in the triplet and the audio features of the positive speech sample.

[0208] Based on the first feature distance and the second feature distance corresponding to each triplet, the second training loss corresponding to each triplet is determined.

[0209] The total training loss is obtained based on the first training loss corresponding to each sample and the second training loss corresponding to each triplet.

[0210] In some embodiments, the sample label of the noise sample also characterizes ambient noise in the invalid speech; the sample label of the counterexample interference sample also characterizes background reverberant speech in the invalid speech.

[0211] The sample detection result includes at least a first result indication information, which indicates whether the sample belongs to valid speech;

[0212] If the first result indication information indicates that the sample does not belong to valid speech, then the sample detection result also includes invalid speech reference information, which indicates that the sample belongs to background reverberant speech or environmental noise in invalid speech.

[0213] In some embodiments, the sample acquisition module, when acquiring at least one positive speech sample, is used to:

[0214] Obtain at least one clean speech data from a pre-configured clean speech library, and obtain at least one noise data from a pre-configured noise library;

[0215] According to at least one signal-to-noise ratio, the acquired clean speech data and noise data are synthesized to obtain at least one positive speech sample.

[0216] The model training method provided in this application obtains a first sample set of ineffective speech and a second sample set of effective speech. The first sample set includes counterexample interference samples and noise samples. The effective speech samples include effective speech data from human voice sources. The counterexample interference samples are obtained by propagating the effective speech data in sample rooms with different reverberation intensities. The noise samples include environmental noise data from non-human voice sources. Based on this, the speech of background human voice sources can be simulated using counterexample interference samples. The first and second sample sets are used for training, so that the trained speech detection model can not only accurately identify effective human voices and ineffective environmental noise, but also effectively identify background noisy human voices that belong to ineffective speech, thereby improving the accuracy and practicality of speech detection.

[0217] Furthermore, the trained speech detection model can effectively assist in speech recognition or voiceprint recognition tasks. For speech recognition tasks, the speech detection model trained using the method described in this application can further improve the robustness of the speech detection process, accurately filtering out background noise in addition to effectively filtering out noise. For voiceprint recognition tasks, the speech detection model described in this application, in addition to filtering out noise, can also finely distinguish between valid human voices and background noise, thereby improving the accuracy of voiceprint recognition. This further enhances the practicality of the model training method and the speech detection method.

[0218] Figure 9 is a schematic diagram of a speech detection device provided in an embodiment of this application. As shown in Figure 9, the device includes:

[0219] The audio signal acquisition module 901 is used to acquire at least one frame of audio signal from the target room where the target object is located.

[0220] The second detection module 902 is used to detect each frame of audio signal based on the trained speech detection model, and obtain the detection result of each frame of audio signal. The detection result indicates whether the corresponding frame of audio signal belongs to valid speech.

[0221] The speech detection model was trained using the aforementioned model training method.

[0222] In some embodiments, the detection result includes audio indication information and reference information, wherein the audio indication information indicates whether the corresponding frame audio signal belongs to valid speech, and the reference information indicates whether the corresponding frame audio signal belongs to background reverberant speech or environmental noise in invalid speech; the device further includes any one of the following:

[0223] The cut-off module is used to cut off the first audio signal corresponding to the background reverberant speech or environmental noise that belongs to non-effective speech from each frame of audio signal based on the audio indication information of each frame of audio signal;

[0224] The recognition module is used to determine the second audio signal that belongs to valid speech and the third audio signal that belongs to background reverberation speech in each frame of audio signal based on the audio indication information and reference information of each frame of audio signal respectively; and to perform voiceprint recognition on each frame of audio signal based on the audio features of the second audio signal and the third audio signal to obtain the audio signals from the target object and other objects outside the target object in each frame of audio signal respectively.

[0225] The method provided in this application, through the speech detection model trained using the aforementioned model training method, can effectively assist in speech recognition or voiceprint recognition tasks. For speech recognition tasks, the speech detection model trained using the method of this application can further improve the robustness of the speech detection process, accurately filtering out background noise in addition to effectively filtering out noise. For voiceprint recognition tasks, the speech detection model of this application, in addition to filtering out noise, can also finely distinguish between valid human voices and background noise, thereby improving the accuracy of voiceprint recognition. This further enhances the practicality of the model training method and the speech detection method.

[0226] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0227] Figure 10 is a schematic diagram of an electronic device provided in an embodiment of this application. As shown in Figure 10, the electronic device includes: a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the model training method and the speech detection method described above.

[0228] In one optional embodiment, an electronic device is provided, as shown in FIG10. The electronic device 1000 shown in FIG10 includes a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, for example, via a bus 1002. Optionally, the electronic device 1000 may further include a transceiver 1004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 1004 is not limited to one type, and the structure of the electronic device 1000 does not constitute a limitation on the embodiments of this application.

[0229] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0230] Bus 1002 may include a pathway for transmitting information between the aforementioned components. Bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 1002 may be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in Figure 10, but this does not indicate that there is only one bus or one type of bus.

[0231] The memory 1003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0232] The memory 1003 is used to store computer programs that execute the embodiments of this application, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the computer programs stored in the memory 1003 to implement the steps shown in the foregoing method embodiments.

[0233] Electronic devices include, but are not limited to, servers, terminals, or cloud computing center equipment.

[0234] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0235] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0236] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0237] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. The terms “comprising” and “including” as used in the embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, or operation, but do not exclude implementation as other features, information, data, steps, or operations supported by this art.

[0238] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0239] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0240] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0241] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A speech detection model training method, executed by an electronic device, the method comprising: Obtain at least one positive speech sample, at least one noise sample, and at least one negative interference sample including speech, wherein the sample label of the positive speech sample represents the positive sample, and the sample labels of the noise sample and the negative interference sample represent the negative sample; Obtain the sample detection results of the speech detection model for the at least one positive speech sample, the at least one noise sample and the at least one negative interference sample, wherein the sample detection result of a sample represents whether the sample belongs to the positive sample; The total training loss is determined based on the sample detection results and the sample labels, and the speech detection model is iteratively trained based on the total training loss to obtain the trained speech detection model.

2. The method according to claim 1, wherein, The acquisition of at least one positive speech sample, at least one noise sample, and at least one negative interference sample including speech includes: By utilizing at least one of preset environmental noise data and preset spatial propagation characteristics, preset clean speech data is processed to obtain the positive example speech sample and the negative example interference sample.

3. The method according to claim 2, wherein, The step of processing the preset clean speech data using at least one of preset environmental noise data and preset spatial propagation characteristics to obtain the positive speech samples and the negative interference samples includes at least one of the following: At least one synthesized sample is obtained by processing the preset clean speech data using the impulse response in the preset spatial propagation characteristics; the sample label of the at least one synthesized sample is determined based on the reverberation intensity of the impulse response. At least one synthetic sample is obtained by mixing preset environmental noise data with preset clean speech data at a preset signal-to-noise ratio; the sample label of the at least one synthetic sample is determined based on the signal-to-noise ratio.

4. The method according to claim 3, wherein, The step of processing the preset clean speech data using at least one of preset environmental noise data and preset spatial propagation characteristics to obtain the positive example speech sample and the negative example interference sample includes at least one of the following: Mix at least two clean speech data from at least two human voice sources to obtain at least one counterexample interference sample. Mix at least two clean speech data from at least two human voice sources, and process the mixing result using a preset impulse response to obtain at least one counterexample interference sample. Based on a preset signal-to-noise ratio, preset environmental noise data is mixed with at least two preset clean speech data from at least two human voice sources to obtain at least one counterexample interference sample. The preset impulse response is used to process at least one preset clean speech data, and the processing result is mixed with preset environmental noise data based on the preset signal-to-noise ratio to obtain at least one counterexample interference sample.

5. The method according to claim 3, wherein, Determining the sample label of the at least one synthesized sample based on the reverberation intensity of the impulse response, or determining the sample label of the at least one synthesized sample based on the signal-to-noise ratio, includes at least one of the following: Based on the reverberation intensity being greater than a preset intensity threshold or the signal-to-noise ratio being less than a preset threshold, the sample label of the at least one synthesized sample is determined to be a negative sample; based on the reverberation intensity of the impulse response being less than a preset intensity threshold or the signal-to-noise ratio being greater than a preset threshold, the sample label of the at least one synthesized sample is determined to be a positive sample. Based on the reverberation intensity or the signal-to-noise ratio, determine the confidence level that the at least one synthesized sample belongs to the positive sample, and use the confidence level as the sample label of the at least one synthesized sample.

6. The method according to any one of claims 2-4, wherein, The obtained positive example speech sample and the negative example interference sample include at least one of the following: At least one second positive speech sample is obtained by adjusting the volume of the first positive speech sample in the positive speech sample using at least one preset adjustment coefficient. By adjusting the volume of the first counterexample interference sample in the counterexample interference sample using at least one preset adjustment coefficient, at least one second counterexample interference sample is obtained.

7. The method according to any one of claims 1-6, wherein, The sample labels of the noise samples further characterize the environmental noise; the sample labels of the counterexample interference samples further characterize the background speech. The sample detection result includes at least a first result indication information, which indicates whether the sample belongs to a positive sample. If the first result indication information indicates that the sample does not belong to the positive sample, then the sample detection result also includes reference information, which indicates that the sample belongs to background speech or environmental noise.

8. The method according to any one of claims 1-7, wherein, The at least one positive example speech sample includes at least one set of clean speech data and at least one positive example speech sample generated using the clean speech data; the at least one negative example interference sample includes at least one negative example interference sample generated using the clean speech data. Determining the total training loss based on the sample detection results and the sample labels includes: Based on the similarity between the sample detection results and sample labels of each sample, the first training loss corresponding to each sample is determined; Determine at least one triplet, each triplet comprising a first clean speech data in the at least one clean speech data, a first positive speech sample generated using the first clean speech data, and a first negative interference sample generated using the first clean speech data; For each triplet, a first feature distance is determined between the audio features of the first clean speech data in the triplet and the audio features of the first negative interference sample; and a second feature distance is determined between the audio features of the first clean speech data and the audio features of the first positive speech sample. Based on the first feature distance and the second feature distance corresponding to each triplet, the second training loss corresponding to each triplet is determined; The total training loss is obtained based on the first training loss corresponding to each sample and the second training loss corresponding to each triplet.

9. A speech detection method, performed by an electronic device, the method comprising: The speech detection model trained using the model training method according to any one of claims 1-8 is used to perform speech detection on at least one audio signal frame, and the detection result of each audio signal frame in the at least one audio signal frame is obtained.

10. The method according to claim 9, wherein, The detection result includes audio indication information and reference information. The audio indication information indicates whether the audio signal frame belongs to a positive example, and the reference information indicates whether the audio signal frame belongs to background speech or environmental noise in the negative examples. The method further includes any one of the following: Based on the audio indication information of each audio signal frame, the first audio signal corresponding to the background speech or environmental noise is removed from each audio signal frame; Based on the audio indication information and reference information of each audio signal frame, the second audio signal belonging to the positive example and the third audio signal belonging to the background speech are determined in each audio signal frame; and voiceprint recognition is performed on each audio signal frame based on the audio features of the second audio signal and the third audio signal to obtain the audio signal from the target object and the audio signal from other objects outside the target object in each audio signal frame.

11. A model training device, comprising: The sample acquisition module is used to acquire at least one positive speech sample, at least one noise sample, and at least one negative interference sample including speech, wherein the sample label of the positive speech sample represents the positive sample, and the sample labels of the noise sample and the negative interference sample represent the negative sample. The first detection module is used to obtain the sample detection results obtained by the speech detection model for the positive speech sample, the noise sample and the negative interference sample, wherein the sample detection result of a sample represents whether the sample belongs to the positive sample; The total loss determination module is used to determine the total training loss based on the sample detection results and the sample labels; The training module is used to iteratively train the speech detection model based on the total training loss to obtain the trained speech detection model.

12. The apparatus according to claim 11, wherein, The sample acquisition module is used for: By utilizing at least one of preset environmental noise data and preset spatial propagation characteristics, preset clean speech data is processed to obtain the positive example speech sample and the negative example interference sample.

13. The apparatus according to claim 12, wherein, The sample acquisition module is used to perform at least one of the following: At least one synthesized sample is obtained by processing the preset clean speech data using the impulse response in the preset spatial propagation characteristics; the sample label of the at least one synthesized sample is determined based on the reverberation intensity of the impulse response. At least one synthetic sample is obtained by mixing preset environmental noise data with preset clean speech data at a preset signal-to-noise ratio; the sample label of the at least one synthetic sample is determined based on the signal-to-noise ratio.

14. The apparatus according to claim 13, wherein, The sample acquisition module is used to perform at least one of the following: Mix at least two clean speech data from at least two human voice sources to obtain at least one counterexample interference sample. Mix at least two clean speech data from at least two human voice sources, and process the mixing result using a preset impulse response to obtain at least one counterexample interference sample. Based on a preset signal-to-noise ratio, preset environmental noise data is mixed with at least two preset clean speech data from at least two human voice sources to obtain at least one counterexample interference sample. A preset pure speech data is processed using a preset impulse response, and the processing result is mixed with preset environmental noise data based on a preset signal-to-noise ratio to obtain at least one counterexample interference sample.

15. The apparatus according to claim 13, wherein, The sample acquisition module is used to perform at least one of the following: When the reverberation intensity is greater than a preset intensity threshold or the signal-to-noise ratio is less than a preset threshold, the sample label of the at least one synthesized sample is determined to be a negative sample; when the reverberation intensity of the impulse response is less than a preset intensity threshold or the signal-to-noise ratio is greater than a preset threshold, the sample label of the at least one synthesized sample is determined to be a positive sample. Based on the reverberation intensity or the signal-to-noise ratio, determine the confidence level that the at least one synthesized sample belongs to the positive sample, and use the confidence level as the sample label of the at least one synthesized sample.

16. The apparatus according to any one of claims 11-15, wherein, The at least one positive example speech sample includes at least one preset clean speech sample and at least one positive example speech sample synthesized using the clean speech sample; the at least one negative example interference sample includes at least one negative example interference sample synthesized using the clean speech sample. The training module is used for: Based on the similarity between the sample detection results and sample labels of each sample, the first training loss corresponding to each sample is determined; Determine at least one triplet, each triplet comprising a first clean speech sample from the at least one clean speech sample, a first positive example speech sample synthesized using the first clean speech sample, and a first negative example interference sample synthesized using the first clean speech sample; For each triplet, a first feature distance is determined between the audio features of the first clean speech sample and the audio features of the first negative interference sample in the triplet; and a second feature distance is determined between the audio features of the first clean speech sample and the audio features of the first positive speech sample. Based on the first feature distance and the second feature distance corresponding to each triplet, the second training loss corresponding to each triplet is determined; The total training loss is obtained based on the first training loss corresponding to each sample and the second training loss corresponding to each triplet.

17. A voice detection device, comprising: The second detection module is used to perform speech detection on at least one audio signal frame using a speech detection model trained according to any one of claims 1-8, and to obtain the detection result of the at least one audio signal frame.

18. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 10.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 10.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 10.

Images