Voice activity detection method, device, equipment and storage medium

Abstract

The present application discloses a method, apparatus, device and storage medium for voice activity detection, belonging to the field of artificial intelligence technology. The method comprises: based on the collected first voice data of the first user, obtaining the acoustic feature template of the first user and the conversation intention of the first user; in response to the first user's conversation intention indicating the occurrence of a voice interruption event, adjusting the voice detection parameters of the voice detection model based on the current voice detection state; wherein the voice interruption event refers to the user's speech being interrupted; storing the acoustic feature template of the first user and the adjusted voice detection parameters in a database; when applied, based on the collected second voice data of the second user and the database, confirming the voice detection parameters of the voice detection model that matches the second user; based on the obtained voice detection parameters, performing voice activity detection on the subsequent voice data of the second user through the voice detection model. The present application can reduce the probability of the device interrupting the user's speech.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a voice activity detection method, apparatus, device, and storage medium. Background Art

[0002] Voice Activity Detection (VAD), also known as voice activity detection, is an important technology in the development of artificial intelligence. It is often used in human-computer dialogue scenarios to detect whether a voice signal exists.

[0003] In related technologies, some users speak slowly, for example, with multiple short pauses in between sentences. This makes it easy for devices to interrupt users during voice activity detection. This means that even before a user finishes speaking, the voice interaction device's VAD model incorrectly determines that the user has finished speaking and then responds.

[0004] This will undoubtedly seriously affect the quality of dialogue during human-computer dialogue, resulting in poor human-computer interaction effects. Summary of the Invention

[0005] The embodiments of the present application provide a method, apparatus, device, and storage medium for detecting voice activity, which can improve the quality of human-computer dialogue and achieve better human-computer interaction. The technical solution is as follows:

[0006] In one aspect, a method for detecting voice activity is provided, the method comprising:

[0007] Acquiring an acoustic feature template of the first user and a conversation intention of the first user based on the collected first voice data of the first user;

[0008] In response to the first user's conversation intention indicating a speech interruption event, adjusting speech detection parameters of a speech detection model based on a current speech detection state; the speech interruption event refers to the user's speech being interrupted; the speech detection state includes a speech start point detection state and a speech end point detection state;

[0009] Storing the acoustic feature template of the first user and the adjusted speech detection parameters in a database;

[0010] Determining, based on the collected second voice data of the second user and the database, voice detection parameters of a voice detection model that matches the second user;

[0011] Based on the speech detection parameters of the speech detection model matched with the second user, speech activity detection is performed on subsequent speech data of the second user through the speech detection model.

[0012] In one possible implementation, adjusting the speech detection parameters of the speech detection model based on the current speech detection state includes:

[0013] In response to not currently entering the voice start point detection state, increasing the value of a first voice detection parameter; wherein the first voice detection parameter is used to indicate a front-end silence detection timeout period;

[0014] In response to currently entering the voice endpoint detection state, the value of the second voice detection parameter is increased; wherein the second voice detection parameter is used to indicate the backend pause time.

[0015] In a possible implementation, the method further includes:

[0016] Acquiring signal energy of the first voice data;

[0017] In response to the acquired signal energy beginning to be greater than a first threshold, determining that the speech start point detection state has been entered;

[0018] In response to the acquired signal energy being less than the second threshold and the duration exceeding the preset duration, it is determined that the voice endpoint detection state has been entered.

[0019] In one possible implementation, adjusting the speech detection parameters of the speech detection model based on the current speech detection state includes:

[0020] Adjusting speech detection parameters of the speech detection model based on a current speech detection state and target constraints;

[0021] The target constraint conditions include: a maximum adjustable range and an adjustment step size for a single adjustment.

[0022] In a possible implementation, the acquiring, based on the collected first voice data of the first user, an acoustic feature template of the first user includes:

[0023] performing acoustic feature extraction on the first speech data;

[0024] performing a user age analysis on the first voice data to obtain an age feature of the first user;

[0025] Performing user gender analysis on the first voice data to obtain gender characteristics of the first user;

[0026] Performing user emotion analysis on the first voice data to obtain an emotional characteristic of the first user;

[0027] performing feature fusion on the extracted acoustic features, the age features, the gender features, and the emotion features to obtain acoustic features of the first user;

[0028] The acoustic features of the first user are input into a voiceprint recognition model to obtain an acoustic feature template of the first user.

[0029] In a possible implementation, obtaining the conversation intention of the first user based on the collected first voice data of the first user includes:

[0030] Determining an environment type of a current sound pickup environment, and performing speech recognition on the first voice data based on a speech recognition model matching the environment type to obtain a speech recognition result;

[0031] Performing intent recognition on the speech recognition result to obtain the conversation intention of the first user;

[0032] Wherein, the environment type includes a quiet environment and a noisy environment;

[0033] The speech recognition model is obtained by retraining a pre-trained model based on a sample set, where the sample set includes speech data collected in a corresponding environment.

[0034] In a possible implementation, performing speech recognition on the first speech data based on a speech recognition model matching the environment type includes:

[0035] In response to the current sound pickup environment being a quiet environment, performing speech recognition on the first voice data based on a first speech recognition model matching the quiet environment;

[0036] In response to the current sound pickup environment being a noisy environment, performing speech recognition on the first speech data based on a second speech recognition model matching the noisy environment;

[0037] The first speech recognition model is obtained by retraining a pre-trained model based on a first sample set, wherein the first sample set includes speech data collected in the quiet environment;

[0038] The second speech recognition model is obtained by retraining the pre-model based on a second sample set, where the second sample set includes speech data collected in the noisy environment.

[0039] In a possible implementation, determining the environment type of the current sound pickup environment includes:

[0040] Determine the venue type, surrounding infrastructure information, and current pickup time of the current pickup environment;

[0041] The environment type of the current sound pickup environment is determined according to the venue type, the surrounding infrastructure information, and the current sound pickup time.

[0042] In one possible implementation, the determining, based on the collected second voice data of the second user and the database, voice detection parameters of a voice detection model that matches the second user includes:

[0043] performing acoustic feature extraction on the second speech data;

[0044] Inputting the acoustic features of the second user into a voiceprint recognition model to obtain an acoustic feature template of the second user;

[0045] comparing the acoustic feature template of the second user with the acoustic feature templates stored in the database;

[0046] In response to the degree of matching between the acoustic feature template of the second user and the target acoustic feature template in the database being greater than a third threshold, the speech detection parameters corresponding to the target acoustic feature template are used as the speech detection parameters of the speech detection model matching the second user.

[0047] In another aspect, a voice activity detection device is provided, the device comprising:

[0048] A first acquisition module is configured to acquire an acoustic feature template of the first user and a conversation intention of the first user based on the collected first voice data of the first user;

[0049] a parameter adjustment module configured to adjust speech detection parameters of a speech detection model based on a current speech detection state in response to a speech interruption event indicated by the first user's conversation intention, wherein the speech interruption event refers to the user's speech being interrupted; and the speech detection state includes a speech start point detection state and a speech end point detection state;

[0050] a storage module configured to store the acoustic feature template of the first user and the adjusted speech detection parameters in a database;

[0051] A second acquisition module is configured to determine the speech detection parameters of the speech detection model that matches the second user based on the collected second speech data of the second user and the database;

[0052] The voice detection module is configured to perform voice activity detection on subsequent voice data of the second user through the voice detection model based on the voice detection parameters of the voice detection model matched with the second user.

[0053] In a possible implementation, the parameter adjustment module is configured to:

[0054] In response to not currently entering the voice start point detection state, increasing the value of a first voice detection parameter; wherein the first voice detection parameter is used to indicate a front-end silence detection timeout period;

[0055] In response to currently entering the voice endpoint detection state, the value of the second voice detection parameter is increased; wherein the second voice detection parameter is used to indicate the backend pause time.

[0056] In a possible implementation, the parameter adjustment module is configured to:

[0057] Acquiring signal energy of the first voice data;

[0058] In response to the acquired signal energy beginning to be greater than a first threshold, determining that the speech start point detection state has been entered;

[0059] In response to the acquired signal energy being less than the second threshold and the duration exceeding the preset duration, it is determined that the voice endpoint detection state has been entered.

[0060] In a possible implementation, the parameter adjustment module is configured to:

[0061] Adjusting speech detection parameters of the speech detection model based on a current speech detection state and target constraints;

[0062] The target constraint conditions include: a maximum adjustable range and an adjustment step size for a single adjustment.

[0063] In a possible implementation, the first acquisition module is configured to:

[0064] performing acoustic feature extraction on the first speech data;

[0065] performing a user age analysis on the first voice data to obtain an age feature of the first user;

[0066] Performing user gender analysis on the first voice data to obtain gender characteristics of the first user;

[0067] Performing user emotion analysis on the first voice data to obtain an emotional characteristic of the first user;

[0068] performing feature fusion on the extracted acoustic features, the age features, the gender features, and the emotion features to obtain acoustic features of the first user;

[0069] The acoustic features of the first user are input into a voiceprint recognition model to obtain an acoustic feature template of the first user.

[0070] In a possible implementation, the first acquisition module is configured to:

[0071] Determining an environment type of a current sound pickup environment, and performing speech recognition on the first voice data based on a speech recognition model matching the environment type to obtain a speech recognition result;

[0072] Performing intent recognition on the speech recognition result to obtain the conversation intention of the first user;

[0073] Wherein, the environment type includes a quiet environment and a noisy environment;

[0074] The speech recognition model is obtained by retraining a pre-trained model based on a sample set, where the sample set includes speech data collected in a corresponding environment.

[0075] In a possible implementation, the first acquisition module is configured to:

[0076] In response to the current sound pickup environment being a quiet environment, performing speech recognition on the first voice data based on a first speech recognition model matching the quiet environment;

[0077] In response to the current sound pickup environment being a noisy environment, performing speech recognition on the first speech data based on a second speech recognition model matching the noisy environment;

[0078] The first speech recognition model is obtained by retraining a pre-trained model based on a first sample set, wherein the first sample set includes speech data collected in the quiet environment;

[0079] The second speech recognition model is obtained by retraining the pre-model based on a second sample set, where the second sample set includes speech data collected in the noisy environment.

[0080] In a possible implementation, the first acquisition module is configured to:

[0081] Determine the venue type, surrounding infrastructure information, and current pickup time of the current pickup environment;

[0082] The environment type of the current sound pickup environment is determined according to the venue type, the surrounding infrastructure information, and the current sound pickup time.

[0083] In a possible implementation, the second acquisition module is configured to:

[0084] performing acoustic feature extraction on the second speech data;

[0085] Inputting the acoustic features of the second user into a voiceprint recognition model to obtain an acoustic feature template of the second user;

[0086] comparing the acoustic feature template of the second user with the acoustic feature templates stored in the database;

[0087] In response to the degree of matching between the acoustic feature template of the second user and the target acoustic feature template in the database being greater than a third threshold, the speech detection parameters corresponding to the target acoustic feature template are used as the speech detection parameters of the speech detection model matching the second user.

[0088] On the other hand, an intelligent voice device is provided, comprising a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the above-mentioned voice activity detection method.

[0089] On the other hand, a computer-readable storage medium is provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the above-mentioned voice activity detection method.

[0090] On the other hand, a computer program product or computer program is provided, which includes computer program code, which is stored in a computer-readable storage medium. The processor of the intelligent voice device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the intelligent voice device performs the above-mentioned voice activity detection method.

[0091] The embodiment of the present application can intelligently adjust the voice detection parameters of the voice detection model based on the speaking habits of the speaking object, thereby reducing the probability of the device interrupting the user's speech. In detail, after collecting the voice data of a certain user, the scheme will first obtain the user's acoustic feature template and conversation intention; if the user's conversation intention indicates that a voice interruption event has occurred, the voice detection parameters of the voice detection model are adjusted based on the current voice detection state, and the user's acoustic feature template and adjusted voice detection parameters are stored in the database. In this way, in the subsequent human-computer dialogue process, if the voice data of a certain user (which may be the same as or different from the above-mentioned user) is collected, the device will use the database to confirm the voice detection parameters of the voice detection model that matches the user; and then based on the voice detection parameters of the voice detection model that matches the user, the subsequent voice data of the user will be subjected to voice activity detection through the voice detection model.

[0092] In other words, by identifying the user's intention, the embodiment of the present application can promptly detect whether the voice stream generated on the device side interrupts the user's speech, and then reduce the probability of voice interruption events by adjusting the voice detection parameters. This method significantly improves the quality of human-computer dialogue; in addition, the embodiment of the present application distinguishes different users through acoustic features, thereby improving the accuracy of identifying different users.

[0093] In summary, the embodiments of the present application reduce the probability of the device interrupting the user's speech, improve the quality of conversation during the human-computer conversation process, and achieve good human-computer conversation effects, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0095] Figure 1 Schematic diagram of an implementation environment involved in a voice activity detection method provided in an embodiment of the present application;

[0096] Figure 2 This is a schematic diagram of the overall execution process of a voice activity detection method provided in an embodiment of the present application;

[0097] Figure 3 This is a flow chart of a voice activity detection method provided in an embodiment of the present application;

[0098] Figure 4 is a structural diagram of a voice activity detection device provided in an embodiment of the present application;

[0099] Figure 5 This is a schematic diagram of the structure of an intelligent voice device provided in an embodiment of the present application;

[0100] Figure 6 This is a structural diagram of another intelligent voice device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0101] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0102] In this application, the terms "first," "second," and the like are used to distinguish identical or similar items having substantially the same role and function. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms.

[0103] These terms are simply used to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the scope of various examples. Both the first element and the second element can be elements, and in some cases, can be separate and different elements.

[0104] Here, at least one refers to one or more than one. For example, at least one element can be one element, two elements, three elements, or any other integer greater than or equal to one. And multiple refers to two or more than two. For example, multiple elements can be two elements, three elements, or any other integer greater than or equal to two.

[0105] The term "and / or" used in this document indicates that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0106] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0107] The following is an introduction to the key terms or abbreviations involved in the embodiments of the present application.

[0108] Voice interruption: In the embodiment of the present application, voice interruption refers to the user's speech being interrupted by the intelligent voice device, rather than the non-intelligent voice device's speech being interrupted by the user.

[0109] Figure 1 Schematic diagram of an implementation environment involved in a voice activity detection method provided in an embodiment of the present application.

[0110] In one possible implementation, see Figure 1 The voice activity detection method provided in the embodiment of the present application is applied to an intelligent voice device with machine learning capabilities.

[0111] For example, the intelligent voice device can be a tablet computer, a smart phone, a smart speaker, a smart outbound call, a smart elevator, or other intelligent voice products, which is not limited in this application.

[0112] In an embodiment of the present application, the intelligent voice device can adjust the relevant parameters (also called voice detection parameters or VAD parameters) of the voice detection model (also called VAD model) according to the speaking habits of the speaker. For example, during a human-computer conversation, if the user expresses a conversation intention such as "Don't interrupt me" or "Don't interrupt me", the intelligent voice device will adjust the voice detection parameters during voice interaction with the user to reduce the probability of interrupting the user's speech.

[0113] In another possible implementation, voice detection parameters typically include two parameters: a front-end silence detection timeout and a back-end pause time. The front-end silence detection timeout, measured in milliseconds, typically defaults to 8000 milliseconds. If VAD is enabled but no user speech is detected, voice pickup will terminate after a certain time. The back-end detection timeout, measured in milliseconds, typically defaults to 500 milliseconds. If a user pauses for more than a certain time, the user is deemed to have finished speaking, and voice pickup will terminate.

[0114] The following is based on Figure 2 The present invention provides a method for detecting voice activity, which supports intelligently adjusting the voice detection parameters of a voice detection model according to the speaking habits of the speaker.

[0115] 1. Collect the user's voice data, perform noise reduction on the collected voice data, extract acoustic features from the noise-reduced voice data, and input the extracted acoustic features into the trained voiceprint recognition model to obtain the user's acoustic feature template (also called acoustic feature value).

[0116] It should be noted that, for the sake of distinction, the user here is also referred to as the first user.

[0117] 2. Perform speech recognition and intent recognition on the noise-reduced speech data in sequence to obtain the user's conversation intention.

[0118] For example, speech recognition can be performed based on an automatic speech recognition (ASR) module, and intent recognition can be performed based on a natural language understanding (NLU) module, which is not limited in this application.

[0119] 3. If the user's conversation intention is such as "don't interrupt me" or "don't interrupt me", adjust the VAD parameters and record the user's acoustic feature template and VAD parameters in the database.

[0120] In the embodiment of the present application, adjusting the VAD parameters refers to adjusting the VAD parameters according to the VAD state (also called the speech detection state).

[0121] In one possible implementation, the speech detection state includes a speech start detection state (vad begin) and a speech end detection state (vad end). For example, if the signal energy of the speech data starts to exceed a certain threshold (the corresponding parameter is the front-end silence detection timeout), the state enters the vad begin state; if the signal energy of the speech data is lower than a certain threshold and has exceeded a certain time (the corresponding parameter is the back-end pause time), the state enters the vad end state.

[0122] In another possible implementation, if vad begin is the first value (the vad begin state has not yet been entered, indicating that the user has not spoken), the parameter value of the front-end silence detection timeout is increased; if vad end is the second value (indicating that the intelligent voice device determines that the user has finished speaking, but the actual user has not finished speaking), the parameter value of the back-end pause time is increased.

[0123] Exemplarily, the first value is 0 and the second value is 1, which is not limited in this application.

[0124] The first point that needs to be explained is that the front-end silence detection timeout and back-end pause time settings have a maximum adjustable value and a single adjustment increment value.

[0125] The second point that needs to be explained is that a table is maintained in the database to record the correspondence between the acoustic feature templates of different users and the corresponding VAD parameters; in other words, one acoustic feature template corresponds to a set of VAD parameters.

[0126] After recording the correspondence between different acoustic feature templates and corresponding VAD parameters in the database, voice activity detection can be performed according to the following steps 4-5 during the human-computer dialogue process.

[0127] 4. Collect the user's voice data, perform noise reduction on the collected voice data, obtain the user's acoustic feature template, and match the user's acoustic feature template in the above database; if there is a matching result in the database with a matching degree higher than a certain threshold, the voiceprint recognition is determined to be successful.

[0128] It should be noted that, for the sake of distinction, the user here is also referred to as the second user.

[0129] 5. If the voiceprint recognition is successful, the set of VAD parameters corresponding to the matching result recorded in the database is used; otherwise, the default VAD parameters are used.

[0130] After acquiring the user's voice, the embodiment of the present application can promptly detect whether the voice stream generated by the device side interrupts the user's speech by identifying the user's intention, and then reduce the probability of voice interruption events by adjusting the voice detection parameters. This method significantly improves the quality of human-computer dialogue. In addition, the embodiment of the present application distinguishes different users through acoustic features, thereby improving the accuracy of distinguishing different users. In summary, the embodiment of the present application reduces the probability of the device interrupting the user's speech, improves the dialogue quality during the human-computer dialogue process, and thus improves the user experience.

[0131] The above briefly introduces the voice activity detection solution of the present application. The voice activity detection solution provided in the embodiments of the present application will be described in detail through the following implementation methods.

[0132] Figure 3 This is a flow chart of a voice activity detection method provided by an embodiment of the present application. The execution subject of this method is an intelligent voice device. Figure 3 The method flow includes:

[0133] 301. Based on the collected first voice data of the first user, the intelligent voice device obtains the acoustic feature template of the first user and the conversation intention of the first user.

[0134] In this article, the first voice data refers to the voice data from the first user collected by the microphone of the intelligent voice device. In addition, the above-mentioned acoustic feature template is also referred to as the acoustic feature value in this article. Here, the first user refers to any user.

[0135] For example, after collecting the voice data, the first step is to enter the noise elimination stage, that is, to perform noise reduction processing on the voice data to filter out background noise and meet the user's needs in different environments. Among them, the voice data mentioned in steps 301-305 are all voice data that have undergone noise reduction processing.

[0136] In one possible implementation, obtaining the acoustic feature template of the first user based on the collected first voice data of the first user includes but is not limited to the following methods:

[0137] 3011. Extract acoustic features from the first speech data.

[0138] During the feature extraction phase, characteristic parameters that characterize the user's specific organ structure or behavioral habits are extracted from the speech data to serve as acoustic features. These characteristic parameters are relatively stable for the same user, unchanged over time or in different environments, and consistent across different utterances of the same user, making them difficult to imitate and highly resistant to noise.

[0139] In addition, in order to ensure the accuracy of subsequent voiceprint recognition, user age characteristics, user gender characteristics and user emotional characteristics can also be extracted and fused with the extracted acoustic features to obtain fused acoustic features, and the fused acoustic features are input into the voiceprint recognition model.

[0140] 3012. Perform a user age analysis on the first voice data to obtain an age characteristic of the first user; perform a user gender analysis on the first voice data to obtain a gender characteristic of the first user; and perform a user emotion analysis on the first voice data to obtain an emotion characteristic of the first user.

[0141] In one possible implementation, the user age of the first voice data can be analyzed based on an age classification model to obtain the above-mentioned age characteristics; the user gender of the first voice data can be analyzed based on a gender classification model to obtain the above-mentioned gender characteristics; and the user emotion of the first voice data can be analyzed based on an emotion classification model to obtain the above-mentioned emotion characteristics.

[0142] In another possible implementation, taking the gender classification model as an example, the training process of the gender classification model is illustrated below. The training process of other models is similar and will not be repeated here.

[0143] A sample set for training a gender classification model is obtained, the sample set including male voice data and female voice data; the sample set is input into a deep learning model to obtain a predicted classification result output by the deep learning model for the sample set; based on a loss function, it is determined whether the labeled classification result of the sample set is consistent with the predicted classification result; in response to the inconsistency between the labeled classification result and the predicted classification result, the network parameter value of the deep learning model is iteratively updated until the labeled classification result is consistent with the predicted classification result, thereby obtaining a gender classification model.

[0144] Exemplarily, based on the gender classification model, the user gender analysis is performed on the first voice data to obtain the above-mentioned gender features, including: inputting the first voice data into the gender classification model for feature extraction, and using the output result of the penultimate layer of the gender classification model as the above-mentioned gender features.

[0145] 3013. Perform feature fusion on the extracted acoustic features, age features, gender features, and emotion features to obtain the acoustic features of the first user.

[0146] Illustratively, the above-mentioned feature fusion may be feature splicing, which is not limited in this application.

[0147] 3014. Input the acoustic features of the first user into the voiceprint recognition model to obtain the acoustic feature template of the first user.

[0148] In an embodiment of the present application, the extracted acoustic features are used to learn a voiceprint recognition model to generate a voiceprint feature template specific to the first user and store it in a database.

[0149] In another possible implementation, obtaining the conversation intention of the first user based on the collected first voice data of the first user includes but is not limited to the following methods:

[0150] 3015. Determine the environment type of the current sound pickup environment, and perform speech recognition on the first voice data based on a speech recognition model that matches the environment type to obtain a speech recognition result.

[0151] For example, the above-mentioned environment types include but are not limited to quiet environments and noisy environments, which are not limited in this application. The speech recognition model is obtained by retraining the pre-trained model based on a sample set, and the sample set includes speech data collected in the corresponding environment.

[0152] In one possible implementation, determining the environmental type of the current sound pickup environment includes, but is not limited to, the following methods: determining the type of venue to which the current sound pickup environment belongs, surrounding infrastructure information, and the current sound pickup time; determining the environmental type of the current sound pickup environment based on the above-mentioned venue type, surrounding infrastructure information, and current sound pickup time. This application does not limit this.

[0153] In another possible implementation, performing speech recognition on the first speech data based on a speech recognition model that matches the environment type includes but is not limited to the following methods:

[0154] In response to the current sound pickup environment being a quiet environment, performing speech recognition on the first speech data based on a first speech recognition model that matches the quiet environment; wherein the first speech recognition model is obtained by retraining a pre-trained model based on a first sample set, the first sample set including speech data collected in the quiet environment;

[0155] In response to the current sound pickup environment being a noisy environment, speech recognition is performed on the first speech data based on a second speech recognition model that matches the noisy environment; the second speech recognition model is obtained by retraining the pre-model based on a second sample set, and the second sample set includes speech data collected in a noisy environment.

[0156] 3016. Perform intent recognition on the speech recognition result to obtain the conversation intention of the first user.

[0157] For example, intent recognition is performed based on the NLU module. Furthermore, before performing intent recognition, intent categories must be predefined. For example, intent categories can be defined based on specific tasks or scenarios. For example, intents can include categories such as voice interruption, ordering takeout, booking a hotel, booking tickets, booking movie tickets, or booking air tickets, though this application does not limit this.

[0158] 302. In response to the first user's dialogue intention indicating a voice interruption event, the intelligent voice device adjusts the voice detection parameters of the voice detection model based on the current voice detection state; wherein the voice interruption event refers to the user's speech being interrupted; the voice detection state includes a voice start point detection state and a voice end point detection state.

[0159] In the embodiment of the present application, based on the current speech detection state, the speech detection parameters of the speech detection model are adjusted, including but not limited to the following methods:

[0160] 3021. In response to not entering the voice start point detection state, that is, not entering the vad begin state, indicating that the user has not spoken, the value of the first voice detection parameter is increased; wherein the first voice detection parameter is used to indicate the front-end silence detection timeout period.

[0161] 3022. In response to the current voice endpoint detection state, that is, the device determines that the user has finished speaking, but the actual user has not finished speaking, the value of the second voice detection parameter is increased; wherein the second voice detection parameter is used to indicate the backend pause time.

[0162] In one possible implementation, the current voice detection status may be determined as follows:

[0163] Acquiring signal energy of first voice data;

[0164] In response to the acquired signal energy beginning to be greater than a first threshold (corresponding to the first voice detection parameter), determining that the voice start point detection state has been entered, that is, entering the vad begin state;

[0165] In response to the acquired signal energy being less than the second threshold and the duration exceeding the preset duration (corresponding to the second voice detection parameter), it is determined that the voice endpoint detection state has been entered, that is, the vad end state has been entered.

[0166] The first point to be explained is that the first threshold can be the same as or greater than the second threshold, and this application does not limit this. In addition, the signal energy can be the root mean square energy of the voice data, which represents the average energy of the voice signal waveform in a short period of time.

[0167] The second point that needs to be explained is that in addition to adjusting the speech detection parameters of the speech detection model based on the current speech detection status, the embodiment of the present application will also adjust the speech detection parameters of the speech detection model in combination with the target constraints; wherein the above-mentioned target constraints include: the maximum adjustable range, the adjustment step size of a single adjustment, so as to avoid adjusting the speech detection parameters too much, thereby affecting the speech data collection.

[0168] 303. The intelligent voice device stores the acoustic feature template of the first user and the adjusted voice detection parameters in a database.

[0169] In an embodiment of the present application, the acoustic feature template is used as the key, and the adjusted speech detection parameters (the speech detection parameters adjusted by the speech detection model matching the first user) are used as the value. The above correspondence is stored in the database through a table, and this application does not limit this.

[0170] After recording the correspondence between different acoustic feature templates and corresponding voice detection parameters in the database, voice activity detection can be performed according to the following steps 304-305 during the human-computer dialogue process.

[0171] 304. The intelligent voice device confirms the voice detection parameters of the voice detection model that matches the second user based on the collected second voice data of the second user and the database.

[0172] In this article, the second voice data refers to the voice data from the second user collected by the microphone of the intelligent voice device. The first user and the second user can be the same user or different users, which is not limited in this application.

[0173] In one possible implementation, based on the collected second voice data of the second user and the database, determining the voice detection parameters of the voice detection model that matches the second user includes but is not limited to the following methods:

[0174] 3041. Extract acoustic features from the second voice data; input the acoustic features of the second user into a voiceprint recognition model to obtain an acoustic feature template of the second user.

[0175] This step is similar to the above step 301 and will not be described again here.

[0176] 3042. Compare the acoustic feature template of the second user with the acoustic feature template stored in the database.

[0177] Exemplarily, the above comparison is to perform similarity distance measurement, such as calculating the cosine distance between the acoustic feature template of the second user and each acoustic feature template in the above database, and setting a distance threshold, that is, by selecting an appropriate distance measurement as the threshold value, acoustic feature matching is performed.

[0178] 3043. In response to the matching degree between the acoustic feature template of the second user and the target acoustic feature template in the above database being greater than a third threshold, the speech detection parameters corresponding to the target acoustic feature template are used as the speech detection parameters of the speech detection model matching the second user.

[0179] As described above, the third threshold may be a distance threshold, which is not limited in this application. The target acoustic feature template may be the first acoustic feature template or an acoustic feature template from another user other than the first user, which is not limited in this application.

[0180] 3044. In response to the fact that the matching degree between the acoustic feature template of the second user and all the acoustic feature templates in the above-mentioned database is less than the third threshold, the speech detection parameters with default values ​​are used as the speech detection parameters of the speech detection model matching the second user.

[0181] 305. The intelligent voice device performs voice activity detection on subsequent voice data of the second user using the voice detection model based on the voice detection parameters of the voice detection model matched with the second user.

[0182] The embodiments of the present application can intelligently adjust the voice detection parameters of the voice detection model based on the speaking habits of the speaker, thereby reducing the probability of the device interrupting the user's speech. In other words, by identifying the user's intention, the embodiments of the present application can promptly detect whether the voice stream generated by the device side interrupts the user's speech, and then reduce the probability of voice interruption events by adjusting the voice detection parameters. This method significantly improves the quality of human-computer dialogue. In addition, the embodiments of the present application distinguish different users through acoustic features, thereby improving the accuracy of distinguishing different users.

[0183] In summary, the embodiments of the present application accurately reduce the probability of the device interrupting the user's speech, improve the quality of conversation during the human-computer conversation process, and achieve good human-computer conversation effects, thereby improving the user experience.

[0184] Figure 4 Schematic diagram of a voice activity detection device provided in an embodiment of the present application. Figure 4 , the device comprises:

[0185] A first acquisition module 401 is configured to acquire an acoustic feature template of the first user and a conversation intention of the first user based on the collected first voice data of the first user;

[0186] The parameter adjustment module 402 is configured to adjust the speech detection parameters of the speech detection model based on the current speech detection state in response to the first user's conversation intention indicating the occurrence of a speech interruption event; wherein the speech interruption event refers to the user's speech being interrupted; and the speech detection state includes a speech start point detection state and a speech end point detection state;

[0187] The storage module 403 is configured to store the acoustic feature template of the first user and the adjusted speech detection parameters in a database;

[0188] A second acquisition module 404 is configured to determine the speech detection parameters of the speech detection model that matches the second user based on the collected second speech data of the second user and the database;

[0189] The voice detection module 405 is configured to perform voice activity detection on subsequent voice data of the second user using the voice detection model based on the voice detection parameters of the voice detection model matched with the second user.

[0190] The embodiment of the present application can intelligently adjust the voice detection parameters of the voice detection model based on the speaking habits of the speaking object, thereby reducing the probability of the device interrupting the user's speech. In detail, after collecting the voice data of a certain user, the scheme will first obtain the user's acoustic feature template and conversation intention; if the user's conversation intention indicates that a voice interruption event has occurred, the voice detection parameters of the voice detection model are adjusted based on the current voice detection state, and the user's acoustic feature template and adjusted voice detection parameters are stored in the database. In this way, in the subsequent human-computer dialogue process, if the voice data of a certain user (which may be the same as or different from the above-mentioned user) is collected, the device will use the database to confirm the voice detection parameters of the voice detection model that matches the user; and then based on the voice detection parameters of the voice detection model that matches the user, the subsequent voice data of the user will be subjected to voice activity detection through the voice detection model.

[0191] In other words, by identifying the user's intention, the embodiment of the present application can promptly detect whether the voice stream generated on the device side interrupts the user's speech, and then reduce the probability of voice interruption events by adjusting the voice detection parameters. This method significantly improves the quality of human-computer dialogue; in addition, the embodiment of the present application distinguishes different users through acoustic features, thereby improving the accuracy of identifying different users.

[0192] In summary, the embodiments of the present application reduce the probability of the device interrupting the user's speech, improve the quality of conversation during the human-computer conversation process, and achieve good human-computer conversation effects, thereby improving the user experience.

[0193] In a possible implementation, the parameter adjustment module 402 is configured to:

[0194] In response to not currently entering the voice start point detection state, increasing the value of a first voice detection parameter; wherein the first voice detection parameter is used to indicate a front-end silence detection timeout period;

[0195] In response to currently entering the voice endpoint detection state, the value of the second voice detection parameter is increased; wherein the second voice detection parameter is used to indicate the backend pause time.

[0196] In a possible implementation, the parameter adjustment module 402 is configured to:

[0197] Acquiring signal energy of the first voice data;

[0198] In response to the acquired signal energy beginning to be greater than a first threshold, determining that the speech start point detection state has been entered;

[0199] In response to the acquired signal energy being less than the second threshold and the duration exceeding the preset duration, it is determined that the voice endpoint detection state has been entered.

[0200] In a possible implementation, the parameter adjustment module 402 is configured to:

[0201] Adjusting speech detection parameters of the speech detection model based on a current speech detection state and target constraints;

[0202] The target constraint conditions include: a maximum adjustable range and an adjustment step size for a single adjustment.

[0203] In a possible implementation, the first obtaining module 401 is configured to:

[0204] performing acoustic feature extraction on the first speech data;

[0205] performing a user age analysis on the first voice data to obtain an age feature of the first user;

[0206] Performing user gender analysis on the first voice data to obtain gender characteristics of the first user;

[0207] Performing user emotion analysis on the first voice data to obtain an emotional characteristic of the first user;

[0208] performing feature fusion on the extracted acoustic features, the age features, the gender features, and the emotion features to obtain acoustic features of the first user;

[0209] The acoustic features of the first user are input into a voiceprint recognition model to obtain an acoustic feature template of the first user.

[0210] In a possible implementation, the first obtaining module 401 is configured to:

[0211] Determining an environment type of a current sound pickup environment, and performing speech recognition on the first voice data based on a speech recognition model matching the environment type to obtain a speech recognition result;

[0212] Performing intent recognition on the speech recognition result to obtain the conversation intention of the first user;

[0213] Wherein, the environment type includes a quiet environment and a noisy environment;

[0214] The speech recognition model is obtained by retraining a pre-trained model based on a sample set, where the sample set includes speech data collected in a corresponding environment.

[0215] In a possible implementation, the first obtaining module 401 is configured to:

[0216] In response to the current sound pickup environment being a quiet environment, performing speech recognition on the first voice data based on a first speech recognition model matching the quiet environment;

[0217] In response to the current sound pickup environment being a noisy environment, performing speech recognition on the first speech data based on a second speech recognition model matching the noisy environment;

[0218] The first speech recognition model is obtained by retraining a pre-trained model based on a first sample set, wherein the first sample set includes speech data collected in the quiet environment;

[0219] The second speech recognition model is obtained by retraining the pre-model based on a second sample set, where the second sample set includes speech data collected in the noisy environment.

[0220] In a possible implementation, the first obtaining module 401 is configured to:

[0221] Determine the venue type, surrounding infrastructure information, and current pickup time of the current pickup environment;

[0222] The environment type of the current sound pickup environment is determined according to the venue type, the surrounding infrastructure information, and the current sound pickup time.

[0223] In a possible implementation, the second obtaining module 404 is configured to:

[0224] performing acoustic feature extraction on the second speech data;

[0225] Inputting the acoustic features of the second user into a voiceprint recognition model to obtain an acoustic feature template of the second user;

[0226] comparing the acoustic feature template of the second user with the acoustic feature templates stored in the database;

[0227] In response to the degree of matching between the acoustic feature template of the second user and the target acoustic feature template in the database being greater than a third threshold, the speech detection parameters corresponding to the target acoustic feature template are used as the speech detection parameters of the speech detection model matching the second user.

[0228] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0229] It should be noted that the voice activity detection device provided in the above embodiment is merely an example of the division of the aforementioned functional modules when performing voice activity detection. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to perform all or part of the functions described above. Furthermore, the voice activity detection device provided in the above embodiment and the voice activity detection method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0230] Figure 5 Schematic diagram of the structure of an intelligent voice device 500 provided in an embodiment of the present application.

[0231] Typically, the intelligent voice device 500 includes a processor 501 and a memory 502 .

[0232] The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In one possible implementation, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In one possible implementation, the processor 501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0233] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In one possible implementation, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one program code, which is executed by the processor 501 to implement the voice activity detection method provided in the embodiments of the present application.

[0234] In one possible implementation, the intelligent voice device 500 may optionally include a peripheral device interface 503 and at least one peripheral device. The processor 501, memory 502, and peripheral device interface 503 may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 503 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 504, a display screen 505, a camera assembly 506, an audio circuit 507, a positioning assembly 508, and a power supply 509.

[0235] The peripheral device interface 503 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 501 and the memory 502. In one possible implementation, the processor 501, the memory 502, and the peripheral device interface 503 are integrated on the same chip or circuit board. In another possible implementation, any one or two of the processor 501, the memory 502, and the peripheral device interface 503 can be implemented on separate chips or circuit boards. This embodiment of the present application is not limited to this.

[0236] The RF circuit 504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 504 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 504 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 504 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In one possible implementation, the RF circuit 504 may also include circuitry related to Near Field Communication (NFC), which is not limited in this application.

[0237] The display screen 505 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 505 is a touch screen display, the display screen 505 also has the ability to collect touch signals on the surface or above the surface of the display screen 505. The touch signal can be input to the processor 501 as a control signal for processing. At this time, the display screen 505 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In one possible implementation, the display screen 505 can be one, set on the front panel of the intelligent voice device 500; in another possible implementation, the display screen 505 can be at least two, respectively set on different surfaces of the intelligent voice device 500 or in a folding design; in another possible implementation, the display screen 505 can be a flexible display screen, set on the curved surface or folding surface of the intelligent voice device 500. Even the display screen 505 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 505 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0238] The camera assembly 506 is used to capture images or videos. Optionally, the camera assembly 506 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In one possible implementation, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In one possible implementation, the camera assembly 506 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0239] The audio circuit 507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 501 for processing, or input them into the radio frequency circuit 504 to realize voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the intelligent voice device 500. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 501 or the radio frequency circuit 504 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In one possible implementation, the audio circuit 507 may also include a headphone jack.

[0240] The positioning component 508 is used to locate the current geographic location of the intelligent voice device 500 to implement navigation or LBS (Location Based Service). The positioning component 508 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Greninja system, or the European Union's Galileo system.

[0241] The power supply 509 is used to power the various components of the intelligent voice device 500. The power supply 509 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When the power supply 509 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0242] In one possible implementation, the intelligent voice device 500 further includes one or more sensors 510. The one or more sensors 510 include, but are not limited to, an acceleration sensor 511, a gyroscope sensor 512, a pressure sensor 513, a fingerprint sensor 514, an optical sensor 515, and a proximity sensor 516.

[0243] The accelerometer 511 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the intelligent voice device 500. For example, the accelerometer 511 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 501 can control the display screen 505 to display the user interface in a horizontal view or a vertical view based on the gravity acceleration signal collected by the accelerometer 511. The accelerometer 511 can also be used to collect game or user motion data.

[0244] The gyroscope sensor 512 can detect the body orientation and rotation angle of the intelligent voice device 500. The gyroscope sensor 512 can work with the accelerometer 511 to collect the user's 3D movements of the intelligent voice device 500. Based on the data collected by the gyroscope sensor 512, the processor 501 can implement the following functions: motion sensing (such as changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0245] The pressure sensor 513 can be set on the side frame of the intelligent voice device 500 and / or the lower layer of the display screen 505. When the pressure sensor 513 is set on the side frame of the intelligent voice device 500, it can detect the user's grip signal of the intelligent voice device 500, and the processor 501 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 513. When the pressure sensor 513 is set on the lower layer of the display screen 505, the processor 501 controls the operability controls on the UI interface based on the user's pressure operation on the display screen 505. The operability controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0246] The fingerprint sensor 514 is used to collect the user's fingerprint, and the processor 501 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 514, or the fingerprint sensor 514 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 501 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 514 can be set on the front, back, or side of the intelligent voice device 500. When a physical button or manufacturer logo is set on the intelligent voice device 500, the fingerprint sensor 514 can be integrated with the physical button or manufacturer logo.

[0247] The optical sensor 515 is used to collect ambient light intensity. In one possible implementation, the processor 501 can control the display brightness of the display screen 505 based on the ambient light intensity collected by the optical sensor 515. Specifically, when the ambient light intensity is high, the display brightness of the display screen 505 is increased; when the ambient light intensity is low, the display brightness of the display screen 505 is decreased. In another possible implementation, the processor 501 can also dynamically adjust the shooting parameters of the camera assembly 506 based on the ambient light intensity collected by the optical sensor 515.

[0248] The proximity sensor 516, also known as a distance sensor, is typically disposed on the front panel of the smart voice device 500. The proximity sensor 516 is used to detect the distance between the user and the front of the smart voice device 500. In one possible implementation, when the proximity sensor 516 detects that the distance between the user and the front of the smart voice device 500 is gradually decreasing, the processor 501 controls the display screen 505 to switch from the screen-on state to the screen-off state; and when the proximity sensor 516 detects that the distance between the user and the front of the smart voice device 500 is gradually increasing, the processor 501 controls the display screen 505 to switch from the screen-off state to the screen-on state.

[0249] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the intelligent voice device 500, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0250] Figure 6 This is a structural diagram of another intelligent voice device provided in an embodiment of the present application.

[0251] The intelligent voice device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 601 and one or more memories 602, wherein the memory 602 stores at least one program code, and the at least one program code is loaded and executed by the processor 601 to implement the voice activity detection method provided by the above-mentioned various method embodiments. Of course, the intelligent voice device 600 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The intelligent voice device 600 may also include other components for implementing device functions, which will not be described in detail here.

[0252] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including program code. The program code can be executed by a processor in an intelligent voice device to perform the voice activity detection method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0253] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer program code, which is stored in a computer-readable storage medium. The processor of the intelligent voice device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the intelligent voice device performs the above-mentioned voice activity detection method.

[0254] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0255] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for detecting voice activity, characterized in that: The method comprises: Acquiring an acoustic feature template of the first user and a conversation intention of the first user based on the collected first voice data of the first user; In response to the first user's conversation intention indicating a speech interruption event, adjusting speech detection parameters of a speech detection model based on a current speech detection state; the speech interruption event refers to the user's speech being interrupted; the speech detection state includes a speech start point detection state and a speech end point detection state; Storing the acoustic feature template of the first user and the adjusted speech detection parameters in a database; Determining, based on the collected second voice data of the second user and the database, voice detection parameters of a voice detection model that matches the second user; Based on the speech detection parameters of the speech detection model matched with the second user, speech activity detection is performed on subsequent speech data of the second user through the speech detection model.

2. The method according to claim 1, characterized in that The adjusting the speech detection parameters of the speech detection model based on the current speech detection state includes: In response to not currently entering the voice start point detection state, increasing the value of a first voice detection parameter; wherein the first voice detection parameter is used to indicate a front-end silence detection timeout period; In response to currently entering the voice endpoint detection state, the value of the second voice detection parameter is increased; wherein the second voice detection parameter is used to indicate the backend pause time.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Acquiring signal energy of the first voice data; In response to the acquired signal energy beginning to be greater than a first threshold, determining that the speech start point detection state has been entered; In response to the acquired signal energy being less than the second threshold and the duration exceeding the preset duration, it is determined that the voice endpoint detection state has been entered.

4. The method according to claim 1, wherein The adjusting the speech detection parameters of the speech detection model based on the current speech detection state includes: Adjusting speech detection parameters of the speech detection model based on a current speech detection state and target constraints; The target constraint conditions include: a maximum adjustable range and an adjustment step size for a single adjustment.

5. The method according to claim 1, wherein The acquiring, based on the collected first voice data of the first user, an acoustic feature template of the first user includes: performing acoustic feature extraction on the first speech data; performing a user age analysis on the first voice data to obtain an age feature of the first user; Performing user gender analysis on the first voice data to obtain gender characteristics of the first user; Performing user emotion analysis on the first voice data to obtain an emotional characteristic of the first user; performing feature fusion on the extracted acoustic features, the age features, the gender features, and the emotion features to obtain acoustic features of the first user; The acoustic features of the first user are input into a voiceprint recognition model to obtain an acoustic feature template of the first user.

6. The method according to claim 1, characterized in that The acquiring the conversation intention of the first user based on the collected first voice data of the first user includes: Determining an environment type of a current sound pickup environment, and performing speech recognition on the first voice data based on a speech recognition model matching the environment type to obtain a speech recognition result; Performing intent recognition on the speech recognition result to obtain the conversation intention of the first user; Wherein, the environment type includes a quiet environment and a noisy environment; The speech recognition model is obtained by retraining a pre-trained model based on a sample set, where the sample set includes speech data collected in a corresponding environment.

7. The method according to claim 6, characterized in that The performing speech recognition on the first speech data based on a speech recognition model matching the environment type includes: In response to the current sound pickup environment being a quiet environment, performing speech recognition on the first voice data based on a first speech recognition model matching the quiet environment; In response to the current sound pickup environment being a noisy environment, performing speech recognition on the first speech data based on a second speech recognition model matching the noisy environment; The first speech recognition model is obtained by retraining a pre-trained model based on a first sample set, wherein the first sample set includes speech data collected in the quiet environment; The second speech recognition model is obtained by retraining the pre-model based on a second sample set, where the second sample set includes speech data collected in the noisy environment.

8. The method according to claim 6, characterized in that Determining the environment type of the current sound pickup environment includes: Determine the venue type, surrounding infrastructure information, and current pickup time of the current pickup environment; The environment type of the current sound pickup environment is determined according to the venue type, the surrounding infrastructure information, and the current sound pickup time.

9. The method according to claim 1, characterized in that The determining, based on the collected second voice data of the second user and the database, voice detection parameters of a voice detection model that matches the second user includes: performing acoustic feature extraction on the second speech data; Inputting the acoustic features of the second user into a voiceprint recognition model to obtain an acoustic feature template of the second user; comparing the acoustic feature template of the second user with the acoustic feature templates stored in the database; In response to the degree of matching between the acoustic feature template of the second user and the target acoustic feature template in the database being greater than a third threshold, the speech detection parameters corresponding to the target acoustic feature template are used as the speech detection parameters of the speech detection model matching the second user.

10. A voice activity detection device, characterized in that: The device comprises: A first acquisition module is configured to acquire an acoustic feature template of the first user and a conversation intention of the first user based on the collected first voice data of the first user; a parameter adjustment module configured to adjust speech detection parameters of a speech detection model based on a current speech detection state in response to a speech interruption event indicated by the first user's conversation intention, wherein the speech interruption event refers to the user's speech being interrupted; and the speech detection state includes a speech start point detection state and a speech end point detection state; a storage module configured to store the acoustic feature template of the first user and the adjusted speech detection parameters in a database; A second acquisition module is configured to determine the speech detection parameters of the speech detection model that matches the second user based on the collected second speech data of the second user and the database; The voice detection module is configured to perform voice activity detection on subsequent voice data of the second user through the voice detection model based on the voice detection parameters of the voice detection model matched with the second user.

11. An intelligent voice device, characterized in that: The device includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the voice activity detection method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the voice activity detection method according to any one of claims 1 to 9.

Images