Voice interaction, model training method, device, equipment and product

Abstract

This application discloses a voice interaction and model training method, apparatus, device, and product, belonging to the field of voice interaction technology. The method includes: extracting features from audio signals collected by a microphone to obtain audio features; inputting the audio features into a wake-up recognition model to obtain a wake-up recognition result output by the wake-up recognition model, wherein the wake-up recognition result includes wake-up and non-wake-up, and the wake-up recognition model is trained based on sample audio signals at different audio receiving distances, where the audio receiving distance is the distance from the sound source to the microphone; and activating the voice interaction function when the wake-up recognition result indicates wake-up. The solution provided by this application simplifies the wake-up process of the voice interaction function and improves the efficiency of voice interaction.

Description

Technical Field

[0001] This application relates to the field of voice interaction technology, and in particular to a voice interaction, model training method, apparatus, device and product. Background Technology

[0002] Voice interaction is a technology in which users issue commands to a terminal using natural language, and the terminal responds to those commands. In related technologies, users can activate a voice interaction assistant using a specific wake word to perform voice interaction functions. Summary of the Invention

[0003] This application provides a voice interaction method, model training method, apparatus, device, and product. The technical solution is as follows:

[0004] On one hand, embodiments of this application provide a voice interaction method, the method comprising:

[0005] The audio signals captured by the microphone are subjected to feature extraction to obtain audio features;

[0006] The audio features are input into the wake-up recognition model to obtain the wake-up recognition result output by the wake-up recognition model. The wake-up recognition result includes wake-up and non-wake-up. The wake-up recognition model is trained based on sample audio signals at different audio receiving distances. The audio receiving distance is the distance from the sound source to the microphone.

[0007] If the wake-up recognition result indicates that the voice interaction function is activated, the voice interaction function is activated.

[0008] On the other hand, embodiments of this application provide a model training method, the method comprising:

[0009] Acquire sample audio signals, wherein the sample audio signals correspond to an audio receiving distance, and the audio receiving distance is the distance from the sound source to the microphone;

[0010] Feature extraction is performed on the sample audio signal to obtain sample audio features;

[0011] The sample audio features are input into the wake-up recognition model to obtain the sample wake-up recognition result output by the wake-up recognition model, and the sample wake-up recognition result includes wake-up and non-wake-up;

[0012] The wake-up recognition model is trained based on the wake-up recognition results of the samples and the wake-up tags corresponding to the sample audio signals.

[0013] On the other hand, embodiments of this application provide a voice interaction device, the device comprising:

[0014] The feature extraction module is used to extract features from the audio signals collected by the microphone to obtain audio features;

[0015] The inference module is used to input the audio features into the wake-up recognition model and obtain the wake-up recognition result output by the wake-up recognition model. The wake-up recognition result includes wake-up and non-wake-up. The wake-up recognition model is trained based on sample audio signals at different audio receiving distances. The audio receiving distance is the distance from the sound source to the microphone.

[0016] The voice interaction module is used to activate the voice interaction function when the wake-up recognition result indicates that the voice interaction function should be activated.

[0017] On the other hand, embodiments of this application provide a model training apparatus, the apparatus comprising:

[0018] The sample acquisition module is used to acquire sample audio signals, wherein the sample audio signals correspond to an audio receiving distance, and the audio receiving distance is the distance from the sound source to the microphone;

[0019] The feature extraction module is used to extract features from the sample audio signal to obtain sample audio features;

[0020] The training module is used to input the sample audio features into the wake-up recognition model to obtain the sample wake-up recognition result output by the wake-up recognition model, the sample wake-up recognition result including wake-up and non-wake-up; and to train the wake-up recognition model based on the sample wake-up recognition result and the wake-up tag corresponding to the sample audio signal.

[0021] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory, the memory storing at least one computer instruction, the at least one computer instruction being loaded and executed by the processor to implement the voice interaction method or the model training method as described above.

[0022] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one computer instruction, which is executed by a processor to implement the voice interaction method or the model training method described above.

[0023] On the other hand, embodiments of this application provide a computer program product, the computer program product including computer instructions stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the voice interaction method or the model training method as described above.

[0024] In this embodiment, a wake-up recognition model is pre-trained based on sample audio signals at different audio reception distances. After this model is deployed to the terminal, the terminal can perform wake-up recognition based on the audio features of the audio signals collected by the microphone, and activate the voice interaction function when indicated by the wake-up recognition result. Leveraging the close-range audio recognition capability of this wake-up recognition model, users can trigger voice interaction simply by speaking close to the terminal, eliminating the need for a specific wake-up word, thus simplifying the wake-up process for the voice interaction function and improving its efficiency. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a voice interaction method provided in an exemplary embodiment of this application is shown;

[0026] Figure 2 This illustration shows an implementation diagram of a wake-up recognition model deployment method provided in an exemplary embodiment of this application;

[0027] Figure 3 This is a schematic diagram illustrating an implementation of the audio feature extraction process in an exemplary embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the structure of a wake-up recognition model provided in an exemplary embodiment of this application;

[0029] Figure 5 A flowchart illustrating a model training method provided in an exemplary embodiment of this application is shown;

[0030] Figure 6 This is a schematic diagram illustrating a breath signal and a short-duration speech signal, as shown in an exemplary embodiment of this application.

[0031] Figure 7 This is a flowchart illustrating a sample audio signal acquisition process in an exemplary embodiment of this application;

[0032] Figure 8 A structural block diagram of a voice interaction device provided in another exemplary embodiment of this application is shown;

[0033] Figure 9 A structural block diagram of a model training apparatus provided in another exemplary embodiment of this application is shown;

[0034] Figure 10 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0036] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0037] In related technologies, when a user needs to interact with the terminal's voice assistant, they first need to say a specific wake-up word (such as "Hi, Xiaobu") to activate the voice interaction function. This specific wake-up word can be a system-preset wake-up word or a user-defined wake-up word. The terminal's low-power microphone continuously collects audio signals and detects whether the collected audio signals contain the specific wake-up word. If they do, the voice interaction function is activated, and voice interaction is performed based on the continuously collected audio signals.

[0038] However, before each use of the voice interaction function, it is necessary to wake it up with a specific wake word. On the one hand, voice interaction is inefficient and does not conform to the natural voice communication habits between people; on the other hand, the standard of pronunciation will directly affect the wake-up success rate.

[0039] In this embodiment, a wake-up recognition model is pre-trained based on sample audio signals at different audio reception distances. This wake-up recognition model can identify the audio reception distance of an audio signal, thereby determining whether the audio signal is a near-range audio signal used to wake up the voice interaction function. After deploying this wake-up recognition model to the terminal, the terminal can perform wake-up recognition based on the audio characteristics of the audio signal collected by the microphone, and wake up the voice interaction function when the wake-up recognition result indicates that it is ready to be activated. After applying this wake-up recognition model, when a user needs to interact with the terminal via voice, they only need to speak close to the terminal, without needing a specific wake-up word, simplifying the wake-up process of the voice interaction function and improving the efficiency of voice interaction.

[0040] The voice interaction method provided in this application can be applied to terminals with voice interaction capabilities. These terminals can be smartphones, tablets, wearable devices, smart TVs, smart speakers, smart home gateways, personal computers, etc., and this application does not limit the application to these devices. Furthermore, the terminal is equipped with a microphone for acquiring external audio signals. This microphone can be a low-power microphone to achieve continuous audio acquisition with extremely low power consumption.

[0041] For ease of explanation, the following embodiments use the application of voice interaction methods to terminals as an example.

[0042] Please refer to Figure 1This document illustrates a flowchart of a voice interaction method provided in an exemplary embodiment of this application. This embodiment uses the method applied to a terminal as an example for illustration, and the method may include the following steps:

[0043] Step 101: Extract features from the audio signal collected by the microphone to obtain audio features.

[0044] When the voice interaction function is enabled, the terminal continuously collects audio signals through the microphone and extracts features from the collected audio signals in real time to obtain audio features. Specifically, the terminal extracts features according to the feature requirements of the wake-up recognition model, ensuring that the extracted audio features meet the model input requirements of the wake-up recognition model.

[0045] Step 102: Input the audio features into the wake-up recognition model to obtain the wake-up recognition result output by the wake-up recognition model. The wake-up recognition result includes wake-up and non-wake-up. The wake-up recognition model is trained based on sample audio signals at different audio receiving distances. The audio receiving distance is the distance from the sound source to the microphone.

[0046] In this embodiment, the wake-up recognition model is trained based on sample audio signals at different audio reception distances. The sample labels for these audio signals are "wake-up" or "not wake-up." Therefore, the wake-up recognition model can output a wake-up recognition result indicating whether the user is awake or not based on audio features, using the audio reception distance as a basis.

[0047] Optionally, the output of the wake-up recognition model can be represented by 0 and 1, where 0 represents wake-up and 1 represents non-wake-up.

[0048] Specifically, the wake-up recognition result output by the wake-up recognition model indicates that the recognized audio signal is a near-field audio signal, meaning that the audio reception distance of this audio signal is within the audio reception distance range that triggers the voice interaction function (the distance between the sound source and the microphone when the voice interaction function is activated). Accordingly, this audio signal is likely collected when the user needs to interact with the terminal via voice and speaks close to the terminal's microphone.

[0049] If the wake-up recognition model outputs a wake-up recognition result indicating no wake-up, it means that the recognized audio signal is not a near-field audio signal, meaning that the audio reception distance of the audio signal is outside the audio reception distance range that triggers the voice interaction function. Accordingly, the audio signal is most likely not collected when the user speaks close to the terminal microphone, but rather is a far-field audio signal.

[0050] In one possible design, the wake-up recognition model is built on a sensorhub platform. The sensorhub platform employs AON (Always On) low-power technology, allowing the model to remain in standby mode for extended periods. The sensorhub platform can be positioned between the application processor (AP) and various MEMS (Micro-Electro-Mechanical Systems).

[0051] Indicative, such as Figure 2 As shown, the sensorhub platform 22 is positioned between the microphone 21 and the AP 23. The microphone 21 receives external audio signals during low-power periods and writes them to a buffer. The sensorhub platform 22 acquires the buffered audio signals through the sensor driver 221 and performs feature extraction, thereby using the wake-up recognition model 222 to perform wake-up recognition based on the extracted features. The wake-up recognition model 222 runs independently on the sensorhub platform 22, utilizing the computing power of the NPU 24 in the sensorhub-side inference framework.

[0052] Step 103: When the wake-up recognition result indicates that the voice interaction function is activated.

[0053] When the wake-up recognition result indicates that the terminal is ready to wake up, the terminal activates the voice interaction function. After activating the voice interaction function, the terminal continuously collects audio through the microphone and performs speech recognition (e.g., using ASR technology) on the collected audio, thereby enabling voice interaction with the user based on the speech recognition results.

[0054] In some embodiments, when the wake-up function is implemented via a sensorhub platform, the voice interaction after the voice interaction function is activated can be performed by the AP. (Illustrative example, such as...) Figure 2 As shown, when a wake-up is detected, the sensorhub platform 22 reports a wake-up command to the AP 23, which then activates the voice interaction function and performs subsequent voice interactions. Compared to AP-side invocation, deploying the model on the sensorhub platform achieves ultra-low power consumption and always-on functionality.

[0055] Since the solution provided in this application allows users to issue voice commands directly without having to say a specific wake-up word first, the audio signal used for wake-up recognition will also be used as the input for voice recognition in order to ensure the integrity of voice command recognition.

[0056] Optionally, if the wake-up recognition result of the audio signal at time T is "wake-up", the terminal takes time Tt as the starting point for speech recognition, that is, it performs speech recognition on the audio collected after time Tt. Here, t is usually set to a small duration, such as 0.5 seconds, 1 second, or 1.5 seconds.

[0057] It should be noted that when using the solution provided in the embodiments of this application, the terminal can still activate the voice interaction function when a specific wake-up word is recognized. The recognition of the specific wake-up word and the model recognition can be implemented using two different processes.

[0058] In summary, in this embodiment, a wake-up recognition model is pre-trained based on sample audio signals at different audio reception distances. After deploying this model to the terminal, the terminal can perform wake-up recognition based on the audio features of the audio signals collected by the microphone, and activate the voice interaction function when indicated by the wake-up recognition result. Leveraging the close-range audio recognition capability of this wake-up recognition model, users can trigger voice interaction simply by speaking close to the terminal, eliminating the need for a specific wake-up word, thus simplifying the wake-up process for the voice interaction function and improving its efficiency.

[0059] The process of audio feature extraction

[0060] Because audio is continuous, the terminal can extract the acquired audio signal using a sliding window and perform feature extraction on the extracted audio signal. In one possible implementation, step 101 above may include the following sub-steps:

[0061] Step 101A: Extract audio signals from the microphone using a sliding window to obtain audio signal segments.

[0062] In one possible implementation, the terminal slides across the acquired audio signal through a fixed-length sliding window with a fixed sliding step size to obtain audio signal segments within the sliding window at different sliding positions. The length of the audio signal segment is less than or equal to the length of the sliding window.

[0063] In an illustrative example, the sliding window is 1 second long, meaning it's used to extract a 1-second audio signal segment from a continuous audio signal. Furthermore, when the sliding window's step size is 0.2 seconds (this is just for illustrative purposes; the sliding can be done in frames), there is an 0.8-second audio signal overlap between adjacent sliding window positions.

[0064] In one possible scenario, during the initial audio signal acquisition phase, since the length of the acquired audio signal may be shorter than the length of the sliding window, to ensure the normal extraction of subsequent audio features (extracting audio features of sufficient length) and improve recognition efficiency (potentially acquiring the wake-up audio signal during the initial acquisition phase), the terminal amplifies the signal within the sliding window if the audio signal length is less than the sliding window length. The length of the amplified signal is then the length of the sliding window. The terminal subsequently identifies the amplified signal as an audio signal segment.

[0065] When the length of the audio signal reaches the length of the sliding window, the signal located within the sliding window is extracted as an audio signal segment.

[0066] Regarding the signal amplification method, in one possible implementation, the terminal performs signal interpolation processing on the audio signal based on the length of the audio signal and the length of the sliding window to obtain the amplified signal.

[0067] During signal interpolation, the terminal selects a specified audio signal frame from the audio signal and determines the interpolated audio signal frame using an interpolation algorithm, thereby inserting the interpolated audio signal frame between the specified audio signal frames. This interpolation algorithm can be a linear interpolation algorithm or a nonlinear interpolation algorithm.

[0068] Of course, in other possible implementations, the terminal may use other time-domain signal stretching techniques to amplify the audio signal, and this application does not limit this.

[0069] To illustrate, when the duration of the signal in the sliding window is 0.5 seconds, the terminal can amplify the signal in the sliding window to 1 second.

[0070] In one possible implementation, the terminal also needs to preprocess the acquired audio signal to reduce the impact of audio loudness on subsequent recognition. The method of audio signal preprocessing varies depending on the microphone setup.

[0071] In some embodiments, when the microphone is a dual microphone (e.g., two microphones respectively located at the top and bottom of the terminal), the terminal performs differential processing on the dual audio signals acquired by the dual microphones to obtain an audio signal. Subsequently, segment extraction is performed on the differentially processed audio signal.

[0072] In other embodiments, when the microphone is a single-channel microphone, the terminal normalizes the single-channel audio signal acquired by the single microphone to obtain an audio signal. Subsequently, segment extraction is performed on the normalized audio signal. Normalizing the single-channel audio signal refers to normalizing the loudness, meaning the loudness of the normalized audio signal is within the loudness range of 0-1.

[0073] Step 101B: Extract features from the audio signal segment to obtain the audio features of the audio signal segment.

[0074] For each extracted audio signal segment, the terminal performs feature extraction on the audio signal segment to obtain its audio features. In one possible implementation, the audio feature extraction process may include the following sub-steps:

[0075] Sub-step 1: Perform a short-time Fourier transform on the audio signal segment to obtain the signal time spectrum.

[0076] For an audio signal segment containing time-domain amplitude information, the terminal first performs a Short-Time Fourier Transform (STFT) on the audio signal segment, decomposing the signal into time and frequency to obtain the signal's time spectrum. The audio signal segment before the STFT transformation contains the relationship between time and time-domain signal amplitude, while the signal's time spectrum obtained from the STFT transformation contains the relationship between time and frequency, as well as the signal's amplitude at specific times and frequencies.

[0077] Indicative, such as Figure 3 As shown, the terminal performs STFT transformation on the audio signal segment 31 to obtain the signal time spectrum 32.

[0078] Sub-step 2 involves filtering the time spectrum of the signal using a triangular filter to obtain the audio characteristics of the audio signal segment.

[0079] After completing the STFT transformation, the terminal further performs triangular filtering on the time spectrum of the signal using a triangular filter, and determines the triangular filtering result as the audio characteristics of the audio signal segment.

[0080] Optionally, the triangular filter used for triangular filtering is a Mel filter bank, which typically consists of multiple triangular filters. This bank is used to transform the frequency axis from a linear scale to a Mel scale, and it better simulates the nonlinear perception of frequency by the human ear. Correspondingly, the triangular filtering result obtained by performing triangular filtering on the signal's time-frequency spectrum is called the Mel spectrum.

[0081] The process of filtering the signal's time spectrum involves multiplying the signal's time spectrum by a triangular filter.

[0082] Indicative, such as Figure 3 As shown, the terminal further performs triangular filtering on the signal time spectrum 32 through triangular filter 33 to obtain audio features 34 in the form of Mel spectrum.

[0083] Of course, in addition to extracting audio features in the above manner, the terminal can also extract features of other dimensions of audio signal segments, and this application embodiment does not limit this.

[0084] In some embodiments, after obtaining the audio features in the form of time-spectrum using the above method, a wake-up recognition model based on a convolutional neural network can be used for wake-up recognition. This wake-up recognition model based on a convolutional neural network can consist of convolutional layers, pooling layers, and fully connected layers.

[0085] Indicative, such as Figure 4 As shown, the audio features 41 in the form of the time spectrum of the signal are input into the wake-up recognition model 42, and then undergo convolution, pooling, convolution and fully connected processing in sequence, and finally the wake-up recognition result 43 is output through the output layer.

[0086] Of course, the wake-up recognition model can also use other network structures, such as recurrent neural networks (RNN), long short-term memory networks (LSTM), etc., and the embodiments of this application do not limit this.

[0087] Training process of wake-up recognition model

[0088] The wake-up recognition model deployed on the terminal side can be pre-trained by a computer device based on sample audio signals. Each sample audio signal corresponds to a specific audio receiving distance and a corresponding wake-up tag, which indicates whether the voice interaction function is activated based on the sample audio signal. The computer device can be a server, workstation, personal computer, or other device with model training capabilities.

[0089] Please refer to Figure 5 This document illustrates a flowchart of a model training method provided in an exemplary embodiment of this application. This embodiment uses the method applied to a computer device as an example for illustration, and the method may include the following steps:

[0090] Step 501: Obtain sample audio signals. The sample audio signals correspond to audio receiving distances, which are the distances from the sound source to the microphone.

[0091] In one possible implementation, the computer device first acquires sample audio signals at various audio reception distances. These include sample audio signals located within the audio reception distance range that triggers the voice interaction function, as well as sample audio signals located outside the audio reception distance range that triggers the voice interaction function.

[0092] For example, when the audio receiving distance range for triggering the voice interaction function is 0-10cm, the samples used to train the wake-up recognition model include audio signals with an audio receiving distance less than or equal to 10cm and audio signals with an audio receiving distance greater than 10cm.

[0093] In some embodiments, the sample audio signal is obtained by manual recording (e.g., by a real person reading arbitrary text), and / or by data augmentation of the manually recorded audio.

[0094] Furthermore, in order to improve the robustness of the wake-up recognition model, the sample audio signals cover natural people of different ages, genders, speaking habits, timbre, and languages.

[0095] Step 502: Extract features from the sample audio signal to obtain sample audio features.

[0096] In this process, the computer equipment extracts features according to the feature requirements of the wake-up recognition model, so that the extracted sample audio features meet the model input requirements of the wake-up recognition model.

[0097] Step 503: Input the sample audio features into the wake-up recognition model to obtain the sample wake-up recognition result output by the wake-up recognition model. The sample wake-up recognition result includes wake-up and non-wake-up.

[0098] During training, the computer device inputs sample audio features into the wake-up recognition model to be trained, obtaining the sample wake-up recognition result corresponding to the sample audio features. Specifically, if the sample wake-up recognition result indicates wake-up, it means the model predicts that the voice interaction function needs to be activated upon receiving the sample audio signal corresponding to that sample audio feature; if the sample wake-up recognition result indicates no wake-up, it means the model predicts that the voice interaction function does not need to be activated upon receiving the sample audio signal corresponding to that sample audio feature.

[0099] Optionally, the output of the wake-up recognition model can be represented by 0 and 1, where 0 represents wake-up and 1 represents non-wake-up.

[0100] Step 504: Train the wake-up recognition model based on the sample wake-up recognition results and the wake-up tags corresponding to the sample audio signals.

[0101] In this embodiment, each sample audio signal is assigned a wake-up tag. This wake-up tag is a truth value tag indicating whether or not the received sample audio signal needs to be woken up; that is, the wake-up tag of the sample audio signal indicates wake-up or no wake-up. Optionally, the wake-up tag indicating wake-up can be represented by 1, and the wake-up tag indicating no wake-up can be represented by 0.

[0102] In some embodiments, the wake-up tag corresponding to the sample audio signal is related to the audio acquisition distance corresponding to the sample audio signal and the range of audio acquisition distances that trigger the wake-up voice interaction function.

[0103] In one possible implementation, for the same sample audio signal, the computer device determines the model's prediction loss for that sample audio signal based on the difference between the sample wake-up recognition result and the wake-up tag corresponding to that sample audio signal. Then, based on the prediction loss of different sample audio signals, the wake-up recognition model is trained until the training termination condition is met.

[0104] The predicted loss can be cross-entropy loss; the backpropagation algorithm of gradient descent can be used when training the wake-up recognition model; the training termination condition can include the upper limit of the number of iterations, the loss convergence condition, etc., but the embodiments of this application do not limit this.

[0105] In summary, in this embodiment, a wake-up recognition model is pre-trained based on sample audio signals at different audio reception distances. After deploying this model to the terminal, the terminal can perform wake-up recognition based on the audio features of the audio signals collected by the microphone, and activate the voice interaction function when indicated by the wake-up recognition result. Leveraging the close-range audio recognition capability of this wake-up recognition model, users can trigger voice interaction simply by speaking close to the terminal, eliminating the need for a specific wake-up word, thus simplifying the wake-up process for the voice interaction function and improving its efficiency.

[0106] Wake-up tag settings

[0107] Regarding the method of setting the wake-up tag for the sample audio signal, in one possible implementation, the computer device sets the wake-up tag for the sample audio signal based on the relationship between the audio acquisition distance corresponding to the sample audio signal and the distance threshold.

[0108] The distance threshold is the maximum audio acquisition distance of the audio signal used to wake up the voice interaction function.

[0109] If the audio receiving distance corresponding to the sample audio signal is less than or equal to the distance threshold, the computer device will determine the wake-up tag corresponding to the sample audio signal as wake-up; if the audio receiving distance corresponding to the sample audio signal is greater than the distance threshold, the computer device will determine the wake-up tag corresponding to the sample audio signal as not wake-up.

[0110] In some embodiments, the terminal can provide multiple distance thresholds for the user to choose from. For example, the user can set the distance threshold for waking up the voice interaction function to 5cm or 10cm.

[0111] Accordingly, to adapt to different user needs, computer devices can pre-set different wake-up labels for the same sample audio signal based on different distance thresholds (for example, for a sample audio signal with an audio acquisition distance of 7cm, the wake-up label for the sample audio signal is "wake-up" when the distance threshold is 10cm, and "not wake-up" when the distance threshold is 5cm), and train a wake-up recognition model suitable for different distance thresholds. Subsequently, the terminal can deploy a wake-up recognition model suitable for the distance threshold set by the user.

[0112] Sample audio signal form

[0113] To improve the accuracy of the timing for activating the voice interaction function, in one possible implementation, the sample audio signal includes a breath signal and a short-duration speech signal. The breath signal is the breath sound initially captured when the microphone is brought close to the terminal, while the short-duration speech signal is a short-duration speech sound emitted after the microphone is brought close (typically the beginning of a voice command).

[0114] By using a sample setup of "breath signal + short-time speech signal", the trained wake-up recognition model can identify audio signals that are close to the terminal microphone (breath signal) and truly have the intention to interact with the voice (short-time speech signal).

[0115] Indicatively, during the process of bringing the mouth close to the microphone and issuing a voice command, the dual microphones capture, for example, the sounds... Figure 6 The audio signals shown are upper and lower channels. The audio signal located in the first region 61 is a breath signal, and the audio signal located in the second region 62 is a short-duration speech signal.

[0116] In one possible implementation, for manually recorded audio signals, the positions of breath signals and short-term speech signals in the manually recorded audio signals are marked by manual annotation and / or feature recognition. Based on the marked positions of breath signals and short-term speech signals, sample audio signals as training samples are extracted from the manually recorded audio signals.

[0117] The process of extracting audio features from samples

[0118] Similar to the audio feature extraction process in the application process, in one possible implementation, step 502 above may include the following sub-steps:

[0119] Step 502A: Extract the sample audio signal from the sample audio signal using a sliding window to obtain a sample audio signal segment.

[0120] In one possible implementation, the computer device slides across the sample audio signal through a fixed-length sliding window with a fixed sliding step size to obtain sample audio signal segments within the sliding window at different sliding positions. The length of the sample audio signal segment is less than or equal to the length of the sliding window.

[0121] In an illustrative example, the sliding window is 1 second long, meaning it is used to extract a 1-second sample audio signal segment from a continuous audio signal. Furthermore, when the sliding window has a step size of 0.2 seconds (this is for illustrative purposes only; the sliding can be done in frames), there is an 0.8-second overlap in audio signal between adjacent sliding window positions.

[0122] Because the duration of the sample audio signal may be shorter than the duration of the sliding window (related to factors such as the speaker's speaking speed, speaking habits, and content), to ensure the normal extraction of subsequent audio features (extracting audio features of sufficient length) and improve recognition efficiency, when the length of the sample audio signal is less than the length of the sliding window, the computer device amplifies the signal within the sliding window to obtain the amplified signal. The length of the amplified signal is the length of the sliding window. The terminal then identifies the amplified signal as an audio signal segment.

[0123] When the length of the sample audio signal reaches the length of the sliding window, the signal located within the sliding window is extracted as the sample audio signal segment.

[0124] Regarding the signal amplification method, in one possible implementation, the computer device performs signal interpolation processing on the sample audio signal based on the length of the sample audio signal and the length of the sliding window to obtain the sample amplified signal.

[0125] In signal interpolation, the computer selects a specified audio signal frame from the sample audio signal and determines the interpolated audio signal frame using an interpolation algorithm, thereby inserting the interpolated audio signal frame between the specified audio signal frames. This interpolation algorithm can be a linear interpolation algorithm or a nonlinear interpolation algorithm.

[0126] Of course, in other possible implementations, the computer device may employ other time-domain signal stretching techniques to amplify the audio signal, and this application does not limit this.

[0127] To illustrate, when the duration of the sample audio signal is 0.5 seconds and the duration of the sliding window is 1 second, the computer device can amplify the sample audio signal to 1 second through signal amplification.

[0128] In one possible implementation, the computer device also needs to preprocess the sample audio signal to reduce the impact of audio loudness on subsequent recognition. In some embodiments, when the sample audio signal is a dual-channel audio signal acquired by dual microphones, the computer device performs differential processing on the dual-channel sample audio signals. Subsequently, segment extraction is performed on the sample audio signal obtained from the differential processing.

[0129] In other embodiments, when the sample audio signal is a single-channel audio signal acquired by a single microphone, the computer device normalizes the single-channel audio signal. Subsequently, segment extraction is performed on the normalized sample audio signal. Normalizing the single-channel audio signal refers to normalizing the loudness, meaning the loudness of the normalized sample audio signal is within the loudness range of 0-1.

[0130] Step 502B: Extract features from the sample audio signal segment to obtain the sample audio features of the sample audio signal segment.

[0131] For each extracted sample audio signal segment, the terminal performs feature extraction to obtain the sample audio features of the sample audio signal segment. It should be noted that if the duration of the sample audio signal is greater than the duration of the sliding window, at least two sample audio signal segments can be extracted from that sample audio signal. Furthermore, sample audio signal segments extracted from the same sample audio signal correspond to the same audio acquisition distance.

[0132] In one possible implementation, during the sample audio feature extraction process, the computer device performs a short-time Fourier transform on the sample audio signal segment to obtain the time spectrum of the sample signal, and then filters the time spectrum of the signal sample using a triangular filter to obtain the sample audio features of the sample audio signal segment. The specific extraction process of the sample audio features can refer to the audio feature extraction process in the application stage, and will not be elaborated upon in this embodiment.

[0133] The process of acquiring sample audio signals

[0134] Since the quality of model training is closely related to the quality and quantity of training samples, data augmentation techniques can be used to acquire sample audio signals to improve model training quality. The process of acquiring sample audio signals is as follows: Figure 7 As shown.

[0135] Step 701: Acquire the first sample audio signal, which is the user audio signal collected at a preset audio receiving distance.

[0136] In one possible implementation, a computer device acquires a user audio signal (manually reads arbitrary text) collected at a preset audio receiving distance, and marks the positions of breath signals and short-term speech signals in the user audio signal, thereby extracting a first sample audio signal from the user audio signal based on the marked positions.

[0137] Step 702: Perform data enhancement processing on the first sample audio signal to obtain the second sample audio signal.

[0138] Since it is impossible to collect audio from all groups of people manually, in order to increase the number of training samples, computer equipment performs data augmentation on the first sample audio signal to obtain the second sample audio signal.

[0139] Data augmentation methods can include at least one of the following:

[0140] Data augmentation method 1: Perform frequency band adjustment processing on the first sample audio signal to obtain the second sample audio signal, wherein the frequency band adjustment processing includes at least one of increasing the signal frequency band and decreasing the signal frequency band.

[0141] In order to simulate the intonation of different groups of people, the computer device changes the frequency band of the first sample audio signal to obtain a second sample audio signal with a different intonation than the first sample audio signal, thereby enriching the intonation covered by the training samples.

[0142] In one possible implementation, the computer device can increase and / or decrease the signal frequency band of the first sample audio signal by a unit frequency band adjustment amount (e.g., 10Hz) (overall increase and / or decrease). The increase and decrease in the signal frequency band are integer multiples of the unit frequency band adjustment amount. Accordingly, the computer device can obtain at least one second sample audio signal with a pitch higher than the first sample audio signal, and / or at least one second sample audio signal with a pitch lower than the first sample audio signal.

[0143] It should be noted that when performing frequency band adjustment, it is necessary to ensure that the signal frequency band of the second sample audio signal is within the human vocal frequency band (e.g., 85Hz-1100Hz).

[0144] Data augmentation method 2: Perform speech inversion processing on the first sample audio signal to obtain the second sample audio signal, wherein the speech inversion processing is used to reverse the relationship between the beginning and end of the speech signal.

[0145] Typically, the first sample audio signal recorded manually contains semantics. Computer equipment can reverse the speech of the first sample audio signal, that is, reverse the signals before and after the first sample audio signal to construct a training sample (i.e., the second sample audio signal) that does not contain semantics but contains speech features.

[0146] In one possible implementation, since the normally acquired audio signals always start with the breath signal and then the speech signal, in order to ensure the rationality of the generated second sample audio signal, the computer device maintains the order of the breath signals in the first sample audio signal and only performs speech reversal processing on the short-term speech signals in the first sample audio signal.

[0147] It should be noted that when both of the above data augmentation methods are used simultaneously, the computer equipment can perform speech inversion processing on the audio signal obtained after frequency band adjustment to obtain a second sample audio signal.

[0148] Of course, in addition to the data augmentation methods mentioned above, computer devices can also use other methods to augment training sample data, which will not be elaborated here.

[0149] Step 703: Determine the first sample audio signal and the second sample audio signal as sample audio signals.

[0150] Furthermore, the computer equipment uses the manually recorded first sample audio signal and the generated second sample audio signal to determine the sample audio signal. The second sample audio signal generated based on the first sample audio signal corresponds to the same audio acquisition distance.

[0151] In this embodiment, in addition to obtaining the first sample audio signal through manual recording, the computer device generates a second sample audio signal based on the first sample audio signal by adjusting the frequency band and / or reversing the voice, thereby increasing the number of training samples, which helps to reduce the cost of collecting training samples and improve the training quality of the model.

[0152] Noise addition processing of sample audio signals

[0153] In real-world voice wake-up scenarios, environmental noise is often present, which can affect the robustness of the wake-up recognition model. Therefore, to improve the robustness of the wake-up recognition model, computer devices can add noise to the sample audio signals (first sample audio signal and / or second sample audio signal), thereby using the noisy sample audio signals for model training.

[0154] In one possible implementation, after generating the second sample audio signal through data enhancement, the computer device adds noise to the first sample audio signal and the second sample audio signal to obtain the noisy first sample audio signal and the noisy second sample audio signal, thereby determining the noisy first sample audio signal and the noisy second sample audio signal as the sample audio signal.

[0155] Regarding the specific method of noise addition processing, the computer equipment uses noise superposition technology to add preset noise to the first and second sample audio signals (energy superposition).

[0156] To improve the diversity of superimposed noise, the computer device can perform noise scaling on the preset noise and then add the scaled noise to the first and second sample audio signals.

[0157] In one possible implementation, the computer device determines the noise power spectrum of a preset noise, and the signal power spectra of a first sample audio signal and a second sample audio signal. The computer device scales the noise power spectrum and superimposes the scaled noise power spectrum onto the signal power spectrum to obtain the noisy first sample audio signal and the noisy second sample audio signal.

[0158] Optionally, the computer device can be set with various preset noises to simulate different noise environments. For example, white noise, traffic noise, building noise, etc., but this application embodiment does not limit this.

[0159] Furthermore, in order to simulate more noise levels, the computer device can scale the noise power spectrum according to various scaling ratios to obtain scaled noise with different powers. For example, the scaling ratio can include 0.7, 0.8, 0.9, 1.1, 1.2, 1.3, etc., and this embodiment does not limit it.

[0160] It should be noted that the first sample audio signal can be manually recorded under different noise environments to further improve the model's ability to combat noise.

[0161] In this embodiment, the computer device adds noise to the sample audio signal using noise superposition technology, thereby using the noisy sample audio signal to train the wake-up recognition model, improving the model's ability to resist noise and enhancing the model's robustness.

[0162] Adaptive fine-tuning of the model

[0163] In the above embodiments, the wake-up recognition models are all trained based on large datasets and have good universality. However, since some users may have special pronunciation or voice interaction habits, models trained based on large datasets may not perform well on these users.

[0164] To further improve the quality of the wake-up recognition model, the terminal or computer device can fine-tune the wake-up recognition model based on whether the voice interaction function on the terminal side is correctly woken up.

[0165] In one possible implementation, during the application process, if the voice interaction is successful, the terminal generates a positive sample based on the audio signal and the wake-up recognition result. Conversely, if a voice interaction cancellation operation is received, the terminal generates a negative sample based on the audio signal and the wake-up recognition result. Subsequently, the terminal fine-tunes the wake-up recognition model based on the positive and negative samples.

[0166] Among them, successful voice interaction means that after waking up the voice interaction function based on the wake-up recognition result output by the model, a voice interaction command is received, which indicates that the wake-up recognition result of the model output is correct; voice interaction cancellation means that after waking up the voice interaction function based on the wake-up recognition result output by the model, a voice interaction cancellation command is received. This voice interaction cancellation command can be a voice command (such as the user immediately expressing that no interaction is needed after waking up the voice interaction function) or a touch command.

[0167] Furthermore, the generated positive samples contain audio signals that correctly trigger the wake-up voice interaction function and the wake-up label "wake up", while the generated negative samples contain audio signals that incorrectly trigger the wake-up voice interaction function and the wake-up label "not wake up".

[0168] Optionally, the wake-up recognition model can be fine-tuned based on positive and negative samples, either by the terminal (which has strong computing power) or by the terminal reporting to the server, which will then execute the model and send the fine-tuned model parameters back to the terminal.

[0169] In one possible design, the computer device receives positive and negative samples reported by the terminal and fine-tunes the wake-up recognition model based on the positive and negative samples.

[0170] In this embodiment, the terminal generates corresponding positive and negative samples based on the success of voice interaction after the voice interaction function is activated and the voice interaction is canceled. The positive and negative samples are then used to fine-tune the model parameters of the wake-up recognition model, making the fine-tuned model more suitable for the current user's voice wake-up habits, which helps to improve the accuracy of voice wake-up.

[0171] Please refer to Figure 8 This illustration shows a structural block diagram of a voice interaction device provided in an exemplary embodiment of this application. The device includes:

[0172] The feature extraction module 801 is used to extract features from the audio signal collected by the microphone to obtain audio features;

[0173] The inference module 802 is used to input the audio features into the wake-up recognition model and obtain the wake-up recognition result output by the wake-up recognition model. The wake-up recognition result includes wake-up and non-wake-up. The wake-up recognition model is trained based on sample audio signals at different audio receiving distances. The audio receiving distance is the distance from the sound source to the microphone.

[0174] The voice interaction module 803 is used to activate the voice interaction function when the wake-up recognition result indicates that the voice interaction function should be activated.

[0175] Optionally, the feature extraction module 801 is used for:

[0176] The audio signal collected by the microphone is extracted using a sliding window to obtain an audio signal segment;

[0177] Feature extraction is performed on the audio signal segment to obtain the audio features of the audio signal segment.

[0178] Optionally, the feature extraction module 801 is used for:

[0179] If the length of the audio signal does not reach the length of the sliding window, the signal located within the sliding window is amplified to obtain an amplified signal, the length of which is the length of the sliding window; the amplified signal is then identified as the audio signal segment.

[0180] If the length of the audio signal reaches the length of the sliding window, the signal located within the sliding window is extracted as the audio signal segment.

[0181] Optionally, the feature extraction module 801 is used for:

[0182] Based on the length of the audio signal and the length of the sliding window, signal interpolation processing is performed on the audio signal to obtain the amplified signal.

[0183] Optionally, the feature extraction module 801 is used for:

[0184] Perform a short-time Fourier transform on the audio signal segment to obtain the signal time spectrum;

[0185] The audio characteristics of the audio signal segment are obtained by filtering the time spectrum of the signal using a triangular filter.

[0186] Optionally, the apparatus further includes a preprocessing module for:

[0187] When the microphone is a dual microphone, the dual audio signals acquired by the dual microphones are differentially processed to obtain the audio signal;

[0188] When the microphone is a single microphone, the single audio signal acquired by the single microphone is normalized to obtain the audio signal.

[0189] Optionally, the device further includes:

[0190] The local sample generation module is used to generate positive samples based on the audio signal and the wake-up recognition result when the voice interaction is successful; and to generate negative samples based on the audio signal and the wake-up recognition result when a voice interaction cancellation operation is received.

[0191] The fine-tuning module is used to fine-tune the wake-up recognition model based on the positive and negative samples.

[0192] Please refer to Figure 9 This illustration shows a structural block diagram of a model training apparatus provided in an exemplary embodiment of this application. The apparatus includes:

[0193] The sample acquisition module 901 is used to acquire sample audio signals, wherein the sample audio signals correspond to an audio receiving distance, and the audio receiving distance is the distance from the sound source to the microphone;

[0194] Feature extraction module 902 is used to extract features from the sample audio signal to obtain sample audio features;

[0195] The training module 903 is used to input the sample audio features into the wake-up recognition model to obtain the sample wake-up recognition result output by the wake-up recognition model, the sample wake-up recognition result including wake-up and non-wake-up; and to train the wake-up recognition model based on the sample wake-up recognition result and the wake-up tag corresponding to the sample audio signal.

[0196] Optionally, the feature extraction module 902 is used for:

[0197] The sample audio signal is extracted by using a sliding window to obtain sample audio signal segments;

[0198] Feature extraction is performed on the sample audio signal segment to obtain the sample audio features of the sample audio signal segment.

[0199] Optionally, the feature extraction module 902 is used for:

[0200] If the length of the sample audio signal is greater than or equal to the length of the sliding window, the signal located within the sliding window is extracted as the sample audio signal segment.

[0201] If the length of the sample audio signal is less than the length of the sliding window, the signal located within the sliding window is amplified to obtain a sample amplified signal, the length of which is the length of the sliding window; the sample amplified signal is then identified as the sample audio signal segment.

[0202] Optionally, the feature extraction module 902 is used for:

[0203] Based on the length of the sample audio signal and the length of the sliding window, signal interpolation processing is performed on the sample audio signal to obtain the sample amplified signal.

[0204] Optionally, the feature extraction module 902 is used for:

[0205] Perform a short-time Fourier transform on the sample audio signal segment to obtain the time spectrum of the sample signal;

[0206] The sample audio features of the sample audio signal segment are obtained by filtering the time spectrum of the sample signal using a triangular filter.

[0207] Optionally, the sample audio signal includes breath signal and short speech signal.

[0208] Optionally, the sample acquisition module 901 is used for:

[0209] Acquire a first sample audio signal, wherein the first sample audio signal is a user audio signal collected at a preset audio receiving distance;

[0210] The first sample audio signal is subjected to data augmentation processing to obtain the second sample audio signal;

[0211] The first sample audio signal and the second sample audio signal are identified as the sample audio signal.

[0212] Optionally, the sample acquisition module 901 is used for:

[0213] The first sample audio signal is subjected to frequency band adjustment processing to obtain the second sample audio signal, wherein the frequency band adjustment processing includes at least one of increasing the signal frequency band and decreasing the signal frequency band;

[0214] The first sample audio signal is subjected to speech inversion processing to obtain the second sample audio signal, wherein the speech inversion processing is used to reverse the sequential relationship of the speech signal.

[0215] Optionally, the device further includes a noise-adding module for:

[0216] The first sample audio signal and the second sample audio signal are subjected to noise addition processing to obtain the noisy first sample audio signal and the second sample audio signal;

[0217] The sample acquisition module 901 is used for:

[0218] The first sample audio signal and the second sample audio signal after adding noise are determined as the sample audio signal.

[0219] Optionally, the noise-adding module is used for:

[0220] Determine the noise power spectrum of the preset noise, and the signal power spectrum of the first sample audio signal and the second sample audio signal;

[0221] The noise power spectrum is scaled and then superimposed on the signal power spectrum to obtain the first sample audio signal and the second sample audio signal after noise addition.

[0222] Optionally, the device further includes:

[0223] The tag determination module is used to determine the wake-up tag corresponding to the sample audio signal as wake-up when the audio receiving distance corresponding to the sample audio signal is less than or equal to a distance threshold;

[0224] If the audio receiving distance corresponding to the sample audio signal is greater than the distance threshold, the wake-up tag corresponding to the sample audio signal will be determined as not to wake up.

[0225] Optionally, the device further includes a fine-tuning module for:

[0226] The receiver receives positive and negative samples reported by the terminal. The positive samples are training samples generated by the terminal based on the audio signal and wake-up recognition result when the voice interaction is successful. The negative samples are training samples generated by the terminal based on the audio signal and wake-up recognition result when the voice interaction is canceled.

[0227] The wake-up recognition model is fine-tuned based on the positive and negative samples.

[0228] In summary, in this embodiment, a wake-up recognition model is pre-trained based on sample audio signals at different audio reception distances. After deploying this model to the terminal, the terminal can perform wake-up recognition based on the audio features of the audio signals collected by the microphone, and activate the voice interaction function when indicated by the wake-up recognition result. Leveraging the close-range audio recognition capability of this wake-up recognition model, users can trigger voice interaction simply by speaking close to the terminal, eliminating the need for a specific wake-up word, thus simplifying the wake-up process for the voice interaction function and improving its efficiency.

[0229] It should be noted that the apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their implementation process can be found in the method embodiments, which will not be repeated here.

[0230] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. This computer device can be implemented as a terminal or computer device in the above embodiments. The computer device may include one or more of the following components: a processor 1010 and a memory 1020.

[0231] Optionally, the processor 1010 connects to various parts within the electronic device using various interfaces and lines. It performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1020, and by calling data stored in the memory 1020. Optionally, the processor 1010 can be implemented in at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA).

[0232] The processor 1010 can integrate one or more of the following: a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), and a baseband chip. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content displayed on the touchscreen; the NPU implements artificial intelligence (AI) functions; and the baseband chip handles wireless communication. It is understood that the baseband chip can also be implemented as a separate chip without being integrated into the processor 1010.

[0233] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1020 may include a non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the electronic device, etc.

[0234] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more (e.g., microphone, speaker, power supply component, display component, sensor component) or fewer components than shown, or combine certain components, or have different component arrangements.

[0235] This application provides a computer-readable storage medium storing at least one computer instruction, which is executed by a processor to implement voice interaction or model training methods as described in the above embodiments.

[0236] On the other hand, embodiments of this application provide a computer program product, the computer program product including computer instructions stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the voice interaction or model training method as described in the above embodiments.

[0237] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0238] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A voice interaction method, characterized in that, The method includes: The audio signals captured by the microphone are subjected to feature extraction to obtain audio features; The audio features are input into the wake-up recognition model to obtain the wake-up recognition result output by the wake-up recognition model. The wake-up recognition result includes wake-up and non-wake-up. The wake-up recognition model is trained based on sample audio signals at different audio receiving distances. The audio receiving distance is the distance from the sound source to the microphone. If the wake-up recognition result indicates that the voice interaction function is activated, the voice interaction function is activated.

2. The method according to claim 1, characterized in that, The process of extracting features from the audio signal captured by the microphone to obtain audio features includes: The audio signal collected by the microphone is extracted using a sliding window to obtain an audio signal segment; Feature extraction is performed on the audio signal segment to obtain the audio features of the audio signal segment.

3. The method according to claim 2, characterized in that, The step of extracting audio signal segments from the microphone via a sliding window includes: If the length of the audio signal does not reach the length of the sliding window, the signal located within the sliding window is amplified to obtain an amplified signal, the length of which is the length of the sliding window; the amplified signal is then identified as the audio signal segment. If the length of the audio signal reaches the length of the sliding window, the signal located within the sliding window is extracted as the audio signal segment.

4. The method according to claim 3, characterized in that, The step of amplifying the signal located within the sliding window to obtain the amplified signal includes: Based on the length of the audio signal and the length of the sliding window, signal interpolation processing is performed on the audio signal to obtain the amplified signal.

5. The method according to claim 2, characterized in that, The step of extracting features from the audio signal segment to obtain the audio features of the audio signal segment includes: Perform a short-time Fourier transform on the audio signal segment to obtain the signal time spectrum; The audio characteristics of the audio signal segment are obtained by filtering the time spectrum of the signal using a triangular filter.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When the microphone is a dual microphone, the dual audio signals acquired by the dual microphones are differentially processed to obtain the audio signal; When the microphone is a single microphone, the single audio signal acquired by the single microphone is normalized to obtain the audio signal.

7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: If the voice interaction is successful, a positive sample is generated based on the audio signal and the wake-up recognition result; Upon receiving a voice interaction cancellation operation, a negative sample is generated based on the audio signal and the wake-up recognition result; The wake-up recognition model is fine-tuned based on the positive and negative samples.

8. A model training method, characterized in that, The method includes: Acquire sample audio signals, wherein the sample audio signals correspond to an audio receiving distance, and the audio receiving distance is the distance from the sound source to the microphone; Feature extraction is performed on the sample audio signal to obtain sample audio features; The sample audio features are input into the wake-up recognition model to obtain the sample wake-up recognition result output by the wake-up recognition model, and the sample wake-up recognition result includes wake-up and non-wake-up; The wake-up recognition model is trained based on the wake-up recognition results of the samples and the wake-up tags corresponding to the sample audio signals.

9. The method according to claim 8, characterized in that, The step of extracting features from the sample audio signal to obtain sample audio features includes: The sample audio signal is extracted by using a sliding window to obtain sample audio signal segments; Feature extraction is performed on the sample audio signal segment to obtain the sample audio features of the sample audio signal segment.

10. The method according to claim 9, characterized in that, The step of extracting the sample audio signal through a sliding window to obtain sample audio signal segments includes: If the length of the sample audio signal is greater than or equal to the length of the sliding window, the signal located within the sliding window is extracted as the sample audio signal segment. If the length of the sample audio signal is less than the length of the sliding window, the signal located within the sliding window is amplified to obtain a sample amplified signal, the length of which is the length of the sliding window; the sample amplified signal is then identified as the sample audio signal segment.

11. The method according to claim 10, characterized in that, The step of amplifying the signal located within the sliding window to obtain the amplified signal includes: Based on the length of the sample audio signal and the length of the sliding window, signal interpolation processing is performed on the sample audio signal to obtain the sample amplified signal.

12. The method according to claim 9, characterized in that, The step of extracting features from the sample audio signal segment to obtain the sample audio features of the sample audio signal segment includes: Perform a short-time Fourier transform on the sample audio signal segment to obtain the time spectrum of the sample signal; The sample audio features of the sample audio signal segment are obtained by filtering the time spectrum of the sample signal using a triangular filter.

13. The method according to any one of claims 8 to 12, characterized in that, The sample audio signals include breath signals and short-duration speech signals.

14. The method according to any one of claims 8 to 12, characterized in that, The acquisition of sample audio signals includes: Acquire a first sample audio signal, wherein the first sample audio signal is a user audio signal collected at a preset audio receiving distance; The first sample audio signal is subjected to data augmentation processing to obtain the second sample audio signal; The first sample audio signal and the second sample audio signal are identified as the sample audio signal.

15. The method according to claim 14, characterized in that, The process of performing data enhancement processing on the first sample audio signal to obtain the second sample audio signal includes at least one of the following: The first sample audio signal is subjected to frequency band adjustment processing to obtain the second sample audio signal, wherein the frequency band adjustment processing includes at least one of increasing the signal frequency band and decreasing the signal frequency band; The first sample audio signal is subjected to speech inversion processing to obtain the second sample audio signal, wherein the speech inversion processing is used to reverse the sequential relationship of the speech signal.

16. The method according to claim 14, characterized in that, The method further includes: The first sample audio signal and the second sample audio signal are subjected to noise addition processing to obtain the noisy first sample audio signal and the second sample audio signal; The step of determining the first sample audio signal and the second sample audio signal as the sample audio signal includes: The first sample audio signal and the second sample audio signal after adding noise are determined as the sample audio signal.

17. The method according to claim 16, characterized in that, The step of adding noise to the first sample audio signal and the second sample audio signal to obtain the noisy first sample audio signal and the second sample audio signal includes: Determine the noise power spectrum of the preset noise, and the signal power spectrum of the first sample audio signal and the second sample audio signal; The noise power spectrum is scaled and then superimposed on the signal power spectrum to obtain the first sample audio signal and the second sample audio signal after noise addition.

18. The method according to any one of claims 8 to 12, characterized in that, The method further includes: If the audio receiving distance corresponding to the sample audio signal is less than or equal to the distance threshold, the wake-up tag corresponding to the sample audio signal is determined to be wake-up; If the audio receiving distance corresponding to the sample audio signal is greater than the distance threshold, the wake-up tag corresponding to the sample audio signal will be determined as not to wake up.

19. The method according to any one of claims 8 to 12, characterized in that, The method further includes: The receiver receives positive and negative samples reported by the terminal. The positive samples are training samples generated by the terminal based on the audio signal and wake-up recognition result when the voice interaction is successful. The negative samples are training samples generated by the terminal based on the audio signal and wake-up recognition result when the voice interaction is canceled. The wake-up recognition model is fine-tuned based on the positive and negative samples.

20. A voice interaction device, characterized in that, The device includes: The feature extraction module is used to extract features from the audio signals collected by the microphone to obtain audio features; The inference module is used to input the audio features into the wake-up recognition model and obtain the wake-up recognition result output by the wake-up recognition model. The wake-up recognition result includes wake-up and non-wake-up. The wake-up recognition model is trained based on sample audio signals at different audio receiving distances. The audio receiving distance is the distance from the sound source to the microphone. The voice interaction module is used to activate the voice interaction function when the wake-up recognition result indicates that the voice interaction function should be activated.

21. A model training device, characterized in that, The device includes: The sample acquisition module is used to acquire sample audio signals, wherein the sample audio signals correspond to an audio receiving distance, and the audio receiving distance is the distance from the sound source to the microphone; The feature extraction module is used to extract features from the sample audio signal to obtain sample audio features; The training module is used to input the sample audio features into the wake-up recognition model to obtain the sample wake-up recognition result output by the wake-up recognition model, the sample wake-up recognition result including wake-up and non-wake-up; and to train the wake-up recognition model based on the sample wake-up recognition result and the wake-up tag corresponding to the sample audio signal.

22. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer instruction, the at least one computer instruction being loaded and executed by the processor to implement the voice interaction method as described in any one of claims 1 to 7, or the model training method as described in any one of claims 8 to 19.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer instruction, which is executed by a processor to implement the voice interaction method as described in any one of claims 1 to 7, or the model training method as described in any one of claims 8 to 19.

24. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement the voice interaction method as described in any one of claims 1 to 7, or the model training method as described in any one of claims 8 to 19.

Images