A voice interaction method and system for an AI security assistant

Abstract

This application relates to the field of speech recognition technology, specifically to a voice interaction method and system for an artificial intelligence security assistant. The method includes: acquiring audio data from the voice assistant's wake-up to sleep state and dividing it into multiple time periods; dividing each time period into multiple time intervals; based on the changing trend of the dispersion of audio data in all time intervals within each time period, as well as the dispersion of the time interval between adjacent abrupt change points, the dispersion of the time interval between all adjacent peak points, and the difference between low-frequency and high-frequency energy, obtaining the step size factor adjustment coefficient for each time period; and then adjusting the preset initial step size factor for each time period to obtain denoised audio data for each time period, thereby enabling voice interaction. This application improves the denoising effect of audio data by adaptively acquiring the step size factor of audio data for each time period.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, specifically to a voice interaction method and system for an artificial intelligence security assistant. Background Technology

[0002] As voice assistants are applied and promoted, the use scenarios and fields of voice interaction are gradually increasing. The challenges faced by voice assistants in recognizing users' voices are also gradually increasing. Noise interference mixed in the user audio data received by voice assistants in different scenarios and fields affects the quality of voice recognition and leads to a decline in the quality of interaction. Therefore, existing technologies usually use various denoising algorithms to denoise the collected audio data. Among them, the Least Mean Square (LMS) algorithm has been applied in various voice assistant interaction scenarios due to its advantages such as small computational load, simple calculation and strong stability.

[0003] However, the traditional LMS algorithm uses a fixed step size for calculation. Since there are many hardware devices that can be equipped with voice assistants, the noise environment encountered when using voice assistants varies greatly. Especially for mobile devices equipped with voice assistants, users may interact with the voice assistant in multiple scenarios with different noise levels and changing patterns within a certain period of time. The noise environment is highly time-varying. When using the LMS algorithm to process the noise environment, it cannot cope with the highly time-varying noise environment. This results in incomplete noise removal in the audio after LMS processing, or the convergence time is too long, which cannot meet the response requirements of the voice assistant, leading to deviations in subsequent audio recognition. Summary of the Invention

[0004] To address the aforementioned technical problems, the purpose of this application is to provide a voice interaction method and system for an artificial intelligence security assistant, the specific technical solution of which is as follows:

[0005] In a first aspect, embodiments of this application provide a voice interaction method for an artificial intelligence security assistant, the method comprising the following steps:

[0006] Acquire audio data from the voice assistant between wake-up and sleep, and divide it into multiple time periods;

[0007] Each time period is divided into multiple time intervals; based on the variation trend of the dispersion of audio data in all time intervals within each time period, and the dispersion of the time interval between all adjacent abrupt change points in the audio data within each time period, the noise drastic change coefficient of each time period is obtained.

[0008] The peak points in the fitted curves of the audio data for each time period are obtained. Based on the dispersion of the time interval between all adjacent peak points in the audio data for each time period, and the difference between low-frequency energy and high-frequency energy in the audio data for each time period, the human voice presence coefficient for each time period is obtained. Combined with the noise variation coefficient for each time period, the step size factor adjustment coefficient for each time period is obtained. Then, the preset initial step size factor for each time period is adjusted to obtain the denoised audio data for each time period, which is then used for voice interaction.

[0009] Preferably, the method for obtaining the noise variation coefficients for each time period is as follows:

[0010] Obtain the goodness-of-fit sequence for each time period;

[0011] Calculate the noise variation coefficient for each time period: In the formula, Let be the noise variation coefficient for the i-th time period. Let be the trend strength of the goodness-of-fit sequence for the i-th time period. Let be the trend strength of the goodness-of-fit sequence in the u-th time period. Let be the variance of the time intervals between all adjacent abrupt changes in the audio data within the i-th time period; This is a preset constant.

[0012] Preferably, the process of obtaining the goodness-of-fit sequence for each time period is as follows: curve fitting is performed on the audio data of each time interval, and the goodness-of-fit of each fitting curve is calculated; the sequence formed by arranging the goodness-of-fit of all time intervals in each time period in chronological order is recorded as the goodness-of-fit sequence for each time period.

[0013] Preferably, the process of obtaining the human voice presence coefficient for each time period is as follows: calculate the variance of the time interval between all adjacent peaks in the audio data of each time period; use the audio data of each time period as the input of the Fourier transform algorithm, and output the spectral sequence of the audio data of each time period; take frequencies less than or equal to a preset frequency threshold as low frequencies, and vice versa as high frequencies; calculate the ratio of low-frequency energy to high-frequency energy of the audio data of each time period; and record the ratio of the ratio to the variance as the human voice presence coefficient for each time period.

[0014] Preferably, the step size factor adjustment coefficient for each time period is negatively correlated with the noise drastic change coefficient and the human voice presence coefficient for each time period.

[0015] Preferably, the calculation formula for adjusting the preset initial step size factor for each time period is as follows: In the formula, Let be the step size factor for the i-th time period. Let be the step size factor adjustment coefficient for the i-th time period. This is the preset initial step size factor.

[0016] Preferably, The range of values ​​is ,in, Let be the largest eigenvalue of the autocorrelation matrix of the audio data in the i-th time period; when the calculated Greater than At that time, then order equal .

[0017] Preferably, the method for obtaining the denoised audio data for each time period is as follows: the audio data for all time periods in the entire audio data are used as the input of the LMS algorithm, the step size factor is the step size factor calculated for each time period, and the denoised audio data for each time period is output.

[0018] Preferably, the specific process of voice interaction is as follows: the entire audio data after noise reduction is translated into text information data, and the actual meaning of the text information is understood through NLP natural language algorithms; based on the obtained actual meaning, the corresponding answer text information is output and converted into voice data using TTS semantic recognition and text-to-speech technology, and then output, thereby completing the voice interaction.

[0019] Secondly, embodiments of this application also provide a voice interaction system for an artificial intelligence security assistant, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-described voice interaction methods for an artificial intelligence security assistant.

[0020] This application has at least the following beneficial effects:

[0021] This application addresses the problem that traditional LMS algorithms, when used for audio data denoising, fail to effectively remove noise when faced with time-varying noise or changes in voice input. By analyzing the changing characteristics of the surrounding noise environment during user voice input, a noise variation coefficient is constructed for each time period, characterizing whether the noise environment in the audio data has changed drastically. Furthermore, by analyzing the effective human voice features contained in the audio data, a human voice presence coefficient is constructed for each time period, and then the step-size factor adjustment coefficient for each time period is calculated. This allows for adaptive adjustment of the preset initial step-size factor in each time period of the LMS algorithm, achieving adaptive adjustment of the step-size factor. This improves the denoising effect of the audio data while preserving the effective information in the audio. Attached Figure Description

[0022] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating the steps of a voice interaction method for an artificial intelligence security assistant, as provided in one embodiment of this application;

[0024] Figure 2 A flowchart illustrating the process of obtaining the step size factor adjustment coefficient for each time period, as provided in one embodiment of this application. Detailed Implementation

[0025] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a voice interaction method and system for an artificial intelligence security assistant proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0027] The following description, in conjunction with the accompanying drawings, details the specific scheme of the voice interaction method and system for an artificial intelligence security assistant provided in this application.

[0028] Please see Figure 1 The diagram illustrates a flowchart of a voice interaction method for an artificial intelligence security assistant according to an embodiment of this application. The method includes the following steps:

[0029] Step 1: Obtain the audio data of the voice assistant from wake-up to sleep and divide it into multiple time periods.

[0030] After the user wakes up the voice assistant, the voice assistant continuously collects audio data from the current environment until it enters sleep mode, with a sampling frequency of 51.2kHz. The data collection performed by the voice assistant from wake-up to sleep mode is recorded as one data collection session. For ease of analysis later, this application uses any one of these data collection sessions as an example. The collected audio data is divided into multiple time periods according to a preset time length T. In this embodiment, T is set to 2 seconds, but the implementer can adjust it according to the actual application scenario.

[0031] Step 2: Divide each time period into multiple time intervals; based on the trend of the dispersion of audio data in all time intervals within each time period, and the dispersion of the time interval between adjacent abrupt change points in the audio data within each time period, obtain the noise variation coefficient for each time period.

[0032] For users of mobile voice assistants, the noise contained in their input audio is more likely to change over time, but it may also remain relatively stable for a period of time. For example, a user might initially work in a relatively quiet indoor environment and then move to an outdoor environment with significantly different noise levels. Because voice assistants are used in a wide range of applications and often require extremely short response times, the noise removal efficiency of the input audio is also crucial. Therefore, for such user environments, simply analyzing real-time and historical noise changes is insufficient to handle sudden changes in noise levels. This leads to an inability to accurately identify the user's input when noise interference changes rapidly. Therefore, it is necessary to make certain judgments about impending changes in noise.

[0033] Specifically, when a user transitions between different environments while using a voice assistant (from low noise to high noise, or from high noise to low noise), the noise environment will change during the transition. For example, if the outdoor noise is higher than the indoor noise, as the user moves from indoors to outdoors, the noise in the input audio will gradually increase from a relatively low noise level as the user approaches the doorway. Once the user is outdoors, the noise will become completely dominated by outdoor noise. In the audio data, this is mainly manifested as a gradual increase in the intensity of the audio data fluctuations as the user moves. Conversely, when the user moves from a high-noise environment to a low-noise environment, the intensity of the audio data fluctuations gradually decreases. In addition, due to the complexity of the scene and the strong uncertainty of the noise situation, in addition to the regular continuous noise, there may also be some sudden noise, such as dragging tables and chairs when using indoors, noise from outside the window, etc., which will cause sudden changes in the amplitude of audio data. If the regularity of these sudden changes is relatively weak, it indicates that the environmental noise faced by the user when using the voice assistant is highly time-varying. Specifically, in the collected audio data, the interval between peaks in the audio data is relatively low.

[0034] To characterize the above features, each time period is divided into multiple time intervals according to a preset duration t. In this embodiment, t is set to 0.2s. For the audio data within a single time interval, curve fitting is performed on the audio data, and the goodness of fit for that single time interval is calculated. If the noise interference in the audio data within the time interval is stronger, the goodness of fit calculated for that time interval is relatively smaller, and vice versa. Further, taking the i-th time period as an example, the goodness of fit of all time intervals contained in the i-th time period is arranged in chronological order to construct the goodness of fit sequence for the i-th time period. The goodness of fit sequences of each time period are used as input to a trend test algorithm to obtain the trend strength of the goodness of fit sequences of each time period. The greater the trend strength, the stronger the increasing or decreasing trend of the corresponding goodness of fit sequence, and the more likely there is a change in the noise environment in the corresponding time period. The audio data under the i-th time period is used as input, and a mutation point detection algorithm is used to obtain all mutation points.

[0035] As a preferred implementation, the noise drastic change coefficient for each time period is obtained based on the variation trend of the dispersion of audio data in all time intervals within each time period, and the dispersion of the time interval between adjacent abrupt change points in the audio data within each time period. This coefficient is used to characterize the possibility of drastic changes in the noise environment under each time period.

[0036] In this implementation, the noise variation coefficient for the i-th time period is denoted as... Its specific expression is: In the formula, Let be the noise variation coefficient for the i-th time period. Let be the trend strength of the goodness-of-fit sequence for the i-th time period. Let be the trend strength of the goodness-of-fit sequence in the u-th time period. Let be the variance of the time intervals between all adjacent abrupt changes in the audio data within the i-th time period; This is a preset constant used to prevent the denominator from being 0; in this embodiment, it is set to 0.01.

[0037] The meaning of this relationship is as follows: when the trend strength of the goodness-of-fit sequence of the audio data in the i-th time period is greater than the trend strength of the historical time periods, it indicates that the noise situation of the audio data in the i-th time period has a stronger increasing or decreasing trend, and the possibility of a drastic change in the noise environment of the audio data in that time period is higher; if the variance of the time interval of the abrupt change points in the audio data in the i-th time period is larger, it indicates that the time interval of the abrupt change in the audio data in that time period is more irregular, and the possibility of a drastic change in the noise environment of the audio data in that time period is higher.

[0038] Step 3: Obtain the peak points in the fitting curves of the audio data for each time period. Based on the dispersion of the time interval between all adjacent peak points in the audio data for each time period, and the difference between low-frequency energy and high-frequency energy in the audio data for each time period, obtain the human voice presence coefficient for each time period. Combined with the noise variation coefficient for each time period, obtain the step size factor adjustment coefficient for each time period. Then, adjust the preset initial step size factor for each time period to obtain the denoised audio data for each time period, and use it for voice interaction.

[0039] During the use of voice assistants, users may be using mobile devices for input, or they may place the mobile device in a location while inputting from a distance. Alternatively, the device itself may be placed in a specific location (such as a smart home device). In these cases, the varying distances between the user and the device can lead to relatively high noise interference. Furthermore, when inputting from a greater distance, the user's voice may become similar in volume to the ambient noise, causing some valid information to be masked by the noise. Consequently, during noise reduction, audio data containing valid information may be filtered out as noise, ultimately reducing the integrity of the input audio information and affecting the accuracy of subsequent speech recognition. Therefore, further analysis is needed.

[0040] Specifically, when noise is strong or the user is far away, the effective information (such as human voice) in the audio data may be very similar to the noise, making the effective information less obvious and difficult to distinguish accurately. For the user-input human voice portion, when the noise is low, it typically presents as multiple high-amplitude waveforms in the time domain, meaning multiple peaks combine to form a large envelope, and the time intervals between envelopes are relatively uniform. When there is no human voice in the audio data, only noise, the time intervals between the large envelopes will be more discrete. In the frequency domain, noise has a wider frequency distribution. If an audio data segment contains only noise, the frequency distribution in the audio frequency domain is relatively balanced. However, for human voice audio data, the frequency is more concentrated in the low frequencies. Therefore, when there is a certain human voice component in the audio, the ratio of low-frequency energy to high-frequency energy in the audio data in the frequency domain is greater.

[0041] To characterize the above features, for audio data in a single time period, all peak values ​​in the time domain are obtained and arranged in chronological order. A curve fitting algorithm is used to obtain the fitted curves for the audio data in each time period, and the peaks and troughs in the fitted curves are obtained. Each peak is the large envelope peak mentioned above. Furthermore, the audio data of each time period is used as input, and a Fourier transform algorithm is used to output the spectral sequence of the audio data for each time period. A preset frequency threshold is used as the division; frequencies less than or equal to the preset frequency threshold are considered low frequencies, and vice versa; in this embodiment, the preset frequency threshold is 20kHz. Further, the energy of the low-frequency and high-frequency frequencies in the frequency domain of the audio data for each time period is calculated. The calculation of frequency domain energy is a well-known technique, and the specific process will not be elaborated further.

[0042] As a preferred implementation, based on the dispersion of the time interval between all adjacent peaks in the audio data of each time period and the difference between low-frequency energy and high-frequency energy in the audio data of each time period, the human voice presence coefficient of each time period is obtained, which is used to characterize the possibility of human voice audio in the audio data of each time period.

[0043] In this embodiment, the human voice presence coefficient for the i-th time period is denoted as... Its specific expression is: In the formula, Let be the human voice presence coefficient for the i-th time period. Let be the variance of the time intervals between all adjacent peaks in the audio data of the i-th time period. Let be the ratio of low-frequency energy to high-frequency energy in the frequency domain of the audio data for the i-th time period.

[0044] The meaning of this relationship is: in the audio data of the i-th time period, the smaller the dispersion of the time interval between adjacent peaks and the larger the ratio of low-frequency energy to high-frequency energy, the more likely there is human voice audio in the audio data collected in that time period.

[0045] Furthermore, based on the noise variation coefficient and human voice presence coefficient for each time period, step size factor adjustment coefficients are obtained for each time period. These coefficients characterize the degree to which the step size factor should be increased when processing the audio data of each time period using the LMS algorithm. The process for obtaining the step size factor adjustment coefficients for each time period is as follows: Figure 2 As shown, the step size factor adjustment coefficient for each time period is negatively correlated with the noise variation coefficient and the human voice presence coefficient for each time period. This negative correlation means that the dependent variable decreases (increases) as the independent variable increases (decreases).

[0046] Preferably, in this embodiment, the step size factor adjustment coefficient for the i-th time period is denoted as... Its specific expression is: In the formula, Let be the step size factor adjustment coefficient for the i-th time period. Let be the noise variation coefficient for the i-th time period. Let be the human voice presence coefficient for the i-th time period. The tanh function is used for normalization in this embodiment.

[0047] The meaning of this relationship is: in the audio data of the i-th time period, the greater the possibility of drastic noise changes and the higher the possibility of human voices, the more the step size factor should be reduced during the LMS algorithm processing to slow down the convergence speed, improve the stability of noise removal, and thus ensure that the noise removal effect in the audio data is good after processing, while preserving the human voice part to the maximum extent.

[0048] Furthermore, the step size factor for the i-th time period is calculated using the following formula: In the formula, Let be the step size factor for the i-th time period. Let be the step size factor adjustment coefficient for the i-th time period. In this embodiment, to preset the initial step size factor, It should be noted that the step size factor... The range of values ​​is ,in, It is the largest eigenvalue of the autocorrelation matrix of the audio data in the i-th time period. Therefore, when calculating... Greater than At that time, then order equal .

[0049] Similarly, the step size factors for all time periods are obtained using the above method. Further, the audio data for each time period in the entire audio dataset is used as input to the LMS algorithm, with the step size factor calculated for each time period. The output is the denoised audio data for each time period. The LMS algorithm is a well-known technique, and its specific process will not be elaborated further.

[0050] Finally, the denoised audio data is input into the intelligent voice assistant system. The intelligent voice assistant system first uses ASR real-time translation technology in IVR voice navigation to translate the input audio into text information data, and then uses NLP natural language understanding technology to understand the actual meaning of the user's input audio. Finally, based on the obtained actual meaning, it uses TTS semantic recognition and text-to-speech technology to output the corresponding answer text information and convert it into voice data, and outputs it to complete the voice interaction.

[0051] Based on the same inventive concept as the above method, this application embodiment also provides a voice interaction system for an artificial intelligence security assistant, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described voice interaction methods for an artificial intelligence security assistant.

[0052] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0053] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0054] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A voice interaction method of an artificial intelligence security assistant, the method comprising: The method includes the following steps: Acquire audio data from the voice assistant between wake-up and sleep, and divide it into multiple time periods; Each time period is divided into multiple time intervals; based on the variation trend of the dispersion of audio data in all time intervals within each time period, and the dispersion of the time interval between all adjacent abrupt change points in the audio data within each time period, the noise drastic change coefficient of each time period is obtained. The peak points in the fitting curves of audio data for each time period are obtained. Based on the dispersion of the time interval between all adjacent peak points in the audio data for each time period, and the difference between low-frequency energy and high-frequency energy in the audio data for each time period, the human voice presence coefficient for each time period is obtained. Combined with the noise variation coefficient for each time period, the step size factor adjustment coefficient for each time period is obtained. Then, the preset initial step size factor for each time period is adjusted to obtain the denoised audio data for each time period, which is used for voice interaction. Curve fitting is performed on the audio data for each time interval, and the goodness of fit of each fitted curve is calculated; the sequence formed by arranging the goodness of fit of all time intervals in each time period in chronological order is denoted as the goodness of fit sequence of each time period. Calculate the noise variation coefficient for each time period: In the formula, Let be the noise variation coefficient for the i-th time period. Let be the trend strength of the goodness-of-fit sequence for the i-th time period. Let be the trend strength of the goodness-of-fit sequence in the u-th time period. Let be the variance of the time intervals between all adjacent abrupt changes in the audio data within the i-th time period; This is a preset constant; The step size factor adjustment coefficient for each time period is negatively correlated with the noise drastic change coefficient and the human voice presence coefficient for each time period. The calculation formula for adjusting the preset initial step size factor for each time period is as follows: In the formula, Let be the step size factor for the i-th time period. Let be the step size factor adjustment coefficient for the i-th time period. Preset initial step size factor; The process of obtaining the human voice presence coefficient for each time period is as follows: calculate the variance of the time interval between all adjacent peaks in the audio data of each time period; use the audio data of each time period as the input of the Fourier transform algorithm, and output the spectral sequence of the audio data of each time period; take frequencies less than or equal to a preset frequency threshold as low frequencies, and vice versa as high frequencies; calculate the ratio of low-frequency energy to high-frequency energy in the audio data of each time period; and record the ratio of the ratio to the variance as the human voice presence coefficient for each time period.

2. The voice interaction method for an artificial intelligence security assistant as described in claim 1, characterized in that, The range of values ​​is ,in, Let be the largest eigenvalue of the autocorrelation matrix of the audio data in the i-th time period; when the calculated Greater than At that time, then order equal .

3. The voice interaction method for an artificial intelligence security assistant as described in claim 1, characterized in that, The method for obtaining the denoised audio data for each time period is as follows: the audio data of all time periods in the entire audio data are used as the input of the LMS algorithm, the step size factor is the step size factor calculated for each time period, and the denoised audio data for each time period is output.

4. The voice interaction method for an artificial intelligence security assistant as described in claim 1, characterized in that, The specific process of voice interaction is as follows: the entire audio data after noise reduction is translated into text information data, and the actual meaning of the text information is understood through NLP natural language algorithms; based on the obtained actual meaning, the corresponding response text information is output and converted into speech data using TTS semantic recognition and text-to-speech technology, and then output, thereby completing the voice interaction.

5. A voice interaction system for an artificial intelligence security assistant, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the voice interaction method for an artificial intelligence security assistant as described in any one of claims 1-4.

Images