Voice recognition method and device, storage medium and electronic equipment

Abstract

The invention provides a voice recognition method and device, a storage medium and electronic equipment, and the method comprises the steps: monitoring the noise state of an environment where target equipment is located, and enabling the noise state to be used for representing the noise intensity of the environment; configuring a smoothing algorithm according to the noise state; according to the noise state and the smoothing algorithm, performing smoothing processing on a sound signal collected by the target device to obtain a target voice signal; and recognizing the target voice signal by using a voice recognition model. The technical problem of low voice recognition accuracy in related technologies is solved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of speech technology, and in particular to a speech recognition method and device, a storage medium, and an electronic device. Background Art

[0002] In related technologies, with the rapid development of IoT technology, more and more devices have voice interaction functions. However, the environments in which IoT devices are located are complex and diverse. For example, smart speakers in smart homes may be exposed to cooking noise in the kitchen and TV noise in the living room; sensor devices in industrial IoT may be disturbed by the roar of machine operation. At the same time, IoT devices often have limited computing resources and power consumption.

[0003] When dealing with the noise problem of IoT devices, the speech recognition noise reduction technology in related technologies has problems such as low efficiency, poor adaptability and inability to fully utilize device characteristics for optimization, resulting in low speech recognition accuracy and affecting user experience.

[0004] With respect to the above-mentioned problems existing in the related technologies, no efficient and accurate solutions have been found yet. Summary of the invention

[0005] The present invention provides a speech recognition method and device, a storage medium, and an electronic device to solve the technical problem of low speech recognition accuracy in related technologies.

[0006] According to one embodiment of the present invention, a speech recognition method is provided, comprising: monitoring the noise state of an environment in which a target device is located, wherein the noise state is used to characterize the noise intensity of the environment; configuring a smoothing algorithm according to the noise state; smoothing a sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target speech signal; and recognizing the target speech signal using a speech recognition model.

[0007] Optionally, monitoring the noise state of the environment where the target device is located includes: acquiring the ambient sound collected by the target device according to a preset period; calculating the average sound intensity of the ambient sound in the preset period; determining whether the average sound intensity is less than a preset threshold; if the average sound intensity is less than the preset threshold, determining that the noise state of the environment where the target device is located is a quiet state; if the average sound intensity is greater than or equal to the preset threshold, analyzing the frequency value of the ambient sound, and if the frequency value is distributed within a preset frequency range, determining that the noise state of the environment where the target device is located is a noisy state.

[0008] Optionally, configuring a smoothing algorithm according to the noise state includes: if the noise state is a quiet state, selecting a first smoothing algorithm of a first order, wherein the first order is greater than a preset order; configuring an interpolation weight parameter of the first smoothing algorithm based on the first order, and setting an adjustment step of the first smoothing algorithm, wherein the interpolation weight parameter is used to characterize a weight coefficient of each order language model in the first smoothing algorithm, and the adjustment step is used to indicate a probability amplitude allowed for adjustment of the first smoothing algorithm.

[0009] Optionally, configuring a smoothing algorithm according to the noise state includes: if the noise state is a noisy state, selecting a second smoothing algorithm of a second order, wherein the second order is less than a preset order; configuring anti-noise parameters and a smoothing factor of the second smoothing algorithm, wherein the anti-noise parameters are used to indicate the attenuation strength of speech signals in different frequency ranges, and the smoothing factor is used to increase the probability of vocabulary combinations whose occurrence frequency is lower than a preset value.

[0010] Optionally, the sound signal collected by the target device is smoothed according to the noise state and the smoothing algorithm to obtain a target voice signal, including: if the noise state is a quiet state, collecting a first sound signal; enhancing the amplitude of the high-frequency component of the first sound signal to obtain a first intermediate voice; performing frame processing on the first intermediate voice to obtain a second intermediate voice; using a window function to perform window processing on each frame in the second intermediate voice to obtain a third intermediate voice, and filtering invalid frames in the third intermediate voice by energy value and zero crossing rate; using the smoothing algorithm to perform interpolation and smoothing calculations on the third intermediate voice to obtain a first target voice signal.

[0011] Optionally, the smoothing algorithm is used to perform interpolation smoothing calculations on the third intermediate speech to obtain a first target speech signal, including: constructing a first language model of an n-order sliding window, where n is greater than 1; counting the frequency of occurrence of each order of phrases in a text corpus based on the first language model to obtain statistical results; determining a first phrase in the third intermediate speech that does not appear in the text corpus based on the statistical results; calculating the probability of occurrence of characters in the first phrase in the n-order sliding window, and determining an initial probability of the first phrase; and weighting the calculation of a final probability of the first phrase in the third intermediate speech using the occurrence probability and the initial probability to obtain a first target speech signal.

[0012] Optionally, the sound signal collected by the target device is smoothed according to the noise state and the smoothing algorithm to obtain a target voice signal, including: if the noise state is a noisy state, collecting a second sound signal; converting the second sound signal into a frequency domain signal; parsing characteristic parameters of the ambient sound in the frequency domain signal; filtering the ambient sound in the second sound signal based on the characteristic parameters to obtain a fourth intermediate voice; and using the smoothing algorithm to perform noise-resistant smoothing calculations on the fourth intermediate voice to obtain a second target voice signal.

[0013] Optionally, the smoothing algorithm is used to perform anti-noise smoothing calculation on the fourth intermediate speech to obtain a second target speech signal, including: attenuating the speech signal in the noise frequency range of the fourth intermediate speech to obtain a fifth intermediate speech; constructing a second language model of an m-order sliding window, where m is greater than 1; counting the frequency of occurrence of each order of phrases in a text corpus based on the second language model to obtain statistical results; based on the statistical results, filtering out second phrases in the fifth intermediate speech whose frequency of occurrence is lower than a preset frequency; and increasing the probability of occurrence of the second phrase in the fifth intermediate speech to obtain a second target speech signal.

[0014] According to another embodiment of the present invention, a speech recognition device is provided, comprising: a monitoring module for monitoring the noise state of the environment in which the target device is located, wherein the noise state is used to characterize the noise intensity of the environment; a configuration module for configuring a smoothing algorithm according to the noise state; a smoothing module for smoothing the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target speech signal; and a recognition module for recognizing the target speech signal using a speech recognition model.

[0015] Optionally, the monitoring module includes: an acquisition unit, used to acquire the ambient sound collected by the target device according to a preset period; a calculation unit, used to calculate the average sound intensity of the ambient sound in the preset period; a judgment unit, used to judge whether the average sound intensity is less than a preset threshold; a determination unit, used to determine that the noise state of the environment where the target device is located is a quiet state if the average sound intensity is less than the preset threshold; if the average sound intensity is greater than or equal to the preset threshold, analyze the frequency value of the ambient sound, and if the frequency value is distributed within a preset frequency range, determine that the noise state of the environment where the target device is located is a noisy state.

[0016] Optionally, the configuration module includes: a first selection unit, used to select a first smoothing algorithm of a first order if the noise state is a quiet state, wherein the first order is greater than a preset order; a first configuration unit, used to configure an interpolation weight parameter of the first smoothing algorithm based on the first order, and set an adjustment step of the first smoothing algorithm, wherein the interpolation weight parameter is used to characterize a weight coefficient of each order language model in the first smoothing algorithm, and the adjustment step is used to indicate a probability amplitude allowed for adjustment of the first smoothing algorithm.

[0017] Optionally, the configuration module includes: a second selection unit, used to select a second smoothing algorithm of a second order if the noise state is a noisy state, wherein the second order is less than a preset order; a second configuration unit, used to configure the anti-noise parameters and smoothing factors of the second smoothing algorithm, wherein the anti-noise parameters are used to indicate the attenuation strength of speech signals in different frequency ranges, and the smoothing factor is used to increase the probability of vocabulary combinations whose occurrence frequency is lower than a preset value.

[0018] Optionally, the smoothing module includes: a first acquisition unit, used to collect a first sound signal if the noise state is a quiet state; an enhancement unit, used to enhance the amplitude of the high-frequency component in the first sound signal to obtain a first intermediate speech; a framing unit, used to perform framing processing on the first intermediate speech to obtain a second intermediate speech; a windowing unit, used to perform windowing processing on each frame in the second intermediate speech using a window function to obtain a third intermediate speech, and filter invalid frames in the third intermediate speech by energy value and zero crossing rate; a first smoothing unit, used to perform interpolation smoothing calculation on the third intermediate speech using the smoothing algorithm to obtain a first target speech signal.

[0019] Optionally, the first smoothing unit includes: a construction subunit, used to construct a first language model of an n-order sliding window, where n is greater than 1; a statistical subunit, used to count the occurrence frequency of each order of phrases in the text corpus based on the first language model to obtain statistical results; a determination subunit, used to determine a first phrase in the third intermediate speech that does not appear in the text corpus based on the statistical results; a first calculation subunit, used to calculate the occurrence probability of the characters in the first phrase in the n-order sliding window respectively, and determine the initial probability of the first phrase; a second calculation subunit, used to use the occurrence probability and the initial probability to weightedly calculate the final probability of the first phrase in the third intermediate speech to obtain a first target speech signal.

[0020] Optionally, the smoothing module includes: a second acquisition unit, used to collect a second sound signal if the noise state is a noisy state; a conversion unit, used to convert the second sound signal into a frequency domain signal; an analysis unit, used to analyze the characteristic parameters of the ambient sound in the frequency domain signal; a filtering unit, used to filter the ambient sound in the second sound signal based on the characteristic parameters to obtain a fourth intermediate speech; a second smoothing unit, used to perform noise-resistant smoothing calculation on the fourth intermediate speech using the smoothing algorithm to obtain a second target speech signal.

[0021] Optionally, the second smoothing unit includes: an attenuation subunit, used to attenuate the speech signal in the noise frequency range in the fourth intermediate speech to obtain a fifth intermediate speech; a construction subunit, used to construct a second language model of an m-order sliding window, where m is greater than 1; a statistical subunit, used to count the occurrence frequency of each order of phrases in the text corpus based on the second language model to obtain statistical results; a screening subunit, used to screen the second phrases in the fifth intermediate speech whose occurrence frequency is lower than a preset frequency based on the statistical results; and an enhancement subunit, used to enhance the occurrence probability of the second phrase in the fifth intermediate speech to obtain a second target speech signal.

[0022] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0023] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the program stored in the memory.

[0024] According to yet another embodiment of the present invention, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned device embodiments when running.

[0025] Through the embodiments of the present invention, the noise state of the environment where the target device is located is monitored, wherein the noise state is used to characterize the noise intensity of the environment; a smoothing algorithm is configured according to the noise state; the sound signal collected by the target device is smoothed according to the noise state and the smoothing algorithm to obtain a target voice signal; a voice recognition model is used to recognize the target voice signal, and the noise state of the environment where the target device is located is monitored, and a smoothing algorithm is adaptively configured based on the noise state, and finally the sound signal is smoothed according to the noise state and the configured smoothing algorithm, so that the voice recognition model processed subsequently can more fully utilize and recognize the voice feature information carried in the sound signal, thereby solving the technical problem of low voice recognition accuracy in related technologies and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 is a hardware structure block diagram of a smart speaker according to an embodiment of the present invention;

[0028] Figure 2 is a flow chart of a speech recognition method according to an embodiment of the present invention;

[0029] Figure 3 is a flow chart of speech recognition performed in an embodiment of the present invention;

[0030] Figure 4 It is a structural block diagram of a speech recognition device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Example 1

[0034] The method embodiment provided in the first embodiment of the present application can be executed in an IoT device, a voice robot, a smart speaker, a voice controller or a similar device management device. Taking running on a smart speaker as an example, Figure 1 is a hardware structure block diagram of a smart speaker according to an embodiment of the present invention. Figure 1 As shown, the smart speaker may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above-mentioned smart speaker may also include a transmission device 106 for communication functions and an input and output device 108. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above-mentioned smart speaker. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0035] The memory 104 can be used to store smart speaker programs, for example, software programs and modules of application software, such as a smart speaker program corresponding to a voice recognition method in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the smart speaker program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the smart speaker via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by the communication provider of the smart speaker. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] In this embodiment, a speech recognition method is provided. Figure 2 is a flow chart of a speech recognition method according to an embodiment of the present invention, such as Figure 2 As shown, the process includes the following steps:

[0038] Step S202, monitoring the noise state of the environment where the target device is located, wherein the noise state is used to characterize the noise intensity of the environment;

[0039] Optionally, the target device can be any device with voice collection function, such as a voice robot, a voice control screen, etc.

[0040] Step S204, configuring a smoothing algorithm according to the noise state;

[0041] In addition to configuring the smoothing algorithm according to the noise status of the environment, it can also be configured according to the operating status of the target device. For example, the smoothing parameters of the smoothing algorithm can be adjusted according to the operating status of the IoT device itself (such as working mode, power, network connection status, etc.). When the device power is low, a smoothing algorithm with lower computational complexity is used to give priority to the basic functions of speech recognition and reduce unnecessary computing consumption. When the device is in a high-load working mode (such as a smart camera performing video recording and voice interaction at the same time), more computing resources are dynamically allocated to the smoothing optimization link of speech processing to improve the accuracy of speech recognition.

[0042] Step S206, smoothing the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target voice signal;

[0043] Step S208: using a speech recognition model to recognize the target speech signal.

[0044] Optionally, the speech recognition model can be deployed on a target device that collects sound signals, or deployed on a cloud connected to the target device. When deployed on the cloud, the target speech signal needs to be transmitted to a cloud server.

[0045] For example, the target speech signal after smoothing is input into a speech recognition model (assuming that a speech recognition system combining a deep neural network and a hidden Markov model is used) for recognition. The recognition result is compared and matched with a locally stored standard speech instruction library (including preset control instructions, query questions, etc.) to obtain the final recognition result.

[0046] Through the above steps, the noise state of the environment where the target device is located is monitored, wherein the noise state is used to characterize the noise intensity of the environment; a smoothing algorithm is configured according to the noise state; the sound signal collected by the target device is smoothed according to the noise state and the smoothing algorithm to obtain a target voice signal; the target voice signal is recognized by using a voice recognition model, and the noise state of the environment where the target device is located is monitored, and a smoothing algorithm is adaptively configured based on the noise state, and finally the sound signal is smoothed according to the noise state and the configured smoothing algorithm, so that the subsequent processed voice recognition model can more fully utilize and recognize the voice feature information carried in the sound signal, thereby solving the technical problem of low voice recognition accuracy in related technologies and improving user experience.

[0047] In one implementation of the present embodiment, monitoring the noise state of the environment where the target device is located includes: acquiring the ambient sound collected by the target device according to a preset period; calculating the average sound intensity of the ambient sound in the preset period; determining whether the average sound intensity is less than a preset threshold; if the average sound intensity is less than the preset threshold, determining that the noise state of the environment where the target device is located is a quiet state; if the average sound intensity is greater than or equal to the preset threshold, analyzing the frequency value of the ambient sound, and if the frequency value is distributed within a preset frequency range, determining that the noise state of the environment where the target device is located is a noisy state.

[0048] Optionally, the sound intensity threshold corresponding to the preset threshold is set to 40 decibels. The preset period is 5 seconds.

[0049] In this embodiment, a sound intensity threshold is pre-set in the IoT device to distinguish between quiet and noisy environments, and the sound intensity threshold for a quiet environment is set to 40 decibels (dB). The built-in microphone of the device monitors the ambient sound intensity in real time, continuously samples and calculates the average sound intensity over a period of time (such as 5 seconds). When the average sound intensity is lower than 40dB, it is determined to be a quiet bedroom environment; if it is higher than this threshold, it starts to evaluate whether it is a noisy living room environment.

[0050] When confirming a noisy environment, it can be determined based on the frequency range of the ambient sound and the fluctuation range of the ambient sound. When the frequency range of the ambient sound is within the preset frequency range and the fluctuation range is less than the preset range, it is a noisy environment, and a frequency fluctuation characteristic threshold is set. Since the sound emitted by devices such as TVs has a specific frequency range and fluctuation pattern, if a continuous signal with relatively stable frequency fluctuations in the range of 100Hz-10000Hz (such as similar TV audio signal characteristics) is detected, and the sound intensity is higher than the set threshold, it is confirmed that the living room has a noisy environment with background noise such as TV and stereo.

[0051] By adopting the solution of this embodiment, the noise state of the environment is identified by the average sound intensity and frequency value of the ambient sound, thereby improving the recognition accuracy of the noise state.

[0052] In one implementation of the present embodiment, configuring a smoothing algorithm according to the noise state includes: if the noise state is a quiet state, selecting a first smoothing algorithm of a first order, wherein the first order is greater than a preset order; configuring an interpolation weight parameter of the first smoothing algorithm based on the first order, and setting an adjustment step of the first smoothing algorithm, wherein the interpolation weight parameter is used to characterize a weight coefficient of each order language model in the first smoothing algorithm, and the adjustment step is used to indicate a probability amplitude allowed for adjustment of the first smoothing algorithm.

[0053] The quiet environment may be a bedroom environment, a living room environment with little interfering sound, etc. Optionally, the first smoothing algorithm is an interpolation smoothing algorithm, and the first order is 4.

[0054] For the smoothing algorithm used in the bedroom, which is more accurate but has a slightly larger amount of calculation, such as the interpolation smoothing algorithm based on the high-order N-gram (assuming 4-gram) model, initialize and configure its interpolation weight parameters, including 1-gram weight, 2-gram weight, 3-gram weight, and 4-gram weight. At the same time, set the probability adjustment step parameter to fine-tune the result when calculating the probability to improve accuracy.

[0055] For example, when using optimization algorithms such as gradient descent to train the language model of the first smoothing algorithm, after calculating the gradient direction of the probability, the probability is not updated directly according to the gradient size, but multiplied by the step size parameter to achieve a small, gradual probability update to avoid missing the optimal probability value or making the model unstable due to a large update. Assuming that the probability of a certain phrase needs to be increased by 0.5 through the gradient calculation of the interpolation weight parameter, but the adjustment step size is set to 0.05, then the actual increase in the probability of the phrase in this iterative training is 0.5*0.05. After multiple such iterative trainings, the probability of each phrase in the language model is gradually adjusted to make it more accurate.

[0056] The solution of this implementation is adopted to configure a smoothing algorithm of a larger order for a quiet environment, and configure the interpolation weight parameters and adjustment step size of the smoothing algorithm. Reasoning is performed through a language model of a larger order, and the reasoning results of the model are fine-tuned, thereby improving the accuracy of speech recognition in a quiet environment.

[0057] In another implementation of the present embodiment, configuring a smoothing algorithm according to the noise state includes: if the noise state is a noisy state, selecting a second smoothing algorithm of a second order, wherein the second order is less than a preset order; configuring anti-noise parameters and a smoothing factor of the second smoothing algorithm, wherein the anti-noise parameters are used to indicate the attenuation strength of speech signals in different frequency ranges, and the smoothing factor is used to increase the probability of vocabulary combinations whose occurrence frequency is lower than a preset value.

[0058] Optionally, the second smoothing algorithm is a noise-resistant fast smoothing algorithm parameter, and the second order may be 2.

[0059] In this embodiment, the noisy state can be the noise state of the living room, public places (such as shopping malls, stations, etc.), etc. When used in the user's living room environment, a fast smoothing algorithm focusing on noise reduction can be used based on a combination of frequency shielding and low-order N-gram (2-gram). The noise reduction parameters include a frequency shielding threshold range, such as for frequency components below 200Hz and above 8000Hz, the shielding coefficient is set to 0.8 (i.e., attenuation of 20% signal strength), and for the part within the range of 200Hz-8000Hz and in line with the characteristic frequency interval of TV audio, the shielding coefficient is set to 0.5. The probability smoothing factor of the initial configuration of the 2-gram model is set to β=0.2, which is used to increase the probability of vocabulary combinations with a frequency of occurrence lower than a preset value.

[0060] The solution of this implementation is adopted to configure a smoothing algorithm with a smaller order for a noisy environment, and configure the anti-noise parameters and smoothing factor of the smoothing algorithm. Fast inference is performed through a smaller-order language model, and anti-noise processing is performed, thereby improving the accuracy of speech recognition in noisy environments.

[0061] In an implementation scenario of the present embodiment, the sound signal collected by the target device is smoothed according to the noise state and the smoothing algorithm to obtain a target voice signal, including: if the noise state is a quiet state, collecting a first sound signal; enhancing the amplitude of the high-frequency component of the first sound signal to obtain a first intermediate voice; performing frame processing on the first intermediate voice to obtain a second intermediate voice; using a window function to perform window processing on each frame in the second intermediate voice to obtain a third intermediate voice, and filtering invalid frames in the third intermediate voice by energy value and zero crossing rate; using the smoothing algorithm to perform interpolation smoothing calculation on the third intermediate voice to obtain a first target voice signal.

[0062] In this implementation scenario, the high-frequency component in the first sound signal is a component in the frequency range of 2560 to 5120 Hz.

[0063] For example, when the device determines that it is in a quiet bedroom environment, the microphone collects voice data at a higher sampling frequency (such as 44.1kHz). The collected data is first pre-emphasized to increase the signal energy of the high-frequency part, and the amplitude of the high-frequency component is enhanced by the pre-emphasis coefficient. Then, the data is processed in frames, with the frame length set to 25ms and the frame shift set to 10ms. Each frame of data is windowed using the Hamming window function to reduce spectrum leakage.

[0064] Taking the pre-emphasis coefficient of 0.97 as an example, when the pre-emphasis coefficient is set to 0.97, the amplitude of the high-frequency component will be enhanced according to a specific mathematical relationship, so that the energy proportion of the high-frequency part in the entire speech signal is relatively more reasonable, allowing subsequent processing to make more full use of the speech feature information carried by the high frequency.

[0065] The energy value and zero-crossing rate of each frame of speech data can also be calculated to remove frames with energy below a threshold (such as 10% of the average energy) and abnormal zero-crossing rate (such as 5 times higher than the average zero-crossing rate). These frames may be noise or invalid signals.

[0066] By adopting the solution of this embodiment, by performing amplitude enhancement, frame windowing and invalid frame filtering on the sound signal collected in a quiet state, the subsequent smoothing algorithm processing can more fully utilize and identify the speech feature information carried in the first sound signal, thereby improving the speech processing efficiency of the smoothing algorithm.

[0067] In one example, the smoothing algorithm is used to perform interpolation smoothing calculations on the third intermediate speech to obtain a first target speech signal, including: constructing a first language model of an n-order sliding window, where n is greater than 1; counting the frequency of occurrence of each order of phrases in a text corpus based on the first language model to obtain statistical results; determining a first phrase in the third intermediate speech that does not appear in the text corpus based on the statistical results; calculating the probability of occurrence of characters in the first phrase in the n-order sliding window, and determining an initial probability of the first phrase; and weighting the final probability of the first phrase in the third intermediate speech using the occurrence probability and the initial probability to obtain a first target speech signal.

[0068] In this example, taking n=4 as an example, a 4-gram language model is constructed based on the interpolation smoothing calculation of high-order N-grams, and the frequency of occurrence of 4-grams, 3-grams, 2-grams and 1-grams is counted from the locally stored text corpus (assuming that it contains a large number of daily conversations, stories, news, etc.). For example, for the corpus "I went to the park today", the number of occurrences of phrases of different orders such as "I went to the park today", "I went to the park today", "Go to the park", and "Park" is counted.

[0069] The probability distribution is calculated based on the statistical results. For 4-gram combinations that do not appear in the corpus, the interpolation smoothing algorithm is used to calculate the probability. Assuming that a 4-gram phrase "watch movies and eat popcorn" that has not appeared before is encountered, the final probability is obtained through weighted calculation based on the occurrence probability of "watch", "electric", "film", "explosion", "rice" and other characters in 1-gram, the probability of combinations such as "watch movies" and "popcorn" in 2-gram, the probability of combinations such as "watch movies and eat" in 3-gram, and the initial probability estimate of 4-gram (set to a minimum value, such as), and the final probability is obtained through combined weighted calculation, such as the final probability P = 0.1*p1+0.2*p2...., P1 is the 1-gram probability, p2 is the 2-gram, and so on, 0.1, 0.2 are the interpolation weight parameters pre-configured by the smoothing algorithm, one for each order language model, and the sum is 1. During the calculation process, the final probability is fine-tuned according to the probability adjustment step size to improve accuracy.

[0070] By adopting the solution of this embodiment, the frequency of occurrence of phrases is counted by using the first language model of a high-order sliding window, and the probability distribution of unfamiliar first phrases is calculated, thereby improving the recognition rate of unfamiliar phrases and further improving the accuracy of speech recognition in a quiet state.

[0071] In another implementation scenario of the present embodiment, the sound signal collected by the target device is smoothed according to the noise state and the smoothing algorithm to obtain a target voice signal, including: if the noise state is a noisy state, collecting a second sound signal; converting the second sound signal into a frequency domain signal; parsing the characteristic parameters of the ambient sound in the frequency domain signal; filtering the ambient sound in the second sound signal based on the characteristic parameters to obtain a fourth intermediate voice; and using the smoothing algorithm to perform noise-resistant smoothing calculations on the fourth intermediate voice to obtain a second target voice signal.

[0072] In this implementation scenario, when the device detects that it is in a noisy living room environment with background noise such as a TV, it first performs a fast Fourier transform (FFT) on the collected mixed sound signal, converts the second sound signal into a frequency domain signal, and then analyzes the frequency domain signal characteristics to determine the main frequency components and energy distribution of the TV audio signal (taking the environmental sound as the TV sound as an example). An adaptive filter is used for signal separation, and the characteristics of the TV audio signal are used as a reference signal to subtract the estimated components of the TV audio signal from the mixed signal. For example, based on the energy concentration characteristics of the TV audio signal in a specific frequency range (such as 500Hz-3000Hz), the filter parameters are set so that the signal in this frequency range is attenuated after passing through the filter.

[0073] By adopting the solution of this implementation scenario, through frequency domain conversion, environmental sound filtering and other processing on the sound signal collected in a noisy state, the subsequent smoothing algorithm processing can make more full use of and identify the speech feature information carried in the second sound signal, thereby improving the speech processing efficiency of the smoothing algorithm.

[0074] In one example, the smoothing algorithm is used to perform anti-noise smoothing calculation on the fourth intermediate speech to obtain a second target speech signal, including: attenuating the speech signal in the noise frequency range of the fourth intermediate speech to obtain a fifth intermediate speech; constructing a second language model of an m-order sliding window, where m is greater than 1; based on the second language model, counting the frequency of occurrence of each order of phrases in a text corpus to obtain statistical results; based on the statistical results, filtering out second phrases in the fifth intermediate speech whose frequency of occurrence is lower than a preset frequency; and increasing the probability of occurrence of the second phrase in the fifth intermediate speech to obtain a second target speech signal.

[0075] Optionally, m=2.

[0076] When performing anti-noise smoothing calculation, fast smoothing based on frequency masking and low-order N-gram is performed on the fourth intermediate speech after signal separation. The speech signal is processed according to the set frequency masking threshold range to further remove the residual TV noise frequency components. At the same time, a 2-gram language model is constructed to calculate the probability of lexical combinations in the speech signal. For the second lexical combination that appears less frequently, its probability estimate value is increased according to the set smoothing factor. For example, when processing the speech signal "turn on the TV", if the combination of "turn on" and "TV" appears less frequently in the training data, its probability in the recognition process is adjusted by the smoothing factor to improve the recognition accuracy.

[0077] The second phrase is a low-frequency phrase, which is a word with a low occurrence probability in the learning corpus and the word bank. It can be determined according to the size of the learning library. For example, if the occurrence times are set to be less than 100 times, they are all low-frequency. When increasing the probability estimate value of the second phrase, λ1 = 0.5, λ2 = 0.3, and λ3 = 0.2 can be set (λ reflects the dependence degree of the smoothing factor on the back-off probability, which is equivalent to giving a certain weight to the back-off probability to increase the overall probability estimate). Suppose in the corpus, for the lexical combination "the sun shines on the earth", the statistical result shows that count("sun", "shines", "earth") = 3, and count("sun", "shines") = 10. According to 2-gram calculation, P("earth" | "shines") = 0.2, and the probability of "earth" in the unigram language model P("earth") = 0.1. Then P("earth" | "sun", "shines") = 0.5×0.3 + 0.3×0.2 + 0.2×0.1 = 0.15 + 0.06 + 0.02 = 0.23. By reasonably setting these interpolation weights (λ is related to the smoothing factor), the probability estimate value of the lexical combination is increased by using the smoothing factor, especially for the trigram lexical combination with a low occurrence frequency, which can play a better role in probability adjustment.

[0078] Adopting the solution of this embodiment, by first processing the speech signal in the attenuation noise frequency range, and then using the second language model with a low-order sliding window to count the occurrence frequency of phrases, the occurrence probability of the second low-frequency phrase is increased, the recognition rate of the low-frequency phrase is improved, and thus the accuracy of speech recognition in a noisy state is improved.

[0079] The solution of this embodiment designs a lightweight and efficient smoothing algorithm, which uses edge computing technology to perform preliminary smoothing processing on the device side to reduce data transmission and cloud computing pressure. For example, smart home devices can adopt customized smoothing strategies based on the noise characteristics of different room environments and users' daily language habits. When the bedroom environment is quiet, a more accurate but slightly more computationally intensive smoothing method is used; when there is background noise such as a TV in the living room, it automatically switches to a fast smoothing algorithm that focuses on noise resistance, improving the user's voice interaction experience in different IoT application scenarios. Figure 3 It is a flow chart of speech recognition in an embodiment of the present invention, including: environmental monitoring and threshold setting, algorithm parameter initialization (including precise smoothing algorithm parameters, anti-noise fast smoothing algorithm parameters), selecting different smoothing algorithm parameters according to user needs, calculating probability distribution according to statistical results, speech recognition and model updating.

[0080] When updating the model, if it is a quiet scene, update the language model parameters of the smoothing algorithm based on the accuracy feedback of the recognition results. If there is a recognition error, analyze whether the error is due to inaccurate language model probability estimation or other factors (such as speech signal acquisition problems). If it is a probability estimation problem, adjust the interpolation weight parameters or re-count the frequency of related phrases to optimize the language model.

[0081] If it is a noisy scene, the anti-noise speech recognition and algorithm adjustment will input the processed speech signal into a speech recognition model specially optimized for noisy environments (this model has been trained with a large amount of noisy environment speech data and has strong anti-noise ability) for recognition. According to the accuracy of the recognition result, the frequency shielding threshold and the smoothing factor of the 2-gram model are dynamically adjusted. If the recognition accuracy is continuously lower than a certain threshold (such as 70%), appropriately increase the frequency shielding coefficient (such as increasing 0.1 on the original basis) or adjust the smoothing factor (such as increasing 0.05) to improve the anti-noise effect and recognition accuracy; if the accuracy is high and stable, keep the current algorithm parameters unchanged and continue the speech recognition process. Through the above specific implementation plans in different environments, it is possible to realize that the Internet of Things devices automatically switch to the appropriate smoothing algorithm according to the environmental noise conditions, thereby improving the accuracy and efficiency of speech recognition.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0083] Example 2

[0084] In the present embodiment, a speech recognition device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can realize a combination of software and hardware of a reservation function. Although the device described in the following embodiments is preferably realized in software, the realization of hardware, or a combination of software and hardware, can also be conceived.

[0085] Figure 4 is a structural block diagram of a speech recognition device according to an embodiment of the present invention, such as Figure 4 As shown, including:

[0086] A monitoring module 40, used to monitor the noise state of the environment where the target device is located, wherein the noise state is used to characterize the noise intensity of the environment;

[0087] A configuration module 42, configured to configure a smoothing algorithm according to the noise state;

[0088] A smoothing module 44, configured to perform smoothing processing on the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target speech signal;

[0089] The recognition module 46 is used to recognize the target speech signal using a speech recognition model.

[0090] Optionally, the monitoring module includes: an acquisition unit, used to acquire the ambient sound collected by the target device according to a preset period; a calculation unit, used to calculate the average sound intensity of the ambient sound in the preset period; a judgment unit, used to judge whether the average sound intensity is less than a preset threshold; a determination unit, used to determine that the noise state of the environment where the target device is located is a quiet state if the average sound intensity is less than the preset threshold; if the average sound intensity is greater than or equal to the preset threshold, analyze the frequency value of the ambient sound, and if the frequency value is distributed within a preset frequency range, determine that the noise state of the environment where the target device is located is a noisy state.

[0091] Optionally, the configuration module includes: a first selection unit, used to select a first smoothing algorithm of a first order if the noise state is a quiet state, wherein the first order is greater than a preset order; a first configuration unit, used to configure an interpolation weight parameter of the first smoothing algorithm based on the first order, and set an adjustment step of the first smoothing algorithm, wherein the interpolation weight parameter is used to characterize a weight coefficient of each order language model in the first smoothing algorithm, and the adjustment step is used to indicate a probability amplitude allowed for adjustment of the first smoothing algorithm.

[0092] Optionally, the configuration module includes: a second selection unit, used to select a second smoothing algorithm of a second order if the noise state is a noisy state, wherein the second order is less than a preset order; a second configuration unit, used to configure the anti-noise parameters and smoothing factors of the second smoothing algorithm, wherein the anti-noise parameters are used to indicate the attenuation strength of speech signals in different frequency ranges, and the smoothing factor is used to increase the probability of vocabulary combinations whose occurrence frequency is lower than a preset value.

[0093] Optionally, the smoothing module includes: a first acquisition unit, used to collect a first sound signal if the noise state is a quiet state; an enhancement unit, used to enhance the amplitude of the high-frequency component in the first sound signal to obtain a first intermediate speech; a framing unit, used to perform framing processing on the first intermediate speech to obtain a second intermediate speech; a windowing unit, used to perform windowing processing on each frame in the second intermediate speech using a window function to obtain a third intermediate speech, and filter invalid frames in the third intermediate speech by energy value and zero crossing rate; a first smoothing unit, used to perform interpolation smoothing calculation on the third intermediate speech using the smoothing algorithm to obtain a first target speech signal.

[0094] Optionally, the first smoothing unit includes: a construction subunit, used to construct a first language model of an n-order sliding window, where n is greater than 1; a statistical subunit, used to count the occurrence frequency of each order of phrases in the text corpus based on the first language model to obtain statistical results; a determination subunit, used to determine a first phrase in the third intermediate speech that does not appear in the text corpus based on the statistical results; a first calculation subunit, used to calculate the occurrence probability of the characters in the first phrase in the n-order sliding window respectively, and determine the initial probability of the first phrase; a second calculation subunit, used to use the occurrence probability and the initial probability to weightedly calculate the final probability of the first phrase in the third intermediate speech to obtain a first target speech signal.

[0095] Optionally, the smoothing module includes: a second acquisition unit, used to collect a second sound signal if the noise state is a noisy state; a conversion unit, used to convert the second sound signal into a frequency domain signal; an analysis unit, used to analyze the characteristic parameters of the ambient sound in the frequency domain signal; a filtering unit, used to filter the ambient sound in the second sound signal based on the characteristic parameters to obtain a fourth intermediate speech; a second smoothing unit, used to perform noise-resistant smoothing calculation on the fourth intermediate speech using the smoothing algorithm to obtain a second target speech signal.

[0096] Optionally, the second smoothing unit includes: an attenuation subunit, used to attenuate the speech signal in the noise frequency range in the fourth intermediate speech to obtain a fifth intermediate speech; a construction subunit, used to construct a second language model of an m-order sliding window, where m is greater than 1; a statistical subunit, used to count the occurrence frequency of each order of phrases in the text corpus based on the second language model to obtain statistical results; a screening subunit, used to screen the second phrases in the fifth intermediate speech whose occurrence frequency is lower than a preset frequency based on the statistical results; and an enhancement subunit, used to enhance the occurrence probability of the second phrase in the fifth intermediate speech to obtain a second target speech signal.

[0097] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0098] Example 3

[0099] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0100] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing:

[0101] S1, monitoring the noise state of the environment where the target device is located, wherein the noise state is used to characterize the noise intensity of the environment;

[0102] S2, configuring a smoothing algorithm according to the noise state;

[0103] S3, performing smoothing processing on the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target voice signal;

[0104] S4, using a speech recognition model to recognize the target speech signal.

[0105] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0106] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0107] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0108] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0109] S1, monitoring the noise state of the environment where the target device is located, wherein the noise state is used to characterize the noise intensity of the environment;

[0110] S2, configuring a smoothing algorithm according to the noise state;

[0111] S3, performing smoothing processing on the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target voice signal;

[0112] S4, using a speech recognition model to recognize the target speech signal.

[0113] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0114] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0115] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a controller or a network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., various media that can store program codes.

[0120] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A speech recognition method, characterized in that: include: Monitoring the noise state of the environment where the target device is located, wherein the noise state is used to characterize the noise intensity of the environment; configuring a smoothing algorithm according to the noise state; Smoothing the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target voice signal; The target speech signal is recognized using a speech recognition model.

2. The method according to claim 1, characterized in that The noise status of the environment where the target device is located includes: Acquire the ambient sound collected by the target device according to a preset period; Calculate the average sound intensity of the ambient sound in the preset period; Determining whether the average sound intensity is less than a preset threshold; If the average sound intensity is less than a preset threshold, it is determined that the noise state of the environment where the target device is located is a quiet state; if the average sound intensity is greater than or equal to the preset threshold, the frequency value of the ambient sound is analyzed, and if the frequency value is distributed within a preset frequency range, it is determined that the noise state of the environment where the target device is located is a noisy state.

3. The method according to claim 1, characterized in that: Configuring a smoothing algorithm according to the noise state includes: If the noise state is a quiet state, selecting a first smoothing algorithm of a first order, wherein the first order is greater than a preset order; An interpolation weight parameter of the first smoothing algorithm is configured based on the first order, and an adjustment step size of the first smoothing algorithm is set, wherein the interpolation weight parameter is used to characterize a weight coefficient of each order language model in the first smoothing algorithm, and the adjustment step size is used to indicate a probability amplitude allowed to be adjusted by the first smoothing algorithm.

4. The method according to claim 1, characterized in that Configuring a smoothing algorithm according to the noise state includes: If the noise state is a noisy state, selecting a second smoothing algorithm of a second order, wherein the second order is smaller than a preset order; Configure the anti-noise parameter and smoothing factor of the second smoothing algorithm, wherein the anti-noise parameter is used to indicate the attenuation strength of speech signals in different frequency ranges, and the smoothing factor is used to increase the probability of vocabulary combinations whose occurrence frequency is lower than a preset value.

5. The method according to claim 1, characterized in that Smoothing the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target voice signal, including: If the noise state is a quiet state, collecting a first sound signal; enhancing the amplitude of the high frequency component in the first sound signal to obtain a first intermediate speech; performing frame processing on the first intermediate speech to obtain a second intermediate speech; Performing windowing processing on each frame of the second intermediate speech by using a window function to obtain a third intermediate speech, and filtering invalid frames in the third intermediate speech by using energy value and zero crossing rate; The smoothing algorithm is used to perform interpolation and smoothing calculation on the third intermediate speech to obtain a first target speech signal.

6. The method according to claim 5, characterized in that The method of performing interpolation and smoothing calculation on the third intermediate speech using the smoothing algorithm to obtain a first target speech signal includes: Construct a first language model of an n-order sliding window, where n is greater than 1; Counting the occurrence frequency of each order of phrases in the text corpus based on the first language model to obtain statistical results; Determining, based on the statistical result, a first phrase in the third intermediate speech that does not appear in the text corpus; Calculating the occurrence probabilities of the characters in the first phrase in the n-order sliding window respectively, and determining the initial probability of the first phrase; The final probability of the first phrase in the third intermediate speech is calculated by weighting the occurrence probability and the initial probability to obtain a first target speech signal.

7. The method according to claim 1, characterized in that Smoothing the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target voice signal, including: If the noise state is a noisy state, collecting a second sound signal; converting the second sound signal into a frequency domain signal; Analyzing characteristic parameters of the ambient sound in the frequency domain signal; Filtering the ambient sound in the second sound signal based on the characteristic parameter to obtain a fourth intermediate speech; The smoothing algorithm is used to perform anti-noise smoothing calculation on the fourth intermediate speech to obtain a second target speech signal.

8. The method according to claim 7, characterized in that The method of performing anti-noise smoothing calculation on the fourth intermediate speech using the smoothing algorithm to obtain a second target speech signal includes: performing attenuation processing on the speech signal in the noise frequency range of the fourth intermediate speech to obtain a fifth intermediate speech; Construct a second language model of an m-order sliding window, where m is greater than 1; Counting the occurrence frequency of each order of phrases in the text corpus based on the second language model to obtain statistical results; Based on the statistical result, a second phrase whose appearance frequency in the fifth intermediate speech is lower than a preset frequency is selected; The occurrence probability of the second phrase in the fifth intermediate speech is increased to obtain a second target speech signal.

9. A speech recognition device, characterized in that: include: A monitoring module, used to monitor the noise state of the environment where the target device is located, wherein the noise state is used to characterize the noise intensity of the environment; A configuration module, used for configuring a smoothing algorithm according to the noise state; A smoothing module, used for smoothing the sound signal collected by the target device according to the noise state and the smoothing algorithm to obtain a target voice signal; The recognition module is used to recognize the target speech signal by using a speech recognition model.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the steps of the speech recognition method in any one of claims 1 to 8 when running.

11. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, the communication interface, and the memory communicate with each other via a communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the steps of the speech recognition method according to any one of claims 1 to 8 by running a program stored in a memory.

Images