Human voice recognition and enhancement method and device for voice device and storage medium

Abstract

The embodiment of the application discloses a human voice recognition and enhancement method and device for a voice device and a storage medium, relates to the field of communication technology, and can be applied to a scene in which sound of a intercom in a mask needs to be amplified and sound enhanced in emergency rescue. The application comprises the following steps: performing echo cancellation on collected sound signals by an NLMS algorithm, wherein the collected sound signals comprise environmental noise and voice signals; performing spectrum analysis on the sound processed in step 1, and extracting sound signals conforming to human voice characteristics; performing enhancement processing on the sound signals conforming to human voice characteristics by an IIR; and sending the sound signals processed by the enhancement processing to a intercom module.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method, apparatus and storage medium for human voice recognition and enhancement in voice devices. Background Technology

[0002] Currently, walkie-talkies are widely used as a means of voice communication in on-site coordination during emergency rescue. Moreover, shoulder microphones are usually required during missions. After connecting the walkie-talkie, there is no need to hold the walkie-talkie; it can be hung on the shoulder for communication. In other words, when using walkie-talkies in emergency missions, a shoulder microphone is required for convenient real-time communication.

[0003] Therefore, rescue personnel need to be equipped with a large number of communication devices, and the operating environment is also quite complex, resulting in poor sound quality and noise reduction. Furthermore, due to the specialized application scenarios of walkie-talkies, extended standby time and reliability are paramount requirements. Consequently, these communication devices generally employ a low-power, lower-performance but relatively reliable system architecture. This architecture has limited computing power, making many current sound optimization algorithms unsuitable for implementation. Moreover, there is currently a lack of effective methods for amplifying and enhancing the sound of walkie-talkies within face masks during emergency rescue operations. Summary of the Invention

[0004] Embodiments of the present invention provide a method, apparatus and storage medium for voice recognition and enhancement of voice devices, so as to be applied to scenarios in emergency rescue where it is necessary to amplify and enhance the sound of a walkie-talkie inside a mask.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] In a first aspect, the method provided by embodiments of the present invention includes:

[0007] Step 1: Perform echo cancellation on the acquired sound signal using the NLMS algorithm. The acquired sound signal includes both ambient noise and speech signal.

[0008] Step 2: Perform spectral analysis on the sound processed in Step 1 and extract sound signals that conform to human voice characteristics;

[0009] Step 3: Enhance the sound signal that matches the characteristics of the human voice using IIR;

[0010] Step 4: Send the enhanced audio signal to the intercom module.

[0011] Further steps include: playing and recording the enhanced audio signal; and then using the recorded audio data to perform echo cancellation on the acquired audio signal through the NLMS algorithm.

[0012] Step 1 includes: inputting the acquired sound signal into a codec for encoding; storing the encoded sound signal in a speech signal and noise buffer, and storing the recorded sound data in a reference buffer, wherein the processor extracts data from the speech signal and noise buffer and the echo reference buffer.

[0013] The audio signal and noise buffer is divided into m buffers. When the nth buffer is recording the audio signal, the processor is simultaneously processing the data in the [(n+m-1)mod m]th buffer and playing the data in the [(n+m-2)mod m]th buffer. The [(n+m-1)mod m]th buffer represents the buffer preceding the nth buffer, and the [(n+m-2)mod m]th buffer represents the buffer two preceding the nth buffer.

[0014] The echo cancellation process using the NLMS algorithm for the acquired sound signal includes: after the acquired sound signal is processed by the codec, NLMS normalization filtering is performed with reference to the system output signal, wherein the data type of the data output by the codec is a 24-bit precision signed integer; then NLMS normalization filtering is performed on the main signal with reference to the noise signal to obtain a pre-processed denoised signal; and notch filtering is performed on the denoised signal using IIR, wherein the IIR operates at a frequency specified during compilation.

[0015] The process of spectrum analysis includes: determining the sound threshold by the maximum power density (also known as the single-tone intensity); distinguishing between sound signals belonging to wind noise and sound signals belonging to human voice by the signal-to-noise ratio; and distinguishing between sound signals belonging to breathing valve resonance and sound signals belonging to microphone resonance by the power spectrum concentration.

[0016] The IIR enhancement process includes: collecting 1024 points each time as a cache, retrieving 4 caches according to the time sequence of the recorded caches to form a set of 4096 sampling points; and using the set of sampling points to enhance the speech activity statistics function.

[0017] The IIR is a five-band equalizer; it enhances sound signals that conform to human voice characteristics according to different sound frequency bands. The sound frequency bands include: ultra-low frequency of 40-200Hz, mid-low frequency of 200-800Hz, mid-high frequency of 800-1600Hz, ultra-high frequency of 1600-4000Hz, and overtones above 4000Hz.

[0018] Secondly, the apparatus provided in the embodiments of the present invention includes:

[0019] The preprocessing module is used to perform echo cancellation on the acquired sound signal using the NLMS algorithm. The acquired sound signal includes ambient noise and speech signal.

[0020] The analysis module is used to perform spectral analysis on the sound processed in step 1 and extract sound signals that conform to human voice characteristics;

[0021] The enhancement module is used to enhance sound signals that conform to the characteristics of human voices through IIR (Infrared Resonance Imaging).

[0022] The transmission module is used to send the enhanced audio signal to the intercom module.

[0023] Thirdly, embodiments of the present invention provide a storage medium storing a computer program or instructions, which, when executed, implement the method of this embodiment.

[0024] The voice recognition and enhancement method, apparatus, and storage medium for voice devices provided in this invention provide echo suppression that is effective at all times, regardless of whether an auxiliary microphone is present. After the acquired signal is converted from an analog to a digital signal (A / D converter), the output data type is a 24-bit signed integer. Noise and the main signal are then subjected to NLMS normalization filtering with the system output signal as a reference. Next, NLMS is applied to the main signal with the noise signal as a reference to obtain a pre-processed denoised signal. The denoised signal is then subjected to IIR notch filtering at a frequency specified during compilation to eliminate noise at specific frequencies. This allows for convenient application in emergency rescue scenarios where amplification and sound enhancement of walkie-talkie audio within a face mask are required. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the system architecture provided in an embodiment of the present invention;

[0027] Figure 2 , 3 A schematic diagram of the principle provided for an embodiment of the present invention;

[0028] Figure 4 A schematic diagram illustrating the process of voice recognition and enhancement provided in an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of the spectrum comparison process provided in an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of the method flow provided in an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of a possible spectrum analysis process provided in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Embodiments of the present invention will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of the present invention means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0033] This invention provides a method for voice recognition and enhancement in voice devices, such as... Figure 6 As shown, it includes:

[0034] Step 1: Perform echo cancellation on the acquired sound signal using the NLMS algorithm.

[0035] The collected sound signals include both ambient noise and speech signals. The NLMS (Normalized Least Mean Square) algorithm is a real-time processing algorithm that filters noise from a target signal based on known noise signal characteristics and with a reference. It has the advantages of fast convergence and low overhead.

[0036] Step 2: Perform spectral analysis on the sound processed in Step 1 and extract sound signals that conform to human voice characteristics.

[0037] Step 3: Enhance the sound signal that matches the characteristics of the human voice using IIR.

[0038] In this context, IIR refers to digital filters, which are linear time-invariant systems that filter digital signals. Digital filtering is essentially a computational process that performs operations on signals. The input digital signal (digital sequence) is transformed into an output digital sequence through specific operations. Therefore, a digital filter is essentially a digital computational process that performs specific operations; it can also be understood as a computer. The convolution and difference equations describing the relationship between the output and input of a discrete system merely provide the digital signal filter with operational rules, enabling it to process the input data according to these rules.

[0039] Step 4: Send the enhanced audio signal to the intercom module.

[0040] In this embodiment, the method further includes: playing and recording the enhanced sound signal, and recording the enhanced sound signal for use as a reference for echo cancellation in the next cycle, wherein the specific sound data is recorded in the echo reference buffer.

[0041] The recorded sound data is then used to perform echo cancellation on the acquired sound signal using the NLMS algorithm. For example, the enhanced sound signal 1 is played and stored in the echo reference buffer. Here, sound signal 1 refers to the enhanced sound after noise cancellation and echo cancellation, which is the sound that can be played and sent to the intercom module. If there are main and secondary microphones, the sound signal 2 picked up by the main microphone includes the voice signal 2-1, the echo signal 3 (i.e., the echo generated after the aforementioned sound signal 1 is played), and the ambient noise 4. After A / D conversion, sound signal 2 is stored in the voice signal and noise buffer. The sound signal 5 picked up by the secondary microphone includes the echo signal 3 and the ambient noise 4. After A / D conversion, sound signal 5 is also stored in the voice signal and noise buffer.

[0042] The echo cancellation process specifically employs methods such as... Figure 4In this method, echo cancellation is effective at all times, regardless of whether an auxiliary microphone is used. For example, the echo cancellation of the acquired sound signal using the NLMS algorithm includes: converting the acquired signal to a 24-bit signed integer after A / D conversion; normalizing and filtering the noise and main signal using NLMS with the system output signal as a reference; and then performing NLMS on the main signal with the noise signal as a reference to obtain a pre-processed noise-reduced signal. The system output signal can be the sound signal played by the system (1). If the following method is used... Figure 2 The dual-microphone (master and secondary microphone) architecture shown collects sound signals including echo signals, ambient noise signals, and speech signals. Because the master microphone is closer to the user's mouth and nose, it captures a stronger speech signal. The secondary microphone also collects echo signals, ambient noise signals, and speech signals, but because it is farther from the user's mouth and nose, the strength of the captured speech signal is weaker than that of the master microphone. An IIR (Infinite Impulse Response) filter operates at a frequency specified at compile time. If a signal needs to filter out noise at a specific frequency, after NLMS (Non-Low Frequency Synthesis) processing, it is fed into an IIR notch filter with parameters specified at compile time. The specific processing principle can be described as follows... Figure 2 As shown, in the following steps: S1: The signal picked up by the main microphone and the previously mentioned sound signal 1 are processed by the NLMS algorithm to eliminate echo signal 3. S2: The signal picked up by the secondary microphone and the previously mentioned sound signal 1 are processed by the NLMS algorithm to eliminate echo signal 3. S3: The two signals after eliminating echo signal 3 in S1 and S2 are processed by the NLMS algorithm to obtain speech signal 2-1. Then, IIR notch filtering is applied to speech signal 2-1 to eliminate noise at specific frequencies (S4). Then, FFT transformation and spectrum analysis are performed (S5). The spectrum analysis mainly analyzes human voice, popping sounds, breathing valves, and white noise. If there is human voice, it is amplified (S6). Then, it is sent to the intercom module or played locally (S7).

[0043] For example, if there is no secondary microphone, the main signal is subjected to NLMS with reference to the ambient noise signal, specifically as follows: Figure 3As shown, if there is no secondary microphone, the enhanced audio signal 1 is played and stored in the echo reference buffer. The audio signal 2 picked up by the main microphone includes the speech signal 2-1, the echo signal 3, and the ambient noise 4. After A / D conversion, the audio signal 2 is stored in the speech signal and noise buffer. S01: The signal picked up by the main microphone and the aforementioned audio signal 1 are processed by the NLMS algorithm to eliminate the echo signal 3. S02: The signal obtained in S01 is then subjected to IIR notch filtering to eliminate noise at specific frequencies, and then FFT transformation and spectrum analysis are performed (S03). The spectrum analysis mainly analyzes human voice, ambient noise, popping, breathing valve, and white noise (in this embodiment, white noise is noise without obvious spectrum characteristics, caused by errors generated during A / D conversion). If there is human voice, it is enhanced (S04), and then sent to the intercom module or played locally (S05).

[0044] Currently, the reference signal is the output signal after voice processing, but it needs to be connected to the output signal of the intercom module in the subsequent walkie-talkie. In this embodiment, echo suppression is effective at all times regardless of whether there is an auxiliary microphone. After the acquired signal is converted by A / D, the output data type is a 24-bit precision signed integer. The noise and main signal are normalized and filtered by NLMS with the system output signal as the reference to obtain the pre-processed noise-canceling signal. Optionally, if a dual-microphone architecture is used, the noise signal can also be used as the reference to perform NLMS on the main signal. The noise-canceling signal will be subjected to IIR notch filtering at a frequency specified at compile time to eliminate noise at specific frequencies (such as single-frequency mechanical sounds), and then sent to CFFT to separate frequency features. In practical applications on walkie-talkies, it can achieve voice-controlled PTT function without the need for a shoulder microphone, and the call quality is good.

[0045] In this embodiment, step 1 includes: inputting the acquired sound signal into a codec (e.g., WM8978) for encoding; storing the encoded sound signal in a speech signal and noise buffer; and storing the recorded sound data in a reference buffer. One method for human voice recognition and enhancement is achieved by analyzing the frequency characteristics differences between human voice and natural noise, such as... Figure 1 As shown, audio encoding is performed by the WM8978, data stream is transmitted in full-duplex using SAI, and signals are processed in real-time using an ARM-DSP. Specifically, the processor extracts data from the speech signal and noise buffer and the echo reference buffer. For example, encoded and decoded audio signals 2 and 5 are stored in the speech signal and noise buffer, while audio signal 1 is stored in the echo reference buffer.

[0046] In this embodiment, m buffers are established in the voice signal and noise buffer. When the nth buffer is recording a sound signal, the processor is simultaneously processing the data in the [(n+m-1)mod m]th buffer and playing the data in the [(n+m-2)mod m]th buffer. Here, [(n+m-1)mod m]th buffer represents the buffer preceding the nth buffer, and [(n+m-2)mod m]th buffer represents the buffer two preceding the nth buffer. "Mod" in the industry refers to the remainder, and in a preferred embodiment, m is 4. When a buffer is recording a sound signal, the processor is simultaneously processing the data in the preceding buffer and playing the data (sound signal 1) from the two preceding buffers. This method of writing to the cache effectively reduces the processor's workload. In this embodiment, full-duplex operation is employed, and the input and output use the same buffer. Therefore, recording and playback must end simultaneously. Since recording is received by the master device and playback is sent by the slave device, the playback data stream is synchronized with the recording data stream. Therefore, the playback data buffer address should be mounted to the DMA before mounting the recording buffer. By using multiple circular buffers, assuming that sound is currently being recorded in the nth buffer, it must be processing data from the [(n+m-1)modm]th buffer. If sound is being played, it could be data from the [(n+m-1)modm]th or [(n+m-2)modm]th block (lowest latency mode).

[0047] Optionally, data from the [(n+m-2)modm] and [(n+m-3)modm] buffers can be played simultaneously to prevent some audio from being lost and to achieve a better playback effect.

[0048] Specifically, the use of the same buffer for input and output means that m buffers are divided in the speech signal and noise buffer area. When a certain buffer is recording a sound signal, the processor is simultaneously processing the data in the previous buffer and playing the data (sound signal 1) in the previous two buffers. The aforementioned buffer that is recording a sound signal is the input, and the simultaneous playback of the data (sound signal 1) in the previous two buffers. The data in the buffers is constantly updated, so the input and output use the same buffer.

[0049] In this embodiment, the method of spectral analysis is as follows: Figure 5The process includes: performing CFFT processing on the sound signal processed in step 1 to convert the sound signal from a time-domain signal to a frequency-domain signal. In the human voice region of the sound signal, determining the frequency of maximum energy concentration in the human voice region and recording the energy intensity at that frequency. Obtaining the amplitude-frequency characteristic information of the human voice region, including the average and standard deviation of the amplitude-frequency of the human voice region. In the noise region of the sound signal, determining the frequency of maximum energy concentration in the noise region and recording the energy intensity at that frequency. Using effective sound intensity, dispersion, speech signal-to-noise ratio (SNR), and total SNR compared with reference data, a preliminary determination is made as to whether a buffer contains human voices. Specifically, effective sound intensity = maximum single-tone energy intensity in the human voice region; dispersion = standard deviation of the amplitude-frequency of the human voice region / average amplitude-frequency of the human voice region; speech signal-to-noise ratio = maximum single-tone energy intensity in the human voice region / average amplitude-frequency of the human voice region; and total SNR = maximum single-tone energy intensity in the human voice region / maximum single-tone energy intensity in the noise region. If it is initially determined that there is human voice, the TTL stabilizer is triggered once, wherein the number of times the TTL stabilizer is triggered is positively correlated with the confidence level. Figure 5 In Chinese, "&" represents an AND gate, and "&." represents a NAND gate.

[0050] In practical implementation, the following can be adopted: Figure 7 The process shown includes: recording the buffer and processing it with CFFT (Complex Fast Fourier Transform) to convert the signal from the time domain to the frequency domain; finding the frequency with the maximum energy concentration in the human voice area and recording the energy intensity at that frequency; calculating the amplitude-frequency characteristics of the human voice area and obtaining its corresponding average value and standard deviation; finding the frequency with the maximum energy concentration in the noise area and recording the energy intensity at that frequency; comparing the dimensionless parameters such as effective sound intensity, dispersion, speech signal-to-noise ratio, and total signal-to-noise ratio with relevant data (experimental values) and making a preliminary judgment on whether a buffer contains human voice; if it is preliminarily determined that there is human voice, a TTL stabilizer is triggered once. Since speech itself is continuous, the TTL stabilizer will automatically stabilize and identify the output result according to the scene and the number of triggers, ensuring the confidence level (i.e., calling the speech activity statistics function to enhance the signal and output it).

[0051] In this embodiment, the process of performing spectral analysis on the processed sound includes:

[0052] The sound threshold is determined by the maximum power density (also known as the intensity of a single tone). The signal-to-noise ratio (SNR) is used to distinguish between wind noise and human voice. The power spectral density is used to distinguish between breath valve resonance and microphone resonance. All other sound signals besides human voice are essentially noise.

[0053] The sound threshold can be understood as the volume threshold, determined by the maximum intensity of a single tone corresponding to the acquired sound signal. A single tone specifically refers to a single frequency. In practical applications, for amplification within a mask, it is necessary to distinguish three types of noise: 1. White noise caused by errors during AD conversion; 2. Airflow whistling caused by the breathing valve, generally concentrated in a few fixed frequency bands; 3. Wind noise caused by airflow impacting the microphone during breathing.

[0054] Most speech recognition technologies in the industry focus on semantic recognition and sound recognition in quiet environments. Because mask-mounted amplification requires high precision in noise filtering, and traditional time-frequency analysis methods like Morlet wavelet transform place excessive demands on processor processing power in real-time, they are not suitable for ARM-Cortex M4F processors with a clock frequency of 60-120MHz. For fast spectrum analysis, the signal must first be converted from the time domain to the frequency domain. At a sampling rate of 48kHz, the kernel time occupied by the Fast Fourier Transform is generally around 33%, and the processor's efficiency in transforming the fundamental sinθ across the entire time domain is far higher than that of the Morlet wavelet. The energy of human voice is mainly concentrated in the 160-2000Hz range, so this frequency range can be the focus of analysis. Mathematical characteristics are used for analysis. First, white noise is noise without obvious spectral characteristics; its energy distribution is uniform, and there are no concentrated segments. Its spectral dispersion is extremely high, reflected in a very steep t-distribution curve in a normal distribution graph, where all energy is concentrated near the expected value (mean). Therefore, white noise can be identified using the spectral standard deviation σ = sqrt(((x1-x)^2+(x2-x)^2+......(xn-x)^2) / n). Setting a reasonable threshold can achieve this function. For airflow impact howling, the concentrated energy in this frequency band can be downweighted. Because of the concentrated energy in airflow impact howling, limiting the abrupt change in the amplitude-frequency curve and observing the characteristics of μ and σ in the t-distribution can distinguish the impact noise. When airflow impacts the microphone, the spectrum of wind noise is similar to white noise after analysis, so it will not be discussed in detail.

[0055] The human voice is a special sound source. Due to the structural characteristics of the vocal cords, we cannot produce a single tone (i.e., a single-frequency sound). The so-called pitch variation is the effect of synchronous frequency shifting of multiple fundamental frequencies. From a distribution perspective, its spectrum has significant energy and numerous overtones, resulting in a relatively dispersed amplitude-frequency distribution. If we take a normal distribution X~N(μ,σ)... 2The curve is relatively steep, but there are no abrupt changes. Therefore, after adjustment, choosing an appropriate standard deviation is sufficient to distinguish the human voice frequency bands. Specifically, human voice timbre is divided into the following segments: 40-200Hz ultra-low frequency, 200-800Hz mid-low frequency, 800-1600Hz mid-high frequency, 1600-4000Hz ultra-high frequency, and 4000Hz+ overtones. Generally, a five-segment IIR lattice filter can be used to process the sound: 1) Ultra-low frequency - attenuation, to avoid muffled sound. 2) Mid-low frequency - amplification, to make the sound clear. 3) Mid-high frequency - amplification, to make the sound transparent. 4) Ultra-high frequency - attenuation, with no significant impact but can filter out mechanical noise. 5) Overtones - notch filtering, with no significant impact, isolating high-frequency noise. The speech activity statistical function is used to compare the current spectral characteristics with a given threshold and derive the activity weight, then combine it with the most recent activity weights for a comprehensive judgment. For example: Figure 5 The spectrum comparison process shown is as follows: The main energy of human voice is concentrated in the fundamental frequency of 160-2000Hz, which has a high signal-to-noise ratio (the ratio of peak value to mean value) and low energy concentration (generally falling within a standard normal distribution (μ, σ)). 2 Within the range of 1σ to 2σ, wind noise energy is extremely dispersed, with a very low signal-to-noise ratio, and can be filtered out by the signal-to-noise ratio. Although the overall energy levels of the breathing valve and the popping microphone resonance also follow a t-distribution and have a relatively high signal-to-noise ratio, the energy peaks are mostly weak and the energy levels are close to the center point μ (less than 1σ), with a small overall standard deviation. Based on the above analysis, the following conclusions can be drawn: a. The sound threshold can be determined by the maximum power density (also known as the single-tone intensity). b. Wind noise and human voice can be determined by the signal-to-noise ratio. c. The breathing valve and popping microphone sounds can be determined by the power spectrum concentration σ / μ.

[0056] In this embodiment, the IIR enhancement process includes: collecting 1024 points as a cache each time, and retrieving four caches in chronological order to form a set of 4096 sampling points. For direct amplification (i.e., local playback), excessively long sampling times can lead to significant latency. When the output latency (playback latency) exceeds 100ms, it severely interferes with the user's auditory feedback, thus affecting their speech ability. However, too few sampling points can affect the accuracy of the system's judgment. Therefore, a sampling circular storage + chained loading scheme is used. Collecting 1024 points as a cache each time, and then retrieving four caches in a first-in-first-out manner to form 4096 most recent sampling points, with each cache corresponding to one time period. Based on the signal characteristics of multiple sampling time periods, the possibility of human voices existing in consecutive time periods is analyzed.

[0057] The aforementioned set of sampling points is used for enhancement through a speech activity statistics function. For direct amplification, excessively long sampling times can lead to significant latency; output delays exceeding 100ms can severely interfere with the user's auditory feedback, thus affecting their speech ability. Conversely, too few sampling points can compromise the system's accuracy. Therefore, a sampling loop storage + chained loading scheme is employed. Each time, 1024 points are collected as a cache, and four caches are retrieved in a first-in-first-out manner to form 4096 of the most recent sampling points. These are then combined with the speech activity statistics function for speech recognition and playback.

[0058] The speech activity statistics function can be modeled by those skilled in the art. The function aims to make the voice smoother, more continuous, and more natural. Without this function, the enhanced sound signal 1 after noise and echo removal would appear discontinuous and fragmented. This discontinuity is due to spectral analysis. In this embodiment, "speech activity" is an industry-specific term.

[0059] In this embodiment, the IIR is a five-band equalizer. Specifically, the IIR can enhance sound signals that conform to human voice characteristics according to different sound frequency bands. The sound frequency bands include: ultra-low frequencies of 40-200Hz, mid-low frequencies of 200-800Hz, mid-high frequencies of 800-1600Hz, ultra-high frequencies of 1600-4000Hz, and overtone frequencies above 4000Hz.

[0060] This embodiment also provides a voice recognition and enhancement device for a voice device, including:

[0061] The preprocessing module is used to perform echo cancellation on the acquired sound signal using the NLMS algorithm. The acquired sound signal includes ambient noise and speech signal.

[0062] The analysis module is used to perform spectral analysis on the sound processed in step 1 and extract sound signals that conform to human voice characteristics.

[0063] The enhancement module is used to enhance sound signals that conform to human voice characteristics through IIR.

[0064] The transmission module is used to send the enhanced audio signal to the intercom module.

[0065] Embodiments of the present invention also provide a storage medium storing a computer program or instructions, which, when executed, implement the method of this embodiment.

[0066] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for voice recognition and enhancement in a voice device, characterized in that, Each cycle of processing includes: Step 1: Perform echo cancellation on the acquired sound signal using the NLMS algorithm; Step 2: Perform spectral analysis on the sound processed in Step 1 and extract sound signals that conform to human voice characteristics; Step 3: Enhance the sound signal that matches the characteristics of the human voice using IIR; Step 4: Send the enhanced audio signal to the intercom module; The spectral analysis performed included: The audio signal processed in step 1 is subjected to CFFT processing to convert it from a time-domain signal to a frequency-domain signal. Within the human voice region of the audio signal, the frequency of maximum energy concentration in the human voice region is determined, and the energy intensity at that frequency is recorded. The amplitude-frequency characteristic information of the human voice region is obtained, including the average value and standard deviation of the amplitude-frequency response. Within the noise region of the audio signal, the frequency of maximum energy concentration in the noise region is determined, and the energy intensity at that frequency is recorded. The effective sound intensity, dispersion, speech signal-to-noise ratio, and total signal-to-noise ratio are compared with reference data to preliminarily determine whether a buffer contains human voices. If a human voice is preliminarily determined to be present, a TTL stabilizer is triggered once, wherein the number of TTL stabilizer triggers is positively correlated with the confidence level. Wherein, effective sound intensity = maximum single-tone energy intensity in the human voice area; dispersion = standard deviation of amplitude frequency in the human voice area / average amplitude frequency in the human voice area; speech signal-to-noise ratio = maximum single-tone energy intensity in the human voice area / average amplitude frequency in the human voice area; total signal-to-noise ratio = maximum single-tone energy intensity in the human voice area / maximum single-tone energy intensity in the noise area; The process of spectrum analysis includes: determining the sound threshold by the maximum power density; distinguishing between sound signals belonging to wind noise and sound signals belonging to human voice by the signal-to-noise ratio; and distinguishing between sound signals belonging to breathing valve resonance and sound signals belonging to microphone resonance by the power spectrum concentration. The speech signal and noise buffer is divided into m buffers. When the nth buffer is recording the sound signal, the processor is simultaneously processing the data of the [(n+m-1)mod m]th buffer and playing the data of the [(n+m-2)mod m]th and [(n+m-3)mod m]th buffers.

2. The method according to claim 1, characterized in that, Also includes: Play and record the enhanced audio signal; The recorded sound data is then used to perform echo cancellation on the acquired sound signal through the NLMS algorithm.

3. The method according to claim 2, characterized in that, Step 1 includes: The collected sound signal is input into a codec for encoding; The encoded audio signal is stored in the speech signal and noise buffer, and the recorded audio data is stored in the echo reference buffer. The processor extracts data from the speech signal and noise buffer and the echo reference buffer.

4. The method according to claim 1, characterized in that, The echo cancellation of the acquired sound signal using the NLMS algorithm includes: After the acquired sound signal is processed by the codec, it is subjected to NLMS normalization filtering with reference to the system output signal. The data type of the acquired sound signal after being processed by the codec is a 24-bit precision signed integer. The system output signal is the enhanced sound signal from the previous cycle. Then, using the noise signal as a reference, the main signal is subjected to NLMS normalization filtering to obtain the pre-processed denoised signal; The noise-reducing signal is notch filtered using an IIR filter, wherein the IIR operates at a frequency specified at compile time.

5. The method according to claim 1, characterized in that, The IIR enhancement process includes: Each time 1024 points are collected as a cache, and according to the time sequence of the recorded cache, 4 caches are retrieved to form a set of 4096 sampling points; The sampling point set is used to enhance the speech activity using a speech activity statistics function; The IIR is a five-band equalizer; IIR enhances sound signals that conform to human voice characteristics according to different sound frequency bands. The sound frequency bands include: ultra-low frequency of 40-200Hz, mid-low frequency of 200-800Hz, mid-high frequency of 800-1600Hz, ultra-high frequency of 1600-4000Hz, and overtones above 4000Hz.

6. A voice recognition and enhancement device for a voice device, characterized in that, include: The preprocessing module is used to perform echo cancellation on the acquired sound signal using the NLMS algorithm; The analysis module is used to perform spectral analysis on the sound processed by the preprocessing module and extract sound signals that conform to human voice characteristics; The enhancement module is used to enhance sound signals that conform to the characteristics of human voices through IIR (Infrared Resonance Imaging). The transmission module is used to send the enhanced audio signal to the intercom module; The spectral analysis performed included: The preprocessed audio signal is subjected to CFFT processing to convert it from a time-domain signal to a frequency-domain signal. Within the human voice region of the audio signal, the frequency of maximum energy concentration in the human voice region is determined, and the energy intensity at this frequency is recorded. The amplitude-frequency characteristic information of the human voice region is obtained, including the average value and standard deviation of the amplitude-frequency response. Within the noise region of the audio signal, the frequency of maximum energy concentration in the noise region is determined, and the energy intensity at this frequency is recorded. The effective sound intensity, dispersion, speech signal-to-noise ratio, and total signal-to-noise ratio are compared with reference data to preliminarily determine whether a buffer contains human voices. If a human voice is preliminarily determined to be present, a TTL stabilizer is triggered once, wherein the number of TTL stabilizer triggers is positively correlated with the confidence level. Wherein, effective sound intensity = maximum single-tone energy intensity in the human voice area; dispersion = standard deviation of amplitude frequency in the human voice area / average amplitude frequency in the human voice area; speech signal-to-noise ratio = maximum single-tone energy intensity in the human voice area / average amplitude frequency in the human voice area; total signal-to-noise ratio = maximum single-tone energy intensity in the human voice area / maximum single-tone energy intensity in the noise area; The process of spectrum analysis includes: determining the sound threshold by the maximum power density; distinguishing between sound signals belonging to wind noise and sound signals belonging to human voice by the signal-to-noise ratio; and distinguishing between sound signals belonging to breathing valve resonance and sound signals belonging to microphone resonance by the power spectrum concentration. The speech signal and noise buffer is divided into m buffers. When the nth buffer is recording the sound signal, the processor is simultaneously processing the data of the [(n+m-1)mod m]th buffer and playing the data of the [(n+m-2)mod m]th and [(n+m-3)mod m]th buffers.

7. A storage medium, characterized in that, It stores a computer program or instructions that, when executed, implement the method as described in any one of claims 1 to 5.

Images