Off-line conference recording box and voice recognition system thereof

Abstract

The invention discloses an off-line conference recording box and a voice recognition system thereof, belongs to the field of voice recognition, and aims to solve the problems of accurately and efficiently capturing and recognizing voices of a plurality of speakers and reducing the influence of environmental noise on recognition accuracy. The method comprises the following steps: capturing an audio signal from a microphone array to form audio data, receiving a voice audio signal through voice activity detection and dynamic volume adjustment, and providing early warning feedback when an audio quality problem is detected; the method comprises the steps of capturing audio data, performing data processing on the captured audio data, performing de-noising processing on original voice audio signals in the audio data, extracting audio features, and performing voiceprint recognition through clustering analysis of the audio features so as to obtain sound source separation signals with voiceprint information marks; and performing independent voice recognition according to the sound source separation signal, and organizing and managing voice data.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition, and more specifically to an off-line conference recording box and a speech recognition system thereof. Background Art

[0002] Traditional meeting recording methods often fail to meet the needs of modern fast-paced work. Although the method of relying on manpower to take real-time notes is direct and low-cost, it is easily affected by the personal status of the recorder. In addition, due to individual differences, important information may be omitted or misunderstood. Although common products such as portable voice recorders can capture the sound of the entire meeting, they generally lack the ability to convert audio into text. With the development of cloud computing technology, more and more online services provide powerful speech recognition and natural language processing capabilities, which can help users quickly complete the conversion from speech to text. However, in actual applications, not all occasions have stable and reliable Internet access conditions. For information security reasons, the use of any form of external communication means is often restricted or even prohibited. Therefore, it is particularly important to find a method that can ensure data security and achieve efficient office work.

[0003] The Chinese patent with the authorization announcement number CN110246501B discloses a voice recognition method and system for conference records. The method includes: receiving an activation instruction sent by the control terminal, and opening the cloud transcription service permission to the first platform according to the account ID; after successful activation, when receiving the instruction to start the meeting, when the network status does not meet the preset network standard, offline collection is performed and stored in the onboard memory; in response to receiving the instruction to end the meeting, the collection is stopped, and when the network status meets the preset network standard, the conference audio is pushed to the first platform; the feedback conference record text is received, forwarded to the push service of the second platform for multi-terminal collaborative push, and the conference record text is sent to the electronic device of the login account ID through multi-terminal collaboration. A voice recognition system for conference records is also provided, which separates sound collection and recognition to avoid the problem of inability to complete recognition records and loss of files when there is a problem with the connected network.

[0004] Although the existing technology separates the development of sound collection and recognition, avoiding the problem of inability to complete recognition records or loss of files due to poor network or the current recording device has no dedicated network to connect to; by adding a storage module, sufficient space is guaranteed for offline collection; the network status is detected, and the offline collected conference audio is pushed to the cloud recognition service as soon as possible for recognition, ensuring the recognition efficiency as much as possible. It still fails to solve the problem of accurately and efficiently capturing and recognizing the voices of multiple speakers and reducing the impact of environmental noise on recognition accuracy. Therefore, in order to overcome these limitations, the present invention proposes an offline conference recording box and its voice recognition system. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an offline conference recording box and a speech recognition system thereof, which solves the problem of accurately and efficiently capturing and recognizing the voices of multiple speakers and reducing the impact of environmental noise on recognition accuracy.

[0006] To achieve the above object, the present invention provides the following technical solutions: A speech recognition system for an offline conference recording box, comprising a speech receiving module, a speech processing module, a data management module and an interactive module; The voice receiving module is used to capture the audio signal from the microphone array to form audio data, identify the valid audio frame based on the energy value and frequency characteristics of the audio signal, and calculate the comprehensive energy value of the valid audio frame to obtain the length of the silence period, determine whether to perform early warning feedback according to the duration of the silence period, and dynamically adjust the microphone gain according to the energy value of the audio signal; The speech processing module is used to process the captured audio data by denoising the original speech audio signal in the audio data, extracting audio features, and performing voiceprint recognition through cluster analysis of the audio features to obtain a sound source separation signal marked with voiceprint information; The data management module is used to perform independent speech recognition based on the sound source separation signal, and organize and manage the speech data, which includes audio data, log information, speech recognition model, and speech recognition data; The interactive module is used to provide an operation interface, and according to the user's command input, the user can view the status of the offline meeting record box and the voice recognition results through the operation interface.

[0007] Specifically, the voice receiving module includes an audio capturing unit and an audio optimizing unit; The audio capture unit is configured with a speech detection strategy, which is used to identify valid speech activities in the environment and determine whether the audio contains valid speech content by analyzing the energy changes and frequency characteristics of the audio signal; The audio optimization unit is configured with a dynamic adjustment strategy, which is used to automatically adjust the gain setting of the microphone according to the energy value of the audio signal in the conference environment.

[0008] Specifically, the specific steps of the voice detection strategy include: Capture multi-channel ambient audio signals through a microphone array, detect audio energy changes, and calculate the energy value of each frame of audio signals; Configure an energy threshold. If the energy value of an audio frame exceeds the energy threshold, the audio frame is determined to contain voice activity. Otherwise, the audio frame is determined to contain no voice activity. The audio frame containing voice activity is marked as potentially valid, and the audio frame not containing voice activity is marked as potentially invalid. Obtain the time domain signal of the audio frame marked as potentially valid, perform frequency feature analysis, use fast Fourier transform to analyze the spectrum feature of the audio signal, and obtain the spectrum signal where the audio frame is located; Configure the spectrum threshold, including the upper spectrum threshold and the lower spectrum threshold. If the spectrum signal of the audio frame is between the upper spectrum threshold and the lower spectrum threshold, the spectrum signal of the audio frame is determined to be in the speech frequency range. Otherwise, the spectrum signal of the audio frame is determined to be not in the speech frequency range. The audio frame that is in the speech frequency range is marked as valid, and the audio frame that is not in the speech frequency range is marked as invalid.

[0009] Specifically, the specific implementation steps of the voice detection strategy also include: Configure a sliding window and calculate the comprehensive energy value of the audio frames marked as valid in the sliding window. The comprehensive energy value calculation formula is: ; in, It is The comprehensive energy value of all audio frames marked as valid in the sliding window at frame time, is the size of the sliding window, It is The first The energy value of the audio frame marked as valid. is the number of audio frames marked as valid in the sliding window; Configure the comprehensive threshold and standby threshold to determine the comprehensive energy state of the audio frame in the sliding window and the length of the silent period. If the comprehensive energy value of the sliding window is less than the comprehensive threshold, the sliding window is determined to be in a silent period, otherwise the sliding window is determined to be in a non-silent period; continuously detect the length of the silent period. If the length of the silent period exceeds the standby threshold, the offline meeting recording box automatically enters low power consumption mode and provides early warning feedback.

[0010] Specifically, the specific steps of the dynamic adjustment strategy include: By analyzing the difference between the energy value of the current audio signal and the energy value of the standard audio signal, the current ambient volume is determined and the gain is adjusted, namely: ; in, It is The frame gain, It is The energy value of the audio signal of the frame, is the energy value of the standard audio signal, is the minimum gain, is the maximum gain, is the gain adjustment coefficient, is the signal-to-noise ratio adjustment coefficient, It is The signal-to-noise ratio of the audio signal of the frame; A nonlinear gain adjustment factor is introduced to adaptively optimize the gain adjustment rate. The calculation formula of the gain adjustment rate is: ; in, It is The frame gain adjustment rate, It is The frame gain, It is The frame gain, is the time interval for gain adjustment, is the gain change adjustment coefficient, is an index of the sensitivity of control gain changes.

[0011] Specifically, the speech processing module includes a noise suppression unit and a feature extraction unit; The noise suppression unit is equipped with an adaptive filtering strategy, which is used to monitor the noise level in the audio signal in real time and remove background noise by dynamically adjusting the parameters of the filter; The feature extraction unit is configured with a deep recognition strategy, which is used to separate the sound source of the audio signal and extract audio features. The speaker in the audio signal is clustered and identified by clustering analysis of the extracted audio features and voiceprint recognition.

[0012] The specific steps of the adaptive filtering strategy include: Combining the optimized audio signals collected by the microphone array into an audio vector signal; Get the audio vector signal segment of the silent period and make a preliminary estimate of the background noise intensity, namely: ; in, is the estimated background noise intensity, is the time length of the audio vector signal segment during the silence period, is the start time frame of the audio vector signal segment during the silence period, is the number of microphones, is the frequency domain transform function, It is The microphone collects the Frame optimized audio signal; Initialize the filter parameters according to the estimated background noise intensity; when the silence period ends, recursively update the estimated value of the background noise intensity, that is: ; in, After the silent period, The estimated background noise intensity of the frame, is the smoothing factor, After the silent period, The estimated background noise intensity of the frame, After the silent period, The microphone collects the Frame audio signal, is the number of microphones, is the frequency domain transform function.

[0013] Specifically, the specific steps of the adaptive filtering strategy also include: The filter parameters are updated according to the updated background noise intensity, namely: ; in, and After the silent period, Frame and The filter parameters of the frame, and After the silent period, Frame and The estimated background noise intensity of the frame, is the parameter suppression factor; Based on the filter parameters, the audio vector signal is denoised by the minimum mean square error filter, and the audio signals after adaptive filtering of each microphone are weighted and synthesized to obtain the denoised audio signal, that is: ; in, is the first Frame audio signal, It is The weighting coefficients of the microphone audio signals, is the number of microphones, It is The filter parameters of the frame, It is The microphone collects the Frame audio signal, is the minimum mean square error filter function.

[0014] Specifically, the specific steps of the depth recognition strategy include: Acquire the denoised audio signal, perform frame processing on the audio signal to form a short-time frame audio signal, apply a window function to each short-time frame audio signal to perform windowing processing, and smooth the edge of the short-time frame audio signal; Performing Fourier transform on each short-time frame audio signal to obtain a spectrum signal, and performing sound source signal separation based on the spectrum signal to obtain a sound source separation signal set of each short-time frame audio signal; Extract audio features from each sound source separation signal in the short-time frame audio signal, and perform feature fusion on the audio features to form an audio feature vector; Based on the extracted audio feature vectors, a clustering algorithm is used to perform preliminary clustering of the sound source separation signal. When a new short-time frame audio signal appears, an incremental clustering algorithm is used to perform real-time clustering of the newly added audio feature vectors and update the existing clustering categories.

[0015] Specifically, the specific steps of the depth recognition strategy also include: Configure a new threshold, monitor the number of cluster categories in real time, and calculate the number of additional categories of the number of cluster categories of each newly added short-time frame audio signal. If the number of additional categories of the newly added short-time frame audio signal is greater than the new threshold, the category discrimination detection is triggered, otherwise clustering continues; Perform category discrimination detection, configure clustering thresholds, and calculate the discrimination between cluster categories in real time through the distance between clusters and the internal density of clusters. When the discrimination is less than the clustering threshold, categories are merged. Otherwise, the division of cluster categories is saved and abnormal warnings are issued. A voiceprint library is established, and the feature vector of each cluster category is matched with the pre-established voiceprint library. If the match is successful, it is marked as the corresponding speaker identity. If the match fails, a new temporary identity is assigned to the cluster category and further updated in subsequent processing. Each short-time frame audio signal is output according to the recognition result, and the sound source separation signal in the short-time frame audio signal is speaker marked.

[0016] An offline conference recording box includes a processor unit, a memory unit, a storage unit, a microphone array, a power management unit and an interactive interface; The processor unit is used to calculate and analyze the captured audio signals, including voice activity detection, noise suppression, feature extraction, voiceprint recognition, sound source separation, and speech recognition; The memory unit is used to store real-time data, including audio cache, audio frames being processed, noise estimation, short-time frame audio signals, speaker tags, speech recognition models, speaker feature libraries, audio feature vectors, and clustering information; The storage unit is used to store original audio data, processed audio features, voiceprint data, speech recognition results, and log information; The microphone array is used to receive audio signals in the conference environment, provide audio input, and optimize the audio signal through voice activity detection and dynamic gain adjustment; The power management unit is used to optimize the energy consumption of the offline conference recording box. It automatically enters low-power mode during silent periods or when there is no voice activity, and adjusts power consumption in real time based on voice activity detection feedback; The interactive interface is used to provide a user interactive operation interface through the touch screen to display the device status and voice recognition results.

[0017] Beneficial effects of the present invention: 1. Through adaptive filtering and deep recognition strategies, it can remove environmental noise in real time and significantly improve the quality of speech signals. In particular, it can effectively separate the audio signals of different speakers in complex environments and perform independent speech recognition, ensuring the accuracy of speech recognition and reducing recognition errors caused by background noise or cross-talk.

[0018] 2. By extracting and clustering the unique audio features of each speaker, different speakers can be effectively distinguished, so that the identity of the speaker can be accurately marked in the meeting minutes, avoiding the problem of confusing different speakers and improving the reliability of the meeting minutes.

[0019] 3. Through dynamic adjustment strategies, the microphone gain can be automatically adjusted according to the real-time environmental volume and signal quality, which not only reduces manual intervention, but also ensures that clear audio signals can be obtained when the volume fluctuates greatly. And it automatically enters a low-power state through silent period detection, effectively extending the use time of the device. At the same time, it automatically resumes normal working state when audio is detected to be restored, improving intelligence and energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a structural diagram of a speech recognition system for an offline conference recording box; Figure 2 A flowchart of the specific steps of the speech detection strategy; Figure 3 is a flowchart of the specific steps of the adaptive filtering strategy; Figure 4 A flowchart of the specific steps of the deep identification strategy; Figure 5 A schematic diagram of the structure of an offline meeting recording box. DETAILED DESCRIPTION

[0021] See also Figure 1,This embodiment introduces a speech recognition system for an offline conference recording box, including a speech receiving module, a speech processing module, a data management module and an interactive module; The voice receiving module is used to capture the audio signal from the microphone array to form audio data, identify the valid audio frame based on the energy value and frequency characteristics of the audio signal, and calculate the comprehensive energy value of the valid audio frame to obtain the length of the silence period, determine whether to perform early warning feedback according to the duration of the silence period, and dynamically adjust the microphone gain according to the energy value of the audio signal; In this embodiment, the voice receiving module is responsible for capturing audio signals in the conference environment and efficiently receiving audio data from all directions by connecting to a microphone array. The module uses voice activity detection and dynamic volume adjustment technology to automatically identify the moment of speech and adjust the gain of the recording device to reduce silence or process background noise, ensuring that the voice signal meets expectations in terms of clarity and loudness. When the system detects audio quality issues, it provides warnings through interactive modules or audio feedback to ensure that meeting records are not affected.

[0022] Preferably, the voice receiving module includes an audio capturing unit and an audio optimizing unit; The audio capture unit is configured with a voice detection strategy, which is used to identify valid voice activities in the environment. By analyzing the energy changes and frequency characteristics of the audio signal, it determines whether the audio contains valid voice content, realizes effective voice triggering and acquisition, avoids interference from irrelevant noise, and ensures that valid voice data is always recorded during the recording time period.

[0023] The audio optimization unit is equipped with a dynamic adjustment strategy, which is used to automatically adjust the microphone gain setting according to the energy value of the audio signal in the conference environment, avoiding distortion or attenuation of the recording signal in high noise or far away from the sound source, achieving a quick response to volume fluctuations in the environment, and optimizing the clarity and audibility of the audio signal to improve the overall sound quality.

[0024] See also Figure 2 Preferably, the specific steps of the voice detection strategy include: The multi-channel ambient audio signal is captured by the microphone array, the audio energy change is detected, and the energy value of each frame of the audio signal is calculated, that is: ; in, For the The energy value of the audio signal of the frame, It is Frame No. The audio signal of sampling points, The value range is {0,1,..., }, is the frame length, ranging from [256, 2048], depending on the required time resolution and spectral resolution; it reveals the strength of the sound signal through the change of energy value, effectively quantifies the strength of the audio signal, and facilitates the subsequent judgment of whether the audio frame contains a speech signal.

[0025] Configure the energy threshold. If the energy value of the audio frame exceeds the energy threshold, the audio frame is determined to contain voice activity. Otherwise, the audio frame is determined to contain no voice activity. At this time, the offline conference recording box is in a silent period. The audio frames containing voice activity are marked as potentially valid, and the audio frames not containing voice activity are marked as potentially invalid. This is to distinguish audio frames with or without voice, in preparation for subsequent frequency analysis.

[0026] Get the time domain signal of the audio frame marked as potentially valid, perform frequency feature analysis, and use fast Fourier transform to analyze the spectral characteristics of the audio signal, namely: ; in, Spectrum signal, It is The time domain signal of the frame, is the time domain signal length, is a window function used to improve the resolution of the spectrum. is frequency; so that the frequency components of the audio signal can be captured, thereby identifying the characteristics of the speech signal within a specific frequency range.

[0027] Configure the spectrum threshold, including the upper spectrum threshold and the lower spectrum threshold. If the spectrum signal of the audio frame is between the upper spectrum threshold and the lower spectrum threshold, the spectrum signal of the audio frame is determined to be in the voice frequency range. Otherwise, the spectrum signal of the audio frame is determined to be not in the voice frequency range. At this time, the offline conference recording box is in a silent period. The audio frames that are in the voice frequency range are marked as valid, and the audio frames that are not in the voice frequency range are marked as invalid.

[0028] Configure a sliding window, calculate the comprehensive energy value of the audio frames marked as valid in the sliding window, and determine whether the sliding window is in a silent period based on the comprehensive energy value of the sliding window. The comprehensive energy value calculation formula is: ; in, It is The comprehensive energy value of all audio frames marked as valid in the sliding window at frame time, is the size of the sliding window, that is, the number of audio frames contained in the window, which depends on the set sliding window length. It is The first The energy value of the audio frame marked as valid. It is the number of audio frames marked as valid in the sliding window. By configuring the sliding window, the energy values ​​of valid audio frames within a certain time range are aggregated. By calculating the comprehensive energy values ​​of all valid audio frames in the window, it can be determined whether there is sufficient voice activity in the time period.

[0029] Configure the comprehensive threshold and standby threshold to determine the comprehensive energy state of the audio frames in the sliding window and the length of the silent period. If the comprehensive energy value of the sliding window is less than the comprehensive threshold, the sliding window is determined to be in a silent period, otherwise the sliding window is determined to be in a non-silent period; to help detect the time period without voice activity, and further provide a basis for the device to enter low power consumption mode or continue standby.

[0030] The length of the silent period is continuously detected. If the length of the silent period exceeds the standby threshold, the offline meeting recording box automatically enters low power consumption mode and provides early warning feedback to notify the user or administrator of the current device status change, optimize the device's energy management, and avoid unnecessary energy consumption. Otherwise, no operation is performed.

[0031] Preferably, the specific steps of the dynamic adjustment strategy include: When the end of the silence period is detected, the current ambient volume is determined by analyzing the difference between the energy value of the current audio signal and the energy value of the standard audio signal, and the gain is adjusted, that is: ; in, It is The frame gain, It is The energy value of the audio signal of the frame, is the energy value of the standard audio signal, is the minimum gain, is the maximum gain, is the gain adjustment coefficient, the value range is (0,1), The specific value is determined by experimental tests of various audio signals in different environments. In a quiet indoor environment, A value between 0.3 and 0.5 usually achieves good results; in noisy outdoor environments, The value between 0.6 and 0.8 is more suitable. In practical applications, you can select an appropriate value within the range according to the specific audio type and usage scenario. value to achieve the best gain adjustment effect. is the signal-to-noise ratio adjustment coefficient, with a value range of (0,1), which is used to control the effect of the signal-to-noise ratio on the gain. The value is determined according to the signal-to-noise ratio of the audio signal. When the signal-to-noise ratio (SNR) of the audio signal is greater than 20dB, in order to avoid excessive noise enhancement, The value can be selected from 0.2 to 0.3; when the signal-to-noise ratio (SNR) is less than 10dB, in order to better improve the signal strength, The value can be selected from 0.6 to 0.8. In practical applications, the appropriate value can be selected in the range of (0,1) according to the real-time detected audio signal noise ratio. value to achieve effective control of the impact on the signal-to-noise ratio, thereby optimizing the gain adjustment effect. It is The signal-to-noise ratio of the audio signal of the frame; by detecting the end of the silent period, it is determined when to start the gain adjustment of the audio signal. The end of the silent period marks the beginning of external sound activity or change. By comparing the energy value of the current audio signal with the energy value of the standard audio signal, the gain value is calculated to control the amplification degree of the audio signal. In the gain adjustment formula, the minimum gain value and the maximum gain value limit the range of gain adjustment, so that the gain adjustment is neither excessive nor insufficient.

[0032] A nonlinear gain adjustment factor is introduced to adaptively optimize the gain adjustment rate to avoid over-adjustment. The calculation formula of the gain adjustment rate is: ; in, It is The frame gain adjustment rate, It is The frame gain, It is The frame gain, is the time interval for gain adjustment, is the gain change adjustment coefficient, with a value range of (0,1), which is used to control the sensitivity of gain change. The value is determined based on the requirements for the naturalness of audio gain changes and the response speed. When the naturalness of audio gain changes is required to be high and the gain change is allowed to be relatively slow, The value can be selected from 0.1 to 0.3; when the gain needs to respond quickly to audio signal changes and the naturalness of audio gain changes is slightly lower, The value can be selected from 0.6 to 0.8. In practical applications, the appropriate value can be selected in the range of (0,1) according to the specific audio application scenario and the requirements for audio quality. value to balance the naturalness and response speed of gain changes. is the exponent of the control gain change sensitivity, which is a positive real number. The specific value is determined according to the type and characteristics of the audio signal. For a stable speech signal, in order to make the gain change smoothly, The value can be selected from 1 to 2. For music signals with obvious rhythm changes, in order to make the gain better follow the rhythm changes, The value can be selected from 3 to 4. In practical applications, an appropriate positive real number can be selected according to the specific characteristics of the audio signal. The gain adjustment speed is controlled by the nonlinear gain adjustment factor to avoid unnatural audio fluctuations caused by excessively fast gain changes.

[0033] The speech processing module is used to process the captured audio data by denoising the original speech audio signal in the audio data, extracting audio features, and performing voiceprint recognition through cluster analysis of the audio features to obtain a sound source separation signal marked with voiceprint information; In this embodiment, the voice processing module further processes the captured audio signal, including denoising, feature extraction, etc. The background noise is removed by using a noise suppression algorithm, and feature data is extracted from the original voice audio signal through technologies such as short-time Fourier transform and Mel frequency cepstral coefficients for voiceprint recognition.

[0034] Preferably, the speech processing module includes a noise suppression unit and a feature extraction unit; The noise suppression unit is equipped with an adaptive filtering strategy, which is used to monitor the noise level in the audio signal in real time. By dynamically adjusting the parameters of the filter, the background noise can be effectively removed to improve the clarity of the audio signal. Through adaptive filtering technology, the filter configuration can be automatically adjusted according to the background noise level in the current environment, thereby reducing unnecessary noise interference while maintaining the quality of the voice signal.

[0035] The feature extraction unit is configured with a deep recognition strategy, which is used to perform feature extraction and voiceprint recognition on audio signals. By performing real-time cluster analysis on the extracted audio features, the speaker's identity can be automatically identified, thereby providing additional speaker identification information and optimizing the accuracy of meeting records.

[0036] See also Figure 3 Preferably, the specific steps of the adaptive filtering strategy include: The optimized audio signals collected by the microphone array are combined into an audio vector signal, which is expressed as ,in, is the number of microphones, It is The microphone collects the Frame optimized audio signal; Get the audio vector signal segment of the silent period and make a preliminary estimate of the background noise intensity, namely: ; in, is the estimated background noise intensity, is the time length of the audio vector signal segment during the silence period, is the start time frame of the audio vector signal segment during the silence period, is the number of microphones, is the frequency domain transform function, It is The microphone collects the The optimized audio signal of the frame; by selecting audio segments during the silent period, calculating and averaging the power spectra of multiple microphones, and performing time averaging to reduce instantaneous fluctuations, the spectrum and intensity estimation of the background noise are obtained as a preliminary estimate of the background noise intensity, providing a reference for noise suppression of the adaptive filter.

[0037] According to the estimated background noise intensity, the filter parameters are initialized. The initial values ​​of the filter parameters are proportional to the background noise intensity. The initial values ​​of the filter parameters are expressed as: ; in, are the initial values ​​of the filter parameters, is a normal number with a value range of (0, 1], The specific value is determined according to different application scenarios and the requirements for the initial performance of the filter. For example, in some scenarios where the initial audio signal quality is required to be high and the filter is expected to be relatively stable in the initial stage, such as the initial establishment stage of voice communication, The value can be selected from 0.1 to 0.3. In some scenarios where the background noise is strong and the filter needs to have strong noise suppression capabilities in the initial stage, such as voice collection in a noisy industrial environment, It can be selected from 0.7 to 0.9. In practical applications, according to the specific usage scenario and the expectation of filter performance, a suitable value can be selected in the range of (0, 1]. The value is used to realize the reasonable initialization of the filter parameters, while ensuring the filter performance, avoiding the adverse effects caused by excessive or too small noise estimation; is the estimated background noise intensity, is the maximum allowed value of the noise estimate to ensure that the filter does not overreact due to excessive noise estimates.

[0038] When the silence period ends and the speech signal begins to appear, the estimated value of the background noise intensity is updated recursively, that is: ; in, After the silent period, The estimated background noise intensity of the frame, is a smoothing factor used to control the response speed of the noise estimate to new noise samples, and its value range is [0, 1]. The value is determined according to the rate of noise change. In an environment where the noise changes slowly, in order to make the background noise intensity estimate more stable, The value can be selected from 0.8 to 1. In an environment where the noise changes rapidly, in order to make the estimated value keep up with the noise changes in time, The value can be selected from 0.1 to 0.3. In practical applications, the appropriate value can be selected in the range of [0, 1] according to the noise changes monitored in real time. value to effectively update the estimated value of background noise intensity, improve the filter's adaptive ability, and better cope with the changing environmental noise; After the silent period, The estimated background noise intensity of the frame, After the silent period, The microphone collects the Frame audio signal, is the number of microphones, It is a frequency domain transformation function that recursively updates the noise estimate to cope with the ever-changing environmental noise and improve the filter's adaptive ability.

[0039] The filter parameters are updated according to the updated background noise intensity, namely: ; in, and After the silent period, Frame and The filter parameters of the frame, and After the silent period, Frame and The estimated background noise intensity of the frame, is a parameter suppression factor, which is used to control the effect of background noise intensity changes on filter parameter changes. Its value range is [0, 1]. The specific value is determined through experiments in various noise environments. In a stable white noise environment, in order to ensure the stability of the filter, The value can be selected from 0.1 to 0.3. In the scenario where the noise changes rapidly, in order to enable the filter to adapt to the noise changes in time, The value can be selected from 0.7 to 0.9. In practical applications, the appropriate value can be selected in the range of [0, 1] according to the specific characteristics of the noise environment. The value is adjusted to balance the filter's response speed to noise changes and the stability of the filtering effect. By updating the filter parameters, the noise suppression effect of the adaptive filter is improved, thereby effectively reducing the impact of noise in a dynamic environment.

[0040] Based on the filter parameters, the audio vector signal is denoised by the minimum mean square error filter, and the audio signals after adaptive filtering of each microphone are weighted and synthesized to obtain the denoised audio signal, that is: ; in, is the first Frame audio signal, It is The weighting coefficient of the microphone audio signal ranges from [0, 1]. The value is determined based on the quality of the audio signal collected by each microphone. For microphones with good signal quality and little noise interference, You can choose 0.7 to 0.9; for microphones with poor signal quality and more noise, The value can be selected from 0.1 to 0.3. In practical applications, a suitable weighting coefficient can be assigned to each microphone in the range of [0, 1] according to the actual signal conditions of each microphone in the microphone array. , in order to achieve effective weighted synthesis of audio signals after adaptive filtering of multiple microphones, and improve the quality of audio signals after denoising; is the number of microphones, It is The filter parameters of the frame, It is The microphone collects the Frame audio signal, is the minimum mean square error filter function, used to represent the filter parameters Acts on audio signals Through multi-step dynamic noise estimation and filter adjustment, effective denoising of audio signals is achieved, adapting to changes in environmental noise and improving the clarity of speech signals, so that the filter strikes a balance between noise suppression and speech retention to achieve the best denoising effect.

[0041] See also Figure 4 , preferably, the specific steps of the deep recognition strategy include: Get the denoised audio signal, configure the frame signal length and frame shift, divide the audio signal into frames, and overlap adjacent frames to form short-time frames, that is: ; in, For the A short-time frame of audio signal, After denoising, Frame audio signal, is the frame shift, which is used to control the overlap degree of adjacent frames. is the frame signal length, which is used to ensure sufficient time resolution while avoiding excessive smoothing or information loss caused by too long or too short windows; Applying a window function to each short-time frame audio signal for windowing processing to reduce the frame boundary effect and smooth the edge of the short-time frame audio signal; Perform Fourier transform on each short-time frame audio signal, convert it into the frequency domain, obtain the spectrum signal, separate the sound source signal based on the spectrum signal, extract multiple independent speaker sound source separation signals, and obtain the sound source separation signal set of each short-time frame audio signal ,in It is The first short-time frame of the audio signal Source separation signal; Extract audio features from each sound source separation signal in the short-time frame audio signal. The audio features include Mel frequency cepstral coefficient features, perceptual linear prediction features, pitch features, formant features, and spectral envelope features. The audio features are fused to form an audio feature vector. Based on the extracted audio feature vector, a clustering algorithm is used to perform preliminary clustering on the sound source separation signal to distinguish the identities of different speakers, and each cluster category represents a speaker; Configure the distance threshold. When a new short-time frame audio signal appears, use the incremental clustering algorithm to cluster the newly added audio feature vector in real time and update the existing cluster category. If the distance between the audio feature vector in the new short-time frame audio signal and the center of the existing cluster category is greater than the distance threshold, a new cluster category is generated. Otherwise, it is classified into the cluster category with the closest distance to it. Configure a new threshold, monitor the number of cluster categories in real time, and calculate the number of additional categories of the number of cluster categories of each newly added short-time frame audio signal. If the number of additional categories of the newly added short-time frame audio signal is greater than the new threshold, the category discrimination detection is triggered, otherwise clustering continues; Perform category discrimination detection, configure clustering threshold, and calculate the discrimination between cluster categories in real time through the distance between clusters and the internal density of clusters. When the discrimination is less than the clustering threshold, merge the categories. Otherwise, save the division of cluster categories and issue an abnormal warning. The discrimination calculation formula is as follows: ; in, Is the cluster category and The degree of differentiation, Is the cluster category and The center distance, and They are cluster categories and The local clustering metric of is used to measure the similarity between samples in a cluster using the K nearest neighbor algorithm. and They are cluster categories and The relative density metric of strengthens the influence of neighbor samples on cluster density by inversely weighting the distance between samples. Specifically: ; ; in, Is the cluster category The relative density measure of , which measures the relative concentration of samples within the cluster, Is the cluster category Mid-frequency feature vector and The distance Is the cluster category The local clustering measure of is the audio feature vector In the clustering category The K nearest neighbor score in is used to measure the audio feature vector and cluster category The similarity of other audio feature vectors in the cluster is used to measure the similarity of samples within the cluster. A higher local clustering measure value indicates that the similarity of samples within the cluster is higher.

[0042] A voiceprint database is established in advance, and the target population is selected according to the target scenario of the offline conference box application. The audio data of the target population is obtained, including daily conversations, text reading, and free speech. The audio data of the target population is unified in audio format and denoised. The audio features of the audio data are extracted, and the feature vectors of each cluster category of the audio features of the target population are obtained through cluster analysis of the audio features. The voiceprint identity is marked and stored in the voiceprint database. The feature vector of each clustering category is matched with the pre-established voiceprint library. If the match is successful, it is marked as the corresponding speaker identity. If the match fails, a new temporary identity is assigned to the category and further updated in subsequent processing. Each short-time frame audio signal is output according to the recognition result, and the sound source separation signal in the short-time frame audio signal is speaker marked.

[0043] The data management module is used to perform speech recognition based on the sound source separation signal and organize and manage speech data, which includes audio data, log information, speech recognition model, and speech recognition data; In this embodiment, the data management module performs speech recognition based on the speech feature data, and effectively manages the recognition results, original audio data, log information, etc. By further recognizing the audio, an accurate text record is generated, and the recognized data is stored and organized into a structured format, including the original audio file, transcribed text, recognition log, speaker identification, etc. In addition, it also includes a management and update mechanism for the speech recognition model so that the system can be regularly trained or tuned according to user needs to improve recognition accuracy.

[0044] Preferably, the data management module includes a speech recognition unit and a speech storage unit, wherein the speech recognition unit is configured with a cluster recognition strategy, and the cluster recognition strategy is used to perform sound source separation signal recognition according to the marked speaker category grouping, and generate speech text data through an offline speech recognition model; The voice storage unit is configured with an associated storage strategy, which is used to associate the recognized text records with the original audio data according to the voice recognition results, speaker identification and audio metadata, and manage the access and update of voice data in a structured manner.

[0045] Preferably, the specific steps of the cluster identification strategy include: After the silence period ends, when the effective audio signal begins to be received, the sound source separation signal in each short-time frame audio signal is processed, and the speaker identity of each sound source separation signal is marked to perform independent speech recognition for each sound source, thereby realizing parallel recognition of sound source separation signals of multiple speakers; When a new speaker category appears, the pre-trained speech recognition model built into the offline meeting record box is reused to convert text data; the speech recognition models include hidden Markov model, recurrent neural network, n-gram model and long short-term memory network language model; Generate meeting records in real time based on the text data of each identified speaker, and add speaker identification to each identified text data in the meeting record. The speaker identification includes specific tags in the voiceprint library and automatically generated temporary tags. Based on the recognition results, the text data corresponding to each speaker is merged into a complete meeting document in sequence, and each text is arranged in chronological order and matches the corresponding speaker information; In the conference document, a timestamp is added to each speaker's speech segment to ensure that the order and time of the speeches can be traced. The timestamp information includes the start and end time of each speech.

[0046] Preferably, the specific steps of associating the storage strategy include: Automatically extract metadata information from meeting documents, including meeting time, location, list of participants, and agenda, to facilitate subsequent rapid retrieval and classification management of voice records; Based on the speaker ID in the speech recognition results, a multi-dimensional index is established to associate each speech segment with its corresponding speaker and related metadata, where each dimension represents an attribute, including speaker, timestamp, and topic, to facilitate subsequent efficient queries; Use a database management system to organize and store voice data and related information according to pre-defined patterns to ensure data integrity and consistency while supporting complex query operations; Implement regular data backup strategies, keep records of historical versions so that they can be restored to a specific state when needed, and set access control to protect sensitive information from unauthorized users; Provide API interface for external applications to call to achieve remote access and batch export.

[0047] The interactive module is used to provide an operation interface, and according to the user's command input, the user can view the status of the offline meeting record box and the voice recognition results through the operation interface.

[0048] In this embodiment, the interactive module provides a user-friendly interface to allow users to operate the offline conference record box and view the results. The interactive module interacts with the user through a touch screen, buttons or voice commands. The user can view the current meeting information, including the recording time, the current speaker, and the progress of voice recognition, view the transcription results in real time, and modify, confirm or mark the inaccurate parts. Automatically mark or provide feedback based on the recognition results, and can input commands through the interface, including starting, pausing, and stopping recording, and view relevant statistical information as needed, including the number of speakers, total duration, etc.

[0049] Example 2 See also Figure 5 , an offline conference recording box, including a processor unit, a memory unit, a storage unit, a microphone array, a power management unit and an interactive interface; The processor unit is used to calculate and analyze the captured audio signals, including voice activity detection, noise suppression, feature extraction, voiceprint recognition, sound source separation, and speech recognition. It supports real-time audio stream processing and speech recognition algorithms, can process large amounts of audio data in real-time and offline modes, generate accurate text records, and supports dynamic gain adjustment and noise suppression, automatically adapting to different conference environments to ensure audio quality; The memory unit is used to store real-time data, including audio cache, audio frames being processed, noise estimation, short-time frame audio signals, speaker tags, cached speech recognition models, speaker feature libraries, audio feature vectors, and clustering information; The storage unit is used to store raw audio data, processed audio features, voiceprint data, speech recognition results, and log information. It supports long-term storage of conference recordings to ensure that data can be stably saved without a network connection. It also supports regular updates and optimizations of speech recognition models to ensure continuous improvement in recognition accuracy. The microphone array is used to receive audio signals in the conference environment and provide high-quality audio input. Through multi-channel signal input, it can effectively achieve noise reduction processing, support voice activity detection and dynamic gain adjustment, so as to automatically adjust the recording quality when the noise changes to avoid signal attenuation or distortion; The power management unit is used to optimize the energy consumption of the offline conference recording box, especially in an environment without external power support, to ensure that the device can work for a long time. In the silent period or when there is no voice activity, the power system can automatically enter the low power mode and adjust the power consumption in real time according to the voice activity detection feedback; it also has intelligent battery management, and timely feedback the battery status to the user or administrator to prevent the normal operation of the device from being affected by low power. The interactive interface is used to provide a user interactive operation interface through a touch screen, displaying device status and voice recognition results for easy user operation and monitoring.

[0050] Working principle and its effect: An offline conference recording box and its speech recognition system realize efficient and accurate speech recognition and session recording functions. First, the microphone array is used to capture the audio signal in the environment, and the speech detection strategy is used to screen the effective speech. The speech content is identified and collected by analyzing the energy change of the audio signal. At the same time, based on the real-time environmental volume and signal strength, the gain setting of the microphone is dynamically adjusted to optimize the audio quality, and early warning feedback is provided when audio quality problems are encountered. When processing the audio signal, the background noise is detected in real time, and the environmental noise is removed by adaptive filtering to ensure a clear speech signal. Then, the audio signal is separated from the sound source and feature extracted, and the speaker is identified and clustered by deep learning technology. Finally, the speaker identity of each speech signal is annotated by voiceprint recognition technology. After the separated audio signal is speech recognized, the recognition results and speech data are stored and managed. Users can also enter instructions through the interactive module to view the status of the conference recording box and the speech recognition results.

[0051] Through denoising, sound source separation and deep recognition strategies, the impact of environmental noise and other interference is significantly reduced, thereby improving the accuracy of speech recognition. In a meeting with multiple participants, the system can accurately distinguish the voice of each speaker and recognize their speech content. Through real-time gain adjustment and silent period detection, the microphone settings can be adaptively adjusted according to the volume fluctuations of the meeting environment to ensure that the voice signal can be accurately captured in noisy or quiet environments. If there is a long silent period in the meeting, it will automatically enter low power mode to effectively save battery and system resources. Through voiceprint recognition and sound source separation, the speech of each speaker is marked and archived to avoid confusion between different people's speeches when viewing the meeting records later. In addition, intelligent voice log management provides structured conversation records, which is convenient for post-meeting review, search and archiving, and improves the efficiency and accuracy of meeting record management. This makes it extremely practical in conference recording, speech recognition and record management, especially in the scenario of multi-person voice interaction, which can achieve efficient, clear and accurate recording and management.

[0052] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A speech recognition system for an offline conference record box, characterized in that: It includes a voice receiving module, a voice processing module, a data management module and an interactive module; The voice receiving module is used to capture the audio signal from the microphone array to form audio data, identify the valid audio frame based on the energy value and frequency characteristics of the audio signal, and calculate the comprehensive energy value of the valid audio frame to obtain the length of the silence period, determine whether to perform early warning feedback according to the duration of the silence period, and dynamically adjust the microphone gain according to the energy value of the audio signal; The speech processing module is used to process the captured audio data by denoising the original speech audio signal in the audio data, extracting audio features, and performing voiceprint recognition through cluster analysis of the audio features to obtain a sound source separation signal marked with voiceprint information; The data management module is used to perform independent speech recognition according to the sound source separation signal, and organize and manage speech data, wherein the speech data includes audio data, log information, speech recognition model, and speech recognition data; The interactive module is used to provide an operation interface, and according to user command input, the user can view the status of the offline meeting record box and the voice recognition results through the operation interface.

2. The speech recognition system for an offline conference recording box as claimed in claim 1, characterized in that: The voice receiving module includes an audio capturing unit and an audio optimizing unit; The audio capture unit is configured with a voice detection strategy, which is used to identify valid voice activities in the environment and determine whether the audio contains valid voice content by analyzing the energy changes and frequency characteristics of the audio signal; The audio optimization unit is configured with a dynamic adjustment strategy, which is used to automatically adjust the gain setting of the microphone according to the energy value of the audio signal in the conference environment.

3. The speech recognition system for an offline conference recording box as claimed in claim 2, characterized in that: The specific steps of the voice detection strategy include: Capture multi-channel ambient audio signals through a microphone array, detect audio energy changes, and calculate the energy value of each frame of audio signals; Configure an energy threshold. If the energy value of an audio frame exceeds the energy threshold, the audio frame is determined to contain voice activity. Otherwise, the audio frame is determined to contain no voice activity. The audio frame containing voice activity is marked as potentially valid, and the audio frame not containing voice activity is marked as potentially invalid. Obtain the time domain signal of the audio frame marked as potentially valid, perform frequency feature analysis, use fast Fourier transform to analyze the spectrum feature of the audio signal, and obtain the spectrum signal where the audio frame is located; Configure the spectrum threshold, including the upper spectrum threshold and the lower spectrum threshold. If the spectrum signal of the audio frame is between the upper spectrum threshold and the lower spectrum threshold, the spectrum signal of the audio frame is determined to be in the speech frequency range. Otherwise, the spectrum signal of the audio frame is determined to be not in the speech frequency range. The audio frame that is in the speech frequency range is marked as valid, and the audio frame that is not in the speech frequency range is marked as invalid.

4. The speech recognition system for an offline conference recording box as claimed in claim 3, characterized in that: The specific implementation steps of the voice detection strategy also include: Configure a sliding window and calculate the comprehensive energy value of the audio frames marked as valid in the sliding window. The comprehensive energy value calculation formula is: ； in, It is The comprehensive energy value of all audio frames marked as valid in the sliding window at frame time, is the size of the sliding window, It is The first The energy value of the audio frame marked as valid. is the number of audio frames marked as valid in the sliding window; Configure the comprehensive threshold and standby threshold to determine the comprehensive energy state of the audio frame in the sliding window and the length of the silent period. If the comprehensive energy value of the sliding window is less than the comprehensive threshold, the sliding window is determined to be in a silent period, otherwise the sliding window is determined to be in a non-silent period; continuously detect the length of the silent period. If the length of the silent period exceeds the standby threshold, the offline meeting recording box automatically enters low power consumption mode and provides early warning feedback.

5. The speech recognition system for an offline conference record box as claimed in claim 2, characterized in that: The specific steps of the dynamic adjustment strategy include: By analyzing the difference between the energy value of the current audio signal and the energy value of the standard audio signal, the current ambient volume is determined and the gain is adjusted, that is: ； in, It is The gain of the frame, It is The energy value of the audio signal of the frame, is the energy value of the standard audio signal, is the minimum gain, is the maximum gain, is the gain adjustment coefficient, is the signal-to-noise ratio adjustment coefficient, It is The signal-to-noise ratio of the audio signal of the frame; A nonlinear gain adjustment factor is introduced to adaptively optimize the gain adjustment rate. The calculation formula of the gain adjustment rate is: ； in, It is The frame gain adjustment rate, It is The gain of the frame, It is The gain of the frame, is the time interval for gain adjustment, is the gain change adjustment coefficient, is an index of the sensitivity of control gain changes.

6. The speech recognition system for an offline conference recording box as claimed in claim 1, characterized in that: The speech processing module includes a noise suppression unit and a feature extraction unit; The noise suppression unit is configured with an adaptive filtering strategy, which is used to monitor the noise level in the audio signal in real time and remove background noise by dynamically adjusting the parameters of the filter; The feature extraction unit is configured with a deep recognition strategy, which is used to separate the sound source of the audio signal and extract audio features. The speaker in the audio signal is clustered and recognized by performing cluster analysis and voiceprint recognition on the extracted audio features.

7. The speech recognition system for an offline conference record box as claimed in claim 6, characterized in that: The specific steps of the adaptive filtering strategy include: Combining the optimized audio signals collected by the microphone array into an audio vector signal; Get the audio vector signal segment of the silent period and make a preliminary estimate of the background noise intensity, namely: ； in, is the estimated background noise intensity, is the time length of the audio vector signal segment during the silence period, is the start time frame of the audio vector signal segment during the silence period, is the number of microphones, is the frequency domain transform function, It is The microphone collects the Frame optimized audio signal; Initialize the filter parameters according to the estimated background noise intensity; when the silence period ends, recursively update the estimated value of the background noise intensity, that is: ； in, After the silent period, The estimated background noise intensity of the frame, is the smoothing factor, After the silent period, The estimated background noise intensity of the frame, After the silent period, The microphone collects the Frame audio signal, is the number of microphones, is the frequency domain transform function.

8. The speech recognition system for an offline conference record box as claimed in claim 7, characterized in that: The specific steps of the adaptive filtering strategy also include: The filter parameters are updated according to the updated background noise intensity, namely: ； in, and After the silent period, Frame and The filter parameters of the frame, and After the silent period, Frame and The estimated background noise intensity of the frame, is the parameter suppression factor; Based on the filter parameters, the audio vector signal is denoised by the minimum mean square error filter, and the audio signals after adaptive filtering of each microphone are weighted and synthesized to obtain the denoised audio signal, that is: ； in, is the first Frame audio signal, It is The weighting coefficients of the microphone audio signals, is the number of microphones, It is The filter parameters of the frame, It is The microphone collects the Frame audio signal, is the minimum mean square error filter function.

9. The speech recognition system for an offline conference record box as claimed in claim 6, characterized in that: The specific steps of the depth recognition strategy include: Acquire the denoised audio signal, perform frame processing on the audio signal to form a short-time frame audio signal, apply a window function to each short-time frame audio signal to perform windowing processing, and smooth the edge of the short-time frame audio signal; Performing Fourier transform on each short-time frame audio signal to obtain a spectrum signal, and performing sound source signal separation based on the spectrum signal to obtain a sound source separation signal set of each short-time frame audio signal; Extract audio features from each sound source separation signal in the short-time frame audio signal, and perform feature fusion on the audio features to form an audio feature vector; Based on the extracted audio feature vectors, a clustering algorithm is used to perform preliminary clustering of the sound source separation signal. When a new short-time frame audio signal appears, an incremental clustering algorithm is used to perform real-time clustering of the newly added audio feature vectors and update the existing clustering categories.

10. The speech recognition system for an offline conference record box as claimed in claim 9, characterized in that: The specific steps of the depth recognition strategy also include: Configure a new threshold, monitor the number of cluster categories in real time, and calculate the number of additional categories of the number of cluster categories of each newly added short-time frame audio signal. If the number of additional categories of the newly added short-time frame audio signal is greater than the new threshold, the category discrimination detection is triggered, otherwise clustering continues; Perform category discrimination detection, configure clustering thresholds, and calculate the discrimination between cluster categories in real time through the distance between clusters and the internal density of clusters. When the discrimination is less than the clustering threshold, categories are merged. Otherwise, the division of cluster categories is saved and abnormal warnings are issued. A voiceprint library is established, and the feature vector of each cluster category is matched with the pre-established voiceprint library. If the match is successful, it is marked as the corresponding speaker identity. If the match fails, a new temporary identity is assigned to the cluster category and further updated in subsequent processing. Each short-time frame audio signal is output according to the recognition result, and the sound source separation signal in the short-time frame audio signal is speaker marked.

11. An offline conference recording box, which is based on a speech recognition system of an offline conference recording box according to any one of claims 1 to 10, characterized in that: It includes a processor unit, a memory unit, a storage unit, a microphone array, a power management unit and an interactive interface; The processor unit is used to calculate and analyze the captured audio signal, including voice activity detection, noise suppression, feature extraction, voiceprint recognition, sound source separation and speech recognition; The memory unit is used to store real-time data, including audio cache, audio frames being processed, noise estimation, short-time frame audio signals, speaker tags, speech recognition models, speaker feature libraries, audio feature vectors, and clustering information; The storage unit is used to store original audio data, processed audio features, voiceprint data, speech recognition results, and log information; The microphone array is used to receive audio signals in a conference environment, provide audio input, and optimize audio signals through voice activity detection and dynamic gain adjustment; The power management unit is used to optimize the energy consumption of the offline conference recording box, automatically enter a low power consumption mode during a silent period or when there is no voice activity, and adjust the power consumption in real time according to the voice activity detection feedback; The interactive interface is used to provide a user interactive operation interface through a touch screen to display device status and voice recognition results.

Images