Variable bit rate quantization speech recognition method for intelligent speech controller

Abstract

The invention relates to a variable bit rate quantization speech recognition method for an intelligent speech controller. The variable bit rate quantization speech recognition method is applied to a first end comprising a target sensing array. The first end dynamically collects noisy voice signals through the voice collection module, and when the signals are detected, time domain features and frequency domain features are collected through the time domain feature unit and the frequency domain feature unit respectively. The features are sent to a second end, the second end fuses the features to generate an enhanced speech signal, variable bit rate quantization coding is carried out, and then a text transcription result is output through a transformation decoding structure. The method realizes efficient and accurate speech recognition in a complex noise environment, has adaptive bit rate allocation capability, remarkably improves transmission efficiency and recognition robustness, and is suitable for scenes such as smart home and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech signal processing, and in particular relates to a variable bit rate quantized speech recognition method for an intelligent speech controller. Background Art

[0002] With the widespread application of intelligent voice control technology in daily life, such as smart home devices and voice assistants, higher requirements are placed on the performance of voice recognition systems.

[0003] Especially in noisy environments, background noise often interferes with the accurate acquisition and processing of speech signals, resulting in reduced recognition rates or misoperation. Traditional speech recognition methods typically use fixed bitrate encoding, which is difficult to adapt to audio clips of varying complexity, limiting transmission efficiency and recognition robustness.

[0004] Furthermore, single feature extraction methods struggle to cope with complex noisy environments, risking signal distortion and information loss. To address these issues, a speech recognition technology that can dynamically adjust the bit rate and incorporate multi-dimensional feature processing is urgently needed to improve performance in complex scenarios. Summary of the Invention

[0005] The purpose of the present invention is to provide a variable bit rate quantized speech recognition method for an intelligent speech controller, aiming to solve the problem of insufficient speech recognition accuracy and transmission efficiency in noisy environments in the prior art.

[0006] The present invention achieves efficient noise suppression and signal optimization by dynamically collecting noisy speech signals at the first end, extracting time and frequency domain features using a target sensor array, and combining this with feature fusion and enhancement processing at the second end. Furthermore, by introducing adaptive bitrate quantization coding technology, time frames are dynamically allocated based on the complexity of the speech signal, improving data transmission efficiency and storage utilization while ensuring recognition robustness. Ultimately, accurate text transcription output is generated by transforming the decoding structure, meeting the needs of real-time voice control in complex scenarios such as smart homes, providing users with a more convenient and intelligent interactive experience.

[0007] Specifically, according to a variable bit rate quantized speech recognition method for an intelligent speech controller, the method includes: When a noisy speech signal is detected, the time domain features of the speech signal are collected according to the time domain feature units in the target sensor array; the frequency domain features of the speech signal are collected according to the frequency domain feature units in the target sensor array; the time domain features and the frequency domain features are sent to a second end connected to the first end, wherein the second end generates an enhanced speech signal according to the time domain features and the frequency domain features, and performs variable bit rate quantization encoding and text transcription output according to the enhanced speech signal.

[0008] Furthermore, convolution processing is performed on the coding unit whose time domain features change to obtain a first feature array, wherein the output points in the first feature array serve as time domain feature units; convolution processing is performed on the coding unit whose frequency domain features change to obtain a second feature array, wherein the output points of the second feature array serve as frequency domain feature units; and a target sensor array is constructed based on the first feature array and the second feature array.

[0009] Furthermore, the step of collecting time domain features of a speech signal based on the time domain feature units in the target sensor array includes: determining time domain change data generated by the first feature array based on the noisy speech signal, wherein the time domain change data includes the amplitude change of the coding unit in the first feature array; determining the feature strength of each time domain feature unit in the first feature array based on the time domain change data; and determining the time domain feature based on the feature strength.

[0010] Furthermore, the step of collecting frequency domain features of the speech signal based on the frequency domain feature units in the target sensor array includes: determining the frequency domain change data generated by the second feature array based on the noisy speech signal, wherein the frequency domain change data includes the spectrum amplitude of the coding unit in the second feature array; converting the spectrum amplitude of each of the frequency domain feature units into frequency domain features.

[0011] Furthermore, the second end generates an enhanced voice signal based on the time domain features and the frequency domain features, and performs variable bit rate quantization encoding and text transcription based on the enhanced voice signal, including: obtaining the time domain features and frequency domain features sent by the first end in real time; generating an enhanced voice signal based on the time domain features and the frequency domain features, and performing variable bit rate quantization encoding and text transcription based on the enhanced voice signal.

[0012] Furthermore, the step of generating an enhanced speech signal based on the time domain features and the frequency domain features includes: fusing the time domain features and the frequency domain features to generate fused features; processing the fused features through an encoder-decoder structure to obtain an enhanced speech signal.

[0013] Furthermore, the step of performing variable bit rate quantization encoding and text transcription based on the enhanced speech signal includes: allocating time frames according to the complexity of the enhanced speech signal to generate variable bit rate quantization encoding; inputting the variable bit rate quantization encoding into a transform decoder to generate corresponding text transcription output.

[0014] Furthermore, the step of performing variable bit rate quantization encoding based on the enhanced speech signal includes: extracting features of the enhanced speech signal through a multi-scale encoder to obtain a first scale feature and a second scale feature; performing importance analysis on the first scale feature and the second scale feature respectively to obtain a first importance distribution map and a second importance distribution map; and calculating a relative importance distribution map based on the first importance distribution map and the second importance distribution map, wherein the relative importance distribution map is calculated according to the formula calculate, represents the first importance distribution map, represents the second importance distribution map, Representing a relative importance distribution graph; determining a time frame allocation method based on the relative importance distribution graph, and generating a variable bit rate quantization code.

[0015] Furthermore, the step of determining the time frame allocation method based on the relative importance distribution diagram includes: generating a threshold value corresponding to the relative importance distribution diagram through one-dimensional convolution processing, wherein the threshold value is calculated according to the formula Calculation, TH represents the threshold, represents a one-dimensional convolution operation, Represents a relative importance distribution map; calculating a mask value based on the relative importance distribution map and the threshold, wherein the mask value is calculated by the formula m = I( - TH > 0) is determined, I represents the indicator function, when - When TH > 0, m is set to 1, otherwise it is set to 0; the time frame allocation method is determined according to the mask value, where when m is 1, a longer time frame is allocated, otherwise a shorter time frame is allocated, generating variable bit rate quantization coding.

[0016] Furthermore, the step of inputting the variable bit rate quantization code into a transform decoder to generate a corresponding text transcription output includes: converting the variable bit rate quantization code into an input vector and superimposing position information; processing the input vector through a multi-layer multi-directional attention mechanism and a feedforward network, wherein each layer of the multi-directional attention mechanism performs attention calculation on the current and previous input vectors; and further processing the attention calculation results through the feedforward network to generate the final text transcription output.

[0017] The present invention provides a variable bit rate quantization speech recognition method for an intelligent voice controller, which has achieved significant improvements and beneficial effects in response to the technical problems existing in the background technology. First, in response to the problem that the traditional method has a reduced recognition rate due to background noise interference in a noisy environment, the present invention dynamically collects time domain and frequency domain features through the target sensor array at the first end, and combines the signal enhancement technology at the second end to effectively suppress noise interference and improve the accuracy of speech recognition. Secondly, in response to the defects that fixed bit rate coding is difficult to adapt to audio clips of different complexities, wastes resources or has insufficient recognition accuracy, the present invention adopts adaptive bit rate quantization coding, dynamically allocates time frames according to the complexity of the voice signal, and optimizes transmission efficiency and storage requirements. Thirdly, in response to the problem that a single feature extraction method reduces recognition robustness due to signal distortion or information loss, the present invention integrates multi-dimensional features and generates accurate text transcription output through a transform decoding structure, thereby enhancing robustness and adaptability in complex noise environments. In summary, the present invention significantly improves the real-time performance and user experience of voice control in scenarios such as smart homes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flow chart of the variable bit rate quantized speech recognition method provided by the present invention; Figure 2 The second enhancement and processing flow chart provided by the present invention; Figure 3 A flow chart of the front-end speech enhancement structure and method provided by the present invention; Figure 4 A schematic diagram of the structure of a multi-scale encoder provided by the present invention; Figure 5 A schematic diagram of the structure of a multi-scale decoder provided by the present invention; Figure 6 This is a comparison table of speech recognition performance of different models provided by the present invention.

[0019] Reference numerals: A functional fiber preparation system 100 based on reaction site regulation, a primary reaction site processing unit 101, a secondary reaction site optimization unit 102, and a functionalization processing unit 103. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated. Thus, features specified as "first" or "second" may explicitly or implicitly include one or more of such features; and in the description of this application, unless otherwise specified, "plurality" means two or more.

[0022] In order to more clearly illustrate the technical solution of the present invention, the present invention is described in detail below in conjunction with specific embodiments, but this should not be construed as limiting the scope of protection of the present invention.

[0023] This embodiment provides an implementation of a variable bit rate quantized speech recognition method for an intelligent speech controller, such as Figure 1 As shown, targeting voice control needs in noisy environments such as smart homes, this paper focuses on the voice signal acquisition and feature extraction process to support subsequent enhancement and recognition. The method is applied to the first end, which includes a voice acquisition module with a target sensor array. This module dynamically acquires noisy input voice signals. The first end serves as the input unit for the entire system and is installed in a smart home environment, such as a voice control device in the living room. The first end includes a compact voice acquisition module with a built-in circuit board containing a target sensor array. The target sensor array consists of multiple high-sensitivity microphones and a signal processing chip, responsible for dynamically acquiring noisy input voice signals. For example, when a user issues the "turn off TV" command while the TV is running, the voice acquisition module uses the microphone array to capture the voice waveform, including background noise, in real time. The module uses adaptive gain control technology to adjust sensitivity based on the ambient sound pressure level (e.g., 50-80 decibels), ensuring full signal capture even in noisy conditions.

[0024] Step S01: When a noisy speech signal is detected, the time domain features of the speech signal are collected according to the time domain feature units in the target sensor array.

[0025] Specifically, when a noisy speech signal is detected, the time domain feature unit in the target sensor array starts working. The time domain feature unit is composed of specific sampling points in the microphone array, and continuously records the amplitude and duration changes of the speech waveform at intervals of every 20 milliseconds. For example, in a living room scenario, when the user issues a command, the acquisition is triggered by monitoring the sound pressure level exceeding a preset threshold (such as 60 decibels), and time domain feature data reflecting the fluctuations and duration of the voice pitch is generated. These data include the timing pattern and dynamic changes of the voice, and are affected by background noise such as TV sound, but the high-sensitivity design ensures that the core voice information is captured, providing a reliable basis for subsequent processing.

[0026] Step S02: collecting frequency domain features of the speech signal according to the frequency domain feature units in the target sensor array.

[0027] Furthermore, the collected noisy speech signal is transmitted to the frequency domain feature units in the target sensor array for processing. These units are composed of dedicated spectrum analysis modules, which are responsible for decomposing the time domain signal into frequency components. For example, in a living room environment containing air conditioning noise, the time domain waveform is converted into a frequency domain representation, the main frequency range of the speech (such as 100Hz to 3000Hz) is identified, and the low-frequency characteristics of the noise are distinguished. The frequency domain feature unit samples at multiple frequency bandwidths, for example, one bandwidth every 50Hz, and generates frequency domain feature data containing speech energy distribution and spectral peaks. The acquisition process adapts to different noise intensities by adjusting the sampling parameters to ensure that the harmonic structure and interference pattern of the speech are captured, providing key input for subsequent feature transmission.

[0028] Step S03: Send the time domain features and the frequency domain features to the second end connected to the first end, wherein the second end generates an enhanced speech signal based on the time domain features and the frequency domain features, and performs variable bit rate quantization encoding and text transcription output based on the enhanced speech signal.

[0029] As can be understood, the first end transmits the time-domain and frequency-domain features to the second end via a high-speed data interface (such as a wireless network or a wired connection). The second end serves as the processing core and is deployed in the smart home's central control unit. Equipped with a high-performance signal processing chip, the second end receives data from the first end and begins integrating the features to generate an enhanced voice signal. For example, data is delivered with millisecond latency via a dedicated transmission protocol to ensure real-time performance. The second end then uses these features for noise suppression and quantization encoding, ultimately outputting a text transcription using a transform decoding structure. This enables accurate execution of commands such as "turn off the TV," meeting the requirements of intelligent voice control.

[0030] In some embodiments, the construction process of the target sensor array is described in detail to meet the voice control needs in noisy environments such as smart homes to support subsequent feature acquisition and recognition.

[0031] Specifically, convolution processing is performed on the coding unit of the time domain feature change to obtain a first feature array, wherein the output points in the first feature array serve as the time domain feature unit.

[0032] It can be understood that the encoding unit is composed of multiple sensor nodes in the voice acquisition module at the first end, which are responsible for detecting the temporal changes of the voice signal. In a smart home scenario, for example, when a user in the living room issues an instruction to "adjust the lights", the encoding unit captures the continuous waveform of the noisy voice signal. Convolution processing uses a one-dimensional sliding window technology through the built-in signal processing chip to extract local features of these waveform data, such as sliding the window every 10 milliseconds to analyze amplitude changes and timing patterns. After processing, a first feature array is generated, which contains multiple output points. These points correspond to time domain feature units, which are used to subsequently capture the dynamic time characteristics of the voice to ensure that key timing information is retained in a noisy environment.

[0033] Furthermore, convolution processing is performed on the coding unit whose frequency domain characteristics change to obtain a second feature array, wherein the output points of the second feature array serve as frequency domain feature units.

[0034] Specifically, the encoding unit receives noisy speech signal data from the first end via a spectrum analysis module. In a living room scenario, the signal containing television noise is decomposed into frequency components. The encoding unit utilizes a similar sliding window technique, but focuses on the energy distribution in the frequency domain. Convolution processing adjusts the window size (e.g., a frequency bandwidth of 50 Hz) to extract spectral peaks and harmonic structure, generating a second feature array. The output points of this array serve as frequency domain feature units, reflecting the frequency characteristics of speech and its differences from noise, providing basic data for subsequent feature fusion.

[0035] Furthermore, a target sensor array is constructed based on the first characteristic array and the second characteristic array.

[0036] Understandably, the first end integrates the data of the generated time domain feature units and frequency domain feature units, and performs mapping and calibration through the signal processing module. For example, in an indoor environment such as a living room, the time series data of the first feature array is combined with the frequency data of the second feature array to generate a unified sensor data framework. The target sensor array assigns specific weights to the time domain and frequency domain units to ensure that the two work together under noisy conditions, constructing an array structure that can fully capture voice features, laying the foundation for subsequent signal enhancement and recognition.

[0037] In some embodiments, targeting the voice control needs in noisy environments such as smart homes, the focus is on describing the acquisition and processing of time domain features to support subsequent signal enhancement and recognition.

[0038] Specifically, time domain change data generated by the first feature array according to the noisy speech signal is determined, wherein the time domain change data includes amplitude changes of coding units in the first feature array.

[0039] As can be understood, the first feature array is composed of multiple encoding units in the voice acquisition module at the first end. These units are integrated into smart home devices, such as smart speakers in the living room, and are used to detect temporal changes in voice signals. When a user issues a "turn up the volume" command in a noisy TV environment, the encoding unit captures the continuous waveform of the noisy voice signal using a highly sensitive sensor. By monitoring the acoustic characteristics of the signal in real time, when the voice amplitude exceeds the background noise threshold (e.g., 60 decibels), the amplitude change of each encoding unit is recorded, for example, from 5 millivolts in a silent state to 40 millivolts, generating time domain change data. This data reflects the temporal dynamics of speech, including fluctuations in pitch and duration, ensuring that critical speech information is retained in noisy environments.

[0040] Furthermore, the feature intensity of each time-domain feature unit in the first feature array is determined according to the time-domain variation data.

[0041] Specifically, the system analyzes the collected time-domain variation data point by point, utilizing the built-in signal processing chip to process the amplitude variation of each coding unit. For example, in a living room scenario, the signal energy of each time-domain feature unit is calculated, and the feature strength is determined based on the peak intensity of the amplitude variation, such as the difference from 5 mV to 40 mV. These strength values ​​are generated by comparing the amplitude variations of adjacent sampling points, such as the peak difference within 20 milliseconds, reflecting the dynamic variation trend of speech. The processing process reduces noise interference through multiple sampling (e.g., taking an average of five times), ensuring the stability and accuracy of feature strength, and providing a reliable basis for subsequent time-domain feature generation.

[0042] Furthermore, the time domain feature is determined according to the feature strength.

[0043] It can be understood that the feature intensity data of each time domain feature unit is input into the signal characterization module for comprehensive processing. For example, in a living room environment containing air conditioning noise, the temporal pattern of speech, such as pitch fluctuations and the boundaries of speech segments, is identified based on the distribution pattern of feature intensity. The processing process generates time domain features by setting an intensity threshold, for example, treating points with an intensity greater than 30 millivolts as valid speech segments. These features are output in the form of a time series, reflecting the dynamic characteristics of speech and the separation effect of the noise background. The voice habits of different users are also taken into account. By adjusting the threshold range, it is ensured that the time domain features are accurately extracted under diverse noisy conditions to provide support for subsequent transmission to the second end.

[0044] In some embodiments, targeting the voice control needs in noisy environments such as smart homes, the focus is on describing the acquisition and processing of frequency domain features to support subsequent signal enhancement and recognition.

[0045] Specifically, frequency domain change data generated by the second feature array according to the noisy speech signal is determined, wherein the frequency domain change data includes the spectrum amplitude of the coding unit in the second feature array.

[0046] Understandably, the second feature array is composed of multiple encoding units in the voice acquisition module at the first end. These units are integrated into smart home devices, such as the smart speaker in the living room, and are responsible for analyzing the frequency characteristics of voice signals. When a user issues the command "turn off the lights" amidst the noise of the air conditioner, the encoding unit receives the noisy voice signal through the spectrum analysis module. Using real-time signal decomposition technology, the time domain waveform is converted to a frequency domain representation. When a frequency component is detected that exceeds the background noise threshold (e.g., 100Hz), the spectral amplitude change of each encoding unit is recorded, for example, from 10 decibels to 50 decibels, generating frequency domain change data. This data reflects the frequency distribution and energy changes of the speech, ensuring that key spectral information is captured in noisy environments.

[0047] Furthermore, the spectrum amplitude of each frequency domain feature unit is converted into a frequency domain feature.

[0048] Specifically, the collected frequency domain change data is processed point by point, using the built-in signal processing chip to analyze the spectral amplitude of each coding unit. For example, in a living room scenario, the spectral amplitude data (e.g., 50 decibels) is compared with a preset frequency range (e.g., 100Hz to 3000Hz) to extract the main harmonic components of the speech. The conversion process generates frequency domain features by setting frequency band thresholds, for example, considering frequencies with amplitudes greater than 30 decibels as valid speech features. These features are output as frequency sequences, reflecting the separation of the speech spectral pattern and noise interference. The processing also includes smoothing filtering techniques to reduce the impact of high-frequency noise and ensure the stability of the frequency domain features.

[0049] As you can understand, the converted frequency domain features are further optimized to adapt to different noise environments. For example, in a kitchen scenario containing dishwasher noise, the frequency band is adjusted to prioritize the low- and mid-frequency components of the voice, generating complete frequency domain feature data. Multi-channel sampling (e.g., sampling every 50Hz) ensures data comprehensiveness, providing reliable input for subsequent transmission to the second end and supporting the accuracy of intelligent voice control.

[0050] In some embodiments, as Figure 2 As shown in the figure, for voice control needs in noisy environments such as smart homes, the focus is on the second-end feature processing and output process to support efficient speech recognition. The following method steps are followed one by one to expand the specific implementation.

[0051] Step S51: Acquire the time domain features and frequency domain features sent by the first end in real time.

[0052] It can be understood that the second end serves as a smart home central control unit, equipped with a high-performance signal processing module, deployed in the living room or home server, and is used to receive data transmitted by the first end. When the user issues a "play music" command amidst the noise of the TV, the time domain features and frequency domain features collected by the voice acquisition module of the first end are sent to the second end in the form of data packets through the wireless network interface. The receiving process uses a high-speed data buffer to ensure that the data arrives within millisecond delays, for example, a feature data packet is received every 20 milliseconds. The integrity of the data is verified through a verification mechanism to ensure that the time domain features (such as the amplitude of the voice waveform) and the frequency domain features (such as the spectrum energy distribution) are not lost due to transmission interruptions, providing reliable input for subsequent processing.

[0053] Step S52: generating an enhanced speech signal based on the time domain features and the frequency domain features, and performing variable bit rate quantization encoding and text transcription based on the enhanced speech signal.

[0054] Specifically, the second end inputs the received time domain features and frequency domain features into the signal fusion module, and combines them with the preset noise suppression method to generate an enhanced voice signal. For example, in the living room scene, the time domain features are used to remove short-term noise pulses, and the frequency domain features are combined to suppress low-frequency background sounds, and a clear voice signal is output. Next, the enhanced voice signal is sent to the quantization coding module, which dynamically adjusts the bit rate allocation according to the complexity of the signal, and then processes it through the transform decoding structure to generate the corresponding text transcription result, such as the instruction text of "play music". The processing process achieves real-time performance through multi-threading technology, ensuring that the accuracy and response speed of voice recognition are improved in complex noise environments.

[0055] Understandably, the generated enhanced speech signal and text transcription results are optimized to suit the smart home environment. For example, in a kitchen containing dishwasher noise, high-frequency speech components are prioritized for enhancement, resulting in stable quantization encoding and transcription output. Data processing also includes error correction mechanisms to ensure that the transcription results are consistent with the user's expectations. Figure 1 Provide continuous support for intelligent voice control.

[0056] In some embodiments, in response to voice control needs in noisy environments such as smart homes, the focus is on describing the process of generating an enhanced voice signal at the second end to support subsequent quantization coding and recognition.

[0057] Specifically, the time domain features and the frequency domain features are fused to generate fused features.

[0058] Understandably, the second end, serving as the smart home central control unit, is equipped with a signal fusion module that receives the time-domain and frequency-domain features transmitted in real time by the first end. When a user issues a "lower volume" command amidst the noise of the living room TV, the second end inputs the time-domain features (e.g., amplitude changes in the speech waveform) and the frequency-domain features (e.g., spectral energy distribution) into the fusion module. The fusion process combines the two sets of feature data using a weighted averaging technique, for example, assigning a 60% weight to the amplitude intensity of the time-domain features and a 40% weight to the energy distribution of the frequency-domain features, to generate a single fused feature data set. A time alignment mechanism is also used to ensure the synchronization of the time-domain and frequency-domain data, generating fused features that reflect the overall characteristics of the speech and provide comprehensive input for subsequent enhancement processing.

[0059] Furthermore, the fused features are processed through an encoder-decoder structure to obtain an enhanced speech signal.

[0060] Specifically, the second end inputs the fused features into an encoder-decoder structure, which consists of a multi-layer neural network processing unit. In the living room scenario, the encoder first performs dimensionality reduction on the fused features to extract the main speech patterns, such as removing low-frequency noise components, and the decoder then reconstructs the signal to enhance the clarity of the speech. For example, the system adjusts the output layer of the decoder to amplify the harmonic components of the speech and suppress background noise such as the sound of the air conditioner. The processing process is completed in real time, for example, a data segment is processed every 40 milliseconds to generate an enhanced speech signal. The system also includes a feedback mechanism to dynamically adjust the encoder parameters based on the quality of the enhanced signal to ensure the output of high-quality speech signals in noisy environments.

[0061] As you can understand, the generated enhanced speech signal is optimized to suit different smart home scenarios. For example, in the presence of kitchen dishwasher noise, the system prioritizes enhancing mid- and high-frequency speech components, generating a stable enhanced signal. The processing also takes into account the diversity of user speech, adjusting the decoder output through multiple iterations to ensure that the enhancement effect meets the requirements of subsequent quantization encoding.

[0062] In addition, if Figure 3 As shown, the second end processes the fusion features through the encoder-decoder structure to obtain an enhanced speech signal. The processing flow is the same as Figure 3 The same as shown. Figure 3This paper presents the overall architecture of front-end speech enhancement, which consists of two main components: input speech enhancement and output speech enhancement. The input speech enhancement component begins with a noisy speech signal and comprises multiple sequential processing layers, gradually generating intermediate features. The output speech enhancement component, based on these intermediate features, further processes the signal through a series of connected processing modules, ultimately generating an enhanced speech signal. The entire process consists of several progressive processing layers, representing the complete path from the original signal to the enhanced output. Specifically, the front-end speech enhancement component comprises an encoder-LSTM-decoder structure. Because purely time-domain speech enhancement models can lead to phase instability and poor noise reduction capabilities, the encoder structure proposed in this paper goes beyond time-domain modeling and incorporates auxiliary frequency-domain information into the time-domain features to improve the robustness of speech enhancement. First, the noisy speech passes through two Conv1D layers and activation layers (ReLU and GLU) to obtain time-domain features. The amplitude spectrum of the noisy speech passes through three Conv2D-ELU-BatchNorm2D structures to obtain frequency-domain features. Next, in the refinement block, the time domain features and the frequency domain features are concatenated and passed through the Linear layer to obtain the refined speech features. Finally, in the fusion block, the original speech features and the refined speech features are fused through Conv1D to obtain the output of each layer of encoder. For the decoder, this embodiment uses the Conv1D layer and the ConvTranspose1D layer to obtain the output of each layer of decoder. Through this fusion of time domain and frequency domain information, the model proposed in the present invention can not only process the noise in the speech signal more stably, but also improve the adaptability to complex noise environments. Finally, after being processed by the encoder and decoder, the output enhanced speech signal has higher speech quality and lower distortion, which can provide clearer and more accurate input for subsequent ASR tasks.

[0063] In some embodiments, in response to voice control needs in noisy environments such as smart homes, the focus is on describing the process of variable bit rate quantization encoding and text transcription at the second end to support efficient voice recognition.

[0064] Specifically, time frames are allocated according to the complexity of the enhanced speech signal, and variable bit rate quantization coding is generated.

[0065] As can be understood, the second end, serving as the smart home central control unit, is equipped with a signal quantization module that receives the enhanced voice signal. When a user issues the command "open the curtains" amidst the noise of the living room TV, the second end analyzes the complexity of the enhanced voice signal, detecting, for example, the frequency change rate and energy fluctuations of the voice segments. More complex segments, such as commands with rapid speech, are assigned longer time frames (e.g., 40 milliseconds), while simpler segments, such as silent intervals, are assigned shorter time frames (e.g., 10 milliseconds). The quantization encoding module dynamically adjusts the bit rate based on these time frames, for example, allocating 256 kbps to complex segments and 64 kbps to simple segments, generating variable bitrate quantization codes. Real-time monitoring ensures that the encoding process adapts to the noisy environment and preserves critical voice information.

[0066] Furthermore, the variable bit rate quantized code is input into a transform decoder to generate a corresponding text transcription output.

[0067] Specifically, the second end transmits the generated quantized code to a transform decoder, which consists of multiple processing units. In the living room scenario, the decoder receives the quantized coded data and gradually restores the speech features, such as parsing the phoneme sequence of a complex segment. The processing process converts the coded data into text output, such as the instruction text "open the curtains", through the speech-to-text mapping rules. After processing each segment of coded data, the result is output, for example, a portion of the transcription is generated every 50 milliseconds to ensure real-time performance. The decoder also includes an error detection mechanism to correct possible noise interference and ensure transcription accuracy.

[0068] The generated text transcription output is understandably optimized to suit the needs of smart homes. For example, in the presence of a noisy kitchen dishwasher, high-priority commands are prioritized, resulting in stable transcription results. The processing also accounts for differences in user accents, adjusting decoder parameters to enhance recognition of diverse speech patterns, providing reliable support for intelligent control.

[0069] In some embodiments, as Figure 4 As shown in the figure, targeting the voice control needs in noisy environments such as smart homes, the focus is on describing the process of variable bit rate quantization encoding at the second end to support efficient speech recognition.

[0070] Specifically, the enhanced speech signal is subjected to feature extraction by a multi-scale encoder to obtain a first scale feature and second-scale features .

[0071] It can be understood that the multi-scale encoder extracts features from the enhanced speech signal through two parallel processing units E1 and E2, where E1 is responsible for extracting first-scale features, focusing on global speech patterns, and E2 is responsible for extracting second-scale features, focusing on local speech details. Figure 4As shown, The outputs of E1 and E2 generate error signals respectively, which are combined with the subsequent mask weighting mechanism to generate the first scale feature and the second scale feature, providing a basis for importance analysis. Furthermore, the second end serves as the smart home central control unit, equipped with a multi-scale encoder module to receive the enhanced speech signal. When the user issues the command "adjust the temperature" under the noise of the TV in the living room, the encoder uses multi-resolution analysis technology to process the signal and generate the first scale feature. and second-scale features . Extract global speech patterns, such as overall pitch changes, using a wider time-frequency window (e.g., 100 milliseconds); A narrow window (e.g., 20 milliseconds) is used to capture local details, such as phoneme boundaries. Parallel processing ensures the integrity of both sets of feature data, reflecting the long-term and short-term characteristics of the signal, respectively, providing a basis for subsequent importance analysis.

[0072] Furthermore, the first scale features and the second scale feature Importance analysis is performed to obtain a first importance distribution map and a second importance distribution map.

[0073] Specifically, the second end uses the signal analysis module to process and , evaluate the speech relevance of each time frame. For example, in the living room scene, the energy distribution and frequency change rate are used to determine The key speech segment generates the first importance distribution map and marks the high energy area; similarly, A secondary importance distribution map is generated through local peak detection, highlighting short-term speech details. The analysis process uses adaptive weighting to emphasize frames containing the core content of the command, ensuring that the distribution map accurately reflects the importance of the speech.

[0074] Further, based on the first importance distribution map and the second importance distribution map, a relative importance distribution map is calculated, wherein the relative importance distribution map is calculated according to the formula calculate, represents the first importance distribution map, represents the second importance distribution map, Represents a relative importance distribution graph.

[0075] It can be understood that the second end will be the first importance distribution map ( ) and the second importance distribution map ( ) is input to the differential processing module for calculation. Formula It means that the importance of the second scale feature (reflecting local details) is subtracted from the importance of the first scale feature (reflecting the global speech structure) to generate a relative importance distribution map ( ). For example, in a living room environment, if The value of a certain frame is 0.8 (indicating global high energy), The value of is 0.5 (indicating local energy), then = 0.8 - 0.5 = 0.3, showing the relative importance of the frame in the global picture. The difference operation in the formula is used to highlight the difference between the global and local parts, eliminate minor noise interference, and ensure Accurately identify key speech areas. This calculation is done point by point, adapting to different noise conditions, such as the sound of a TV or air conditioner.

[0076] Furthermore, a time frame allocation mode is determined according to the relative importance distribution diagram, and variable bit rate quantization coding is generated.

[0077] It can be understood that the second end is distributed according to the relative importance diagram ( ). For example, in a kitchen scene containing dishwasher noise, Frames with higher values ​​(e.g., greater than 0.3) are identified as important speech segments, allocated longer time frames (e.g., 40 milliseconds), and encoded at a higher bit rate (e.g., 256 kbps); Frames with lower values ​​(e.g., less than 0.1) are assigned shorter frames (e.g., 10 milliseconds) and encoded at a lower bitrate (e.g., 64 kbps). The quantization encoding module dynamically adjusts the bitrate to optimize data transmission efficiency and storage space. Frame allocation is adjusted by real-time monitoring of noise changes to ensure voice quality in complex environments and provide reliable input for subsequent text transcription.

[0078] In some embodiments, in response to voice control requirements in noisy environments such as smart homes, the focus is on describing a process in which the second end determines a time frame allocation method to support efficient variable bit rate quantization coding.

[0079] Specifically, the threshold corresponding to the relative importance distribution map is generated by one-dimensional convolution processing, wherein the threshold is calculated according to the formula Calculation, TH represents the threshold, represents a one-dimensional convolution operation, Represents a relative importance distribution graph.

[0080] Specifically, the second end serves as a smart home central control unit, equipped with a signal processing module, and receives a relative importance distribution map ( ). When the user issues a "turn off TV" command amidst the noise of the TV in the living room, the second end uses a one-dimensional convolution operation to Processing is performed by scanning along the time axis using a sliding window technique For example, if The values ​​of a certain area are 0.3, 0.4, 0.5, 0.4, 0.3, and the convolution kernel calculates the local weighted sum to generate a dynamic threshold TH. The convolution kernel is designed with a window size of 5 time frames, the weights are set to [0.1, 0.2, 0.4, 0.2, 0.1], and the stride is 1 frame to ensure sensitivity to changes in local importance. Formula Indicates the threshold edge (second scale time steps) generated, with IM R The processing process adapts to noise changes by adjusting the weight parameters of the convolution kernel. For example, in a high-noise environment, the center weight is increased to 0.5 to generate a threshold that reflects the local importance level.

[0081] Furthermore, a mask value is calculated based on the relative importance distribution map and the threshold value, wherein the mask value is calculated by the formula m=I(IM R - TH > 0) is determined, I represents the indicator function, when IM R - When TH > 0, m is set to 1, otherwise it is set to 0.

[0082] Specifically, the second end will IM R Compare TH point by point and apply the indicator function to calculate the mask value m. For example, in the living room scene, if IM R If a frame value is 0.4 and TH is 0.3, then IM R - TH = 0.4 - 0.3 = 0.1 > 0, m is set to 1; if IM R is 0.2, TH is 0.3, then IM R -TH = -0.1 < 0, m is set to 0. Formula m = I(IM R - TH >0) indicates that by comparing IM R The difference between TH and TH generates a binary mask to mark the important speech segments (m = 1) and the minor segments (m = 0). Timeline enforcement ensures that the mask values ​​accurately reflect the relative importance of speech and noise.

[0083] Furthermore, a time frame allocation mode is determined according to the mask value, wherein when m is 1, a longer time frame is allocated, otherwise a shorter time frame is allocated, thereby generating a variable bit rate quantization code.

[0084] As you can understand, the second end dynamically adjusts the time frame length based on the mask value m. For example, in a kitchen scene containing dishwasher noise, when m = 1 (indicating a significant speech segment), a longer time frame (e.g., 40 milliseconds) is allocated and encoded at a higher bitrate (e.g., 256 kbps); when m = 0 (indicating a less significant or noisy segment), a shorter time frame (e.g., 10 milliseconds) is allocated and encoded at a lower bitrate (e.g., 64 kbps). The quantization encoding module generates variable bitrate quantization codes based on these frame allocations, optimizing data transmission efficiency. By updating the mask value in real time to adapt to noise variations, speech quality is maintained in complex environments, providing reliable support for subsequent text transcription.

[0085] In some embodiments, the focus is on describing the process of generating text transcription output by the second end to support efficient speech recognition in response to voice control needs in noisy environments such as smart homes.

[0086] Specifically, the variable bit rate quantization code is converted into an input vector, and position information is superimposed.

[0087] Understandably, the second end, serving as a smart home central control unit, is equipped with a transform decoder module that receives variable bit rate quantization codes. When a user issues a "turn on the lights" command amidst the noise of the living room TV, the second end decodes the quantized coded data into a continuous sequence of audio features and converts it into a multidimensional input vector through an embedding layer. For example, each frame of data is mapped into a 300-dimensional vector representing the acoustic characteristics of the speech. When superimposing position information, the system adds a time series position identifier to each vector and generates a position code using sine and cosine functions, marking the relative position of the vector in the sequence (e.g., frame 1, frame 2), ensuring that the decoder understands the temporal relationship of the speech. This process is accomplished using a preset position code table, providing structured input for subsequent processing.

[0088] Furthermore, the input vector is processed by a multi-layer multi-directional attention mechanism and a feedforward network, wherein each layer of the multi-directional attention mechanism performs attention calculation on the current and previous input vectors.

[0089] Specifically, the second end uses a multi-layer neural network structure to process the input vector, for example, setting up a 3-layer multi-directional attention mechanism. In the living room scenario, the first layer of attention mechanism calculates the correlation between the current input vector (such as the 10th frame) and the previous vector (such as frames 1 to 9), focusing on the key parts of the voice command, such as the phoneme sequence of the word "open". The attention calculation generates a contextual representation through a weighted sum, for example, assigning a higher weight to frames related to "open". Next, the feedforward network applies a nonlinear transformation to the attention result to enhance the feature expression capability. This process is repeated at each layer, gradually refining the voice features to ensure that the command information is retained under noise interference.

[0090] Furthermore, the attention calculation results are further processed by the feedforward network to generate the final text transcription output.

[0091] Understandably, the second end inputs the output of the multi-layer attention mechanism into the last layer of the feedforward network for optimization. In a living room environment containing air-conditioning noise, the feedforward network processes the attention results through multiple layers of fully connected layers and activation functions, and maps them into text probability distribution. For example, the system outputs the text sequence "turn on the lights" based on the result with the highest probability. The processing process is updated every 50 milliseconds to ensure real-time performance, and possible errors are corrected through the post-processing module, such as merging consecutive phoneme segments. The system adopts a Decoder-Only decoder structure, such as Figure 5 The multi-scale decoder design shown in the figure starts with variable bitrate quantization encoding. The audio discrete code is first converted into a vector through an embedding layer and superimposed with a positional code. It is then processed by multiple layers of Transformer layers consisting of multi-head self-attention and feedforward networks to complete the generation of transcripts. Figure 5 The multi-scale decoder contains multiple sequentially connected processing layers, reflecting the integration of information at different time scales. To verify the effect, the present invention conducted experiments on the Librispeech dataset and compared it with the Hubert + DecoderOnly and Encodec + DecoderOnly models. The evaluation criteria were word error rate (WER) and real-time rate (RTF). Figure 6 As shown, Hubert + DecoderOnly achieves a WER of 2.08% and an RTF of 0.44; Encodec + DecoderOnly achieves a WER of 2.14% and an RTF of 0.35; while Ours achieves a WER of 1.69% and an RTF of 0.25. These results demonstrate that the proposed method achieves superior RTF performance while maintaining a low WER, making it suitable for real-time voice control in noisy smart home environments. The system also adapts to different user accents by adjusting network parameters to generate accurate transcription output.

[0092] The above description is merely an exemplary embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformation made by utilizing the contents of the present invention's description and drawings under the technical concept of the present invention, or directly / indirectly applied in other related technical fields, is included in the patent protection scope of the present invention.

Claims

1. A variable bit rate quantized speech recognition method for an intelligent speech controller, characterized in that: Applied to a first end, the first end includes a voice acquisition module having a target sensor array, the voice acquisition module dynamically acquires an input noisy voice signal, and the method includes: When a noisy speech signal is detected, collecting time domain features of the speech signal according to the time domain feature units in the target sensor array; collecting frequency domain features of a speech signal according to a frequency domain feature unit in the target sensor array; The time domain features and the frequency domain features are sent to a second end connected to the first end, wherein the second end generates an enhanced speech signal based on the time domain features and the frequency domain features, and performs variable bit rate quantization encoding and text transcription output based on the enhanced speech signal.

2. The method according to claim 1, wherein The method further comprises: Performing convolution processing on the coding unit of the time domain feature change to obtain a first feature array, wherein the output points in the first feature array serve as the time domain feature unit; Performing convolution processing on the coding unit with frequency domain feature changes to obtain a second feature array, wherein the output points of the second feature array serve as frequency domain feature units; A target sensor array is constructed based on the first characteristic array and the second characteristic array.

3. The method according to claim 2, wherein The step of collecting the time domain features of the speech signal based on the time domain feature units in the target sensor array includes: Determining time-domain variation data generated by the first feature array based on the noisy speech signal, wherein the time-domain variation data includes amplitude variation of coding units in the first feature array; determining a feature intensity of each time-domain feature unit in the first feature array according to the time-domain variation data; A time domain feature is determined according to the feature strength.

4. The method according to claim 2, wherein The step of collecting the frequency domain features of the speech signal based on the frequency domain feature units in the target sensor array includes: Determining frequency domain change data generated by the second feature array based on the noisy speech signal, wherein the frequency domain change data includes spectrum amplitudes of coding units in the second feature array; The spectrum amplitude of each frequency domain feature unit is converted into a frequency domain feature.

5. The method according to claim 1, wherein The second end generates an enhanced speech signal based on the time domain features and the frequency domain features, and performs variable bit rate quantization encoding and text transcription based on the enhanced speech signal, including: Obtaining time domain features and frequency domain features sent in real time by the first end; An enhanced speech signal is generated according to the time domain features and the frequency domain features, and variable bit rate quantization encoding and text transcription are performed according to the enhanced speech signal.

6. The method according to claim 5, wherein The step of generating an enhanced speech signal based on the time domain features and the frequency domain features includes: fusing the time domain features and the frequency domain features to generate fused features; The fused features are processed through an encoder-decoder structure to obtain an enhanced speech signal.

7. The method according to claim 5, wherein The step of performing variable bit rate quantization encoding and text transcription based on the enhanced speech signal includes: Allocate time frames according to the complexity of the enhanced speech signal and generate variable bit rate quantization code; The variable bit rate quantized code is input into a transform decoder to generate a corresponding text transcription output.

8. The method according to claim 5, wherein The step of performing variable bit rate quantization encoding according to the enhanced speech signal comprises: Performing feature extraction on the enhanced speech signal using a multi-scale encoder to obtain first-scale features and second-scale features; Performing importance analysis on the first scale feature and the second scale feature respectively to obtain a first importance distribution map and a second importance distribution map; According to the first importance distribution map and the second importance distribution map, the relative importance distribution map is calculated, wherein the relative importance distribution map is calculated according to the formula calculate, represents the first importance distribution map, represents the second importance distribution map, Represents a relative importance distribution graph; A time frame allocation mode is determined according to the relative importance distribution diagram, and variable bit rate quantization coding is generated.

9. The method according to claim 8, wherein The step of determining the time frame allocation method based on the relative importance distribution diagram includes: The threshold corresponding to the relative importance distribution map is generated by one-dimensional convolution processing, wherein the threshold is according to the formula: Calculation, TH represents the threshold, represents a one-dimensional convolution operation, Represents a relative importance distribution graph; The mask value is calculated based on the relative importance distribution map and the threshold, wherein the mask value is calculated by the formula Determine, I represents the indicator function, when - When > 0, m is set to 1, otherwise it is set to 0; The time frame allocation mode is determined according to the mask value, wherein when m is 1, a longer time frame is allocated, otherwise a shorter time frame is allocated, thereby generating a variable bit rate quantization code.

10. The method according to claim 7, wherein: The step of inputting the variable bit rate quantized code into a transform decoder to generate a corresponding text transcription output comprises: Converting the variable bit rate quantization code into an input vector and superimposing position information; Processing the input vector through multiple layers of multi-directional attention mechanisms and feedforward networks, wherein each layer of multi-directional attention mechanisms performs attention calculations on the current and previous input vectors; The attention calculation results are further processed by the feedforward network to generate the final text transcription output.

Images