Text generation method and device based on end-side speech recognition, equipment and medium

Abstract

This invention relates to the field of audio processing technology, addressing the serious user privacy leakage risks inherent in existing cloud-based speech recognition solutions. It provides a text generation method, apparatus, device, and medium based on edge speech recognition. The method includes: performing analog-to-digital conversion and sampling on a user-generated speech signal to obtain raw audio data; preprocessing the raw audio data on a terminal device to obtain target audio data; performing speech endpoint detection on the target audio data and determining valid speech segments based on the detection results; extracting acoustic features from the valid speech segments to obtain an audio feature sequence; and inputting the audio feature sequence into a pre-trained text recognition model to obtain a text recognition result. The text recognition model is a locally deployed, lightweight model. This invention improves data security and privacy protection during the speech recognition process.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to a method, apparatus, device, and medium for text generation based on edge speech recognition. Background Technology

[0002] With the rapid development of mobile computing, smart homes, and the Internet of Things, voice, as a natural and efficient human-computer interaction method, is becoming increasingly important. Real-time and accurate conversion of voice signals into text is fundamental for subsequent intelligent understanding, command execution, content generation, and information storage, and is a key technological link in realizing device intelligence and natural interaction. Therefore, high-efficiency and high-precision speech recognition and text generation technologies have become one of the core requirements for promoting the application of artificial intelligence.

[0003] Traditional speech recognition text generation solutions, especially in high-precision scenarios, typically rely on the powerful computing resources of the cloud. The typical process involves the terminal device acquiring a speech signal, then uploading the audio data or pre-processed feature data to a cloud server via the network. A large-scale speech recognition model deployed in the cloud then performs complex acoustic modeling and language decoding, generating the text result which is then returned to the terminal. This cloud-dependent architecture has several inherent drawbacks: First, its recognition performance is limited by network conditions, completely failing in weak or no-network environments, resulting in insufficient reliability. Second, there is a risk of privacy leaks during speech data transmission and cloud processing, making it difficult to meet the needs of applications with strict data security requirements. Finally, network transmission and cloud processing introduce additional latency, affecting the real-time interactive experience. To address these issues, the industry has begun exploring on-device (local) speech recognition solutions. However, directly deploying large-scale cloud models to terminal devices with limited computing power, storage space, and power consumption presents significant challenges.

[0004] How to ensure the model runs efficiently on the terminal while maintaining recognition accuracy, and how to design a complete optimization process that is executed entirely locally, are the technical challenges that urgently need to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, device and medium for text generation based on edge speech recognition, in order to solve the problem of serious user privacy leakage risks in existing cloud speech recognition solutions.

[0006] In a first aspect, embodiments of the present invention provide a text generation method based on edge speech recognition, the method comprising: The system performs analog-to-digital conversion and sampling on the user's voice signal to obtain the raw audio data. The target audio data is obtained by preprocessing the original audio data on the terminal device. Speech endpoint detection is performed on the target audio data, and valid speech segments are determined based on the detection results; Acoustic features are extracted from the effective speech segments to obtain an audio feature sequence; The audio feature sequence is input into a pre-trained text recognition model to obtain the text recognition result, wherein the text recognition model is a lightweight model deployed locally.

[0007] In an optional embodiment, the step of performing analog-to-digital conversion and sampling on the user's voice signal to obtain raw audio data includes: The terminal's microphone array is used to acquire the user's voice signal. The speech signal is subjected to anti-aliasing filtering to obtain the filtered speech signal; The filtered speech signal is converted from analog to digital to obtain a pulse code modulation data stream; The pulse code modulation data stream is sampled and quantized according to the preset sampling rate and quantization bit depth to obtain the original audio data.

[0008] In an optional embodiment, the preprocessing of the original audio data by the terminal device to obtain the target audio data includes: The original audio data is subjected to pre-emphasis filtering to obtain a pre-emphasis audio signal; The pre-emphasized audio signal is processed into frames to obtain multiple short-time audio frame sequences; A windowing function is applied to each frame of audio data in the short-time audio frame sequence to obtain a windowed audio frame sequence. The windowed audio frame sequence is subjected to noise reduction processing to obtain the target audio data.

[0009] In an optional embodiment, the step of performing voice endpoint detection on the target audio data and determining valid voice segments based on the detection results includes: Short-time energy and zero-crossing rate are calculated on the target audio data to obtain the short-time energy value and zero-crossing rate value corresponding to each frame of audio data, forming an energy sequence and a zero-crossing rate sequence. Based on the energy sequence and the zero-crossing rate sequence, each frame of audio data in the target audio data is jointly discriminated to generate a preliminary labeling sequence containing speech frame labels and non-speech frame labels; Based on the preliminary labeling sequence, the preset dynamic energy threshold, and the preset silence length, the labeling results are smoothed and corrected to generate a final set of start and end points containing start and end point information of continuous speech segments. Based on the final set of start and end points, the corresponding valid speech segments are extracted from the target audio data.

[0010] In an optional embodiment, the step of smoothing and correcting the marking results based on the initial marking sequence, a preset dynamic energy threshold, and a preset silence length to generate a final set of start and end points containing start and end point information of continuous speech segments includes: Based on the energy sequence and the preliminary label sequence, the background noise energy is calculated; Based on the background noise energy, the dynamic energy threshold is updated to obtain the updated dynamic energy threshold; Using the updated dynamic energy threshold, the initial labeled sequence is re-scanned and discriminated to generate a first corrected labeled sequence; In the first correction marker sequence, all consecutive segments marked as speech frames are identified as candidate speech segments; For each candidate speech segment, determine whether the frame length of the continuous silence area marked as a non-speech frame before its start endpoint and after its end endpoint is less than the preset silence length. If the frame length of the continuous silence zone is less than the preset silence length, then the tags of all frames in the corresponding silence zone are corrected to speech frames to expand the candidate speech segment and generate a second correction tag sequence. Based on the second correction marker sequence, the start frame index and end frame index of all corrected continuous speech segments are extracted to form the final start and end point set.

[0011] In an optional embodiment, the step of extracting acoustic features from the effective speech segment to obtain an audio feature sequence includes: Perform a Fast Fourier Transform on each frame of speech data in the effective speech segment to obtain the linear spectrum of each frame of speech data; The linear spectrum is passed through a set of Mel-scale triangular filter banks to calculate the filter bank energy of each frame of speech data in the Mel band. Taking the logarithm of the filter bank energy yields the logarithmic Mel spectrum energy; Performing a discrete cosine transform on the logarithmic Mel frequency energy yields the Mel frequency cepstral coefficients; Dynamic feature calculations are performed on the Mel frequency cepstral coefficients to obtain a feature vector containing both static and dynamic features; The feature vectors of all frames in the effective speech segment are arranged in temporal order to form the audio feature sequence.

[0012] In an optional embodiment, the step of inputting the audio feature sequence into a pre-trained text recognition model to obtain a text recognition result, wherein the text recognition model is a locally deployed lightweight model including: Based on the audio feature sequence, the text recognition model's encoder performs feature encoding to obtain an advanced acoustic feature sequence; Initialize the internal state of the decoder and set the initial decoding input; Based on the advanced acoustic feature sequence and the current decoding input, the context vector of the current decoding step is calculated through an attention mechanism. Based on the context vector and the current internal state of the decoder, the decoder performs decoding calculations to obtain the symbol prediction result for the current decoding step; Based on the symbol prediction results, determine the text symbol output in the current decoding step; Determine whether the text symbol output in the current decoding step is a preset termination symbol; If the text symbol output by the current decoding step is not the termination symbol, then the text symbol output by the current decoding step is used as the new decoding input, and the internal state of the decoder is updated. Then, the process returns to the step of calculating the context vector of the current decoding step based on the high-level acoustic feature sequence and the current decoding input through an attention mechanism. If the text symbol output in the current decoding step is the termination symbol, then, in the decoding order, all text symbols output from the initial decoding input to the termination symbol are combined to generate the text recognition result.

[0013] Secondly, embodiments of the present invention provide a text generation device based on edge speech recognition, the device comprising: The raw audio data acquisition module is used to perform analog-to-digital conversion and sampling on the user's voice signal to acquire raw audio data; The preprocessing module is used to preprocess the raw audio data through the terminal device to obtain the target audio data; The voice endpoint detection module is used to perform voice endpoint detection on the target audio data and determine the valid voice segments based on the detection results. An acoustic feature extraction module is used to extract acoustic features from the effective speech segments to obtain an audio feature sequence; The text recognition module is used to input the audio feature sequence into a pre-trained text recognition model to obtain the text recognition result, wherein the text recognition model is a lightweight model deployed locally.

[0014] Thirdly, embodiments of the present invention provide an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect described above.

[0015] Fourthly, embodiments of the present invention provide a storage medium storing computer program instructions, which, when executed by a processor, implement the method of the first aspect described above.

[0016] In summary, the beneficial effects of the present invention are as follows: This invention provides a method, apparatus, device, and medium for text generation based on edge speech recognition. The method includes: performing analog-to-digital conversion and sampling on a user-generated speech signal to obtain raw audio data; preprocessing the raw audio data on a terminal device to obtain target audio data; performing speech endpoint detection on the target audio data and determining valid speech segments based on the detection results; extracting acoustic features from the valid speech segments to obtain an audio feature sequence; and inputting the audio feature sequence into a pre-trained text recognition model to obtain a text recognition result. The text recognition model is a locally deployed, lightweight model. This invention avoids the privacy risks associated with uploading raw audio data to a remote server, as is required in traditional cloud-based speech recognition, by deploying all key processing steps of speech recognition on the terminal device. Specifically, the user's speech signal undergoes analog-to-digital conversion, preprocessing, endpoint detection, and acoustic feature extraction locally. The resulting audio feature sequence does not need to be transmitted externally but is directly input into a locally deployed, lightweight text recognition model to generate text. Throughout the entire recognition process, no audio or feature data is transmitted to the cloud, fundamentally eliminating the risks of interception, eavesdropping, and misuse that may occur during network transmission and cloud storage of voice content, thus achieving local closed-loop processing of voice information. Therefore, this invention, through an on-device self-processing mechanism, ensures that user voice data is always protected by the physical boundaries of the terminal device, significantly improving data security and privacy protection capabilities during voice recognition, and resolving the common user privacy leakage risks present in existing cloud-based voice recognition solutions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0018] Figure 1 This is a schematic diagram of the overall process of the text generation method based on edge speech recognition in Embodiment 1 of the present invention; Figure 2 This is a flowchart of Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the text generation device based on end-side speech recognition in Embodiment 2 of the present invention. Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0020] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0021] It should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and regulations of the locality and with authorization from the owner of the relevant device.

[0022] Example 1 Please see Figure 1 This invention provides a text generation method based on edge speech recognition, the method comprising: The system performs analog-to-digital conversion and sampling on the user's voice signal to obtain the raw audio data. Specifically, speech signals are essentially air vibrations that change continuously over time. They need to be converted into discrete digital signals that can be directly manipulated by digital processing modules through analog-to-digital conversion (ADC) and sampling. ADC converts analog sound waves into digital form, while sampling records the amplitude information of the sound waves at fixed time intervals. By digitizing this physical sound wave, a unified data foundation can be established for subsequent digital signal processing. In practice, the audio acquisition module of the terminal device amplifies, filters, and quantizes the analog sound signal picked up by the microphone, ultimately forming a raw audio data stream that can be directly read in the software. Since all subsequent operations in speech recognition depend on the integrity and temporal continuity of the audio data, this step, through strict local digital acquisition, ensures stable input data quality while preventing audio from being uploaded or processed externally during the acquisition stage. This forms a locally controlled processing loop, laying the foundation for privacy and security.

[0023] The target audio data is obtained by preprocessing the original audio data on the terminal device. Specifically, preprocessing refers to a series of digital signal processing operations performed on the raw audio data to improve its quality and standardize its format, thereby enhancing the recognizability of speech components and suppressing irrelevant noise. While the specific algorithms involved in preprocessing may vary depending on device capabilities and usage scenarios, their core purpose is to improve the clarity and stability of effective speech information, enabling subsequent recognition models to operate under better input conditions. In terms of implementation, preprocessing typically begins by framing and windowing the audio, ensuring that continuous audio is stably fed into the processing chain in time segments. Then, algorithms such as noise reduction, gain control, and dynamic range adjustment are used to suppress background noise and non-speech components. By performing this series of standardization processes locally on the terminal, the accuracy of subsequent endpoint detection and feature extraction can be significantly improved. Furthermore, it avoids transmitting unprocessed noisy audio to external systems, thereby reducing potential privacy exposure and ensuring secure data flow within the device.

[0024] Speech endpoint detection is performed on the target audio data, and valid speech segments are determined based on the detection results; Specifically, speech endpoint detection is used to identify which segments of audio contain actual speech content and which are quiet or noisy intervals. Its significance lies in dividing a continuous audio stream into semantically meaningful segments, reducing the burden of processing invalid data. This step typically combines energy, frequency domain change rate, or lightweight acoustic discrimination models to determine the state of speech activity, and then filters out stable speech segments based on continuity conditions. Through this segmentation mechanism, terminal devices can filter out a large number of blank or noisy segments early in the recognition process, ensuring that subsequent feature extraction and text recognition are performed only within the time range that actually contains speech, thereby reducing computational overhead and improving recognition efficiency. Because endpoint detection is performed locally, the device does not need to transmit the entire long audio segment to the cloud for speech localization, effectively reducing the possibility of sensitive audio content leakage and improving the real-time responsiveness of edge recognition.

[0025] Acoustic features are extracted from the effective speech segments to obtain an audio feature sequence; Specifically, acoustic feature extraction transforms the temporal waveforms of speech segments into more expressive feature representations, enabling subsequent models to more accurately understand the speech structure within the sound. This feature extraction typically involves segmenting effective speech segments into short frames, performing spectral analysis on each frame, and extracting representation vectors that reflect vocal tract deformation, energy distribution, and speech structure. The aim is to transform high-dimensional, redundant raw waveforms into more compact and stable representations, allowing recognition models to achieve higher recognition capabilities with less computation. In implementation, the system can use operations such as filtering, spectral transformation, and statistical compression to generate feature sequences, ensuring good consistency across different devices, environments, and pronunciation methods. Since feature extraction is performed on the device side, the system does not need to upload large amounts of raw audio or intermediate sensitive data, further limiting privacy leakage paths. Simultaneously, it provides structured, high-quality input data for local models, improving recognition efficiency and overall performance.

[0026] The audio feature sequence is input into a pre-trained text recognition model to obtain the text recognition result, wherein the text recognition model is a lightweight model deployed locally.

[0027] Specifically, the lightweight text recognition model is a speech recognition model that, after structural compression and efficiency optimization, can run independently on a terminal device. Its function is to convert acoustic feature sequences into the final text output. By deploying this model locally, the system can complete the entire process of speech-to-text inference without relying on an external server. In implementation, the model typically employs an optimized network structure, capable of completing acoustic modeling, pattern matching, and decoding under limited storage space and computing power, and generating the final text content based on language rules. Since the entire recognition process is executed within the terminal, there is no need to upload speech features or inference-related data to the cloud. This solution fundamentally eliminates the risk of leakage that may occur during user voice transmission, server storage, and multi-party interaction. At the same time, the local model has a faster response speed and can maintain stable operation in weak network or no network scenarios, further improving user experience and the system's autonomous security capabilities.

[0028] In an optional embodiment, the step of performing analog-to-digital conversion and sampling on the user's voice signal to obtain raw audio data includes: The terminal's microphone array is used to acquire the user's voice signal. Specifically, the microphone array consists of multiple pickup units located at different spatial positions. Its function is not only to capture the airborne sound waves emitted by the user, but also to utilize the small time differences between multiple sound sources to obtain a higher signal-to-noise ratio and more accurate sound source direction information. Acquiring speech through an array structure allows the system to more easily distinguish user speech from background noise in noisy environments, thus providing a cleaner input for subsequent signal processing. In implementation, each microphone in the array synchronously picks up sound waves, which are then initially processed by the acquisition module and converged into a unified analog speech signal channel. This array-based pickup method not only enhances the stability and clarity of audio data but also reduces noise interference at the source, thereby ensuring that subsequent analog-to-digital conversion and sampling stages obtain higher-quality speech signals, laying a solid foundation for the overall performance of edge-side speech recognition.

[0029] The speech signal is subjected to anti-aliasing filtering to obtain the filtered speech signal; Specifically, anti-aliasing filtering is used to remove high-frequency components exceeding the Nyquist frequency before analog-to-digital conversion, preventing these components from being incorrectly mapped to the low-frequency region during digitization and causing aliasing, thus disrupting the spectral structure of the speech signal. Its purpose is to provide a bandwidth-controlled, structurally clear analog input for analog-to-digital conversion, enabling the digital signal to strictly reflect the effective frequency characteristics of real speech. This is typically achieved by using a low-pass filter to clean the input waveform, ensuring it conforms to the bandwidth supported by the system's sampling rate before entering the digitization link. By performing anti-aliasing processing locally at the terminal, the fidelity and stability of subsequent digital audio can be significantly improved, making the spectral representation of speech more consistent with the processing characteristics of the recognition model, while avoiding recognition errors caused by spectral pollution, thereby improving the overall accuracy and adaptability of edge speech recognition.

[0030] The filtered speech signal is converted from analog to digital to obtain a pulse code modulation data stream; Specifically, analog-to-digital conversion (ADC) transforms continuously changing speech waveforms into discrete digital forms, serving as the core starting point for digital speech signal processing. Pulse code modulation (PCM) is a commonly used digital representation method that periodically captures the amplitude of analog signals and represents them with digital codes, achieving a complete mapping from analog sound waves to digital data. This process is typically performed by a dedicated ADC chip, which samples, holds, quantizes, and encodes the waveform at the hardware level to generate a stable PCM data stream. By completing this crucial digitization process locally, the system ensures that speech data does not need to be transmitted or processed externally in analog mode, avoiding the risk of leakage caused by exposed analog signal interfaces. Simultaneously, high-quality PCM output provides a clear and structured foundation for subsequent sampling, quantization, and overall feature extraction, making the digital link in the speech recognition process more stable and reliable.

[0031] The pulse code modulation data stream is sampled and quantized according to the preset sampling rate and quantization bit depth to obtain the original audio data.

[0032] Specifically, the sampling rate determines the number of times speech amplitude is recorded per unit time, while the quantization bit depth determines the amplitude precision that can be expressed in each recording; together, they constitute the fundamental parameters of audio quality. By sampling and quantizing the PCM data stream according to preset system requirements, it can be ensured that the final generated raw audio data meets the input specifications of subsequent speech recognition models in terms of both temporal precision and amplitude representation. During implementation, the system selects appropriate parameters based on the terminal's performance and the needs of the recognition task, ensuring that the audio has sufficient clarity without incurring computational burden due to excessively high resolution. This step ensures that the end device obtains digital audio with standardized structure and format, providing a stable data foundation for subsequent preprocessing, feature extraction, and recognition inference. Since sampling and quantization are performed locally without relying on any cloud data processing flow, the entire voice data link remains under the control of the terminal, further reducing potential privacy risks.

[0033] In an optional embodiment, the preprocessing of the original audio data by the terminal device to obtain the target audio data includes: The original audio data is subjected to pre-emphasis filtering to obtain a pre-emphasis audio signal; Specifically, pre-emphasis filtering refers to a linear filtering operation that applies a first-order high-frequency gain to the audio signal in the time domain. Its function is to compensate for the attenuation of high-frequency energy in the speech signal during propagation through the vocal tract and microphone frequency response, thereby enhancing the high-frequency components of the speech and improving the resolution of subsequent spectral analysis. In implementation, a feedback mechanism with coefficients less than but close to one can be used in the linear filter to recursively process each sampling point, or the sample stream can be processed sample-by-sample in a digital signal processor with a fixed gain factor to form a pre-emphasis signal. The parameters can be adjusted according to device characteristics and target applications. The technical feature of performing this operation at the edge ensures that the generated pre-emphasis signal matches the quantization and noise characteristics of the specific acquisition link, resulting in more stable short-time spectral analysis, richer feature representation, and improved robustness of edge-side feature extraction and recognition. Simultaneously, it avoids the privacy risks associated with uploading unstandardized raw waveforms for processing.

[0034] The pre-emphasized audio signal is processed into frames to obtain multiple short-time audio frame sequences; Specifically, frame segmentation divides a continuous temporal audio signal into a series of overlapping short time segments. This segmentation is based on the assumption of short-term stationarity of speech, meaning that the statistical characteristics of the speech signal remain approximately unchanged within a short time window, thus providing stable input for frequency domain analysis and feature calculation. In practice, the system slides the sampled stream according to predetermined frame length and frame shift parameters, typically using an overlapping method to ensure a smooth transition of temporally continuous information. During segmentation, the timestamps and sequential integrity of frame boundaries are maintained to facilitate subsequent streaming processing. Performing frame segmentation on the terminal not only facilitates parallel or pipelined processing of each frame, reducing latency and memory usage, but also allows control over the triggering of more expensive subsequent operations at different time granularities, thereby optimizing edge computing resources and power consumption, while ensuring that the temporal resolution of the feature sequence meets the requirements of the recognition model.

[0035] A windowing function is applied to each frame of audio data in the short-time audio frame sequence to obtain a windowed audio frame sequence. Specifically, windowing involves multiplying the frame data by a smooth window function on each short-time frame to reduce spectral leakage and improve the numerical characteristics of the frequency domain transform. Its purpose is to reduce spectral energy diffusion caused by directly truncating the time-domain signal, making the transformed spectrum more reflective of the true speech components. In terms of implementation, after framing, each frame element is multiplied point-by-point according to the coefficients of the adopted window function. The window function can be selected based on system requirements, choosing a function with good sidelobe suppression and main lobe width characteristics, and the window shape and length can be statically or dynamically selected at the terminal. By performing windowing at the terminal, the basic quality of spectral analysis is guaranteed, making the short-time Fourier transform and filter bank calculation outputs at the terminal more robust. This approach of ensuring the quality of the frequency domain representation locally is beneficial to improving the accuracy of subsequent feature extraction and model inference, and avoids introducing misjudgments or additional processing at the remote end due to spectral leakage caused by uploading unwindowed frames.

[0036] The windowed audio frame sequence is subjected to noise reduction processing to obtain the target audio data.

[0037] Specifically, noise reduction aims to suppress background noise, echoes, and other interfering components to enhance the signal-to-noise ratio of speech components and improve the robustness of the recognition system in complex environments. On-device noise reduction can be implemented using various algorithms and strategies, including classical methods based on spectral subtraction or statistical models, as well as lightweight deep learning models, coherent noise suppression, or array-based beamforming. Typically, it incorporates speech activity information to adaptively update the noise estimation interval and suppresses or reconstructs the spectral components of each frame. Placing noise reduction at the terminal and coordinating it with previous frame-level processing and windowing can improve the data quality delivered to the feature extraction module at the source, significantly reducing feature distortion and recognition errors caused by noise. Simultaneously, on-device noise reduction avoids transmitting unprocessed frames containing sensitive speech content to an external processing platform, thereby further tightening privacy protection boundaries while improving recognition performance.

[0038] In an optional embodiment, the step of performing voice endpoint detection on the target audio data and determining valid voice segments based on the detection results includes: Short-time energy and zero-crossing rate are calculated on the target audio data to obtain the short-time energy value and zero-crossing rate value corresponding to each frame of audio data, forming an energy sequence and a zero-crossing rate sequence. Specifically, short-time energy and zero-crossing rate are important statistics for measuring speech activity from a time-domain perspective. Short-time energy measures the overall energy intensity of the signal within each time frame, used to distinguish speech from silent backgrounds. Zero-crossing rate measures the number of times the signal crosses zero points within a frame, and is uniquely valuable for distinguishing clear speech, coarse noise, and certain types of consonants. In implementation, when calculating the energy of each frame according to the time sequence of the pre-defined frames, the system sums the squares of the sample values ​​and normalizes them to obtain the frame energy. When calculating the zero-crossing rate, the system counts the number of symbol changes of adjacent samples and normalizes them to a frame-level ratio. These frame-level values ​​are then organized into an energy sequence and a zero-crossing rate sequence in chronological order. The technical feature of performing this step on the edge is that it can capture changes in audio at a microscopic time scale in real time, providing a direct and low-cost criterion for subsequent accurate discrimination, thereby obtaining a preliminary quantitative judgment of speech activity with a small computational burden on resource-constrained terminals.

[0039] Based on the energy sequence and the zero-crossing rate sequence, each frame of audio data in the target audio data is jointly discriminated to generate a preliminary labeling sequence containing speech frame labels and non-speech frame labels; Specifically, joint discrimination refers to classifying frames based on a combination of logic or lightweight models using multiple frame-level metrics, such as energy and zero-crossing rate, rather than relying on a single feature. The aim is to leverage the complementarity of different metrics under varying noise or speech types to reduce the misclassification rate caused by a single criterion. Implementation can employ rule-based decision logic, such as labeling frames as speech if energy exceeds a dynamic threshold and the zero-crossing rate is within a reasonable range, or using a simple linear or nonlinear discriminator to perform binary classification on standardized features, incorporating strategies to suppress short-term burst noise during the decision-making process to avoid misclassification due to transient interference. By performing this joint discrimination locally, the system can filter out a large number of non-speech frames at an early stage, reducing the frequency of subsequent expensive feature calculations and model inference. Simultaneously, this localized decision-making reduces the need to externalize large amounts of intermediate data, thereby mitigating potential privacy risks and improving overall energy efficiency.

[0040] Based on the preliminary labeling sequence, the preset dynamic energy threshold, and the preset silence length, the labeling results are smoothed and corrected to generate a final set of start and end points containing start and end point information of continuous speech segments. Specifically, smoothing and correction aim to rectify breaks and jumps caused by transient noise or short speech segments, thereby extracting semantically coherent speech segments as the final recognition units. Implementation typically includes applying short-term connectivity filtering to the initial tags, minimum speech duration and minimum silence duration constraints, and adaptive threshold adjustment based on a dynamic energy threshold. The dynamic energy threshold is updated in real-time according to background noise levels to adapt to environmental changes, while silence length is used to define whether two speech segments should be merged or considered a breakpoint. Furthermore, hysteresis and forward retention mechanisms can be introduced to avoid prematurely cutting off ongoing speech. This correction step, completed at the edge, can significantly reduce the problem of incomplete or over-segmented slices caused by incorrect start and end points, thereby improving the coherence and accuracy of subsequent feature aggregation and recognition. Simultaneously, it saves computational resources by reducing unnecessary short segment processing and further closes privacy-risk channels that might upload unnecessary audio segments.

[0041] Based on the final set of start and end points, the corresponding valid speech segments are extracted from the target audio data.

[0042] Specifically, after obtaining the smoothed and corrected set of start and end points, corresponding intervals are extracted from the target audio data according to time indexes to form effective speech segments. The purpose is to restrict subsequent calculations to time intervals that truly contain semantic information, thereby saving computing power and improving recognition accuracy. During implementation, it is necessary to ensure that the boundaries of the extracted segments are aligned with the original timestamps for subsequent frame-level feature alignment. A small amount of pre- and post-buffering can be added at the boundaries to preserve edge information of the pronunciation start and end points. If necessary, upper and lower limits are applied to the segment length to prevent extremely short or abnormally long segments from affecting subsequent processing. The beneficial effects of this end-side extraction mechanism include significantly reducing the amount of data to be processed, improving the real-time performance and stability of end-to-end recognition, and further avoiding the transmission of large amounts of raw or semi-processed audio to external systems by completing the entire extraction and trimming process within the device, continuously strengthening the local privacy protection of user voice data.

[0043] In an optional embodiment, the step of smoothing and correcting the marking results based on the initial marking sequence, a preset dynamic energy threshold, and a preset silence length to generate a final set of start and end points containing start and end point information of continuous speech segments includes: Based on the energy sequence and the preliminary label sequence, the background noise energy is calculated; Specifically, background noise energy is a numerical description of the overall intensity of non-speech components in the current audio environment, serving as a direct reference for subsequent adaptive thresholding and noise suppression. To obtain this metric, the system can use non-speech frames identified by the initial labeled sequence as noise samples, employing statistical convergence methods to robustly estimate the short-time energy of these frames. For example, it can use moving averages or exponential smoothing, median estimation after removing abnormally high-energy samples, or estimate the noise spectrum energy in the frequency domain and summarize it into an equivalent energy value. Performing this calculation locally on the terminal allows for timely reflection of changes in environmental noise, avoiding misjudgments caused by sudden noise changes in static thresholds. This provides a reliable basis for adaptive updates of dynamic energy thresholds, significantly improving endpoint detection stability and robustness in changing environments.

[0044] Based on the background noise energy, the dynamic energy threshold is updated to obtain the updated dynamic energy threshold; Specifically, the dynamic energy threshold is a discrimination threshold used to distinguish between speech and noise. Its adaptive adjustment based on environmental noise levels aims to maintain appropriate sensitivity and robustness under varying noise conditions, preventing the underestimation of weak speech in low-noise environments and the misclassification of noise as speech in high-noise environments. In implementation, threshold updates can employ linear or nonlinear mapping rules to proportionally or weightedly map background noise energy to a new threshold. Smoothing filters are used to avoid frequent threshold jitter, while upper and lower bound constraints and rate-of-change limits are set to ensure threshold stability. Implementing this adaptive update at the endpoint not only enables endpoint detection to adapt to the surrounding acoustic environment in real time but also reduces false positives and false negatives caused by inappropriate thresholds. This ensures recognition recall while controlling the false alarm rate, improving the system's reliability in real-world scenarios.

[0045] Using the updated dynamic energy threshold, the initial labeled sequence is re-scanned and discriminated to generate a first corrected labeled sequence; Specifically, rescanning applies new discrimination criteria to previously obtained frame-level annotations to correct biases caused by mismatched initial thresholds. Its purpose is to recover or correct misclassified frames through secondary judgment of the entire sequence or local sequences. In practice, each frame's energy can be compared with the updated threshold in chronological order. Simultaneously, zero-crossing rate or other auxiliary features are used to make weighted decisions for frames with blurred boundaries. Short-term smoothing rules are applied when necessary to avoid frequent jumps in isolated single frames. By implementing this correction process at the terminal, the system can promptly correct inaccuracies in early annotations based on real-time estimated noise levels, reducing the loss of speech segments or the inclusion of noise segments due to initial judgment oversights. This provides a more reliable basis for subsequent segment merging and boundary fine-tuning, thereby improving the overall accuracy and continuity of endpoint detection.

[0046] In the first correction marker sequence, all consecutive segments marked as speech frames are identified as candidate speech segments; Specifically, the process of identifying continuous speech segments involves aggregating frame-level speech markers into temporally continuous segment units. This allows for higher-level smoothing and correction of each segment, transitioning from frame-level to segment-level granularity and providing context for determining segment integrity. In implementation, the system searches for adjacent frames marked as speech in the first correction marker sequence and merges them into continuous segments, recording the start and end indices of each segment. During the merging process, minimum duration filtering can be applied to eliminate extremely short noise triggers, and inter-segment intervals can be pre-marked for subsequent silence length determination. Executing this segment recognition process on the device side effectively aggregates speech information, suppresses fragmented output, and provides clear candidate objects for subsequent silence extension correction, thereby improving the coherence and accuracy of subsequent segment extraction. Simultaneously, it reduces unnecessary recognition calls, saving terminal computing power and energy.

[0047] For each candidate speech segment, determine whether the frame length of the continuous silence area marked as a non-speech frame before its start endpoint and after its end endpoint is less than the preset silence length. Specifically, the core purpose of this judgment process is to identify false silence regions on both sides of candidate speech segments that may be caused by noise fluctuations, abnormal zero-crossing rates, or sudden energy drops, so as to expand and correct segment boundaries when necessary. To achieve this, the system extends its scan to adjacent frames before and after the candidate segment, counts consecutive frames marked as non-speech, and compares the count value with a preset silence length threshold. This threshold is usually determined based on empirical phonetics rules; for example, silences as short as tens of milliseconds are insufficient to constitute a real speech interval and therefore should not be considered as segment boundaries. By performing this judgment process on the terminal side, the system can identify "false silences" formed by slight pauses, weakened pronunciation ends, or transient changes in environmental noise, thereby avoiding erroneous truncation of continuous speech content, improving the integrity of speech segments, and further enhancing the stability of endpoint detection in natural dialogue environments.

[0048] If the frame length of the continuous silence zone is less than the preset silence length, then the tags of all frames in the corresponding silence zone are corrected to speech frames to expand the candidate speech segment and generate a second correction tag sequence. Specifically, when the silence area on one side is determined to be insufficient to constitute a true segment boundary, the system reclassifies this silence area as speech, thus expanding the speech segment outward. This correction includes not only simple label replacement but may also incorporate energy trends, zero-crossing rate changes, or spectral feature changes from adjacent frames to ensure the expanded segment has reasonable acoustic coherence. This type of expansion can effectively correct erroneous truncation caused by overly sensitive endpoint detection, making the speech segment boundary closer to actual pronunciation. For example, breath sounds at the beginning of a sentence, weakened codas at the end of a sentence, and brief natural pauses may all be incorrectly identified as silence by the initial system, but this correction mechanism can reintegrate them into the speech segment. The second correction label sequence generated through this dynamic compensation process is more natural and coherent in temporal structure, significantly improving the continuity, completeness, and comprehensibility consistency of subsequently extracted speech segments.

[0049] Based on the second correction marker sequence, the start frame index and end frame index of all corrected continuous speech segments are extracted to form the final start and end point set.

[0050] Specifically, the frame-level markers, after silence expansion and boundary correction, are converted into explicit speech segment boundary indices for direct use in subsequent audio truncation, feature extraction, and model inference. During implementation, the system traverses the second correction marker sequence, retrieving all consecutively marked speech frames and recording the start and end frames of each segment. To further ensure result quality, the system can also apply minimum segment length filtering, boundary smoothing rules, or context-based fine-tuning mechanisms for final refinement of the start and end points. The resulting set of start and end points not only accurately indicates the actual temporal range of the speech but can also be combined with the audio sampling rate and frame shift parameters to convert into time coordinates, directly guiding the extraction of target speech segments. By completing this boundary extraction process locally, the system can generate highly reliable speech segment intervals with low latency, providing clean, complete, and low-error input for the edge speech recognition model, significantly improving speech recognition accuracy and reducing performance losses caused by environmental noise and segment truncation common in cloud-based solutions.

[0051] In an optional embodiment, the step of extracting acoustic features from the effective speech segment to obtain an audio feature sequence includes: Perform a Fast Fourier Transform on each frame of speech data in the effective speech segment to obtain the linear spectrum of each frame of speech data; Specifically, in the acoustic feature extraction process, the time-domain waveform of the speech signal must first be transformed into a frequency-domain representation that reflects the frequency distribution. Therefore, performing a Fast Fourier Transform (FFT) on each frame of speech is crucial. The FFT can efficiently calculate the linear spectrum of the speech signal, clearly revealing the energy distribution of different frequency components in the sound. To achieve this transformation, each frame of speech data, after windowing and frame shifting, enters the Fourier Transform module. By constructing a complex frequency-domain representation at a finite number of sampling points, the amplitude values ​​that originally fluctuated over time are transformed into amplitude and phase information arranged by frequency. This spectrum not only provides the necessary foundation for subsequent filter bank processing but also enables the acquisition of a relatively stable time-frequency distribution with extremely low computational cost in an edge environment, thus providing an efficient and usable frequency-domain input for lightweight speech recognition.

[0052] The linear spectrum is passed through a set of Mel-scale triangular filter banks to calculate the filter bank energy of each frame of speech data in the Mel band. Specifically, the introduction of the Mel filter bank aims to map the linear frequency axis to a Mel scale that is more consistent with human auditory perception, thereby obtaining a frequency band energy distribution that better reflects the characteristics of speech perception. In its implementation, the system pre-constructs multiple sets of triangular filters covering different frequency ranges. By weighted summation of each frequency band on the spectrum, each filter focuses only on the acoustic energy near its center frequency band. Since the Mel scale has a sparser distribution in the high-frequency band, this filtering structure can effectively compress the frequency domain dimension and enhance the response to key speech frequency bands. The energy of the filter bank obtained through this processing can significantly reduce the interference of background noise or irrelevant frequency components on the recognition results, making the acoustic features closer to the essential perceptual laws of human speech.

[0053] Taking the logarithm of the filter bank energy yields the logarithmic Mel spectrum energy; Specifically, to further reduce the amplitude differences caused by energy variations in the speech signal while highlighting the relative energy relationships between different frequency bands, a logarithmic operation needs to be performed on the filter bank energy. This processing has the dual effect of dynamic range compression and acoustic feature normalization, preventing the difference between high-energy and low-energy frequency bands from becoming excessively amplified, thereby improving feature stability. In implementation, a natural logarithmic transformation is performed on the energy of each frequency band, reducing the influence of additive noise in the logarithmic domain and making the subtle structure of the speech easier to capture by subsequent feature transformations. The log-Mel spectral energy remains highly compact under end-side computation conditions, while enhancing the noise resistance of the features, laying a solid foundation for further cepstral processing.

[0054] Performing a discrete cosine transform on the logarithmic Mel frequency energy yields the Mel frequency cepstral coefficients; Specifically, the purpose of performing a discrete cosine transform (DCT) on the logarithmic Mel-frequency energy is to further eliminate inter-band correlations, making features more independent and easier for the model to process. The DCT maps frequency domain energy to the cepstral domain, effectively encoding the spectral envelope information as a small number of cepstral coefficients, thereby achieving feature dimensionality compression and information abstraction. In implementation, the system applies cosine basis function expansion to the frequency band energy sequence of each frame, concentrating the overall spectral information of the speech in the first few cepstral coefficients. This structure not only improves the sensitivity of the edge model to speech variations but also gives the features stronger temporal smoothness. The resulting Mel-frequency cepstral coefficients become the most core, stable, and low-cost acoustic representation for speech recognition tasks.

[0055] Dynamic feature calculations are performed on the Mel frequency cepstral coefficients to obtain a feature vector containing both static and dynamic features; Specifically, to enhance the system's ability to capture speech changes over time, velocity and acceleration information, i.e., dynamic features, need to be calculated in addition to static cepstral coefficients. This process typically involves linear regression derivation of the cepstral coefficients of adjacent frames to obtain first- and second-order changes. This allows the features to reflect not only the instantaneous spectral structure of speech in a single frame but also the changing trends of speech across consecutive frames. The introduction of dynamic features effectively improves the sensitivity of the edge recognition model to speech coherence and semantic transitions, while also enhancing the robustness of the features under conditions of fast speech rate, weak articulation, or channel variations. The resulting feature vector combines static structure with temporal variation information, providing the model with a more complete speech representation capability.

[0056] The feature vectors of all frames in the effective speech segment are arranged in temporal order to form the audio feature sequence.

[0057] Specifically, after frame-by-frame feature extraction, all feature vectors need to be combined according to their temporal order within the speech segment to form a continuous feature sequence that can be directly input into the speech recognition model. The construction of this sequence essentially involves temporal reorganization of the features, enabling the model to decode based on the natural evolution of speech over time. Arranging features in temporal order not only maintains the overall coherence of the speech but also provides a stable, ordered, and semantically clear input source for the lightweight on-device model. Through this organization, the system can complete high-quality acoustic feature construction locally on the terminal without relying on cloud preprocessing, thereby significantly improving recognition accuracy and enhancing privacy protection capabilities.

[0058] In an optional embodiment, the step of inputting the audio feature sequence into a pre-trained text recognition model to obtain a text recognition result, wherein the text recognition model is a locally deployed lightweight model including: Based on the audio feature sequence, the text recognition model's encoder performs feature encoding to obtain an advanced acoustic feature sequence; Specifically, the encoder is a computational module in the model used to map the raw frame-level acoustic features into higher-level, more semantically and temporally contextual information representations. Its purpose is to aggregate local frame information and extract long-term dependency features across frames, so that the subsequent decoder can obtain more stable and discriminative inputs in fewer time steps. In implementation, the encoder can use a series of temporal convolutions, gated recurrent units, or self-attention layers to transform and downsample the input feature sequence layer by layer, and maintain temporal alignment information and optional intra-layer normalization strategies at each layer to stabilize the training and inference process. Deploying the encoder on the terminal to lightweight the model and completing feature encoding locally means that the information required for subsequent semantic inference is already in place within the device. This reduces the model's dependence on external corpora or remote computation, and reduces the data volume and potential privacy leakage by pre-compressing and abstracting acoustic data on the terminal, thereby improving the system's real-time performance and security while ensuring recognition performance.

[0059] Initialize the internal state of the decoder and set the initial decoding input; Specifically, the decoder's internal state refers to the hidden vectors and related gating variables used to store historical information and contextual memory. The initial decoding input is typically a predefined start symbol or a blank marker. The purpose of this step is to establish definite startup conditions for the decoder, ensuring that the subsequent symbol generation process has a controllable initial context and supports both streaming and batch inference modes. Implementation methods include mapping the encoder's first-timestamp output or the pooled digest vector and writing it into the decoder's hidden state, or using zero-vector initialization and introducing a special start symbol as input in the initial step to guide the decoding process. Simultaneously, the implementation optimizes the numerical precision, quantization strategy, and memory layout of the internal state to adapt to the terminal's computing power and storage limitations. By explicitly initializing the decoder state and setting the initial input on the edge, the system can start progressive decoding in a stable and predictable manner, reducing recognition fluctuations caused by random or inconsistent initialization, and helping to maintain decoding quality and inference consistency on resource-constrained devices.

[0060] Based on the advanced acoustic feature sequence and the current decoding input, the context vector of the current decoding step is calculated through an attention mechanism. Specifically, the attention mechanism is a mapping structure used to dynamically calculate which positional information from the encoder the decoder should focus on at each time step. Its purpose is to generate a weighted, aggregated context vector for each step during decoding, allowing the decoder to flexibly utilize the encoder's representations at different time points without being constrained by fixed alignment. In implementation, a scoring function that combines similarity and learnability can be calculated between the decoder's current state and the encoder's representations at each time step. After normalization, the encoder representations are weighted and summed to form the context vector. This can be achieved using dot product attention, learnable feedforward scoring, or scaled dot product, among other methods. On the edge, optimization strategies such as caching, sparsity, or window constraints are employed to reduce computational and storage overhead. By integrating attention computation into a lightweight edge model and optimizing it sufficiently, the decoder can achieve fine temporal alignment and semantic focus without relying on cloud-based alignment information. This improves transcription accuracy under varying speech rates and irregular sentence breaks, while avoiding the need to send all intermediate encoder representations out for remote alignment, thus balancing recognition performance and data privacy.

[0061] Based on the context vector and the current internal state of the decoder, the decoder performs decoding calculations to obtain the symbol prediction result for the current decoding step; Specifically, decoding computation refers to the decoder inputting its current internal state and the context vector provided by the attention mechanism into its generation module to calculate the probability distribution of the next symbol or directly output symbol prediction. Its purpose is to convert acoustic semantic information into discrete text symbols. In terms of implementation, the decoder can employ a recurrent structure, a gated structure, or a transformer-based autoregressive generation module. It obtains the symbol distribution through one or more layers of nonlinear mapping and normalization operations, and combines lightweight decoding strategies such as greedy decoding or small-scale bundle search for inference on the edge. Furthermore, to adapt to terminal resources, model compression techniques such as parameter pruning, quantization, and knowledge distillation can be used on the decoder. The frequency and numerical accuracy of the decoder's internal state updates are controlled in the implementation to balance latency and energy efficiency. The technical feature of completing the entire decoding computation locally ensures that the closed-loop processing from acoustic input to symbol output is completed within the device. This not only reduces the necessity of transmitting sensitive speech information externally but also achieves fast response and stable output through optimized edge decoding, thereby enhancing user privacy protection and system robustness while ensuring text recognition quality.

[0062] Based on the symbol prediction results, determine the text symbol output in the current decoding step; Specifically, the probability distribution of symbol predictions output by the decoder is transformed into a definitive text symbol to form a final readable character sequence. Specifically, at the current decoding step, the decoder outputs a probability distribution of all candidate symbols (such as letters, pinyin, Chinese character components, or sub-word units). The system can determine the final output symbol for this step based on maximum probability selection, beam search filtering, or constrained decoding strategies. In implementation, a greedy strategy can be used on the terminal side to reduce computational overhead, or a lightweight beam search can be used to improve stability in complex scenarios as needed. Simultaneously, this step can filter out illegal symbols, non-semantic symbols, or abnormal outputs caused by model quantization, ensuring that the generated text symbols satisfy structural and linguistic rules. Through this explicit decision-making mechanism of probability-to-symbol conversion, the system can extract standardized text units from model predictions in a consistent manner, thus providing traceable and deterministic input for subsequent autoregressive generation.

[0063] Determine whether the text symbol output in the current decoding step is a preset termination symbol; Specifically, the termination symbol is generally a special marker set during model training to indicate that the text sequence has been fully generated. The judgment method typically involves comparing the output symbol at each decoding step with the preset termination symbol; if they are equal, the system is considered to have reached a natural end point. This mechanism effectively avoids infinite loops while ensuring that the length and content of the output sequence remain reasonable. Deploying this judgment strategy on the terminal side can also avoid redundant inference while ensuring real-time performance, saving computing power and battery consumption, making it particularly suitable for mobile devices and offline recognition scenarios. By setting a clear termination symbol, this decoding process can achieve controllable text generation boundaries, ensuring the structural integrity of the recognition results.

[0064] If the text symbol output by the current decoding step is not the termination symbol, then the text symbol output by the current decoding step is used as the new decoding input, and the internal state of the decoder is updated. Then, the process returns to the step of calculating the context vector of the current decoding step based on the high-level acoustic feature sequence and the current decoding input through an attention mechanism. Specifically, if the current output symbol is not a terminator, this step will proceed to the next autoregressive generation loop. Specifically, the currently output text symbol will be used as input for the next decoding step, guiding the decoder to continue generating sequences. Simultaneously, the decoder's internal state will be updated based on the calculation results of the current decoding step to preserve historical information and serve as contextual reference for the next generation. Internal state updates typically include hidden state vectors, memory units, and gating variables to ensure the model maintains coherence during step prediction. The system then returns to the attention calculation step, allowing the decoder to refocus on the encoder output in the new context and continue generating subsequent symbols. Through this cyclical autoregressive generation method, the model can progressively advance based on previous content, improving language consistency and contextual plausibility. Furthermore, since each step of the calculation is completed locally on the terminal, multiple rounds of remote communication are avoided, significantly improving response speed and privacy security.

[0065] If the text symbol output in the current decoding step is the termination symbol, then, in the decoding order, all text symbols output from the initial decoding input to the termination symbol are combined to generate the text recognition result.

[0066] Specifically, after detecting the terminator, this step performs the final text aggregation and output. The system concatenates all valid text symbols output from previous decoding steps in the order of generation to form the final recognition result, omitting the initial decoding input and the terminator itself. To ensure output quality, this step may include basic symbol cleaning, whitespace processing, word merging, or character-level post-processing to obtain a well-structured, directly usable text string. The termination condition allows the model to naturally converge when recognizing phrases, instructions, or complete sentences, thus achieving dynamically sized text output. Deploying the final combination and output process on the edge not only improves processing speed but also ensures that complete recognition results are obtained without transmitting audio content externally, further enhancing data security and application immediacy.

[0067] Example 2 Please see Figure 3 This invention provides a text generation device based on edge speech recognition, the device comprising: The raw audio data acquisition module is used to perform analog-to-digital conversion and sampling on the user's voice signal to acquire raw audio data; The preprocessing module is used to preprocess the raw audio data through the terminal device to obtain the target audio data; The voice endpoint detection module is used to perform voice endpoint detection on the target audio data and determine the valid voice segments based on the detection results. An acoustic feature extraction module is used to extract acoustic features from the effective speech segments to obtain an audio feature sequence; The text recognition module is used to input the audio feature sequence into a pre-trained text recognition model to obtain the text recognition result, wherein the text recognition model is a lightweight model deployed locally.

[0068] It should be noted that each module and unit in the text generation device based on edge speech recognition in this embodiment corresponds one-to-one with each step in the text generation method based on edge speech recognition in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned text generation method based on edge speech recognition, and will not be repeated here.

[0069] Example 3 In addition, combined Figure 1 The text generation method based on edge speech recognition described in this embodiment of the invention can be implemented by an electronic device. Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention is shown.

[0070] Electronic devices may include processors and memory storing computer program instructions.

[0071] Specifically, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0072] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0073] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated communication signals and carrier waves.

[0074] The processor reads and executes computer program instructions stored in memory to implement any of the text generation methods based on edge speech recognition in the above embodiments.

[0075] In one example, the electronic device may also include a communication interface and a bus. For example, Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0076] The communication interface is mainly used to enable communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0077] A bus, including hardware, software, or both, couples components of an electronic device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, a bus may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0078] Example 4 Furthermore, in conjunction with the text generation method based on edge speech recognition in the above embodiments, this invention can be implemented using a computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the text generation methods based on edge speech recognition in the above embodiments.

[0079] In summary, the embodiments of the present invention provide a text generation method, apparatus, device, and medium based on edge speech recognition.

[0080] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0081] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0086] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A text generation method based on edge speech recognition, characterized in that, The method includes: The system performs analog-to-digital conversion and sampling on the user's voice signal to obtain the raw audio data. The target audio data is obtained by preprocessing the original audio data on the terminal device. Speech endpoint detection is performed on the target audio data, and valid speech segments are determined based on the detection results; Acoustic features are extracted from the effective speech segments to obtain an audio feature sequence; The audio feature sequence is input into a pre-trained text recognition model to obtain the text recognition result, wherein the text recognition model is a lightweight model deployed locally.

2. The text generation method based on edge speech recognition according to claim 1, characterized in that, The process of performing analog-to-digital conversion and sampling on the user's voice signal to obtain raw audio data includes: The terminal's microphone array is used to acquire the user's voice signal. The speech signal is subjected to anti-aliasing filtering to obtain the filtered speech signal; The filtered speech signal is converted from analog to digital to obtain a pulse code modulation data stream; The pulse code modulation data stream is sampled and quantized according to the preset sampling rate and quantization bit depth to obtain the original audio data.

3. The text generation method based on edge speech recognition according to claim 1, characterized in that, The step of preprocessing the original audio data through a terminal device to obtain the target audio data includes: The original audio data is subjected to pre-emphasis filtering to obtain a pre-emphasis audio signal; The pre-emphasized audio signal is processed into frames to obtain multiple short-time audio frame sequences; A windowing function is applied to each frame of audio data in the short-time audio frame sequence to obtain a windowed audio frame sequence. The windowed audio frame sequence is subjected to noise reduction processing to obtain the target audio data.

4. The text generation method based on edge speech recognition according to claim 1, characterized in that, The step of performing speech endpoint detection on the target audio data and determining valid speech segments based on the detection results includes: Short-time energy and zero-crossing rate are calculated on the target audio data to obtain the short-time energy value and zero-crossing rate value corresponding to each frame of audio data, forming an energy sequence and a zero-crossing rate sequence. Based on the energy sequence and the zero-crossing rate sequence, each frame of audio data in the target audio data is jointly discriminated to generate a preliminary labeling sequence containing speech frame labels and non-speech frame labels; Based on the preliminary labeling sequence, the preset dynamic energy threshold, and the preset silence length, the labeling results are smoothed and corrected to generate a final set of start and end points containing start and end point information of continuous speech segments. Based on the final set of start and end points, the corresponding valid speech segments are extracted from the target audio data.

5. The text generation method based on edge speech recognition according to claim 4, characterized in that, The step of smoothing and correcting the marking results based on the preliminary marking sequence, a preset dynamic energy threshold, and a preset silence length to generate a final set of start and end points containing start and end point information for continuous speech segments includes: Based on the energy sequence and the preliminary label sequence, the background noise energy is calculated; Based on the background noise energy, the dynamic energy threshold is updated to obtain the updated dynamic energy threshold; Using the updated dynamic energy threshold, the initial labeled sequence is re-scanned and discriminated to generate a first corrected labeled sequence; In the first correction marker sequence, all consecutive segments marked as speech frames are identified as candidate speech segments; For each candidate speech segment, determine whether the frame length of the continuous silence area marked as a non-speech frame before its start endpoint and after its end endpoint is less than the preset silence length. If the frame length of the continuous silence zone is less than the preset silence length, then the tags of all frames in the corresponding silence zone are corrected to speech frames to expand the candidate speech segment and generate a second correction tag sequence. Based on the second correction marker sequence, the start frame index and end frame index of all corrected continuous speech segments are extracted to form the final start and end point set.

6. The text generation method based on edge speech recognition according to claim 1, characterized in that, The acoustic feature extraction of the effective speech segment to obtain the audio feature sequence includes: Perform a Fast Fourier Transform on each frame of speech data in the effective speech segment to obtain the linear spectrum of each frame of speech data; The linear spectrum is passed through a set of Mel-scale triangular filter banks to calculate the filter bank energy of each frame of speech data in the Mel band. Taking the logarithm of the filter bank energy yields the logarithmic Mel spectrum energy; Performing a discrete cosine transform on the logarithmic Mel frequency energy yields the Mel frequency cepstral coefficients; Dynamic feature calculations are performed on the Mel frequency cepstral coefficients to obtain a feature vector containing both static and dynamic features; The feature vectors of all frames in the effective speech segment are arranged in temporal order to form the audio feature sequence.

7. The text generation method based on edge speech recognition according to any one of claims 1-7, characterized in that, The step of inputting the audio feature sequence into a pre-trained text recognition model to obtain a text recognition result, wherein the text recognition model is a locally deployed lightweight model, including: Based on the audio feature sequence, the text recognition model's encoder performs feature encoding to obtain an advanced acoustic feature sequence; Initialize the internal state of the decoder and set the initial decoding input; Based on the advanced acoustic feature sequence and the current decoding input, the context vector of the current decoding step is calculated through an attention mechanism. Based on the context vector and the current internal state of the decoder, the decoder performs decoding calculations to obtain the symbol prediction result for the current decoding step; Based on the symbol prediction results, determine the text symbol output in the current decoding step; Determine whether the text symbol output in the current decoding step is a preset termination symbol; If the text symbol output by the current decoding step is not the termination symbol, then the text symbol output by the current decoding step is used as the new decoding input, and the internal state of the decoder is updated. Then, the process returns to the step of calculating the context vector of the current decoding step based on the high-level acoustic feature sequence and the current decoding input through an attention mechanism. If the text symbol output in the current decoding step is the termination symbol, then, in the decoding order, all text symbols output from the initial decoding input to the termination symbol are combined to generate the text recognition result.

8. A text generation device based on edge speech recognition, characterized in that, The device includes: The raw audio data acquisition module is used to perform analog-to-digital conversion and sampling on the user's voice signal to acquire raw audio data; The preprocessing module is used to preprocess the original audio data through the terminal device to obtain the target audio data; The voice endpoint detection module is used to perform voice endpoint detection on the target audio data and determine the valid voice segments based on the detection results. An acoustic feature extraction module is used to extract acoustic features from the effective speech segments to obtain an audio feature sequence; The text recognition module is used to input the audio feature sequence into a pre-trained text recognition model to obtain the text recognition result, wherein the text recognition model is a lightweight model deployed locally.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, The method as described in any one of claims 1-7 is implemented when the computer program instructions are executed by the processor.

Images