Deep learning-based adaptive speech recognition system
By using a deep learning-based adaptive speech recognition system, the original speech signal is purified and features are extracted through a speech processing module, which solves the problems of low recognition efficiency and accuracy in existing technologies and achieves efficient speech recognition in dynamic environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- IANGSU COLLEGE OF ENG & TECH
- Filing Date
- 2026-04-02
- Publication Date
- 2026-07-03
AI Technical Summary
Existing speech recognition systems are not efficient or accurate enough when faced with dynamically changing users and environments.
An adaptive speech recognition system based on deep learning is adopted. The speech processing module preprocesses and cleans the original speech signal, obtains user identification information and environmental embedding feature data, performs dynamic personalized adaptation and feature recognition processing, and uses recognition feature coefficients to judge the speech quality.
It improves the accuracy and efficiency of speech recognition in dynamically changing users and environments.
Smart Images
Figure CN122337201A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of speech recognition technology, specifically to an adaptive speech recognition system based on deep learning. Background Technology
[0002] Currently, mainstream speech recognition systems typically employ deep learning models, and their workflow is as follows: front-end signal processing → acoustic feature extraction → acoustic model → language model → decoding output. These systems are usually trained on large, general-purpose corpora to achieve broad coverage.
[0003] However, current speech recognition systems use static and fixed models, which are not efficient and accurate enough when faced with dynamically changing users, environments, and domains. Summary of the Invention
[0004] To address the shortcomings mentioned in the background section, the present invention aims to provide an adaptive speech recognition system based on deep learning.
[0005] Firstly, the objective of this invention can be achieved through the following technical solution: an adaptive speech recognition system based on deep learning, comprising: The voice acquisition module is used to acquire raw voice signal data and send it to the voice processing module; The speech processing module is used to preprocess the raw speech signal data to obtain processed raw speech signal data, perform adaptive purification processing on the processed raw speech signal data to obtain clean speech signal data, obtain corresponding user identification information based on the clean speech signal data, and perform dynamic personalized adaptation based on the clean speech signal data and the corresponding user identification information to obtain speech coding feature data. Collect environmental metadata; extract features based on the environmental metadata to obtain environmental embedded feature data; perform feature recognition processing and calculation based on the environmental embedded feature data and speech coding feature data to obtain recognition feature coefficients, and send them to the speech recognition module; The speech recognition module is used to compare the recognition feature coefficients with preset recognition feature thresholds and to determine the speech quality based on the comparison results.
[0006] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the preprocessing of the speech processing module includes one or more of noise suppression, echo cancellation, or gain adjustment.
[0007] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the process of dynamically personalizing the adaptation based on clean voice signal data and corresponding user identification information, as follows: Pure speech signal data and corresponding user identification information are inserted into the network layer of a pre-established static backbone acoustic model. The few trainable parameters contained therein are activated or weighted according to the user embedding vector corresponding to the user identification information to obtain speech coding feature data.
[0008] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the process of feature extraction based on environmental metadata, as follows: Modality-specific feature extraction and standardization are performed on environmental meta-related data to obtain the extracted environmental modality feature vectors. Cross-modal attention fusion is then performed on the environmental modality feature vectors to obtain a unified intermediate representation. Multilayer perception and compression encoding are then performed on the unified intermediate representation to obtain environmental embedded feature data.
[0009] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the process of performing feature recognition processing calculation based on environmental embedded feature data and speech coding feature data, as follows: Label the environmental embedding feature data and the speech coding feature data: The environmental embedding feature data is labeled as Hi, and the speech coding feature data is labeled as Yi. In the formula, i is the sampling number of the environmental embedding feature data and the speech coding feature data, and i = 1, 2, 3, ..., n, where n is the total number of sampling times of the environmental embedding feature data and the speech coding feature data. In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: calculating using labeled context embedding feature data Hi and speech coding feature data Yi, as shown in the following formula: In the formula, Si is the recognition feature coefficient; Y0 is the preset standard environment feature coefficient; H0 is the preset standard speech feature coefficient; k1 is the environmental influence coefficient; k2 is the speech extraction influence coefficient; q is the preset factor; ln() is the logarithmic function; and exp() is the exponential function.
[0010] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the analysis process of the speech recognition module is as follows: If Si≥S0, the speech corresponding to the recognition feature coefficients will be labeled as standard quality speech; If Si < S0, the speech corresponding to the recognition feature coefficients will be marked as non-standard quality speech.
[0011] In another aspect of the present invention, in order to achieve the above-mentioned objective, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor, and when the processor loads and executes the computer program, it employs the deep learning-based adaptive speech recognition system described above.
[0012] In another aspect of the present invention, in order to achieve the above objectives, a computer-readable storage medium is disclosed, wherein a computer program is stored in the computer program, and when the computer program is loaded and executed by a processor, it employs the deep learning-based adaptive speech recognition system described above.
[0013] The beneficial effects of this invention are: This invention processes the raw speech signal data acquired by the speech acquisition module through a speech processing module and dynamically adapts it with user identification information to obtain speech coding feature data. Furthermore, it extracts features from environmental metadata to obtain environmental embedding feature data. Based on the environmental embedding feature data and the speech coding feature data, feature recognition processing is performed to calculate recognition feature coefficients. The speech recognition module compares these recognition feature coefficients with preset recognition feature thresholds and determines the speech quality based on the comparison results. This enables recognition under dynamically changing user and environmental conditions, improving the accuracy of speech recognition. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the system structure of the present invention. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] Example 1: The following is a description of the relevant terms used in the embodiments of this application: Adaptive purification process: Adaptive audio purification is a dynamic signal optimization process driven by meta-learning. Its core lies in the following: The system first analyzes the real-time acquired audio stream and its associated device metadata (such as microphone model, noise spectrum, and motion state) through an environment perception unit, generating a high-dimensional environment embedding vector. This vector is then input into a pre-trained lightweight meta-network, which dynamically generates a set of optimal signal processor parameters (including noise suppression coefficients, echo cancellation filters, and adaptive gain control curves) based on the current environmental characteristics. These parameters are then used to configure the front-end processing unit (such as a deep neural network-based noise reduction model or digital filter bank) to perform tailored audio purification, ultimately outputting a clean speech signal optimized for the current scene. The entire process is completed in milliseconds, achieving an intelligent adaptive leap from "fixed processing" to "environment perception - dynamic parameter generation - precise purification."
[0017] like Figure 1 As shown, the deep learning-based adaptive speech recognition system includes: Voice acquisition module, voice processing module, and voice recognition module; The voice acquisition module is used to acquire raw voice signal data and send the raw voice signal data to the voice processing module for processing. Specifically, the acquisition process of the voice acquisition module includes the following steps: In this embodiment, a corpus containing clean speech and multi-channel noisy speech is simultaneously recorded in multiple scenarios (such as anechoic chambers, noisy streets, and in-vehicle environments) using professional acquisition equipment, and environmental parameters and precise text alignment are annotated. During the application phase after the system goes live, the speech stream is collected in real time using the microphones of user terminal devices (such as mobile phones and smart speakers), while automatically binding environmental metadata such as device model, motion sensor data, and location information. Intelligent triggering of acquisition is achieved through voice activity detection to reduce power consumption. All acquired data undergoes standardized preprocessing (resampling, volume normalization, and framing) and encrypted transmission to form a standardized digital signal that drives the subsequent three-layer adaptive modules, serving as the final raw speech signal data. After receiving the raw voice signal data sent by the voice acquisition module, the voice processing module performs voice data processing. Specifically, the processing procedure of the voice processing module includes the following steps: Preprocessing the original speech signal data yields processed original speech signal data. Specifically, at least one processing method, including noise suppression, echo cancellation, or gain adjustment, is applied to the original speech signal data to remove noise components from the original speech signal. The processed raw speech signal data is subjected to adaptive purification processing to obtain clean speech signal data; the adaptive purification process can optimize the signal within the raw speech signal data to obtain cleaner speech signal data. The corresponding user identification information is obtained based on the clean speech signal data; dynamic personalized adaptation is performed based on the clean speech signal data and the corresponding user identification information to obtain speech coding feature data. Specifically, the dynamic personalized adaptation process involves inserting clean speech signal data and corresponding user identification information into the network layer of the static backbone acoustic model. The small number of trainable parameters contained therein are activated or weighted according to the user embedding vector corresponding to the user identification information, which is used to adjust its forward propagation features without changing the core parameters of the static backbone acoustic model, and finally obtain speech coding feature data. Collect environmental metadata; extract features based on the environmental metadata to obtain environmental embedded feature data; The process of feature extraction based on environmental metadata is as follows: Modality-specific feature extraction and standardization are performed on environmental meta-related data to obtain the extracted environmental modality feature vectors. Cross-modal attention fusion is performed on the environmental modality feature vectors to obtain a unified intermediate representation. Multilayer perception and compression encoding are performed on the unified intermediate representation to obtain environmental embedded feature data. Feature recognition coefficients are calculated based on environmental embedded feature data and speech coding feature data. The specific process is as follows: Label the environmental embedding feature data and the speech coding feature data: The environmental embedding feature data is labeled as Hi, and the speech coding feature data is labeled as Yi. In the formula, i is the sampling number of the environmental embedding feature data and the speech coding feature data, and i = 1, 2, 3, ..., n, where n is the total number of sampling times of the environmental embedding feature data and the speech coding feature data. Specifically, the number of environmental embedding feature data and the number of speech coding feature data correspond one-to-one. The calculation is performed using labeled context embedding feature data Hi and speech coding feature data Yi, as shown in the following formula: In the formula, Si is the recognition feature coefficient; Y0 is the preset standard environment feature coefficient; H0 is the preset standard speech feature coefficient; k1 is the environmental influence coefficient; k2 is the speech extraction influence coefficient; q is the preset factor; ln() is the logarithmic function; exp() is the exponential function. Furthermore, in the specific implementation process, the preset standard environmental feature coefficients and preset standard speech feature coefficients are obtained by repeatedly simulating and calculating the average value of the data after collecting environmental embedding feature data and speech coding feature data in daily life. In this embodiment, the environmental impact coefficient and the speech extraction impact coefficient are calculated by comprehensively evaluating the impact of external factors when acquiring environmental embedding feature data and speech coding feature data in daily operations. These factors include human factors, machine detection, and environmental factors. Human factors refer to those caused by human operation or improper scanning. The recognition feature coefficients are sent to the speech recognition module for analysis. After receiving the recognition feature coefficients from the speech processing module, the speech recognition module performs speech analysis. Specifically, the analysis process of the speech recognition module includes the following steps: The recognition feature coefficient Si is compared with the preset recognition feature threshold S0, and the degree of speech recognition result is determined based on the comparison result. The process is as follows: If Si≥S0, it means that the speech recognition quality is standard quality at this time, and the speech corresponding to the recognition feature coefficient is marked as standard quality speech. If Si < S0, it means that the speech recognition is of non-standard quality at this time, and the speech corresponding to the recognition feature coefficient is marked as non-standard quality speech. Specifically, the distinction between standard quality and non-standard quality voice is based on whether it meets the standards for subsequent use.
[0018] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.
[0019] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0020] The above formulas are all numerical calculations after removing dimensions. The formulas are obtained by software simulation based on a large amount of data and are closest to the real situation. The preset parameters and preset thresholds in the formulas are set by those skilled in the art according to the actual situation or obtained by simulation based on a large amount of data.
[0021] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0022] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.
Claims
1. A deep learning-based adaptive speech recognition system, characterized in that, include: The voice acquisition module is used to acquire raw voice signal data and send it to the voice processing module; The speech processing module is used to preprocess the raw speech signal data to obtain processed raw speech signal data, perform adaptive purification processing on the processed raw speech signal data to obtain clean speech signal data, obtain corresponding user identification information based on the clean speech signal data, and perform dynamic personalized adaptation based on the clean speech signal data and the corresponding user identification information to obtain speech coding feature data. Collect environmental metadata; Feature extraction is performed based on environmental metadata to obtain environmental embedded feature data; Feature recognition processing is performed based on environmental embedded feature data and speech coding feature data to obtain recognition feature coefficients, which are then sent to the speech recognition module. The speech recognition module is used to compare the recognition feature coefficients with preset recognition feature thresholds and to determine the speech quality based on the comparison results.
2. The deep learning-based adaptive speech recognition system according to claim 1, characterized in that, The preprocessing of the speech processing module includes one or more of the following: noise suppression, echo cancellation, or gain adjustment.
3. The deep learning-based adaptive speech recognition system according to claim 1, characterized in that, The process of dynamically personalizing the adaptation based on clean voice signal data and corresponding user identification information is as follows: Pure speech signal data and corresponding user identification information are inserted into the network layer of a pre-established static backbone acoustic model. The few trainable parameters contained therein are activated or weighted according to the user embedding vector corresponding to the user identification information to obtain speech coding feature data.
4. The deep learning-based adaptive speech recognition system according to claim 1, characterized in that, The process of feature extraction based on environmental metadata is as follows: Modality-specific feature extraction and standardization are performed on environmental meta-related data to obtain the extracted environmental modality feature vectors. Cross-modal attention fusion is then performed on the environmental modality feature vectors to obtain a unified intermediate representation. Multilayer perception and compression encoding are then performed on the unified intermediate representation to obtain environmental embedded feature data.
5. The deep learning-based adaptive speech recognition system according to claim 1, characterized in that, The process of feature recognition processing based on environmental embedded feature data and speech coding feature data is as follows: Label the environmental embedding feature data and the speech coding feature data: The environmental embedding feature data is labeled as Hi, and the speech coding feature data is labeled as Yi. In the formula, i is the number of times the environmental embedding feature data and the speech coding feature data are collected, and i = 1, 2, 3, ..., n, where n is the total number of times the environmental embedding feature data and the speech coding feature data are collected.
6. The deep learning-based adaptive speech recognition system according to claim 5, characterized in that, The calculation is performed using labeled context embedding feature data Hi and speech coding feature data Yi, as shown in the following formula: In the formula, Si is the recognition feature coefficient; Y0 is the preset standard environment feature coefficient; H0 is the preset standard speech feature coefficient; k1 is the environmental influence coefficient; k2 is the speech extraction influence coefficient; q is the preset factor; ln() is the logarithmic function; and exp() is the exponential function.
7. The deep learning-based adaptive speech recognition system according to claim 1, characterized in that, The analysis process of the speech recognition module is as follows: If Si≥S0, the speech corresponding to the recognition feature coefficients will be labeled as standard quality speech; If Si < S0, the speech corresponding to the recognition feature coefficients will be marked as non-standard quality speech.
8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, it employs the deep learning-based adaptive speech recognition system as described in any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the deep learning-based adaptive speech recognition system as described in any one of claims 1 to 7.