Speech recognition method for multiple languages

CN118553231BActive Publication Date: 2024-12-06NANJING HEARING TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410998260.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-12-06
Estimated Expiration
2044-07-24

Smart Images

  • Figure CN118553231B_ABST
    Figure CN118553231B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for multi-language speech recognition, comprising: pre-processing and feature extraction of collected multi-source digital speech signals to obtain feature vectors; inputting the feature vectors into an acoustic model for decoding processing to obtain a text representation of the speech signal; inputting the text representation into a natural language processing model for optimizing grammar and semantics to obtain a knowledge representation composed of subsequences; performing format conversion and error correction processing on the knowledge representation to output a final recognition result. The present invention can process multi-source digital speech signals more accurately, and through advanced acoustic models and natural language processing technologies, effectively improves the accuracy of speech recognition and semantic understanding capabilities; at the same time, the proposed natural language processing model performs in-depth optimization and adjustment for grammar and semantics, especially showing strong adaptability in complex language environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition and natural language processing, and in particular to a speech recognition method for multiple languages. Background Art

[0002] In today's world of rapid globalization and informatization, speech recognition technology, as a key technology that allows computers to interpret and transcribe human speech, has been widely used in various fields, including but not limited to mobile communications, smart homes, cross-language translation, and human-computer interaction. Especially in multilingual environments, the demand for speech recognition technology is growing. With the rise of deep learning, acoustic models and natural language processing models have made significant progress. However, the challenges faced by multilingual speech recognition are also becoming increasingly prominent, such as the diversity of speech, accent differences, and grammatical and semantic differences between languages.

[0003] Existing multilingual speech recognition technologies mainly rely on large-scale data sets and complex model training to achieve high-precision recognition of different languages. Although these technologies have improved the accuracy and adaptability of recognition to a certain extent, they still have several shortcomings. For example, when processing multi-source digital speech signals, the existing technologies are often not sophisticated enough in signal preprocessing and feature extraction, making it difficult to fully utilize the speech information in the audio, especially in noisy environments. In addition, the processing of grammar and semantics usually relies on a single natural language processing model, which limits the depth and breadth of semantic understanding, especially when understanding complex multilingual sentences.

[0004] In this regard, the present invention provides a multi-language speech recognition method that can more accurately process multi-source digital speech signals and effectively improve the accuracy of speech recognition and semantic understanding capabilities through advanced acoustic models and natural language processing techniques; at the same time, the proposed natural language processing model performs in-depth optimization and adjustment for grammar and semantics, showing strong adaptability, especially in complex language environments. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] In view of the above existing problems, the present invention is proposed.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: preprocessing and feature extraction are performed on the collected multi-source digital speech signals to obtain feature vectors;

[0008] Inputting the feature vector into an acoustic model for decoding processing to obtain a text representation of the speech signal;

[0009] The decoding process includes:

[0010] Taking the feature vector as input to represent the features of the speech signal;

[0011] Select Viterbi algorithm as the decoding algorithm for calculation;

[0012] Use the Viterbi algorithm to decode the input feature vector sequence and find the most likely text representation sequence;

[0013] The mathematical expression formula for its decoding process is as follows:

[0014] A=Viterbi( (v))

[0015] Among them, A is the text representation sequence, Viterbi ( ) is the Viterbi algorithm, (v) is the functional expression of the acoustic model;

[0016] The text representation is input into a natural language processing model for grammatical and semantic optimization and adjustment to obtain a knowledge representation consisting of subsequences; the mathematical expression formula of the natural language processing model is as follows:

[0017]

[0018] Among them, T is the original text representation, is the knowledge representation consisting of subsequences, is a syntax analysis function used to convert T into a syntax structure. is a semantic analysis function used to convert T into a semantic representation. is a pre-trained language model, is a learning function used to convert T into a high-dimensional vector representation;

[0019] Optimize syntax and semantics, including:

[0020] Preprocessing the input text representation, including but not limited to removing noise and punctuation;

[0021] decomposing the text representation into a sequence of words or phrases;

[0022] Use natural language processing models to perform grammatical analysis and identify the various components in a sentence, including but not limited to the subject, predicate, and object;

[0023] Perform structural analysis based on grammatical rules to ensure that the sentence structure is reasonable;

[0024] Conduct semantic analysis to understand the meaning and logical relationships of sentences;

[0025] Use semantic models to represent words or phrases semantically, taking into account word meaning and contextual factors;

[0026] Based on the results of grammatical and semantic analysis, optimization and adjustment are performed to improve the accuracy and coherence of sentence expression;

[0027] Adjust sentence structure to ensure grammatical correctness;

[0028] Optimize the semantic representation of words or phrases to make them more consistent with the context;

[0029] The knowledge representation is format converted and error corrected, and a final recognition result is output.

[0030] As a preferred solution of the multi-language speech recognition method described in the present invention, the multi-source digital speech signal includes but is not limited to language type, signal source characteristics, environmental noise, and speaker attributes.

[0031] As a preferred solution of the multi-language speech recognition method of the present invention, the pre-processing includes silence removal, noise suppression and echo elimination, and gain control;

[0032] The silence removal calculates the short-time energy of the speech signal to identify the speech segment and the silence segment, sets an energy threshold, and regards the energy above the threshold as a speech segment and the energy below the threshold as a silence segment. The detected silence segment is removed and only the speech segment is retained.

[0033] The noise suppression and echo cancellation include: performing spectrum analysis on the noise in the silent segment, estimating the characteristics of the ambient noise, subtracting the estimated noise spectrum from the spectrum of the speech segment to eliminate background noise, and using adaptive filtering technology to dynamically adjust according to the relationship between the input signal and the output signal to eliminate the echo;

[0034] The gain control includes: performing dynamic range compression on the speech signal to reduce the dynamic range of the signal, so that weak signals are enhanced and strong signals are suppressed, and performing volume standardization processing on the entire speech signal to ensure that the overall volume of the speech is consistent.

[0035] As a preferred solution of the multi-language speech recognition method of the present invention, the feature extraction includes:

[0036] Divide the preprocessed speech signal into small segments, each of which is called a frame;

[0037] Apply a window function to each frame of speech signal to reduce discontinuity at the edge of the frame;

[0038] Apply fast Fourier transform to each frame, convert to frequency domain and get the spectrum;

[0039] Extracting Mel-frequency cepstral coefficients from the spectrum based on the nonlinear frequency perception of the human ear;

[0040] Combining the extracted Mel-frequency cepstral coefficients and other features into a feature vector;

[0041] The other features include but are not limited to pitch and volume.

[0042] As a preferred solution of the multi-language speech recognition method of the present invention, the mathematical expression formula of the feature vector is as follows:

[0043]

[0044] Among them, N is N frames of speech signal, that is, the total number of frames, v is the feature vector, is the signal of the i-th frame, To apply a window function to the i-th frame signal, such as a Hamming window, FFT ( ) is the fast Fourier transform applied to the signal, which is used to convert the time domain signal into the frequency domain signal. MFCC ( ) represents the Mel-frequency cepstral coefficients extracted from the spectrum after FFT transformation, that is, the audio features.

[0045] As a preferred solution of the multi-language speech recognition method of the present invention, the format conversion includes:

[0046] Abstract representation of input knowledge representation, including but not limited to high-dimensional vectors or other forms of abstract representation;

[0047] Convert high-dimensional vectors into text sequences;

[0048] Mapping an abstract representation to a specific data structure, such as a graph or table.

[0049] As a preferred solution of the multi-language speech recognition method of the present invention, the error correction process includes:

[0050] The text sequence or data structure after input format conversion;

[0051] Identify and mark potential errors in the text, such as spelling errors and grammatical errors;

[0052] Adaptively correct detected errors based on context;

[0053] For incomplete or ambiguous text parts, inference or completion is performed to improve recognition accuracy.

[0054] Beneficial effects of the present invention:

[0055] 1. The present invention preprocesses and extracts features from the collected multi-source digital voice signals, effectively reduces the interference of background noise and non-voice elements, improves the purity of the voice signal, and uses signal processing algorithms to extract key feature vectors from complex voice signals, thereby realizing the conversion from original voice signals to feature vectors, thereby improving the accuracy and efficiency of acoustic model decoding;

[0056] 2. By converting feature vectors into text representation, this not only increases the adaptability and flexibility of the speech recognition system to different languages, dialects, and accents, but also uses acoustic models based on deep learning or neural network technology to more accurately match the semantic and structural characteristics of speech, ensuring a high degree of recognition accuracy and providing strong technical support for applications in complex and changing language environments;

[0057] 3. Through natural language processing technology, such as syntax analysis and semantic understanding algorithms, the recognized text is deeply analyzed and structured, which not only improves the fluency and logic of the language, but also improves the accuracy of semantics. In particular, when dealing with complex expressions in a multilingual environment, the knowledge representation of subsequences is adjusted to provide richer and more accurate information content for the final output results.

[0058] 4. Through error detection and correction, errors that may be introduced by acoustic models and natural language processing, such as homophone errors or grammatical inconsistencies, are effectively corrected. Format conversion ensures that the output data is compatible with a variety of applications and devices, enhancing the versatility and user experience of the system and improving the reliability and practicality of the recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0060] Figure 1 The figure is a flow chart of the multi-language speech recognition method shown in the present invention. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0062] Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without making any creative work should fall within the scope of protection of the present invention.

[0063] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Example 1

[0064] According to an embodiment of the present invention, Figure 1 The flowchart shown in the figure is a method for multi-language speech recognition, which specifically includes the following steps:

[0065] S1. Preprocess and extract features of the collected multi-source digital speech signals to obtain feature vectors. Among them, what needs to be explained in this step is:

[0066] The multi-source digital speech signal includes but is not limited to language type, signal source characteristics, environmental noise, and speaker attributes.

[0067] As an example, the language types are multiple languages, such as English, Chinese, French and Spanish, and the speech waveform and spectrum characteristics of each language are different.

[0068] As an example, signal source characteristics include speech signals generated by different recording devices, with different sampling rates and bit depths.

[0069] As an example, environmental noise refers to different types and levels of background noise in different recording environments, such as indoor, outdoor, and traffic noise.

[0070] As an example, speaker attributes include the speaker's gender, age, accent, ie, features that may affect speech.

[0071] Furthermore, preprocessing includes silence removal (breakpoint detection), noise suppression + echo cancellation, and gain control, which specifically includes the following steps:

[0072] (1) Silence removal (breakpoint detection)

[0073] Energy analysis: Calculate the short-term energy of speech signals to identify high-energy areas (speech segments) and low-energy areas (silent segments);

[0074] Threshold setting: Set the energy threshold. Segments above the threshold are considered as speech segments, and those below the threshold are considered as silence segments.

[0075] Silence removal: Remove detected silence segments and keep only speech segments.

[0076] (2) Noise suppression + echo cancellation

[0077] Noise spectrum estimation: Perform spectrum analysis on noise in silent segments to estimate the characteristics of ambient noise;

[0078] Spectral subtraction: subtract the estimated noise spectrum from the spectrum of the speech segment to remove background noise;

[0079] Echo cancellation: Use adaptive filtering technology to dynamically adjust according to the relationship between input and output signals to eliminate echoes.

[0080] (3) Gain control

[0081] Dynamic range compression: Implement dynamic range compression on speech signals to reduce the dynamic range of the signal, so that weak signals are enhanced and strong signals are suppressed;

[0082] Volume Normalization: Perform volume normalization on the entire speech signal to ensure that the overall volume of the speech is consistent.

[0083] It should be noted that the purpose of feature extraction is to extract a feature vector that can effectively represent the speech content from the preprocessed speech signal. The feature extraction includes the following steps:

[0084] (1) Frame segmentation: The preprocessed speech signal is segmented into small segments (e.g., 20-30 ms in length), each of which is called a frame.

[0085] (2) Window function application: Apply a window function (such as a Hamming window) to each frame of speech signal to reduce the discontinuity at the frame edge;

[0086] (3) Fast Fourier transform (FFT): Apply fast Fourier transform to each frame, convert it to the frequency domain, and obtain the spectrum;

[0087] (4) Mel-frequency cepstral coefficient (MFCC) extraction: MFCC is extracted from the spectrum based on the nonlinear frequency perception of the human ear;

[0088] (5) Feature vector generation: The extracted MFCC and other features (such as pitch and volume) are combined into a feature vector.

[0089] Specifically, its mathematical expression formula is as follows:

[0090]

[0091] Among them, N is N frames of speech signal, that is, the total number of frames, v is the feature vector, is the signal of the i-th frame, To apply a window function to the i-th frame signal, such as a Hamming window, FFT ( ) is the fast Fourier transform applied to the signal, which is used to convert the time domain signal into the frequency domain signal. MFCC ( ) represents the Mel-frequency cepstral coefficients extracted from the spectrum after FFT transformation, that is, the audio features.

[0092] This step aims to improve the accuracy and efficiency of multilingual speech recognition. The aforementioned process includes frame segmentation, window function application, fast Fourier transform (FFT), Mel-frequency cepstral coefficient (MFCC) extraction, and feature vector generation, where:

[0093] Frame segmentation: Decompose continuous speech signals into manageable small units, each of which is usually 20-30ms. This segmentation allows the time-varying characteristics of speech to be effectively captured in a shorter time window, reducing the complexity of signal processing and improving the temporal locality of feature extraction. It is crucial to solving the problem of time variation in speech recognition, especially in a multilingual environment. The pronunciation speed and rhythm of different languages ​​vary. By controlling the frame length, these differences can be uniformly processed to a certain extent.

[0094] Window function application: used to reduce the edge effect of the frame. In speech signal processing, the edge discontinuity of the frame may cause spectrum leakage during frequency domain analysis, affecting the accuracy of feature extraction. By applying the Hamming window, the signal at the edge of the frame can be smoothed, the spectral characteristics of the signal can be optimized, and the subsequent frequency domain conversion can be more accurate.

[0095] Fast Fourier Transform (FFT): A key technology for converting time domain signals to frequency domain. It provides a basis for MFCC extraction by analyzing the frequency domain characteristics of each frame of the signal. In multi-source signal processing, FFT not only improves the data processing speed, but also can better distinguish and process signal characteristics from different sources, such as background noise, echo, etc., through detailed spectrum analysis.

[0096] Mel-frequency cepstral coefficient (MFCC) extraction: It is based on the nonlinear perception of sound frequency by the human ear. The main advantage of MFCC extraction is that it simulates the auditory characteristics of humans and can effectively extract the features that best represent the speech content from a complex spectrum. This is particularly important in a multilingual environment because the speech spectrum features of different languages ​​can be effectively captured through MFCC, thereby greatly improving the accuracy of speech recognition.

[0097] Preferably, the embodiments of the present invention are optimized for improving the accuracy and efficiency of multi-source digital speech signal processing, which not only solve the limitations of the prior art, such as the time-variability of signals, edge discontinuity, and inaccuracy of frequency-domain feature extraction, but also improve the performance of multi-language speech recognition systems by simulating the human ear's hearing and integrating various speech features.

[0098] S2. Input the feature vector into the acoustic model for decoding processing to obtain the text representation of the speech signal. Herein, it should be noted that the decoding processing includes the following steps:

[0099] Use the feature vector obtained in step S1 as the input to represent the features of the speech signal;

[0100] Select the Viterbi algorithm as the decoding algorithm for calculation;

[0101] Use the Viterbi algorithm to decode the input feature vector sequence and find the most likely text representation sequence.

[0102] As an example, the mathematical expression formula for its decoding processing is as follows:

[0103] A = Viterbi( (v))

[0104] where A is the text representation sequence, Viterbi( ) is the Viterbi algorithm, (v) is the functional expression of the acoustic model.

[0105] As an example, the text representation sequence represents the text representation of the speech signal obtained through decoding processing.

[0106] As an example, the text representation is presented in the form of a string through text mapping, where the text mapping maps the text representation sequence obtained by decoding to the corresponding text words or phrases.

[0107] As an example, assuming there is a dataset containing various different voices, after being processed by the acoustic model and the Viterbi algorithm, the following text representation sequences are obtained:

[0108] For a speech with the spoken content "hello", the text representation sequence is the string "你好";

[0109] For a speech with the spoken content "apple", the text representation sequence is the string "苹果";

[0110] For a speech with the spoken content "goodbye", the text representation sequence is the string "再见".

[0111] Preferably, this step can effectively improve the speed and accuracy of decoding by adopting an efficient decoding algorithm, thereby accelerating the processing speed of speech recognition and improving the recognition accuracy.

[0112] The acoustic model can use the hidden Markov model (HMM) and deep learning model (such as recurrent neural network, convolutional neural network) in the existing technology to realize calculation and processing, that is, it can be carried out using the methods and means in the existing technology, which will not be repeated in this example.

[0113] S3. Input the text representation into the natural language processing model to optimize the syntax and semantics, and obtain the knowledge representation composed of subsequences. The mathematical expression formula of the natural language processing model is as follows:

[0114]

[0115] Among them, T is the original text representation, is the knowledge representation consisting of subsequences, is a syntax analysis function used to convert T into a syntax structure. is a semantic analysis function used to convert T into a semantic representation. is a pre-trained language model, is a learning function used to transform T into a high-dimensional vector representation.

[0116] Further, the syntax and semantics are optimized and adjusted, which specifically includes the following steps:

[0117] (1) Preprocess the input text representation, such as removing noise and punctuation;

[0118] (2) Decomposing text representation into a sequence of words or phrases;

[0119] (3) Use natural language processing models to perform grammatical analysis and identify the various components in a sentence, such as subject, predicate, and object;

[0120] (4) Perform structural analysis based on grammatical rules to ensure that the sentence structure is reasonable;

[0121] (5) Conduct semantic analysis to understand the meaning and logical relationships of sentences;

[0122] (6) Use semantic models to represent words or phrases semantically, taking into account word meaning and contextual factors;

[0123] (7) Based on the results of grammatical and semantic analysis, make optimization adjustments to improve the accuracy and coherence of sentence expression;

[0124] (8) Adjust sentence structure to ensure grammatical correctness;

[0125] (9) Optimize the semantic representation of words or phrases to make them more consistent with the context.

[0126] It should be noted that knowledge representation is to optimize the semantic representation of words or phrases to make them more in line with the context, or to extract key information from the text to build a knowledge graph or semantic network.

[0127] As an example, the knowledge representation may be in the form of a graph structure, a vector representation, or other forms, which are used to represent the relationship and meaning between words and phrases.

[0128] As an example, the form of knowledge representation can be an abstract representation composed of subsequences, where each subsequence can represent a grammatical unit, a semantic unit, or a language model unit in the text. In this embodiment, the form of knowledge representation is described by the following example:

[0129] Assume there is an original text representation T, for example: "I want to book a flight from NewYork to Paris." After grammatical analysis, the following grammatical structure is obtained:

[0130] S→NPVP

[0131] NP→PRP

[0132] VP→VNPPP

[0133] PRP→"I"

[0134] V→"want"

[0135] NP→"tobookaflight"

[0136] PP→"fromNewYorktoParis"

[0137] After semantic analysis, the following semantic representation is obtained: Intent: "bookaflight", Origin: "NewYork", Destination: "Paris". After representation learning, a high-dimensional vector representation is obtained, for example: KnowledgeRepresentation: [0.10.50.3…0.2]. This vector represents important information such as the grammatical structure, semantic information, and language model score of the text. It is an abstraction and representation of the original text.

[0138] S4: Perform format conversion and error correction on the knowledge representation and output the final recognition result. It should be noted that the format conversion includes:

[0139] An abstract representation of the input knowledge representation, such as a high-dimensional vector or other forms of abstract representation;

[0140] Convert high-dimensional vectors into text sequences;

[0141] Mapping an abstract representation to a specific data structure, such as a graph or table.

[0142] As an example, considering the multi-lingual characteristics, the representation may be converted into formats of multiple languages.

[0143] Furthermore, the error correction process includes:

[0144] The text sequence or data structure after input format conversion;

[0145] Identify and mark potential errors in the text, such as spelling errors and grammatical errors;

[0146] Adaptively correct detected errors based on context;

[0147] For incomplete or ambiguous text parts, inference or completion is performed to improve recognition accuracy.

[0148] As an example, the recognition result is text content (recognized text information, indicating the meaning of the speech signal) and tag information (including grammatical tags and semantic tags) after format conversion and error correction processing.

[0149] As an example, the recognition result is expressed as a text string.

[0150] As an example, the recognition results are output in the form of a data structure to facilitate subsequent processing and analysis.

[0151] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for multilingual speech recognition, It is characterized in that include: Preprocessing and feature extraction are performed on the collected multi-source digital speech signals to obtain feature vectors; Inputting the feature vector into an acoustic model for decoding processing to obtain a text representation of the speech signal; The decoding process includes: Taking the feature vector as input to represent the features of the speech signal; Select Viterbi algorithm as the decoding algorithm for calculation; Use the Viterbi algorithm to decode the input feature vector sequence and find the most likely text representation sequence; The mathematical expression formula for its decoding process is as follows: Where A is the text representation sequence, Viterbi(·) is the Viterbi algorithm, is the function expression of the acoustic model; The mathematical expression formula of the eigenvector is as follows: Among them, N is N frames of speech signal, that is, the total number of frames, v is the feature vector, s i (t) is the signal of the i-th frame, Window(s i (t)) is the window function applied to the i-th frame signal, FFT(·) is the fast Fourier transform applied to the signal, which is used to convert the time domain signal into the frequency domain signal, and MFCC(·) represents the Mel frequency cepstrum coefficient extracted from the spectrum after FFT transformation, that is, the audio feature; The text representation is input into a natural language processing model for grammatical and semantic optimization and adjustment to obtain a knowledge representation consisting of subsequences; the mathematical expression formula of the natural language processing model is as follows: K(T)=O[G(T), S(T), L(T), E(T)] Where T is the original text representation, K(T) is the knowledge representation composed of subsequences, G(T) is the syntax analysis function used to convert T into a syntax structure, S(T) is the semantic analysis function used to convert T into a semantic representation, L(T) is the pre-trained language model, and E(T) is the learning function used to convert T into a high-dimensional vector representation; Optimize syntax and semantics, including: Preprocessing the input text representation, including but not limited to removing noise and punctuation; decomposing the text representation into a sequence of words or phrases; Use natural language processing models to perform grammatical analysis and identify the various components in a sentence, including but not limited to the subject, predicate, and object; Perform structural analysis based on grammatical rules to ensure that the sentence structure is reasonable; Conduct semantic analysis to understand the meaning and logical relationships of sentences; Use semantic models to represent words or phrases semantically, taking into account word meaning and contextual factors; Based on the results of grammatical and semantic analysis, optimization and adjustment are performed to improve the accuracy and coherence of sentence expression; Adjust sentence structure to ensure grammatical correctness; Optimize the semantic representation of words or phrases to make them more consistent with the context; The knowledge representation is format converted and error corrected, and a final recognition result is output.

2. The multi-language speech recognition method according to claim 1, It is characterized in that The multi-source digital voice signals include but are not limited to language types, signal source characteristics, environmental noise, and speaker attributes.

3. The multi-language speech recognition method according to claim 1, It is characterized in that The pre-processing includes silence removal, noise suppression, echo cancellation, and gain control; The silence removal calculates the short-time energy of the speech signal to identify the speech segment and the silence segment, sets an energy threshold, and regards the energy above the threshold as a speech segment and the energy below the threshold as a silence segment. The detected silence segment is removed and only the speech segment is retained. The noise suppression and echo cancellation include: performing spectrum analysis on the noise in the silent segment, estimating the characteristics of the ambient noise, subtracting the estimated noise spectrum from the spectrum of the speech segment to eliminate background noise, and using adaptive filtering technology to dynamically adjust according to the relationship between the input signal and the output signal to eliminate the echo; The gain control includes: performing dynamic range compression on the speech signal to reduce the dynamic range of the signal, so that weak signals are enhanced and strong signals are suppressed, and performing volume standardization processing on the entire speech signal to ensure that the overall volume of the speech is consistent.

4. The multi-language speech recognition method according to claim 1, It is characterized in that The feature extraction comprises: Divide the preprocessed speech signal into small segments, each of which is called a frame; Apply a window function to each frame of speech signal to reduce discontinuity at the edge of the frame; Apply fast Fourier transform to each frame, convert to frequency domain and get the spectrum; Extracting Mel-frequency cepstral coefficients from the spectrum based on the nonlinear frequency perception of the human ear; Combining the extracted Mel-frequency cepstral coefficients and other features into a feature vector; The other features include but are not limited to pitch and volume.

5. The multi-language speech recognition method according to claim 1, It is characterized in that Format conversion includes: Abstract representation of input knowledge representation, including but not limited to high-dimensional vectors or other forms of abstract representation; Convert high-dimensional vectors into text sequences; Mapping an abstract representation to a specific data structure.

6. The multi-language speech recognition method according to claim 1, It is characterized in that Error correction processing includes: The text sequence or data structure after input format conversion; Identify and flag potential errors in the text; Adaptively correct detected errors based on context; For incomplete or ambiguous text parts, inference or completion is performed to improve recognition accuracy.