A speech recognition system and method based on a double-weighted directed graph

Abstract

A speech recognition system and method based on a double-weight directed graph, comprising the following steps: 1) a language flow module divides continuous speech into multiple audio frames, and sends the obtained multiple audio frames to an acoustic module; 2) the acoustic module converts the audio frames into an acoustic model; 3) a language module generates a directed and weighted decoding graph according to a text knowledge base, and sends the weighted directed decoding graph to a double-weight wide search module; 4) the double-weight wide search module processes the acoustic model and the weighted directed decoding graph to preliminarily generate a recognition result; 5) an identification output module receives the recognition result and path total loss value sent by the double-weight wide search module, and further processes the two to obtain a language recognition result, and outputs the language recognition result. The application can consider the recognition of logical language output and non-logical speech, retain the use of a language model to remove the influence of pronunciation proximity, and improve the recognition rate of non-logical language such as a telephone number.

Description

A speech recognition system and method based on dual-weighted directed graphs Technical Field

[0001] This invention relates to the field of speech recognition, specifically to a speech recognition system and method based on a dual-weighted directed graph. Background Technology

[0002] By employing a weighted finite state converter based on a directed graph, and fusing acoustic and text models, balanced weights are used to control whether speech recognition favors the acoustic or text model. Increasing the weight of the text model results in outputs that are more consistent with linguistic logic, but it affects the recognition of non-logical speech such as telephone numbers. Increasing the weight of the acoustic model improves the recognition rate of non-logical speech, but it affects the recognition rate of speech with similar pronunciations. It is not possible to guarantee improvements in both aspects at the same time. Therefore, it is essential to propose a speech recognition system and method based on a dual-weighted directed graph. Summary of the Invention

[0003] The purpose of this invention is to provide a speech recognition system and method based on dual-weighted directed graphs, which can improve the recognition rate of non-logical speech by using a language model to remove the influence of similar pronunciation during the speech recognition process, thereby solving the technical defects and unmet technical requirements of the existing technology.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a speech recognition system based on a dual-weighted directed graph, comprising:

[0005] The speech stream module converts speech into audio frames and then sends the audio frames to the acoustic module.

[0006] The acoustic module receives audio frames sent from the speech stream module, processes the audio frames, inputs them into the acoustic model, obtains the acoustic model output, and then sends the acoustic model output to the dual-weighted wide search module.

[0007] The language module converts the text knowledge base into a directed weighted decoding graph and sends the directed weighted decoding graph to the dual-weighted wide-search module;

[0008] The dual-weighted wide-search module receives the acoustic model sent by the acoustic module and the directed weighted decoding graph sent by the language module, sets two balanced weights, uses the two balanced weights to perform a width search on the directed weighted decoding graph, finds all possible results, and sends all possible results to the recognition output module.

[0009] The recognition output module is used to receive all possible results sent by the dual-weighted wide search module, recognize all possible results, and output language recognition results.

[0010] A speech recognition method based on a dual-weighted directed graph includes the following steps:

[0011] 1) The speech stream module divides continuous speech into multiple audio frames and sends the resulting audio frames to the acoustic module;

[0012] 2) The acoustic module converts audio frames into acoustic model outputs;

[0013] 2.1) The acoustic module receives audio frames sent by the speech stream module;

[0014] 2.2) The acoustic module multiplies the audio frames by the Hamming window function;

[0015] 2.3) The audio frames processed in step 2.2) of the acoustic module are converted into frequency domain data using FFT;

[0016] 2.4) The acoustic module processes the frequency domain data to obtain Fbank features, and inputs the obtained Fbank features into the generated acoustic model to obtain the acoustic model output;

[0017] 2.5) The acoustic module sends the acoustic model output to the dual-weighted wide-search module;

[0018] 3) The language module generates a directed weighted decoding graph based on the text knowledge base and sends this directed weighted decoding graph to the dual-weighted wide-search module;

[0019] 4) The dual-weighted wide-search module processes the acoustic model output and the weighted and directed decoding graph to initially generate recognition results.

[0020] 4.1) The dual-weighted wide-search module receives the acoustic model output sent by the acoustic module and the weighted and directed decoding graph sent by the language module;

[0021] 4.2) The dual-weighted wide-search module sets two balanced weights;

[0022] 4.3) The dual-weighted wide-search module starts the width search from the starting node of the directed weighted decoding graph and adds the starting node and the total path loss value to the wide-search queue for the next round.

[0023] 4.4) For each audio frame's acoustic model output received, the dual-weighted wide-search module traverses the wide-search queue to retrieve the current processing node and the total path loss value, processes them sequentially, and obtains a new node.

[0024] 4.5) The dual-weighted wide search module uses new nodes to obtain the loss value of the words, and further processes the loss value of the words to obtain the loss value of the words in two ways.

[0025] 4.6) The dual-weighted wide-search module further processes the total path loss value in step 4.3 to obtain two total path loss values;

[0026] 4.7) The dual-weighted wide-search module compares the total loss value of the two paths with the truncation loss value, filters out the paths that need to be discarded, and further processes the paths that are not discarded.

[0027] 4.8) When the audio of a speech segment ends, the dual-weighted wide-search module generates the path recognition result and the total path loss value;

[0028] 4.9) The dual-weighted wide-search module sends the recognition results and the total path loss value to the recognition output module;

[0029] 5) The recognition output module receives the recognition result and the total path loss value sent by the dual-weighted wide search module, processes them further to obtain the language recognition result, and outputs the language recognition result.

[0030] It is necessary to further explain in this application that there is a length filtering process between steps 4.2) and 4.3): the phoneme of the current processing node representing a word outputs a query probability value in the acoustic model. If the probability value is greater than the same-word threshold, it is considered that the pronunciation of the word has not ended, and the current processing node and the total path loss value are added to the next round of wide search queue. If the probability value is less than the same-word threshold, it means that the current processing node has no next or next node in the directed weighted decoding graph, and it is considered that some partial recognition results have been found, and the path loss value is added to the total path loss value. Then the processing of step 4.3) is performed again, that is, the initial node of the starting node of the directed weighted decoding graph and the total path loss value are initialized to zero, and the total path loss value and the initial node are added to the next round of wide search queue for recognizing the remaining speech. The remaining speech here is the speech remaining after filtering after the above steps. It can be understood that the content processed in 4.3 is the speech after length filtering in the above steps.

[0031] Preferably, in step 1), the specific requirement for dividing continuous speech into multiple audio frames is as follows:

[0032] There is a 1 / 2 overlap between two adjacent frames.

[0033] Preferably, the specific content of step 2.4) includes:

[0034] 2.4.1) The acoustic module uses the Mel filter bank to filter the power spectrum in the frequency domain data to simulate the characteristics of human hearing;

[0035] The Mel filter bank consists of a series of triangular filters;

[0036] 2.4.2) The acoustic module takes the logarithm of the filtered signal to obtain the Fbank feature;

[0037] 2.4.3) The acoustic module uses Fbank features as input to the Transformer neural network encoder and outputs the acoustic model output, which is represented as the probability value of a phoneme series. Each audio frame corresponds to a phoneme series probability value.

[0038] Preferably, the specific contents processed sequentially in step 4.4) are as follows:

[0039] 4.4.1) The dual-weight wide-search module extracts the weights of all the next single nodes and the edges between two nodes from the directed weighted decoding graph according to the current processing node, and processes all the next single nodes as new nodes one by one.

[0040] 4.4.2) The dual-weight wide-search module subtracts the weights of the two node edges from one to obtain the loss value of the two node edges.

[0041] Preferably, the specific content of step 4.5) is as follows:

[0042] 4.5.1) The dual-weighted wide search module uses a new node to represent the probability value of a word's phoneme in the acoustic model output that it can be queried. This probability value is the likelihood of the word. The dual-weighted wide search module subtracts this probability value to obtain the word's loss value.

[0043] 4.5.2) The dual-weighted wide-search module uses the two balancing weights in step 4.2) to multiply the loss value of the word in step 4.5.1) to obtain the two loss values ​​of the word, namely the loss value of the higher balancing weight and the loss value of the lower balancing weight.

[0044] Preferably, the specific content of step 4.6) is as follows: the dual-weighted wide search module adds the total path loss value in step 4.3) to the loss value of the two node edges in step 4.4.2), and then adds the loss values ​​of the two paths of the new node's vocabulary in step 4.5.2) respectively to obtain the two total path loss values.

[0045] 8. The speech recognition method based on a dual-weighted directed graph according to claim 2, characterized in that the specific content of step 4.7) is as follows:

[0046] 4.7.1) The dual-weighted wide-search module compares the total loss value of the two paths in step 4.6) with the truncation loss value;

[0047] 4.7.2) Set a truncation loss value. If the total loss value of a path is greater than the truncation loss value, then discard the path.

[0048] 4.7.3) If the total path loss is less than the truncation loss, the dual-weighted wide-search module adds the new node and the total path loss to the next round of wide-search queue. A new node can be added to the next round a maximum of twice.

[0049] Preferably, the specific content of the path identification result and the total path loss value generated by the dual-weighted wide-search module in step 4.8) is as follows:

[0050] 4.8.1) The dual-weighted wide-search module traverses the wide-search queue to retrieve the total path loss value;

[0051] 4.8.2) Take the words above the path and combine them to form the preliminary recognition result of the path.

[0052] The specific details of further processing the identification results and total path loss value in step 5) include:

[0053] 5.1) Set a reduction coefficient and a character number threshold. The recognition output module performs special word statistics on the preliminary recognition results in step 4.8.2). If the number of characters exceeds the set character number threshold, the total path loss value is divided by the reduction coefficient. The character number threshold means the set value for the number of characters.

[0054] 5.2) The identification output module sorts the total path loss values ​​from smallest to largest;

[0055] 5.3) The recognition output module outputs the recognition result with the minimum total path loss value as the final language recognition result.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] 1. This application can take into account both logical language output and non-logical speech recognition, while retaining the use of language models to remove the influence of similar pronunciation, and improving the recognition rate of non-logical language such as telephone numbers. Attached Figure Description

[0058] Figure 1 is a logical relationship diagram between the modules in this invention;

[0059] In the diagram: Language Flow Module 1; Acoustic Model 2; Language Module 3; Dual-Weight Wide Search Module 4; Recognition Output Module 5. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to Figure 1. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Please refer to Figure 1 for an embodiment of the present invention:

[0062] Example:

[0063] As shown in Figure 1: A speech recognition system based on a dual-weighted directed graph includes:

[0064] The speech stream module 1 converts speech into audio frames and then sends the audio frames to the acoustic module 2.

[0065] Acoustic module 2 receives audio frames sent from language stream module 1, processes the audio frames and inputs them into the acoustic model to obtain the acoustic model output, and then sends the acoustic model output to dual-weighted wide search module 4.

[0066] Language module 3 converts the text knowledge base into a directed weighted decoding graph and sends the directed weighted decoding graph to the dual-weighted wide search module 4;

[0067] The dual-weighted wide-search module 4 receives the acoustic model sent by the acoustic module 2 and the directed weighted decoding graph sent by the language module 3, sets two balanced weights, uses the two balanced weights to perform a width search on the directed weighted decoding graph, finds all possible results, and sends all possible results to the recognition output module 5.

[0068] The recognition output module 5 is used to receive all possible results sent by the dual-weighted wide search module 4, recognize all possible results, and output language recognition results.

[0069] A speech recognition method based on a dual-weighted directed graph includes the following steps:

[0070] 1) The speech stream module 1 divides the continuous speech into multiple audio frames and sends the resulting multiple audio frames to the acoustic module 2;

[0071] Specifically, the speech stream module 1 divides the continuous speech into multiple audio frames. Each frame consists of N sampling points that are grouped into one observation unit. There is a 1 / 2 overlap between two adjacent frames to avoid excessive changes between adjacent frames.

[0072] 2) Acoustic module 2 converts audio frames into acoustic model output;

[0073] 2.1) Acoustic module 2 receives audio frames sent by speech stream module 1;

[0074] 2.2) Acoustic module 2 multiplies the audio frame by the Hamming window function to increase the continuity of the left and right ends of the frame and reduce spectral leakage;

[0075] 2.3) The audio frames processed in step 2.2) of acoustic module 2 are converted into frequency domain data by FFT;

[0076] 2.4) After processing the frequency domain data, the acoustic module 2 inputs it into the generated acoustic model to obtain the acoustic model output;

[0077] The specific content of step 2.4) includes:

[0078] 2.4.1) Acoustic Module 2 uses the Mel filter bank to filter the power spectrum in the frequency domain data to simulate the characteristics of human hearing. The Mel filter bank consists of a series of triangular filters.

[0079] 2.4.2) Acoustic module 2 takes the logarithm of the filtered signal to obtain the Fbank feature;

[0080] 2.4.3) Acoustic module 2 uses Fbank features as input to the output acoustic model of the Transformer neural network encoder. The output acoustic model is represented as the probability value of a phoneme series, with each audio frame corresponding to a phoneme series probability value.

[0081] Specifically, it can be understood that the acoustic module 2 uses Fbank features as input to the Transformer neural network encoder to obtain the acoustic model output. The acoustic model output is represented as a series of phoneme probability values, with each audio frame corresponding to a series of phoneme probability values.

[0082] 2.5) Acoustic module 2 sends the acoustic model to dual-weighted wide-search module 4;

[0083] 3) Language module 3 generates a directed weighted decoding graph based on the text knowledge base and sends this weighted directed decoding graph to the dual-weighted wide search module 4;

[0084] Specifically, based on the occurrence of numerous sentence patterns and words in the text knowledge base, Language Module 3 uses a statistical language model to automatically learn syntactic rules from text data, ultimately generating a directed weighted decoding graph. Since numbers such as years have no logical connection, Language Module 3 performs preprocessing to remove number-related sentence patterns before using the statistical language model.

[0085] 4) The dual-weighted wide-search module 4 processes the acoustic model output and the weighted and directed decoding graph to initially generate recognition results.

[0086] 4.1) The dual-weighted wide-search module 4 receives the acoustic model output sent by the acoustic module 2 and the weighted and directed decoding graph sent by the language module 3;

[0087] 4.2) The dual-weighted wide-search module 4 sets two balanced weights to take into account the high-weighted output of the output acoustic model and the directed weighted decoding graph, and to take into account the logical language output and the recognition of non-logical speech.

[0088] 4.3) The dual-weighted wide-search module 4 starts the width search from the starting node of the directed weighted decoding graph and adds the starting node and the total path loss value to the next round of the wide-search queue. The starting node and the total path loss value are zero.

[0089] 4.4) For each audio frame's acoustic model output received, the dual-weighted wide-search module 4 traverses the wide-search queue to retrieve the current processing node and the total path loss value, and processes them sequentially to obtain a new node.

[0090] The specific processes in step 4.4) are as follows:

[0091] 4.4.1) The dual-weight wide search module 4 extracts the weights of all the next one node and the two nodes from the directed weighted decoding graph according to the current processing node, and processes all the next one nodes as new nodes one by one.

[0092] In a directed weighted decoding graph, each node represents a word, and the weight of an edge between two nodes represents the probability of one word being associated with another word.

[0093] 4.4.2) The dual-weight wide-search module 4 uses one to subtract the weights of the two node edges to obtain the loss values ​​of the two node edges.

[0094] The loss value of the two-node edge represents the cost of going from one word to another, that is, the cost represents one minus the probability of the two nodes.

[0095] 4.5) The dual-weighted wide-search module 4 uses new nodes to obtain the loss value of the words, and further processes the loss value of the words to obtain the two-way loss value of the words. The loss value with a high balance weight will increase the weight of the acoustic model, which will increase the loss value of the words and shorten the path length found by the wide search in the directed weighted decoding graph, thus making it more conducive to the recognition of content without linguistic logic. The loss value with a low balance weight will reduce the loss value of the words, and the wide search can find a longer path, recognizing content with more linguistic logic.

[0096] The specific content of step 4.5) is as follows:

[0097] 4.5.1) The dual-weighted wide search module 4 uses a new node to represent the probability value that a word's phoneme can be found in the acoustic model output. This probability value is the likelihood of the word. The dual-weighted wide search module 4 subtracts this probability value to obtain the word's loss value.

[0098] 4.5.2) The dual-weighted wide-search module 4 uses the two balanced weights in step 4.2) to multiply the loss value of the words in step 4.5.1) to obtain the loss value of the words in both directions.

[0099] 4.6) The dual-weighted wide-search module 4 further processes the total path loss value in step 4.3 to obtain two total path loss values;

[0100] The specific content of step 4.6) is as follows: the dual-weighted wide search module 4 adds the total path loss value in step 4.3) to the loss value of the two node edges in step 4.4.2), and then adds the loss values ​​of the two paths of the vocabulary of the new node in step 4.5.2) respectively to obtain the two total path loss values.

[0101] 4.7) The dual-weighted wide-search module 4 compares the total loss value of the two paths with the truncation loss value, filters out the paths that need to be discarded, and further processes the paths that are not discarded.

[0102] The specific content of step 4.7) is as follows:

[0103] 4.7.1) The dual-weighted wide-search module 4 compares the total loss value of the two paths in step 4.6) with the truncation loss value;

[0104] 4.7.2) Set a truncation loss value. If the total loss value of a path is greater than the truncation loss value, then discard the path.

[0105] 4.7.3) If the total path loss is less than the truncation loss, the dual-weight wide search module (4) adds the new node and the total path loss to the next round of wide search queue. A new node can be added to the next round at most twice, so as to realize the simultaneous recognition of logical language output and non-logical speech.

[0106] 4.8) When the audio of a speech segment ends, the dual-weighted wide search module 4 generates the path recognition result and the total path loss value;

[0107] The specific content of the path identification result and total path loss value generated by the dual-weighted wide-search module 4 in step 4.8) is as follows:

[0108] 4.8.1) The dual-weighted wide-search module traverses the wide-search queue four times to retrieve the total path loss value;

[0109] 4.8.2) Take the words above the path and combine them to form the preliminary recognition result of the path.

[0110] 4.9) The dual-weighted wide-search module 4 sends the recognition result and the total path loss value to the recognition output module 5.

[0111] 5) The recognition output module 5 receives the recognition result and the total path loss value sent by the dual-weighted wide search module 4, processes them further to obtain the language recognition result, and outputs the language recognition result.

[0112] The specific details of further processing the identification results and total path loss value in step 5) include:

[0113] 5.1) Set a threshold for the number of characters to reduce the coefficient. The recognition output module 5 performs special word statistics on the preliminary recognition results in step 4.8.2), such as numbers and other words without language logic. If the number of characters exceeds the set threshold, the total path loss value is divided by the reduction coefficient to reduce the total path loss value and improve the selection rate of the recognition result.

[0114] 5.2) The identification output module 5 sorts the adjusted total path loss values ​​from smallest to largest; "adjusted" refers to the values ​​after processing in step 5.1.

[0115] 5.3) The recognition output module 5 outputs the recognition result with the minimum total path loss value as the final language recognition result.

[0116] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0117] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A speech recognition system based on a dual-weighted directed graph, characterized in that, include: The language flow module (1) converts speech into audio frames and sends the audio frames to the acoustic module (2); the acoustic module (2) receives the audio frames sent by the language flow module (1), processes the audio frames and inputs them into the acoustic model to obtain the acoustic model output, and then sends the acoustic model output to the dual-weighted wide-search module (4); the language module (3) converts the text knowledge base into a directed weighted decoding graph and sends the directed weighted decoding graph to the dual-weighted wide-search module (4); the dual-weighted wide-search module (4) receives the acoustic model output sent by the acoustic module (2) and the directed weighted decoding graph sent by the language module (3), sets two balancing weights, uses the two balancing weights to perform a width search on the directed weighted decoding graph, and adds the starting node and the total path loss value to the next round of the wide-search queue. The acoustic model output of each received audio frame is then used to perform a width search. Each node is traversed through the wide search queue to retrieve the current processing node and the total path loss value, and then processed sequentially to obtain a new node. The dual-weight wide search module (4) then uses the new node to represent the phoneme of a word in the acoustic model output probability value that can be found. This probability value is the possibility of the word. The dual-weight wide search module (4) subtracts this probability value to obtain the word loss value, and then uses two balanced weights to multiply the word loss value to obtain the loss values ​​of the two paths of the word. The total path loss value is compared with the truncation loss value to filter out the paths that need to be discarded. The paths that are not discarded are then further processed. After the audio of a speech segment ends, the dual-weight wide search module (4) generates the path recognition result and the total path loss value. Finally, the recognition result and the total path loss value are sent to the recognition output module (5). The recognition output module (5) is used to receive all possible results sent by the dual-weight wide search module (4), and to recognize all possible results and output the language recognition result.

2. A speech recognition method based on a dual-weighted directed graph, characterized in that, Includes the following steps: 1) The speech stream module (1) divides the continuous speech into multiple audio frames and sends the resulting audio frames to the acoustic module (2); 2) The acoustic module (2) converts the audio frames into an acoustic model output; 2.1) The acoustic module (2) receives the audio frames sent by the speech stream module (1); 2.2) The acoustic module (2) multiplies the audio frames by a Hamming window function; 2.3) The acoustic module (2) performs FFT on the processed audio frames in step 2.2) to convert them into frequency domain data; 2.4) The acoustic module (2) processes the frequency domain data and then inputs it into the generated acoustic model to obtain the acoustic model. Model output; 2.5) The acoustic module (2) sends the acoustic model output to the dual-weighted wide search module (4); 3) The language module (3) generates a directed weighted decoding graph based on the text knowledge base and sends this directed weighted decoding graph to the dual-weighted wide search module (4); 4) The dual-weighted wide search module (4) processes the acoustic model output and the directed weighted decoding graph to generate preliminary recognition results; 4.1) The dual-weighted wide search module (4) receives the acoustic model output sent by the acoustic module (2) and the directed weighted decoding graph sent by the language module (3); 4.2) The dual-weighted wide search module (4) sets two balanced weights; 4.3) The dual-weighted wide-search module (4) starts the width search from the starting node of the directed weighted decoding graph and adds the starting node and the total path loss value to the next round of the wide-search queue; 4.4) For each audio frame of acoustic model output received, the dual-weighted wide-search module (4) traverses the wide-search queue to retrieve the current processing node and the total path loss value, and processes them sequentially to obtain a new node; 4.5) The dual-weighted wide-search module (4) uses the new node to obtain the loss value of the word, and further processes the loss value of the word to obtain the loss values ​​of the two paths of the word; 4.6) The dual-weighted wide-search module (4) further processes the total path loss value in step 4.3 to obtain two total path loss values; 4.7) The dual-weighted wide-search module (4) compares the two total path loss values ​​with the truncation loss value, filters out the paths that need to be discarded, and further processes the paths that are not discarded; 4.8) When the audio of a speech segment ends, the dual-weighted wide-search module (4) generates the path recognition result and the total path loss value; 4.9) The dual-weighted wide-search module (4) sends the recognition result and the total path loss value to the recognition output module (5); the specific content of step 4.5) is as follows: 4.5.1) The dual-weighted wide search module (4) uses the new node to represent the probability value of the phoneme of a word in the acoustic model output that can be found. This probability value is the possibility of the word. The dual-weighted wide search module (4) subtracts this probability value to obtain the loss value of the word. 4.5.2) The dual-weighted wide search module (4) uses the two balanced weights in step 4.2) to multiply the loss value of the word in step 4.5.1) to obtain the loss values ​​of the two paths of the word. 5) The recognition output module (5) receives the recognition result and the total path loss value sent by the dual-weighted wide search module (4), and further processes the two to obtain the language recognition result and outputs the language recognition result.

3. The speech recognition method based on a dual-weighted directed graph according to claim 2, characterized in that, In step 1), the specific requirement for dividing continuous speech into multiple audio frames is that there is a 1 / 2 overlap between two adjacent frames.

4. The speech recognition method based on a dual-weighted directed graph according to claim 3, characterized in that, The specific contents of step 2.4) include: 2.4.1) The acoustic module (2) applies the Mel filter bank to filter the power spectrum in the frequency domain data to simulate the hearing characteristics of the human ear; 2.4.2) The acoustic module (2) takes the logarithm of the filtered signal to obtain the Fbank feature; 2.4.3) The acoustic module (2) uses the Fbank feature to input into the Transformer neural network encoder and outputs the acoustic model output, which is represented as the probability value of a phoneme series, and each audio frame corresponds to a phoneme series probability value.

5. The speech recognition method based on a dual-weighted directed graph according to claim 4, characterized in that, The specific contents processed in step 4.4) are as follows: 4.4.1) The double-weighted wide search module (4) extracts the weights of all the next one node and the two nodes from the directed weighted decoding graph according to the current processing node; 4.4.2) The double-weighted wide search module (4) uses one to subtract the weight of the two node edges to obtain the loss value of the two node edges, and then processes each of the next nodes as new nodes one by one.

6. The speech recognition method based on a dual-weighted directed graph according to claim 2, characterized in that, The specific content of step 4.6) is as follows: the dual-weighted wide search module (4) adds the total path loss value in step 4.3) to the loss value of the two node edges in step 4.4.2), and then adds the loss values ​​of the two paths of the vocabulary of the new node in step 4.5.2) respectively to obtain the two total path loss values.

7. The speech recognition method based on a dual-weighted directed graph according to claim 6, characterized in that, The specific content of step 4.7) is as follows: 4.7.1) The dual-weighted wide-search module (4) compares the total loss value of the two paths in step 4.6) with the truncation loss value; 4.7.2) Set the truncation loss value. If the total loss value of the path is greater than the truncation loss value, the path is discarded; 4.7.3) If the total loss value of the path is less than the truncation loss value, the dual-weighted wide-search module (4) adds the new node and the new path loss value to the next round of wide-search queue. A new node can be added to the next round at most twice.

8. The speech recognition method based on a dual-weighted directed graph according to claim 7, characterized in that, The specific contents of the path identification result and total path loss value generated by the dual-weighted wide-search module (4) in step 4.8) are as follows: 4.8.1) The dual-weighted wide search module (4) traverses the wide search queue to retrieve the total path loss value; 4.8.2) Take the words on the path and combine them to form the preliminary recognition result of the path.

9. The speech recognition method based on a dual-weighted directed graph according to claim 8, characterized in that, The specific content of further processing of the recognition result and the total path loss value in step 5) includes: 5.1) setting a threshold for the number of characters to reduce the coefficient, and the recognition output module (5) performing special word statistics on the preliminary recognition result in step 4.8.2). If the number of characters exceeds the set threshold, the total path loss value is divided by the reduction coefficient; 5.2) the recognition output module (5) sorts the adjusted total path loss value from smallest to largest; 5.3) the recognition output module (5) outputs the recognition result with the smallest total path loss value as the final language recognition result.

Images