Speech recognition optimization method and system, storage medium and equipment

Abstract

The invention relates to the technical field of voice recognition, and discloses a voice recognition optimization method and system, a storage medium and equipment, and the method employs a double-channel audio signal processing and parallel wake-up detection mechanism, and improves the wake-up rate and anti-interference capability of the system. An adaptive selection mechanism is introduced, and an optimal voice recognition input path is dynamically selected according to a wake-up result, so that the recognition accuracy is improved; a microphone fault detection and adaptation strategy is introduced, the system performance is maintained under the condition that part of microphones are damaged, and the robustness and reliability of the system are enhanced; through contextual information integration, adaptive acoustic model selection and dynamic decoding graph construction, the speech recognition process is further optimized; a performance evaluation and self-optimization mechanism is introduced, system parameters can be continuously improved according to actual use data, and long-term stable operation is achieved. According to the invention, the overall performance of the voice interaction system is improved, the application scene is expanded, and more intelligent, reliable and humanized voice interaction experience is provided.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition optimization method, system, storage medium and device. Background Art

[0002] As electronic devices that perform various functions in combination have been developed, electronic devices equipped with voice wake-up and recognition functions have been released to improve operability. The voice wake-up and recognition functions can easily control the device by recognizing the user's voice in a hands-free manner (e.g., without separate button operation or contact of a touch module).

[0003] Existing voice wake-up systems usually only use single-channel data for wake-up judgment, which is easily affected by environmental noise and hardware failure, resulting in unstable wake-up rate. At the same time, a fixed channel of data is selected as the voice recognition input, which cannot adapt to different scenarios and hardware conditions, affecting the accuracy of voice recognition. At the same time, this method performs particularly poorly when the microphone array is partially damaged, seriously affecting the user experience. This method can achieve certain performance under ideal conditions, but faces many challenges in practical applications.

[0004] First, environmental noise and hardware failures may cause unstable wake-up rates and affect system reliability. Secondly, fixed selection of a certain data path as speech recognition input cannot adapt to different usage scenarios and hardware conditions, which may lead to a decrease in recognition accuracy. Especially in multi-microphone arrays, when some microphones are damaged, existing technologies are difficult to deal with effectively, which may cause a sharp decline in the performance of the entire system. These problems seriously restrict the application scope and user experience of voice interaction devices. Therefore, there is an urgent need for a new method that can improve the wake-up rate, improve speech recognition performance, and have strong fault tolerance. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the above-mentioned prior art and to provide a speech recognition optimization method, system, storage medium and device.

[0006] A speech recognition optimization method includes the following steps: S1: collects multiple channels of original audio signals from a multi-microphone array and generates two-channel audio signals after enhancement processing; S2: extracting features from the two-way audio signals, performing parallel detection using a pre-trained wake-up word detection model, and comprehensively determining whether to wake up based on the detection results; S3: selecting an optimal voice recognition input path according to the wake-up detection result, and extracting audio data for voice recognition according to the input path; S4: The acoustic model is selected based on the contextual information and historical recognition results, while the decoding graph is dynamically constructed.

[0007] Furthermore, before performing the speech recognition in step S3, the microphone for inputting the speech is subjected to fault detection and adaptation, including: Performing quality assessment on the multiple channels of original audio signals, and identifying microphones that may be faulty based on the signal quality assessment results; If the microphone is faulty, the audio signal processing strategy is adjusted according to the identified faulty microphone condition.

[0008] The step of performing quality assessment on the multiple channels of original audio signals includes: The short-time energy and spectrum flatness of each signal are calculated, and the microphones that may have faults are identified based on the signal quality evaluation results. If the short-time energy of a signal is lower than the threshold or the spectrum flatness is abnormal, it is determined that the corresponding microphone has a probability of being faulty.

[0009] Furthermore, in step S1, a two-way audio signal is generated after enhancement processing, including: The enhancement process uses an adaptive filtering algorithm to suppress noise; Based on the enhanced multi-channel audio signals, two differently processed audio signals are generated. The first channel uses beamforming technology to generate a directionally enhanced audio signal, and the second channel uses blind source separation technology to extract the audio signal of the main sound source.

[0010] Furthermore, in step S2, feature extraction is performed on the two-way audio signal, including: Based on the feature extraction algorithm, feature extraction is performed on the two-way audio signal respectively to obtain two feature vectors for wake-up word detection.

[0011] Furthermore, in step S2, a pre-trained wake-up word detection model is used to perform parallel detection, including: At the same time, two independent keyword detection models based on deep neural networks are used to perform parallel detection on the extracted feature vectors, and each model processes one feature vector; The keyword detection model based on the deep neural network outputs the probability of the presence of the wake-up word and generates a detection result.

[0012] Further, in step S2, whether to wake up is determined comprehensively according to the detection result, including: A threshold is set between 0 and 1 to determine whether the wake-up word exists, and the two detection results are determined in turn to determine whether they exceed the preset threshold. If they exceed the preset threshold, the path of the path is recorded as the wake-up path that triggers wake-up; If any detection result exceeds a preset threshold, it is determined that the wake-up is successful.

[0013] Further, in step S3, selecting the optimal speech recognition input path according to the wake-up detection result includes: If both paths are wake-up paths, compare the detection results of the two paths and take the one with the higher score as the optimal speech recognition input path; If there is only one wake-up path, then this wake-up path is taken as the optimal speech recognition input path; If not awakened, the default path is used.

[0014] Furthermore, in step S4, an acoustic model is selected according to the context information and the historical recognition results, including: The context information includes the wake-up detection result and the information of the speech recognition input path, maintains multiple acoustic models optimized for different noise environments and hardware states, and selects the best model based on the context information and historical recognition results.

[0015] Furthermore, in step S4, the dynamically constructing a decoding graph includes: Based on the context information and historical recognition results, a decoding graph is dynamically constructed, several models and resources are combined together, and the weights of the models and resources in the dynamic construction are adjusted according to the current scenario.

[0016] Furthermore, speech recognition results and user feedback are collected as a basis for system performance evaluation, key performance indicators of the system are calculated, and system parameters are automatically adjusted according to the performance indicators to achieve continuous optimization.

[0017] Preferably, speech recognition results and user feedback are collected as a basis for system performance evaluation, including: Record and collect the text results, confidence scores, and user confirmation and correction operations of each recognition, and calculate the wake-up rate, false wake-up rate, and speech recognition accuracy based on the collected data; Automatically adjust system parameters according to the performance indicators, including: Adjust wakeup thresholds, feature extraction parameters, and model selection strategies.

[0018] On the other hand, the present invention also discloses a speech recognition optimization system, characterized in that it comprises: The acquisition and preprocessing module is used to acquire multiple channels of original audio signals from a multi-microphone array and generate two-channel audio signals after enhancement processing; A wake-up detection module is used to extract features from the two-way audio signal, perform parallel detection using a pre-trained wake-up word detection model, and comprehensively determine whether to wake up based on the detection results; A speech recognition module, used to select an optimal speech recognition input path according to the wake-up detection result, and extract audio data according to the input path for speech recognition; The speech recognition optimization module is used to select the acoustic model based on contextual information and historical recognition results, while dynamically building a decoding graph.

[0019] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention improves the system's wake-up rate and anti-interference ability and reduces missed wake-ups by adopting dual-channel audio signal processing and parallel wake-up detection mechanism; (2) The present invention introduces an adaptive selection mechanism to dynamically select the optimal speech recognition input path based on the wake-up results, effectively improving the recognition accuracy; (3) The present invention introduces a microphone fault detection and adaptation strategy, which can maintain system performance when some microphones are damaged, thereby enhancing the robustness and reliability of the system; (4) The present invention further optimizes the speech recognition process by integrating context information, selecting the optimal adaptive acoustic model, and constructing a dynamic decoding graph; The present invention introduces a performance evaluation and self-optimization mechanism, which can continuously improve system parameters according to actual usage data and achieve long-term stable operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a flow chart of a speech recognition optimization method based on dual-path wake-up according to the present invention; Figure 2 A schematic diagram of a Mel-frequency cepstral coefficient MFCC feature extraction process of the present invention; Figure 3 This is a structural block diagram of a speech recognition optimization system based on dual-path wake-up according to the present invention; Figure 4 The figure is a schematic diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The present invention processes two different audio signals through dual-channel audio signal processing at the same time to improve the reliability and performance of the system, increase the system wake-up rate, and reduce missed wake-ups; the mechanism of adaptively selecting voice recognition input data improves the accuracy of voice recognition; and the voice recognition decoding structure is adjusted according to the current scene and historical recognition results by dynamically constructing a decoding graph. The method can also effectively deal with the situation where the microphone array is partially damaged, and improve the fault tolerance and reliability of the system. These improvements greatly enhance the user experience and expand the application scenarios of voice interaction devices.

[0023] The specific implementation of the present invention is described below with reference to the accompanying drawings and embodiments.

[0024] Example 1 like Figure 1 As shown, the present embodiment provides a method for optimizing speech recognition, and the technical solution includes the following steps: S1: collects multiple channels of original audio signals from a multi-microphone array and generates two-channel audio signals after enhancement processing; S2: extracting features from the two-way audio signals, performing parallel detection using a pre-trained wake-up word detection model, and comprehensively determining whether to wake up based on the detection results; S3: selecting an optimal voice recognition input path according to the wake-up detection result, and extracting audio data for voice recognition according to the input path; S4: The acoustic model is selected based on the contextual information and historical recognition results, while the decoding graph is dynamically constructed.

[0025] In this embodiment, before performing the speech recognition in step S3, the microphone for inputting speech is subjected to fault detection and adaptation, which specifically includes: Performing quality assessment on the multiple channels of original audio signals, and identifying microphones that may be faulty based on the signal quality assessment results; If the microphone is faulty, the audio signal processing strategy is adjusted according to the identified faulty microphone condition.

[0026] The step of performing quality assessment on the multiple channels of original audio signals includes: The short-time energy and spectrum flatness of each signal are calculated, and the microphones that may have faults are identified based on the signal quality evaluation results. If the short-time energy of a signal is lower than the threshold or the spectrum flatness is abnormal, it is determined that the corresponding microphone has a probability of being faulty.

[0027] The short-time energy calculation formula is as follows: in, is the audio sample, N is the frame length; The spectrum flatness calculation formula is as follows: Here, g(k) is the geometric mean, a(k) is the arithmetic mean, and k is the frequency index.

[0028] Then, based on the signal quality evaluation result, the microphone that may have a fault is identified. If the short-term energy of a certain signal is lower than the threshold or the spectrum flatness is abnormal, it is determined that the microphone corresponding to the signal has a probability of being faulty. The judgment logic of this embodiment is as follows: for i in range(num_mics): if energy[i] < energy_threshold or sfm[i] > sfm_threshold: faulty_mics.append(i).

[0029] Wherein, adjusting the audio signal processing strategy according to the identified faulty microphone situation further includes: If mic2 failure is detected, use output channel2 as wake-up recognition and cloud recognition input; If other faults are detected, use the processed channel3 / 4 for wake-up recognition and channel1 / 4 as cloud recognition input.

[0030] In this embodiment, the processing logic is as follows: if 2 in faulty_mics: wake_up_channel = asr_channel = 2 else: wake_up_channel = random.choice([3, 4]) asr_channel = random.choice([1, 4]).

[0031] In step S1, this embodiment collects 4 audio signals simultaneously in a 4-microphone array system.

[0032] We enhance the collected original audio signal to improve the signal-to-noise ratio, including: Use adaptive filtering algorithms such as the Least Mean Square (LMS) algorithm or the Normalized Least Mean Square (NLMS) algorithm to suppress noise.

[0033] This embodiment uses the minimum mean square error algorithm LMS to suppress noise, and the formula is as follows: , Where w(n) is the filter coefficient of the nth iteration, μ is the step size factor, which controls the step size of each update and is usually in the range of [0,1], e(n) is the error signal of the nth iteration, and x(n) is the input signal of the nth iteration.

[0034] Then, based on the enhanced multi-channel audio signals, two differently processed audio signals are generated. The first channel uses beamforming technology to generate a directionally enhanced audio signal, and the second channel uses blind source separation technology to extract the audio signal of the main sound source, specifically including: The first path uses beamforming technology to generate a directionally enhanced audio signal, and the second path uses blind source separation technology to extract the audio signal of the main sound source. Specifically, in this embodiment, the beamforming technology uses a delay and sum (Delay-and-Sum) method, and the blind source separation technology uses an independent component analysis (ICA) algorithm.

[0035] The beamforming technique uses a delay and sum method, and the formula is as follows: in, is the output signal, is the weight of the i-th microphone, is the signal of the i-th microphone, is the delay of the i-th microphone.

[0036] In step S2, extracting features from the two-way audio signal further includes: Based on the feature extraction algorithm, feature extraction is performed on the two-way audio signal respectively to obtain two feature vectors for wake-up word detection.

[0037] This embodiment uses Mel frequency cepstral coefficients MFCC as features, such as Figure 2 As shown, the extraction steps are as follows: S21: pre-emphasize the dual-channel audio signal, perform framing and windowing (such as Hamming window) on the framing and windowing audio signal, and perform fast Fourier transform (FFT) on the framing and windowing audio signal to obtain a spectrum; S22: converting the spectrum into a Mel spectrum, designing a group of Mel filters, each filter corresponding to a specific Mel frequency range, passing the Mel spectrum through the filter group, and calculating output energy of the Mel filter group; S23: taking the logarithm of the energy of the Mel filter bank to simulate the nonlinear perception of the human ear to the sound, and at the same time, performing discrete cosine transform (DCT) on the logarithmic energy to extract important spectral features.

[0038] Then, the pre-trained wake-up word detection model is used to perform parallel detection on the two feature vectors, including: At the same time, two independent keyword detection models based on deep neural network (DNN) are used to perform parallel detection on the two Mel-frequency cepstral coefficient MFCC feature vectors, and each model processes one feature vector; The probability of the existence of the wake-up word is output by the keyword detection model based on the deep neural network to generate a detection result. In this embodiment, in the keyword detection model based on the deep neural network, the hidden layer adopts a multi-layer fully connected layer, and the ReLU activation function is used in the hidden layer to introduce nonlinear characteristics. At the same time, the detection result is generated in the output layer, and the detection result is the probability of the existence of the wake-up word output using the Sigmoid activation function.

[0039] Then, based on the two detection results, it is comprehensively judged whether to wake up and determine the audio path that triggers wake up, including: A threshold is set between 0 and 1 to determine whether the wake-up word exists, and the two detection results are determined in turn to determine whether they exceed the preset threshold. If they exceed the preset threshold, the path of the path is recorded as the wake-up path that triggers wake-up; If any detection result exceeds a preset threshold, it is determined that the wake-up is successful.

[0040] The wake-up judgment logic of this embodiment is as follows: if (score_path1 > threshold or score_path2 > threshold): wake_up = True else: wake_up = False; At the same time, the logic of wake-up path determination is as follows: if (score_path1 > threshold and score_path2 > threshold): triggered_path = "both" elif (score_path1 > threshold): triggered_path = "path1" elif (score_path2 > threshold): triggered_path = "path2" else: triggered_path = "none".

[0041] In step S3, the optimal speech recognition input path is selected according to the wake-up detection result, including: If both paths are wake-up paths, compare the detection results of the two paths and take the one with the higher score as the optimal speech recognition input path; If there is only one wake-up path, then this wake-up path is taken as the optimal speech recognition input path; If not awakened, the default path is used.

[0042] The optimal path selection logic in this embodiment is as follows: if (triggered_path == "both"): asr_path = "path1" if score_path1 > score_path2 else "path2" elif (triggered_path == "path1" or triggered_path == "path2"): asr_path = triggered_path else: asr_path = "default_path" # Use the default path.

[0043] Then, based on the input path, the audio data for speech recognition is prepared, including: Extract the most recent N seconds of audio data from the audio buffer of the selected optimal speech recognition input path.

[0044] In step S4, an acoustic model is selected according to the context information and the historical recognition results, including: The context information includes the wake-up detection result and the information of the speech recognition input path, maintains multiple acoustic models optimized for different noise environments and hardware states, and selects the best model based on the context information and historical recognition results.

[0045] Specifically, in this embodiment, we add metadata to the speech recognition request, the metadata including the wake-up detection result and the selected audio path information, maintain multiple acoustic models optimized for different noise environments and hardware states, and select the best model based on the metadata, wherein: If the score of the wake-up detection result is greater than the high confidence threshold, the high signal-to-noise ratio model is selected; If the score of the wake-up detection result is less than the high confidence threshold and greater than the low confidence threshold, the medium signal-to-noise ratio model is selected; If the score of the wake-up detection result is less than the low confidence threshold, the low signal-to-noise ratio model is selected.

[0046] The logic of selecting the model in this embodiment is as follows: if wake_up_score > high_confidence_threshold: acoustic_model = "high_snr_model" elif wake_up_score > low_confidence_threshold: acoustic_model = "medium_snr_model" else: acoustic_model = "low_snr_model".

[0047] Furthermore, in step S4, the dynamically constructing a decoding graph includes: Based on the context information and historical recognition results, a decoding graph is dynamically constructed, several models and resources are combined together, and the weights of the models and resources in the dynamic construction are adjusted according to the current scenario.

[0048] Specifically, in this embodiment, a weighted finite state transformer WFST framework is used to adjust the language model weight according to the current scenario. When constructing a WFST-based decoding graph, four different models and resources are combined together, and the models and resources include an acoustic model H, a context correlation C, a dictionary L, and a language model G; The acoustic model H converts the input speech signal into the probability distribution of phoneme or syllable sequence, the context correlation C considers the relationship between the current phoneme and the previous and next phonemes, the dictionary L is a mapping table that maps the phoneme sequence to words or phrases, and the language model G evaluates the probability of a sentence.

[0049] In this embodiment, we use the compose function to combine the acoustic model H, context relevance C, dictionary L and language model G in a specified order, and the specified order is ((H*C)*L)*G. Specifically expressed as: decode_graph = compose(H, C, L, G).

[0050] The optimization method further comprises: Collect speech recognition results and user feedback as the basis for system performance evaluation, calculate the key performance indicators of the system, and automatically adjust system parameters according to the performance indicators to achieve continuous optimization.

[0051] Among them, speech recognition results and user feedback are collected as the basis for system performance evaluation, including: Record and collect the text results, confidence scores, and user confirmation and correction operations of each recognition, and calculate the wake-up rate, false wake-up rate, and speech recognition accuracy based on the collected data; Automatically adjust system parameters according to the performance indicators, including: Adjust wakeup thresholds, feature extraction parameters, and model selection strategies.

[0052] Specifically, in this embodiment, the text results, confidence scores, and user confirmation and correction operations of each recognition are recorded and collected, and the wake-up rate WR (Wake-up Rate, WR), false alarm rate FAR (False Alarm Rate, FAR) and speech recognition accuracy SRA (Speech Recognition Accuracy, SRA) are calculated based on the collected data. The calculation formula is as follows: WR = number of successful wake-up attempts / total number of wake-up attempts, FAR = number of false wake-ups / total detection time, SRA = number of words correctly recognized / total number of words.

[0053] Secondly, according to the performance indicators, the system parameters are automatically adjusted to achieve continuous optimization, including: The Bayesian optimization algorithm is used to adjust the wake-up threshold, feature extraction parameters, and model selection strategy. The Bayesian optimization objective function is expressed as follows: , in, is the parameter vector to be optimized, , , is the weight coefficient.

[0054] Through the above steps, the method of the present invention realizes the optimization of speech recognition performance based on dual-path wake-up and adaptive selection. This method not only improves the system's wake-up rate and speech recognition accuracy, but also effectively copes with hardware problems such as microphone failure, significantly improving the overall performance and user experience of the voice interaction system.

[0055] This method provides a comprehensive voice interaction optimization solution by integrating multiple technologies. It aims to improve the accuracy of wake-up and recognition, while taking into account hardware failure situations that may be encountered in actual applications. The method introduces technologies such as context information integration, adaptive acoustic model selection, and dynamic decoding graph construction to further optimize the speech recognition process. In addition, the method also includes performance evaluation and self-optimization mechanisms to continuously improve system parameters by analyzing actual usage data, thereby improving the overall performance and adaptability of the system.

[0056] This embodiment also discloses a speech recognition optimization system, such as Figure 3 As shown, including: The acquisition and preprocessing module 100 is used to acquire multiple channels of original audio signals from a multi-microphone array and generate two-channel audio signals after enhancement processing; A wake-up detection module 200 is used to extract features from the two-way audio signal, perform parallel detection using a pre-trained wake-up word detection model, and comprehensively determine whether to wake up based on the detection results; A speech recognition module 300, configured to select an optimal speech recognition input path according to the wake-up detection result, and extract audio data according to the input path for speech recognition; The speech recognition optimization module 400 is used to select an acoustic model based on context information and historical recognition results, and dynamically construct a decoding graph.

[0057] The wake-up detection module includes a feature extraction unit 201, which is used to perform feature extraction on the two-way audio signal based on a feature extraction algorithm to obtain two feature vectors for wake-up word detection.

[0058] The speech recognition module 300 includes a path selection unit 301 for selecting an optimal speech recognition input path according to the wake-up detection result.

[0059] The speech recognition optimization module 400 includes a performance evaluation unit 401, which is used to collect speech recognition results and user feedback as a basis for system performance evaluation, calculate key performance indicators of the system, and automatically adjust system parameters according to the performance indicators to achieve continuous optimization.

[0060] In addition, the system also includes a microphone fault detection unit 250, which is used to evaluate the quality of the multiple original audio signals and identify the microphone that may have a fault based on the signal quality evaluation results; if the microphone has a fault, adjust the audio signal processing strategy according to the identified faulty microphone. The functional implementation of each module and each unit in the above-mentioned speech recognition optimization system based on dual-way wake-up corresponds to the steps in the above-mentioned speech recognition optimization system method embodiment based on dual-way wake-up, and their functions and implementation processes will not be repeated here one by one.

[0061] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above audio control method. The electronic device can be a server or a terminal device.

[0062] like Figure 4 As shown, the electronic device includes a processor 14 and a memory 13. The memory 13 stores machine executable instructions that can be executed by the processor 14. The processor 14 executes the machine executable instructions to implement the above audio control method.

[0063] Further, Figure 4 The electronic device shown further includes a bus 12 and a communication interface 11 , and the processor 14 , the communication interface 11 and the memory 13 are connected via the bus 12 .

[0064] The memory 13 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 11 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 12 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0065] The processor 14 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 14. The above processor 14 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in this embodiment can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in this embodiment can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1001, and the processor 1000 reads the information in the memory 1001 and completes the steps of the audio control method in combination with its hardware.

[0066] The present disclosure also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program runs on a computer, the computer executes the steps of the audio control method.

[0067] Finally, it should be noted that the above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0068] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A speech recognition optimization method, characterized in that: The steps include: S1: collects multiple channels of original audio signals from a multi-microphone array and generates two-channel audio signals after enhancement processing; S2: extracting features from the two-way audio signals, performing parallel detection using a pre-trained wake-up word detection model, and comprehensively determining whether to wake up based on the detection results; S3: selecting an optimal voice recognition input path according to the wake-up detection result, and extracting audio data for voice recognition according to the input path; S4: The acoustic model is selected based on the contextual information and historical recognition results, while the decoding graph is dynamically constructed.

2. The speech recognition optimization method according to claim 1, characterized in that: Before performing the speech recognition in step S3, the microphone for inputting the speech is subjected to fault detection and adaptation, including: Performing quality assessment on the multiple channels of original audio signals, and identifying microphones that may be faulty based on the signal quality assessment results; If the microphone is faulty, the audio signal processing strategy is adjusted according to the identified faulty microphone condition.

3. The speech recognition optimization method according to claim 2, characterized in that: Performing quality assessment on the multiple channels of original audio signals, including: The short-time energy and spectrum flatness of each signal are calculated, and the microphones that may have faults are identified based on the signal quality evaluation results. If the short-time energy of a signal is lower than the threshold or the spectrum flatness is abnormal, it is determined that the corresponding microphone has a probability of being faulty.

4. The speech recognition optimization method according to claim 1, characterized in that: In step S1, a two-way audio signal is generated after enhancement processing, including: The enhancement process uses an adaptive filtering algorithm to suppress noise; Based on the enhanced multi-channel audio signals, two differently processed audio signals are generated. The first channel uses beamforming technology to generate a directionally enhanced audio signal, and the second channel uses blind source separation technology to extract the audio signal of the main sound source.

5. The speech recognition optimization method according to claim 1, characterized in that: In step S2, extracting features from the two-way audio signal further includes: Based on the feature extraction algorithm, feature extraction is performed on the two-way audio signal respectively to obtain two feature vectors for wake-up word detection.

6. The speech recognition optimization method according to claim 5, characterized in that: In step S2, parallel detection is performed using a pre-trained wake-up word detection model, including: At the same time, two independent keyword detection models based on deep neural networks are used to perform parallel detection on the extracted feature vectors, and each model processes one feature vector; The keyword detection model based on the deep neural network outputs the probability of the presence of the wake-up word and generates a detection result.

7. The method for optimizing speech recognition according to claim 6, characterized in that: In step S2, whether to wake up is determined comprehensively based on the detection results, including: A threshold is set between 0 and 1 to determine whether the wake-up word exists, and the two detection results are determined in turn to determine whether they exceed the preset threshold. If they exceed the preset threshold, the path of the path is recorded as the wake-up path that triggers wake-up; If any detection result exceeds a preset threshold, it is determined that the wake-up is successful.

8. The speech recognition optimization method according to claim 7, characterized in that: In step S3, the optimal speech recognition input path is selected according to the wake-up detection result, including: If both paths are wake-up paths, compare the detection results of the two paths and take the one with the higher score as the optimal speech recognition input path; If there is only one wake-up path, then this wake-up path is taken as the optimal speech recognition input path; If not awakened, the default path is used.

9. The speech recognition optimization method according to claim 1, characterized in that: In step S4, an acoustic model is selected according to the context information and the historical recognition results, including: The context information includes the wake-up detection result and the information of the speech recognition input path, maintains multiple acoustic models optimized for different noise environments and hardware states, and selects the best model based on the context information and historical recognition results.

10. The speech recognition optimization method according to claim 1, characterized in that: In step S4, the dynamic construction of the decoding graph includes: Based on the context information and historical recognition results, a decoding graph is dynamically constructed, several models and resources are combined together, and the weights of the models and resources in the dynamic construction are adjusted according to the current scenario.

11. The speech recognition optimization method according to claim 1, characterized in that: Collect speech recognition results and user feedback as the basis for system performance evaluation, calculate the key performance indicators of the system, and automatically adjust system parameters according to the performance indicators to achieve continuous optimization.

12. The speech recognition optimization method according to claim 11, characterized in that: Collect speech recognition results and user feedback as the basis for system performance evaluation, including: Record and collect the text results, confidence scores, and user confirmation and correction operations of each recognition, and calculate the wake-up rate, false wake-up rate, and speech recognition accuracy based on the collected data; Automatically adjust system parameters according to the performance indicators, including: Adjust wakeup thresholds, feature extraction parameters, and model selection strategies.

13. A speech recognition optimization system, characterized in that: include: The acquisition and preprocessing module is used to acquire multiple channels of original audio signals from a multi-microphone array and generate two-channel audio signals after enhancement processing; A wake-up detection module is used to extract features from the two-way audio signal, perform parallel detection using a pre-trained wake-up word detection model, and comprehensively determine whether to wake up based on the detection results; A speech recognition module, used to select an optimal speech recognition input path according to the wake-up detection result, and extract audio data according to the input path for speech recognition; The speech recognition optimization module is used to select the acoustic model based on contextual information and historical recognition results, while dynamically building a decoding graph.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the speech recognition optimization method according to any one of claims 1 to 12 is implemented.

15. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the speech recognition optimization method as described in any one of claims 1-12.

Images