A vehicle computer voice recognition method and system

Abstract

The present invention relates to the field of speech recognition technology, and in particular to a vehicle-mounted speech recognition method and system, comprising collecting speech data in a cab through a microphone, which is recorded as initial speech data; using a VAD algorithm to detect and analyze the initial speech data, and identifying speech frames in the initial speech data; analyzing each speech frame to obtain defective speech data; repairing the defective speech data, which is recorded as corrected speech data; and evaluating the repair result based on the corrected speech data. By analyzing the defective speech data and generating corrected speech data, the system can optimize the recognition result and reduce the risk of misoperation. At the same time, by judging the accuracy of the corrected speech data, it can intelligently decide whether to directly execute instructions or return confirmation, thereby improving user experience and operational safety. This method helps to reduce improper operations caused by speech recognition errors, especially during driving, and can better ensure safety.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a vehicle computer speech recognition method and system. Background Art

[0002] In-vehicle voice recognition technology allows drivers to control in-vehicle devices through voice commands, avoiding manual operation, thereby reducing accidents caused by improper operation during driving. However, although in-vehicle voice recognition technology has made significant progress in many application scenarios, it still faces some challenges, especially the recognition accuracy problem in complex environments and irregular pronunciation.

[0003] The Chinese patent application with the authorization announcement number CN104737226B discloses speech recognition in a motor vehicle. A processing device inside the vehicle receives a user's speech input from a detection device inside the vehicle, and the processing device transmits at least one received speech input to an automatic speech recognition system to generate a language text without asking the user. For this purpose, the processing device transmits the speech input to an on-board speech recognizer inside the vehicle and receives a first recognition result from the on-board speech recognizer. The processing device transmits the speech input in whole or in part to an off-board speech recognizer outside the vehicle, and receives a second recognition result from the off-board speech recognizer. The processing device determines the language text based on the first recognition result and the second recognition result.

[0004] As mentioned in the above application, in the prior art, vehicle-mounted voice recognition generally directly collects voice data, and simply performs data filtering and amplification processing to directly generate voice text as a recognition structure. However, the vehicle-mounted environment is usually noisy, and when the driver issues voice commands, the voice recognition system may not be able to accurately capture valid voice data due to factors such as unclear pronunciation, too fast speaking speed, or accent differences. In this case, the risk of misidentification and incorrect execution of commands increases significantly, especially during driving, where voice recognition errors may lead to unnecessary operations and even safety hazards. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a vehicle machine speech recognition method and system.

[0006] The present invention adopts the following technical solution, a vehicle machine voice recognition method, comprising:

[0007] The voice data in the cab is collected through a microphone and recorded as initial voice data;

[0008] Use the VAD algorithm to detect and analyze the initial speech data, identify the speech frames in the initial speech data, and remove non-speech frames;

[0009] Analyze each speech frame, obtain speech feature parameters of the speech frame, and input them into a pre-built defective speech recognition first machine learning model to obtain defective speech data;

[0010] Repair the defective voice data and record it as corrected voice data;

[0011] The restoration result is evaluated based on the corrected speech data, and a direct confirmation instruction or a return confirmation instruction is generated based on the evaluation result.

[0012] As a further description of the above technical solution: the method of using the VAD algorithm to detect and analyze the initial voice data and identify the voice frames in the initial voice data includes:

[0013] Signal preprocessing, converting the audio signal to mono and pre-emphasizing it to enhance the high frequency part and remove the low frequency component;

[0014] Framing and using window functions, the audio signal is divided into several short-time frames, each frame contains N number of sampling points, in order to reduce the edge effect, the window function is used to perform windowing processing on each frame;

[0015] Calculate the signal energy. For each frame, calculate the signal energy. The calculation formula is:

[0016] ;

[0017] In the formula, It is The energy of the frame, is the sampled value of the signal at time point t+n, is the number of samples per frame, is the sample position within the current frame, usually ranging from arrive , that is, all the sampling points in the current frame; x is the value representing the audio signal;

[0018] Set an energy threshold. According to the energy calculation result, set an energy threshold Emin and compare it with the energy of each frame. If the frame energy is greater than the energy threshold, the frame is considered to be a speech frame, otherwise it is a non-speech frame.

[0019] As a further description of the above technical solution: the speech feature parameters include MFCC coefficients, formants and pitch;

[0020] The method for obtaining the MFCC coefficients includes:

[0021] Perform short-time Fourier transform (STFT) on each speech frame to convert the time domain signal into a frequency domain signal;

[0022] The spectrum of each frame is filtered through the Mel frequency filter bank and converted into energy value on the Mel scale;

[0023] After the energy values ​​of each frequency band are obtained by the Mel frequency filter bank, these energy values ​​are logarithmically transformed;

[0024] Calculate the discrete cosine transform of the logarithmic energy to extract the MFCC coefficients;

[0025] The formula for discrete cosine transform is:

[0026] ;

[0027] In the formula, is the energy of the ith Mel band, is the MFCC coefficient after discrete cosine transformation, is the number of Mel frequency filter banks.

[0028] As a further description of the above technical solution: the training method of the first machine learning model for defective speech recognition includes:

[0029] Under the experimental state, speech feature parameters of normal speech frames and defective speech frames are collected, and corresponding actual annotations are generated for the state values ​​of the speech frames when each group of speech feature parameters is collected. The actual annotations corresponding to normal speech frames are 0, and the actual annotations corresponding to defective speech frames are 1. Each group of speech feature parameters is used as the input of the first machine learning model for defective speech recognition. The first machine learning model for defective speech recognition uses the predicted annotations for each group of speech feature parameters as output, the actual annotations as prediction targets, and minimizing the sum of the prediction accuracies of all speech feature parameters as training targets.

[0030] As a further description of the above technical solution: the method for repairing defective speech data includes:

[0031] The defective speech data and speech feature parameters of the defective speech data are input into a pre-built second machine learning model for repairing speech data to obtain repaired speech data.

[0032] As a further description of the above technical solution: the training method of the second machine learning model for repairing the voice data includes:

[0033] Collect historical defective speech data and speech feature parameters of the historical defective speech data and restored speech data;

[0034] Converting the historical defective speech data, speech feature parameters of the historical defective speech data, and the repaired speech data into a set of feature vectors;

[0035] Each group of feature vectors is used as the input of the second machine learning model for repairing speech data. The second machine learning model for repairing speech data takes the repaired speech data corresponding to each group of historical defective speech data and the speech feature parameters of the historical defective speech data as output, takes the repaired speech data actually corresponding to each group of historical defective speech data and the speech feature parameters of the historical defective speech data as the prediction target, and takes minimizing the loss function value of the speech data repair machine learning model as the training target; and stops training when the loss function value of the machine learning model is less than or equal to the preset target loss value.

[0036] As a further description of the above technical solution: the method of evaluating the repair result based on the corrected voice data and generating a direct determination instruction or a return confirmation instruction based on the evaluation result includes:

[0037] Acquire restoration feature parameters of the corrected speech data, wherein the restoration feature parameters include a speech quality perception evaluation value, a short-time objective intelligibility evaluation value, and a speech quality objective evaluation value;

[0038] Generate a repair assessment coefficient based on the repair feature parameters;

[0039] A repair assessment coefficient threshold XFm is preset, and the repair assessment coefficient is compared with the repair assessment coefficient threshold to generate a direct confirmation instruction or return a confirmation instruction.

[0040] As a further description of the above technical solution: the calculation formula of the repair evaluation coefficient is:

[0041] ;

[0042] In the formula, To repair the evaluation coefficient, is the perceptual evaluation value of speech quality, is the short-term objective intelligibility assessment value, is the objective evaluation value of voice quality, , and is the weight coefficient, , and Both are greater than 0.

[0043] As a further description of the above technical solution: the method of comparing and analyzing the repair evaluation coefficient with the repair evaluation coefficient threshold to generate a direct determination instruction or return a confirmation instruction includes:

[0044] when ≥XFm, generate direct determination instructions;

[0045] when <XFm, generate return confirmation instruction.

[0046] A vehicle machine speech recognition system, which is used to implement the vehicle machine speech recognition method, comprises:

[0047] The data acquisition module collects the voice data in the cab through the microphone and records it as the initial voice data;

[0048] The data processing module uses the VAD algorithm to detect and analyze the initial voice data, identify the voice frames in the initial voice data, and remove the non-voice frames;

[0049] The data analysis module analyzes each speech frame, obtains speech feature parameters of the speech frame, and inputs them into the pre-built first machine learning model for defective speech recognition to obtain defective speech data;

[0050] The data repair module repairs the defective voice data and records it as corrected voice data;

[0051] The instruction generation module evaluates the repair result based on the corrected speech data, and generates a direct confirmation instruction or a return confirmation instruction based on the evaluation result.

[0052] Beneficial effects:

[0053] The present invention provides a vehicle machine speech recognition method and system, which can detect the start and end time of speech activity in real time through the VAD algorithm, and mark the effective speech area, so as to overcome other noises in the cab (wind noise, noise generated by other people and the sound of music playing), and only recognize and process the effective speech area, thereby reducing unnecessary computing resource consumption, reducing the processing of noise and unnecessary speech frame calculation during recognition, and enhancing the real-time performance of the system. The effective speech segment marking provided can effectively reduce false triggering and erroneous recognition in processing, and improve the accuracy of speech recognition.

[0054] Secondly, by analyzing defective voice data and generating corrected voice data, the system can optimize the recognition results and reduce the risk of misoperation. At the same time, by judging the accuracy of the corrected voice data, it can intelligently decide whether to directly execute the command or return for confirmation, thereby improving user experience and operational safety. This method helps reduce improper operations caused by voice recognition errors, especially during driving, and can better ensure safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention will be further explained below in conjunction with the accompanying drawings and embodiments:

[0056] Figure 1 A flow chart of a vehicle machine voice recognition method provided by an embodiment of the present invention;

[0057] Figure 2A flowchart for identifying speech frames in initial speech data provided by an embodiment of the present invention;

[0058] Figure 3 A flow chart of generating a direct confirmation instruction or a return confirmation instruction provided by an embodiment of the present invention;

[0059] Figure 4 A structural schematic diagram of a vehicle-mounted voice recognition system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the technical means, creative features, objectives and effects of the present invention easy to understand, the present invention is further described below with reference to specific diagrams. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0061] Example 1

[0062] See also Figure 1 The embodiment of the present invention provides a technical solution: a vehicle machine voice recognition method, comprising:

[0063] The voice data in the cab is collected through a microphone and recorded as initial voice data;

[0064] Use the VAD algorithm to detect and analyze the initial speech data, identify the speech frames in the initial speech data, and remove non-speech frames;

[0065] Analyze each speech frame, obtain speech feature parameters of the speech frame, and input them into a pre-built defective speech recognition first machine learning model to obtain defective speech data;

[0066] The speech feature parameters include MFCC coefficients, formants and pitch;

[0067] It should be noted that the formant is a high-frequency peak in the speech signal, which is usually related to the shape of the oral cavity and pharynx and the vibration pattern of the airflow. In speech processing, the formant is usually extracted by analyzing the spectrum of the signal. The common method for collecting formants is linear predictive coding (LPC). LPC is a technology commonly used to analyze speech signals and can effectively extract formants from speech signals. The LPC model predicts the value of the current sample by fitting a linear model of the speech signal and can estimate the formant of the signal.

[0068] The basic steps of LPC:

[0069] Signal framing: The speech signal is divided into small frames (usually 20-30ms) and each frame is processed.

[0070] Calculate the autocorrelation function: Describe the characteristics of the signal by calculating the autocorrelation function.

[0071] LPC analysis: Using the autocorrelation function, the LPC coefficients are obtained by solving the linear prediction equation. The coefficients can describe the resonance characteristics of the signal.

[0072] Decoding formants: The formant frequency of the speech signal can be calculated through the LPC coefficient.

[0073] Pitch refers to the periodic basic frequency in the speech signal, which is usually directly related to the frequency of vocal cord vibration. By performing Fourier transform on the signal, the spectrum of the signal can be obtained. The strongest frequency component in the spectrum usually corresponds to the fundamental frequency, that is, the pitch.

[0074] The method for obtaining the MFCC coefficients includes:

[0075] Perform short-time Fourier transform (STFT) on each speech frame to convert the time domain signal into a frequency domain signal;

[0076] It should be noted that the short-time Fourier transform describes the frequency characteristics of the signal by calculating the spectrum of each frame. Through the short-time Fourier transform, the spectrum obtained reflects the frequency components of the signal;

[0077] The spectrum of each frame is filtered through the Mel frequency filter bank and converted into energy value on the Mel scale;

[0078] It should be noted that the Mel frequency filter bank is used to weight the spectrum of the signal by dividing the spectrum into a series of Mel-scale filters (usually triangular filters). The calculation formula of the Mel frequency is as follows:

[0079] ;

[0080] ;

[0081] in, is the actual frequency value, is the mel frequency value, is the energy output of the cth Mel frequency filter band, is the dth frequency component of the spectrum signal obtained by fast Fourier transform; is the response of the cth Mel frequency filter in frequency.

[0082] After the energy values ​​of each frequency band are obtained by the Mel frequency filter bank, these energy values ​​are logarithmically transformed (because the human ear perceives the loudness of sound in a logarithmic rather than a linear way. Therefore, taking the logarithm can better simulate the auditory response of the human ear). The transformation formula is:

[0083] ;

[0084] in, It is The output of energy in each Mel band; It means Perform a logarithmic transformation.

[0085] Example: The energy value of a certain frequency band obtained by the Mel frequency filter bank is =1000 and the result after logarithmic transformation (base 10) is:

[0086] ;

[0087] This means that the logarithmic energy value of this frequency band is 3. Through this logarithmic transformation, the gap between the low-energy part and the high-energy part of the signal can be effectively reduced, making the representation of the signal more in line with the auditory characteristics of the human ear.

[0088] Calculate the discrete cosine transform (DCT) of the logarithmic energy to extract MFCC. The purpose of the discrete cosine transform is to convert the Mel band energy representation into more compact coefficients (usually 15 to 20) to obtain the MFCC features of each frame.

[0089] The formula for discrete cosine transform is:

[0090] ;

[0091] In the formula, is the energy of the ith Mel band, is the MFCC coefficient after discrete cosine transformation, is the number of Mel frequency filter banks.

[0092] The training method of the first machine learning model for defective speech recognition includes:

[0093] Under the experimental state, speech feature parameters of normal speech frames and defective speech frames are collected, and the state value of each group of speech feature parameters when the speech frame is collected generates a corresponding actual annotation, the actual annotation corresponding to the normal speech frame is 0, and the actual annotation corresponding to the defective speech frame is 1. Each group of speech feature parameters is used as the input of the first machine learning model for defective speech recognition. The first machine learning model for defective speech recognition uses the predicted annotation of each group of speech feature parameters as output, the actual annotation as the prediction target, and minimizing the sum of the prediction accuracies of all speech feature parameters as the training target; wherein the calculation formula for the prediction accuracy is: ,in, is the number of the speech feature parameter, For prediction accuracy, For the The predicted annotation value corresponding to the group speech feature parameter, For the The actual annotations corresponding to the group speech feature parameters are obtained; the first machine learning model for defective speech recognition is trained until the sum of the prediction accuracies reaches convergence and the training is stopped; preferably, the first machine learning model for defective speech recognition can be a deep neural network model or a deep belief network model;

[0094] Other model parameters of the first machine learning model for defective speech recognition, such as the depth of the network model, the number of neurons in each layer, the activation function used by the network model, the convergence condition, the ratio of the training set, the test set, and the validation set, and the loss function, are all obtained through actual engineering implementation and continuous experimental tuning.

[0095] Repair the defective voice data and record it as corrected voice data;

[0096] Methods for repairing defective voice data include:

[0097] Inputting the defective speech data and speech feature parameters of the defective speech data into a pre-built second machine learning model for repairing speech data to obtain repaired speech data;

[0098] The training method of the second machine learning model for voice data repair includes:

[0099] Converting the historical defective speech data, speech feature parameters of the historical defective speech data, and the repaired speech data into a set of feature vectors;

[0100] Each group of feature vectors is used as the input of the second machine learning model for repairing speech data. The second machine learning model for repairing speech data uses the repaired speech data corresponding to each group of historical defective speech data and the speech feature parameters of the historical defective speech data as output, uses the repaired speech data actually corresponding to each group of historical defective speech data and the speech feature parameters of the historical defective speech data as prediction targets, and uses minimizing the loss function value of the speech data repairing machine learning model as a training target; and stops training when the loss function value of the machine learning model is less than or equal to a preset target loss value;

[0101] The loss function value of the second machine learning model for voice data restoration is mean square error;

[0102] Mean square error is one of the commonly used loss functions.

[0103] = The model is trained with minimization as the goal, so that the speech data repair second machine learning model better fits the data, thereby improving the performance and accuracy of the model;

[0104] In the loss function Fix the loss function value of the second machine learning model for the speech data, is the feature vector group number; is the number of eigenvector groups; For the The restored speech data corresponding to the group feature vector, For the The restored speech data corresponding to the group feature vector;

[0105] The other model parameters of the second machine learning model for voice data restoration, the target loss value, the optimization algorithm, the ratio of the training set, the test set, and the validation set, and the optimization of the loss function are all achieved through actual engineering and obtained after continuous experimental tuning.

[0106] The restoration result is evaluated based on the corrected speech data, and a direct confirmation instruction or a return confirmation instruction is generated based on the evaluation result.

[0107] Example 2

[0108] See also Figure 2 Based on the above embodiments, this embodiment further discloses a method for detecting and analyzing initial speech data using a VAD algorithm and identifying speech frames in the initial speech data;

[0109] The method of using the VAD algorithm to detect and analyze the initial voice data and identify the voice frames in the initial voice data includes:

[0110] Signal preprocessing, converting the audio signal to mono and pre-emphasizing it to enhance the high frequency part and remove the low frequency component;

[0111] Framing and using window functions to divide the audio signal into several short-time frames (usually 20ms to 40ms). Each frame contains N number of sampling points. In order to reduce edge effects, a window function (such as Hanning window) is used to perform windowing on each frame.

[0112] Calculate the signal energy. For each frame, calculate the signal energy. The calculation formula is:

[0113] ;

[0114] In the formula, It is The energy of the frame, is the sampled value of the signal at time point t+n, is the number of samples per frame, is the sample position within the current frame, usually ranging from arrive , that is, all the sampling points in the current frame, x is the value representing the audio signal;

[0115] Example: Assume there is an audio signal, the frame starting at a certain time t contains 5 sampling points (ie N=5), and the signal sample values ​​are:

[0116] , , , , , then the energy calculation of the frame is:

[0117] ;

[0118] Calculate the square of each sampling point and add them up to get the total energy of the current frame.

[0119] Set an energy threshold. According to the energy calculation result, set an energy threshold Emin and compare it with the energy of each frame. If the frame energy is greater than the energy threshold, the frame is considered to be a speech frame, otherwise it is a non-speech frame.

[0120] In this embodiment, the VAD algorithm can detect the start and end time of voice activities in real time, and mark the valid voice area, overcoming other noises in the cab (wind noise, noise generated by other people, and the sound of music playing), and only recognize and process the valid voice area, reducing unnecessary computing resource consumption, and reducing noise processing and unnecessary voice frame calculations during recognition, thereby enhancing the real-time performance of the system. The effective voice segment marking provided can effectively reduce false triggering and erroneous recognition during processing, and improve the accuracy of voice recognition.

[0121] Example 3

[0122] Please refer to Figure 3 Based on the above embodiment, this embodiment further discloses: evaluating the repair result based on the corrected voice data, and generating a direct confirmation instruction or a return confirmation instruction based on the evaluation result, including:

[0123] Acquire restoration feature parameters of the corrected speech data, wherein the restoration feature parameters include a speech quality perception evaluation value, a short-time objective intelligibility evaluation value, and a speech quality objective evaluation value;

[0124] It should be noted that the calculation of the speech quality perception evaluation value requires the defective speech data before repair and the corrected speech data to be given a score by simulating the human ear's perception of speech signals. The range is usually -0.5 to 4.5. The higher the value, the better the speech quality.

[0125] The short-term objective intelligibility evaluation value is a standard for evaluating speech intelligibility, especially focusing on the degree to which the listener understands the speech content. It can evaluate the speech intelligibility during noise suppression or speech restoration, which can be calculated by the stoiPython library or based on the model calculated by stoi. The score range of stoi is usually 0 to 1, and the closer to 1, the easier the speech is to understand;

[0126] It should be noted that the stoi Python library, in Python, stoi is an indicator for evaluating the intelligibility of speech signals, mainly used to measure the intelligibility of speech signals after noise suppression or other processing and the reference (original) speech signals;

[0127] STOI evaluates the similarity of the time-frequency features of two speech signals (usually a reference signal and an enhanced signal) within a short time window. The score ranges from 0 to 1, where:

[0128] 1 means the enhanced speech is very similar to the original reference speech and the content is fully intelligible.

[0129] 0 means the enhanced speech is barely intelligible.

[0130] The objective evaluation value of speech quality can quantify the pronunciation quality of speech by analyzing the spectral features, formants, and pitch parameters of speech. The specific methods include using LPC to extract formants and using a weighted method to obtain the objective evaluation value of speech quality. The larger the objective evaluation value of speech quality, the better the speech quality. The larger the objective evaluation value of speech quality, the better the pronunciation quality of the speech signal, with clear timbre, moderate resonance, and stable pitch.

[0131] The repair evaluation coefficient is generated based on the repair feature parameters. The calculation formula of the repair evaluation coefficient is:

[0132] ;

[0133] In the formula, To repair the evaluation coefficient, is the perceptual evaluation value of speech quality, is the short-term objective intelligibility assessment value, is the objective evaluation value of voice quality, , and is the weight coefficient, , and Both are greater than 0.

[0134] It should be noted that the size of the weight coefficient is a specific value obtained by quantifying each data to facilitate subsequent comparison. The size of the weight coefficient depends on the number of repair feature parameters and the preliminary setting of the corresponding weight coefficient for each set of repair feature parameters by technical personnel in this field.

[0135] Preset the repair assessment coefficient threshold XFm, compare and analyze the repair assessment coefficient with the repair assessment coefficient threshold to generate a direct confirmation instruction or return a confirmation instruction;

[0136] when ≥XFm, generate direct determination instructions;

[0137] when <XFm, generate return confirmation instruction.

[0138] In this embodiment, by obtaining the repair feature parameters of the corrected voice data, generating a repair evaluation coefficient based on the repair feature parameters, presetting the repair evaluation coefficient threshold XFm, and comparing and analyzing the repair evaluation coefficient with the repair evaluation coefficient threshold to generate a direct determination instruction or a return confirmation instruction, it is possible to intelligently decide whether to directly execute the instruction or return confirmation, thereby improving user experience and operational safety. This method helps reduce improper operations caused by voice recognition errors, especially during driving, and can better ensure safety.

[0139] Example 4

[0140] See also Figure 4 The embodiment of the present invention provides a technical solution: a vehicle-mounted computer voice recognition system, which is used to implement the vehicle-mounted computer voice recognition method, comprising:

[0141] The data acquisition module collects the voice data in the cab through the microphone and records it as the initial voice data;

[0142] The data processing module uses the VAD algorithm to detect and analyze the initial voice data, identify the voice frames in the initial voice data, and remove the non-voice frames;

[0143] The data analysis module analyzes each speech frame, obtains speech feature parameters of the speech frame, and inputs them into the pre-built first machine learning model for defective speech recognition to obtain defective speech data;

[0144] The data repair module repairs the defective voice data and records it as corrected voice data;

[0145] The instruction generation module evaluates the repair result based on the corrected speech data, and generates a direct confirmation instruction or a return confirmation instruction based on the evaluation result.

[0146] The basic principles, main features and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and the description in the specification are only to illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A vehicle machine voice recognition method, characterized in that: include: The voice data in the cab is collected through a microphone and recorded as initial voice data; Use the VAD algorithm to detect and analyze the initial speech data, identify the speech frames in the initial speech data, and remove non-speech frames; Analyze each speech frame, obtain speech feature parameters of the speech frame, and input them into a pre-built defective speech recognition first machine learning model to obtain defective speech data; Repair the defective voice data and record it as corrected voice data; The method for repairing defective voice data comprises: Inputting the defective speech data and speech feature parameters of the defective speech data into a pre-built second machine learning model for repairing speech data to obtain repaired speech data; The training method of the second machine learning model for repairing speech data includes: Collect historical defective speech data and speech feature parameters of the historical defective speech data and restored speech data; Converting the historical defective speech data, speech feature parameters of the historical defective speech data, and the repaired speech data into a set of feature vectors; Each group of feature vectors is used as the input of the second machine learning model for repairing speech data. The second machine learning model for repairing speech data uses the repaired speech data corresponding to each group of historical defective speech data and the speech feature parameters of the historical defective speech data as output, uses the repaired speech data actually corresponding to each group of historical defective speech data and the speech feature parameters of the historical defective speech data as prediction targets, and uses minimizing the loss function value of the speech data repairing machine learning model as a training target; and stops training when the loss function value of the machine learning model is less than or equal to a preset target loss value; The restoration result is evaluated based on the corrected speech data, and a direct confirmation instruction or a return confirmation instruction is generated based on the evaluation result.

2. A vehicle machine voice recognition method according to claim 1, characterized in that: The method of using the VAD algorithm to detect and analyze the initial voice data and identify the voice frames in the initial voice data includes: Signal preprocessing, converting the audio signal to mono and pre-emphasizing it to enhance the high frequency part and remove the low frequency component; Framing and using window functions, the audio signal is divided into short time frames, each frame contains N number of sampling points, in order to reduce the edge effect, the window function is used to perform windowing processing on each frame; Calculate the signal energy. For each frame, calculate the signal energy. The calculation formula is: ； In the formula, It is The energy of the frame, is the sampled value of the signal at time point t+n, is the number of samples per frame, is the sample position in the current frame, ranging from arrive , that is, all the sampling points in the current frame, x is the value representing the audio signal; Set an energy threshold. According to the energy calculation result, set an energy threshold Emin and compare it with the energy of each frame. If the frame energy is greater than the energy threshold, the frame is considered to be a speech frame, otherwise it is a non-speech frame.

3. A vehicle machine voice recognition method according to claim 2, characterized in that: The speech feature parameters include MFCC coefficients, formants and pitch; The method for obtaining the MFCC coefficients includes: Perform short-time Fourier transform on each speech frame to convert the time domain signal into a frequency domain signal; The spectrum of each frame is filtered through the Mel frequency filter bank and converted into energy value on the Mel scale; After the energy value of each frequency band is obtained by the Mel frequency filter bank, it is logarithmically transformed; Calculate the discrete cosine transform of the logarithmic energy to extract the MFCC coefficients; The formula for discrete cosine transform is: ； In the formula, is the energy of the ith Mel band, is the MFCC coefficient after discrete cosine transformation, is the number of Mel frequency filter banks.

4. The vehicle machine voice recognition method according to claim 1, characterized in that: The training method of the first machine learning model for defective speech recognition includes: Under the experimental state, speech feature parameters of normal speech frames and defective speech frames are collected, and corresponding actual annotations are generated for the state values ​​of the speech frames when each group of speech feature parameters is collected. The actual annotations corresponding to normal speech frames are 0, and the actual annotations corresponding to defective speech frames are 1. Each group of speech feature parameters is used as the input of the first machine learning model for defective speech recognition. The first machine learning model for defective speech recognition uses the predicted annotations for each group of speech feature parameters as output, the actual annotations as prediction targets, and minimizing the sum of the prediction accuracies of all speech feature parameters as training targets.

5. The vehicle machine voice recognition method according to claim 1, characterized in that: The method of evaluating the repair result based on the corrected voice data and generating a direct confirmation instruction or a return confirmation instruction based on the evaluation result includes: Acquire restoration feature parameters of the corrected speech data, wherein the restoration feature parameters include a speech quality perception evaluation value, a short-time objective intelligibility evaluation value, and a speech quality objective evaluation value; Generate a repair assessment coefficient based on the repair feature parameters; A repair assessment coefficient threshold XFm is preset, and the repair assessment coefficient is compared with the repair assessment coefficient threshold to generate a direct confirmation instruction or return a confirmation instruction.

6. A vehicle machine voice recognition method according to claim 5, characterized in that: The calculation formula of the repair assessment coefficient is: ； In the formula, To repair the evaluation coefficient, is the perceptual evaluation value of speech quality, is the short-term objective intelligibility assessment value, is the objective evaluation value of voice quality, , and is the weight coefficient, , and Both are greater than 0.

7. The vehicle machine voice recognition method according to claim 5, characterized in that: The method of comparing and analyzing the repair assessment coefficient with the repair assessment coefficient threshold to generate a direct determination instruction or return a confirmation instruction includes: when ≥XFm, generate direct determination instructions; when <XFm, generate return confirmation instruction.

8. A vehicle machine speech recognition system, used to implement a vehicle machine speech recognition method according to any one of claims 1 to 7, characterized in that: include: The data acquisition module collects the voice data in the cab through the microphone and records it as the initial voice data; The data processing module uses the VAD algorithm to detect and analyze the initial voice data, identify the voice frames in the initial voice data, and remove the non-voice frames; The data analysis module analyzes each speech frame, obtains speech feature parameters of the speech frame, and inputs them into the pre-built first machine learning model for defective speech recognition to obtain defective speech data; The data repair module repairs the defective voice data and records it as corrected voice data; The instruction generation module evaluates the repair result based on the corrected speech data, and generates a direct confirmation instruction or a return confirmation instruction based on the evaluation result.

Images