AI intelligent voice device speech recognition method and system

Abstract

The present application relates to the technical field of speech recognition, in particular to an AI intelligent voice device speech recognition method and system, the method comprising the following steps: collecting a driver's pronunciation simulation signal, separating the voice content, the in-vehicle background noise and the pronunciation characteristics, extracting the cepstrum coefficient, the fundamental frequency, the speech speed, the volume and the spectral tilt, combining the vehicle speed and the engine speed to generate the noise composition intensity and the Lombard intensity index, determining the acoustic scene, generating the target cepstrum coefficient and the dialect enhancement compensation characteristics, and fusing the data to decode the speech recognition sentence.In the present application, through the separation of the voice noise pronunciation characteristics, the multi-dimensional acoustic feature extraction, the linkage analysis of the vehicle operating state and the pronunciation change, the noise environment and the Lombard effect are quantified in real time, the cepstrum characteristics are compensated around the acoustic scene and the dialect characteristics are enhanced, the feature distortion caused by the superposition of the dynamic noise dialect difference and the pronunciation variation is reduced, the speech recognition accuracy and robustness in complex environments are improved, and the misrecognition, non-response and operation risk are reduced.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a speech recognition method and system for an AI intelligent speech device. Background Technology

[0002] The field of speech recognition technology belongs to the field of human-computer speech interaction and sound information analysis. It mainly revolves around core aspects such as speech acquisition, acoustic feature extraction, articulator matching, word order combination, semantic command correspondence, and recognition result output. It typically involves microphone arrays to collect ambient sound, pre-emphasizing, framing, windowing, endpoint detection, and noise estimation on the acquired speech frames, extracting speech features such as Mel frequency cepstral coefficients, filter bank energy, fundamental frequency, and short-time energy, and then determining the speech content in text or command form based on acoustic models, pronunciation dictionaries, and language sequence rules.

[0003] Traditional AI intelligent voice device speech recognition methods refer to the speech content recognition process applied to devices such as intelligent terminals, intelligent speakers, in-vehicle voice devices, and home control devices with voice input and interaction functions. The technical issues it addresses are usually the process of corresponding user speech from acquisition to recognition of text or control commands. Traditional methods generally acquire user speech through a microphone, extract silent segments and divide effective speech segments from continuous audio, extract Mel-frequency cepstral coefficients or filter bank energy according to a fixed frame length, compare the extracted speech features with the pre-trained acoustic state frame by frame, and then combine phoneme tables, initial and final combination tables, common word dictionaries, grammar rule tables, and instruction word lists to determine candidate recognition content. The candidate content is then filtered through confidence thresholds, keyword matching, contextual word order constraints, and preset device control command tables to form text content or device commands corresponding to the interaction scenario of the intelligent voice device.

[0004] Traditional speech recognition methods rely on fixed frame lengths and frame shifts for speech processing. However, they struggle to capture instantaneous spectral changes caused by rapid fluctuations in wind, tire, and engine noise during urban commutes due to speed changes. Background noise and speech content overlap at the feature level, and drivers experience changes in volume, vowel length, fundamental frequency shift, and spectral tilt due to noise stimulation. Furthermore, the inherent differences in the length and pitch of dialects are intertwined, making it difficult for acoustic models to distinguish between the adaptation of real semantic pronunciation and noise interference. This can easily lead to misrecognition and unresponsiveness of key commands, forcing drivers to repeat themselves or perform manual operations while driving, thus weakening the reliability of interactions and increasing safety risks. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing an AI-powered intelligent voice device speech recognition method and system.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a speech recognition method for an AI intelligent voice device, comprising the following steps: S1: Collect the driver's simulated speech signal, convert and separate the simulated speech signal to obtain a multi-channel separated acoustic signal set containing speech content signal, in-vehicle background noise signal and driver's speech feature signal; S2: Based on the multi-channel separated acoustic signal set, extract the speech Mel frequency cepstral coefficients, driver's pronunciation fundamental frequency data, driver's speech rate characteristic data, driver's speaking volume data, and driver's spectral tilt data; S3: Based on the driver's fundamental frequency data, speaking volume data, spectrum tilt data, and in-vehicle background noise signal, combined with the vehicle's real-time speed data and the engine's real-time rotational speed data, the current environmental noise composition data, in-vehicle real-time noise intensity data, and the driver's Lombard intensity index are obtained. S4: Based on the driver's Lombard intensity index, the current environmental noise composition data, and the real-time noise intensity data inside the vehicle, determine the acoustic scene association state inside the vehicle and obtain the cepstral coefficients of the target speech and the dialect enhancement compensation feature set; S5: Based on the cepstral coefficients of the target speech, the current environmental noise composition data, the driver's Lombard intensity index, and the dialect enhancement compensation feature set, decode and generate the target speech recognition sentence.

[0007] The present invention is improved in that the specific steps of S1 are as follows: S111: While the vehicle is in motion, the driver's voice simulation signal is collected in real time through the in-vehicle miniature digital microphone array. The analog-to-digital conversion program is called to perform continuous waveform sampling operation to extract discrete time signal sequence. Based on the device's preset quantization order, the discrete time signal sequence is subjected to discrete level amplitude quantization processing to extract digital quantization signal features. The digital quantization signal features are combined and encoded to obtain the original mixed sound digital signal stream. S112: Input the original mixed-sound digital signal stream into the digital signal processor, perform frequency band component isolation and screening operation on the original mixed-sound digital signal stream according to the multi-path frequency band decomposition rules built into the digital signal processor, extract independent speech content signals and in-vehicle background noise signals, and compare the acoustic energy distribution change characteristics of the original mixed-sound digital signal stream and the in-vehicle background noise signals to separate and extract the driver's pronunciation feature signals. S113: Perform a timestamp alignment operation under a reference clock on the voice content signal, the in-vehicle background noise signal and the driver's pronunciation feature signal, and perform matrix tensor dimension splicing and encapsulation encoding on the multi-dimensional channel feature data extracted in the timestamp alignment operation stage to establish a multi-channel separated acoustic signal set.

[0008] The present invention is improved in that the specific steps of S2 are as follows: S211: Call the speech content signal from the multi-channel separated acoustic signal set, perform a short-time Fourier transform operation on the speech content signal to extract different frequency components, input the different frequency components into a preset frequency domain filter bank for frequency band energy mapping processing, calculate the discrete cosine transform values ​​of the different frequency components, and generate speech Mel frequency cepstral coefficients. S212: Based on the speech content signal, perform time-domain waveform scanning to extract continuous repeating bands, regard the continuous repeating bands as periodic feature objects of the YIN fundamental frequency estimation algorithm, call the YIN fundamental frequency estimation algorithm to calculate the band repetition frequency of the periodic feature objects, obtain the driver's pronunciation fundamental frequency data, extract the pronunciation segment duration and silence segment duration of the speech content signal, calculate the ratio of the pronunciation segment duration to the silence segment duration, and generate driver speech speed feature data; S213: Extract the driver's vocal feature signal from the multi-channel separated acoustic signal set, calculate the short-time energy mean of the driver's vocal feature signal to generate driver's speaking volume data, extract low-frequency energy data and high-frequency energy data from the driver's vocal feature signal according to the preset frequency band interval rule, calculate the quotient between the low-frequency energy data and the high-frequency energy data, and generate driver's spectral tilt data.

[0009] The present invention is improved in that the specific steps of S3 are as follows: S311: Acquire real-time vehicle speed data and real-time engine speed data; perform short-time Fourier transform on the in-vehicle background noise signal to extract amplitude values ​​of different frequency bands to generate in-vehicle noise spectrum distribution data; calculate the root mean square amplitude of the in-vehicle background noise signal per unit time to generate in-vehicle acoustic energy data; based on the real-time vehicle speed data and the real-time engine speed data, filter high-frequency wind noise frequency band energy data and low-frequency engine roar frequency band energy data within the in-vehicle noise spectrum distribution data, combine them to generate current environmental noise composition data; calculate the differences between the driver's speaking volume data, the driver's fundamental frequency data, and the driver's spectrum tilt data at the current time node and the previous time node respectively, to obtain the driver's volume change deviation, the fundamental frequency change deviation, and the spectrum tilt change deviation. S312: Calculate the real-time noise intensity data inside the vehicle based on the in-vehicle acoustic energy data, call the extreme value normalization mapping function to map and transform the real-time noise intensity data inside the vehicle, the driver's volume change deviation, the fundamental frequency change deviation, and the spectral tilt change deviation to the reference interval, and obtain the normalized mapping values ​​of the in-vehicle noise intensity, the driver's volume change, the fundamental frequency change, and the spectral tilt change. Calculate the weighted combination difference of the normalized mapping values ​​of the in-vehicle noise intensity, the driver's volume change, the fundamental frequency change, and the spectral tilt change according to the preset weight configuration items, and generate the driver's Lombard intensity index.

[0010] The present invention is improved in that the process of calculating the real-time noise intensity data inside the vehicle based on the in-vehicle acoustic energy data is as follows: An energy smoothing sliding window is constructed according to a predetermined collection duration. The in-vehicle acoustic energy data is extracted at the current time point through this window to generate a local time-segment energy sample sequence. The minimum average pickup energy recorded during the silent calibration phase is set as the decibel conversion reference base. Amplitude characteristic values ​​of multiple consecutive acoustic sampling points within the local time-segment energy sample sequence are extracted. The square mean of the amplitude characteristic values ​​is calculated to extract the real-time root mean square energy parameter. The energy ratio quotient between the real-time root mean square energy parameter and the decibel conversion reference base is calculated. A logarithmic conversion program is called to perform logarithmic calculation processing on the energy ratio quotient to extract initial logarithmic intensity decibel data. The sensitivity attenuation calibration value within the hardware configuration specification file is read to generate a sound pressure level conversion multiplier. The initial logarithmic intensity decibel data and the sound pressure level conversion multiplier are multiplied to generate real-time in-vehicle noise intensity data.

[0011] The present invention is improved in that the specific steps of S4 are as follows: S411: Combine the driver's Lombard intensity index, current environmental noise composition data, and real-time in-vehicle noise intensity data into an environmental acoustic feature vector. Input the environmental acoustic feature vector into a preset scene clustering model to calculate the Euclidean distance with each reference scene node. Output the scene category with the smallest Euclidean distance and determine it as the in-vehicle acoustic scene association state. Generate a multi-channel feature fusion weight vector based on the weight distribution of the in-vehicle acoustic scene association state as speech content signal, in-vehicle background noise signal, and driver's pronunciation feature signal. Determine the high-frequency attenuation coefficient based on the multi-channel feature fusion weight vector. Use the high-frequency attenuation coefficient to attenuate the high-frequency coefficient value in the speech Mel-frequency cepstral coefficient to obtain the target speech cepstral coefficient. S412: For the driver's pronunciation fundamental frequency data and driver's speech rate characteristic data, calculate the initial deviation amount relative to the reference pronunciation data in silent mode, and perform adaptive weighting adjustment on the initial deviation amount according to the driver's Lombard intensity index to obtain the environmental adaptive pronunciation compensation difference. Subtract the environmental adaptive pronunciation compensation difference from the driver's pronunciation fundamental frequency data and the driver's speech rate characteristic data to obtain the driver's normal pronunciation fundamental frequency data and the driver's normal speech rate characteristic data. S413: Calculate the first-order difference rate of change of the driver's normal pronunciation fundamental frequency data along the time axis to separate the driver's dialect tone feature parameters that characterize tone fluctuations, and assign a first enhancement weight coefficient to obtain enhanced dialect tone feature parameters. Call the driver's normal speech rate feature data, extract the ratio of the relative duration of the pronunciation segments to separate the driver's dialect long vowel feature parameters that characterize the habit of prolonging sounds, and assign a second enhancement weight coefficient to obtain enhanced dialect long vowel feature parameters. Combine the enhanced dialect tone feature parameters and the enhanced dialect long vowel feature parameters to establish a dialect enhancement compensation feature set.

[0012] The present invention is improved in that the process of obtaining the environmental adaptive pronunciation compensation difference is as follows: In silent mode, the driver's vocal samples are analyzed to extract noise-free fundamental frequency reference parameters and noise-free speech rate reference parameters. The difference between the driver's vocal fundamental frequency data and the noise-free fundamental frequency reference parameters is calculated to obtain the initial fundamental frequency deviation. The difference between the driver's speech rate feature data and the noise-free speech rate reference parameters is calculated to obtain the initial speech rate deviation. The historical Lombard intensity index corresponding to multiple sets of historical vocal samples is extracted as input feature samples. The optimal fundamental frequency adjustment ratio and the optimal speech rate adjustment ratio that restore the historical vocal features to normal are extracted as target output labels. A multilayer perceptron network is trained using the backpropagation algorithm to construct a nonlinear mapping model. The current driver Lombard intensity index is input into the trained nonlinear mapping model for forward propagation calculation, and the predicted fundamental frequency adjustment coefficient and speech rate adjustment coefficient are output. The product of the initial fundamental frequency deviation and the fundamental frequency adjustment coefficient is calculated to extract the fundamental frequency compensation difference. The product of the initial speech rate deviation and the speech rate adjustment coefficient is calculated to extract the speech rate compensation difference. The fundamental frequency compensation difference and the speech rate compensation difference are combined to generate an environmentally adaptive vocal compensation difference.

[0013] The present invention is improved in that the specific steps of S5 are as follows: S511: Based on the multi-channel feature fusion weight vector, for the target speech cepstral coefficient, the current environmental noise composition data, and the driver's Lombard intensity index, combined with the enhanced dialect tone feature parameters and enhanced dialect long vowel feature parameters extracted from the dialect enhancement compensation feature set, a numerical product operation is performed to obtain a weighted feature parameter array. The weighted feature parameter array is then spliced ​​and connected along the time axis reference dimension to establish a multi-channel comprehensive feature vector. S512: Obtain a local preset syllable template, call the multi-channel integrated feature vector into the acoustic matching feature classifier, calculate the Euclidean distance difference variable between the multi-channel integrated feature vector and the spatial nodes inside the preset syllable template, determine the acoustic feature similarity, extract audio segments in the extreme value range of acoustic feature similarity according to the extreme value search comparison rule, extract the optimal matching pronunciation unit, obtain the preset grammar and vocabulary constraint rules, call the optimal matching pronunciation unit into the language decoding program, calculate the word temporal jump cost in combination with the preset grammar and vocabulary constraint rules and filter out conflicting branches, coherently reconstruct the text unit, and generate the target speech recognition sentence.

[0014] The present invention is improved in that the specific process of generating the target speech recognition sentence is as follows: A word sequence transition probability matrix is ​​constructed based on the word co-occurrence frequency of the local corpus and set as the preset grammatical lexical constraint rule; the predecessor and successor word nodes of the optimal matching pronunciation unit at adjacent time nodes are extracted, and the acoustic observation probability corresponding to the successor word node is obtained; the conditional probability value of the jump from the predecessor word node to the successor word node is retrieved in the preset grammatical lexical constraint rule; the logarithmic decay value corresponding to the acoustic observation probability is calculated, and the negative logarithmic transformation value of the conditional probability value is added to the logarithmic decay value to generate the word temporal jump cost; the historical sequence transition cost is extracted. The mean and standard deviation of the total cost data of the correct decoding path are used to construct a dynamic pruning baseline value by summing the mean and standard deviation. The cumulative cost sum of the temporal jump costs of all the words in the current decoding path branch is calculated. The corresponding decoding path branch with a cumulative cost sum greater than the dynamic pruning baseline value is identified as a conflicting branch and removed. The global optimal path with the smallest cumulative cost sum is extracted from the remaining decoding path branches. The word nodes mapped in the global optimal path are concatenated in temporal order to coherently reconstruct text units and generate the target speech recognition sentence.

[0015] An AI-powered intelligent voice device speech recognition system is provided, the system being used to implement the aforementioned AI-powered intelligent voice device speech recognition method, the system comprising: The sound source analysis module collects the driver's simulated pronunciation signal, converts and separates the simulated pronunciation signal to obtain a multi-channel separated acoustic signal set containing speech content signal, in-vehicle background noise signal and driver pronunciation feature signal; The speech representation module extracts the speech Mel-frequency cepstral coefficients, driver's fundamental frequency data, driver's speech rate feature data, driver's speaking volume data, and driver's spectral tilt data based on the multi-channel separated acoustic signal set. The driving noise perception module obtains the current environmental noise composition data, real-time in-vehicle noise intensity data, and driver Lombard intensity index based on the driver's voice fundamental frequency data, speech volume data, spectrum tilt data, and in-vehicle background noise signal, combined with the vehicle's real-time speed data and engine's real-time rotation speed data. The scene compensation module determines the in-vehicle acoustic scene association state based on the driver's Lombard intensity index, the current environmental noise composition data, and the real-time noise intensity data inside the vehicle, and obtains the cepstral coefficients of the target speech and the dialect enhancement compensation feature set. The recognition and recombination module decodes and generates the target speech recognition sentence based on the cepstral coefficients of the target speech, the current environmental noise composition data, the driver's Lombard intensity index, and the dialect enhancement compensation feature set.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by converting and separating the driver's voice simulation signal, a multi-channel acoustic signal is formed in which the voice content, in-vehicle background noise, and voice features are independent. The voice state is represented by the cepstral coefficients, fundamental frequency, speech rate, volume, and spectral tilt. The noise composition and real-time intensity are identified by integrating vehicle speed and engine speed. The correlation between Lombard intensity and in-vehicle acoustic scene is quantified, guiding the construction of cepstral feature compensation and dialect enhancement features. This allows noise fluctuations and pronunciation to adapt to dialect differences to be corrected in layers before decoding, resulting in clearer true semantic boundaries. The stability of navigation, communication media, and control command recognition is improved, the need for repetitive interaction and manual operation is reduced, and the safety of driving voice control is enhanced. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system module diagram of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] Please see Figure 1This invention provides a technical solution: a voice recognition method for an AI intelligent voice device, comprising the following steps: S1: Collect the driver's speech simulation signal, convert and separate the speech simulation signal to obtain a multi-channel separated acoustic signal set containing speech content signal, in-vehicle background noise signal and driver speech feature signal; S2: Based on the multi-channel separated acoustic signal set, extract the speech Mel frequency cepstral coefficients, driver's pronunciation fundamental frequency data, driver's speech rate characteristic data, driver's speaking volume data, and driver's spectral tilt data; S3: Based on the driver's voice fundamental frequency data, speaking volume data, spectrum tilt data, and in-vehicle background noise signal, combined with the vehicle's real-time speed data and the engine's real-time speed data, the current environmental noise composition data, in-vehicle real-time noise intensity data, and the driver's Lombard intensity index are obtained. S4: Based on the driver's Lombard intensity index, current environmental noise composition data, and real-time noise intensity data inside the vehicle, determine the in-vehicle acoustic scene association state and obtain the cepstral coefficients of the target speech and the dialect enhancement compensation feature set; S5: Based on the cepstral coefficients of the target speech, the current environmental noise composition data, the driver's Lombard intensity index, and the dialect enhancement compensation feature set, decode and generate the target speech recognition sentence.

[0020] The specific steps of S1 are as follows: S111: While the vehicle is in motion, the in-vehicle miniature digital microphone array collects the simulated sound signal of the driver's voice in real time. The analog-to-digital conversion program is called to perform continuous waveform sampling operation to extract the discrete-time signal sequence. Based on the device's preset quantization order, the discrete-time signal sequence is subjected to discrete level amplitude quantization processing to extract digital quantization signal features. The digital quantization signal features are combined and encoded to obtain the original mixed sound digital signal stream. With the vehicle in motion, with the engine running and in either forward or reverse gear, a miniature digital microphone array installed in the front headliner collects the driver's vocalizations in real time. The miniature digital microphone array uses internal piezoelectric sensors to capture airborne sound wave vibrations and converts them into continuous analog voltage waveforms. For this vocalization signal, the analog-to-digital converter (ADC) module of the onboard audio processing microcontroller performs continuous waveform sampling. The ADC module's sampling rate is set to 44,100 Hz, based on the requirement of covering the highest perceptible range of 20,000 Hz. According to the Nyquist sampling theorem, the sampling rate must be greater than twice the highest signal frequency to prevent signal distortion. The voltage amplitude of the vocalization signal is read at fixed time intervals, and a discrete-time signal sequence is obtained by extracting the voltage values ​​at multiple discrete time points. The system reads the device's preset 16-bit quantization order specification, which divides the continuous voltage amplitude range into 65,536 fixed discrete level levels. It compares the voltage value at each time point in the discrete-time signal sequence with the upper and lower boundary extremes of each discrete level level. If the voltage value is greater than or equal to the lower limit extreme value of a level level but less than its upper limit extreme value, the voltage value is replaced with the standard representative value corresponding to the center point of that level level. If it does not belong to this range, the comparison continues with the next level level. All replaced standard representative values ​​are extracted, thus completing the discrete level amplitude quantization process and extracting the corresponding digital quantized signal features. Subsequently, all digital quantized signal features are input into the binary pulse code modulation module of the Python software environment. The standard representative values ​​are converted into pulse code binary data streams in chronological order and combined for encoding to obtain the original mixed-audio digital signal stream.

[0021] S112: Input the original mixed sound digital signal stream into the digital signal processor, perform frequency band component isolation and screening operation on the original mixed sound digital signal stream according to the multipath frequency band decomposition rules built into the digital signal processor, extract independent speech content signals and in-vehicle background noise signals, and compare the acoustic energy distribution change characteristics of the original mixed sound digital signal stream and the in-vehicle background noise signals to separate and extract the driver's pronunciation feature signals. The original mixed-audio digital signal stream is input into the digital signal processor chip of the vehicle terminal. The multipath frequency band decomposition rule built into the digital signal processor is read. This rule predefines the human voice frequency band range of 300 Hz to 3400 Hz, as well as the background noise frequency band range of less than 300 Hz and greater than 3400 Hz. The basis for this setting is that the main acoustic energy of human daily speech is concentrated between 300 Hz and 3400 Hz. Below 300 Hz, it is mostly low-frequency vibration sound of the chassis when the vehicle is running, and above 3400 Hz, it is mostly high-frequency wind noise or electronic background noise. According to this rule, the frequency band component isolation and screening operation is performed on the original mixed-audio digital signal stream. In the Python software environment, the bandpass filter function of the SciPy signal processing library is called to compare the frequency values ​​of each component in the original mixed-audio digital signal stream. If the frequency value is greater than or equal to 300 Hz and less than or equal to 3400 Hz, the signal component corresponding to that frequency is retained and extracted as an independent speech content signal. If the frequency value is less than 300 Hz or greater than 3400 Hz, the signal component corresponding to that frequency is intercepted, classified, and extracted as the in-vehicle background noise signal. The average sound pressure level (SPL) value of the original mixed-audio digital signal stream within a fixed time window is extracted, and the average SPL value of the in-vehicle background noise signal within the same time window is extracted. The acoustic energy distribution variation characteristic is calculated as: average SPL value of the original mixed-audio digital signal stream minus average SPL value of the in-vehicle background noise signal. The acoustic energy distribution variation characteristic is obtained by comparing the original mixed-audio digital signal stream and the in-vehicle background noise signal. Based on the acoustic energy distribution variation characteristics, data segments that completely overlap with the peak and trough shapes of the in-vehicle background noise signal are searched in the data sequence of the original mixed acoustic digital signal stream and removed, so as to separate and extract the pure driver's vocal feature signal.

[0022] S113: Perform timestamp alignment operation under the reference clock for the voice content signal, the in-vehicle background noise signal and the driver's pronunciation feature signal, combine the multi-dimensional channel feature data extracted in the timestamp alignment operation stage, perform matrix tensor dimension splicing and encapsulation encoding, and establish a multi-channel separated acoustic signal set; The speech content signal, in-vehicle background noise signal, and driver's vocal feature signal are input into the NumPy matrix processing library in the Python software environment. The master control reference clock node marker of the miniature digital microphone array is read, and data segments with the same time node marker in the speech content signal, in-vehicle background noise signal, and driver's vocal feature signal are retrieved and timestamped under the reference clock. The amplitude values ​​of the speech content signal, in-vehicle background noise signal, and driver's vocal feature signal at the same time node obtained during the timestamping operation are extracted and used as multi-dimensional channel feature data. A three-dimensional tensor space is constructed, with the time sequence as the first dimension, the signal type identifier as the second dimension, and the amplitude value as the third dimension. The multi-dimensional channel feature data are sequentially filled into the corresponding row and column coordinate positions of this three-dimensional tensor space for matrix tensor dimension concatenation. The concatenated three-dimensional matrix data is translated into a continuous serialized string format using a data stream conversion function for encapsulation and encoding, establishing a multi-channel separated acoustic signal set.

[0023] The specific steps of S2 are as follows: S211: Call the speech content signal from the multi-channel separated acoustic signal set, perform short-time Fourier transform operation on the speech content signal to extract different frequency components, input the different frequency components into the preset frequency domain filter bank for frequency band energy mapping processing, calculate the discrete cosine transform values ​​of different frequency components, and generate speech Mel frequency cepstral coefficients. The speech content signal from the multi-channel separated acoustic signal set is input into the Python software environment. Continuous speech content signals are extracted at fixed intervals of 20 milliseconds to form multiple independent data frame sequences. The SciPy signal processing library is used to multiply the data sampling points at both ends of each data frame sequence by a Hamming window function attenuation coefficient to eliminate spectral leakage at the data truncation edges. A short-time Fourier transform is performed on the smoothed time-domain data frames to map the time-domain waveform into a frequency-domain data sequence composed of multiple different frequencies and corresponding amplitude values, extracting different frequency components. A pre-defined frequency-domain filter bank is constructed using 24 triangular frequency-domain filters with a logarithmic distribution of center frequencies. The logarithmic distribution of center frequencies is based on the Mel acoustic scale characteristic, which simulates the sensitivity of human auditory organs to low-frequency sound frequency changes but insensitivity to high-frequency sound frequency changes. The extracted frequency values ​​of different frequency components are compared with the extreme values ​​of the coverage area of ​​each triangular frequency domain filter. If the frequency value of a frequency component falls within the coverage area of ​​a certain triangular frequency domain filter, the corresponding amplitude value is added to the frequency band energy value of that filter. After traversing and comparing, 24 independent frequency band energy values ​​are obtained. The frequency band energy mapping logarithm value = ln(frequency band energy value). The natural logarithm operation is performed on each of the 24 extracted frequency band energy values ​​to complete the frequency band energy mapping process. Discrete cosine transform value = (Logarithmic value of band energy mapping) (Corresponding to the cosine basis coefficients), the cosine series expansion of the mapped logarithmic energy value is used to extract the discrete cosine transform (DCT) values. The first 12 low-order DCT values ​​are retained in the order of the series expansion terms to generate and output the speech Mel-frequency cepstral coefficients.

[0024] S212: Extract continuous repeating bands by time-domain waveform scanning based on speech content signals. Treat the continuous repeating bands as periodic feature objects of the YIN fundamental frequency estimation algorithm. Call the YIN fundamental frequency estimation algorithm to calculate the band repetition frequency of the periodic feature objects, obtain the driver's speech fundamental frequency data, extract the duration of the speech segment and the duration of the silence segment of the speech content signal, calculate the ratio of the duration of the speech segment to the duration of the silence segment, and generate driver speech speed feature data. Based on the speech content signal from the multi-channel separated acoustic signal set, a signal waveform scanning component is invoked in the Python software environment to compare the coordinates of the amplitude extreme points of adjacent waveform periods one by one, extracting continuous waveform segments whose amplitude values ​​are consistent with the time interval between peaks and troughs, and obtaining continuous repeating bands. These continuous repeating bands are considered as periodic feature objects for the YIN fundamental frequency estimation algorithm. The variance data of the fluctuation of the pure speech waveform under silent conditions is read, and a redundancy coefficient of 1.5 is multiplied to set a minimum threshold, typically 0.1. The basis for this setting is that if the sum of squares of the waveform differences is below this threshold, it indicates that the two waveform segments have a very high degree of periodic overlap. In the YIN fundamental frequency estimation algorithm, the sum of squared waveform differences before and after different delay time points of the periodic feature object is calculated. The relationship between this sum of squared waveform differences and the minimum threshold is compared. If the sum of squared waveform differences is less than the minimum threshold, the reciprocal of the delay time value at this point is taken as the band repetition frequency of the periodic feature object, and the delay time is stopped from increasing further. If the sum of squared waveform differences is greater than or equal to the minimum threshold, the delay time is increased by a step size, the sum of squared waveform differences is calculated again, and the comparison is repeated until the band repetition frequency is found, thus obtaining the driver's fundamental frequency data. The mean value of the microphone equipment's noise floor test record is read, and 50 mV is set as the extreme value for sound state judgment. The number of consecutive sampling points with an absolute amplitude value greater than 50 mV in the speech content signal is counted and multiplied by the single sampling time length to extract the duration of the speech segment. The number of consecutive sampling points with an absolute amplitude value less than or equal to 50 mV in the speech content signal is counted and multiplied by the single sampling time length to extract the duration of the silence segment. Driver speech rate feature data = duration of pronunciation segment / duration of silence segment, generating driver speech rate feature data.

[0025] S213: Extract the driver's vocal feature signal from the multi-channel separated acoustic signal set, calculate the short-time energy mean of the driver's vocal feature signal to generate driver's speaking volume data, and extract low-frequency energy data and high-frequency energy data from the driver's vocal feature signal according to the preset frequency band interval rules. Calculate the quotient between the low-frequency energy data and the high-frequency energy data to generate driver's spectral tilt data. The driver's vocal feature signal is extracted from the multi-channel separated acoustic signal set and divided into multiple short-time data frames with a duration of 20 milliseconds. The sum of squares of the short-time data frames = (Amplitude value of a single sampling point) (Amplitude value of a single sampling point), calculate the amplitude characteristic data of all sampling points within each short-time data frame. Driver's speaking volume data = The arithmetic mean of the sum of squares of the amplitudes of all short-time data frames (sum of squares of short-time data frames) / total number of short-time data frames is used to generate the driver's speech volume data. A frequency range greater than or equal to 0 Hz and less than 1000 Hz is defined as the low-frequency band, and a frequency range greater than or equal to 1000 Hz and less than or equal to 4000 Hz is defined as the high-frequency band, forming a preset frequency band rule. This band is set based on the fact that the energy of vowels in human speech signals is mostly concentrated in the low-frequency band below 1000 Hz, while the energy of consonants and voiceless fricatives is mostly distributed in the high-frequency band between 1000 Hz and 4000 Hz. The frequency and amplitude sequence of the driver's vocal characteristic signal after short-time Fourier transform is read, and the values ​​of each frequency component are compared with the preset frequency band rule. If the frequency falls into the low-frequency band, the corresponding amplitude value is accumulated, and low-frequency energy data is extracted from the driver's vocal characteristic signal. If the frequency falls into the high-frequency band, the corresponding amplitude value is accumulated, and high-frequency energy data is extracted. Driver spectrum tilt data = low-frequency energy data / high-frequency energy data, generating driver spectrum tilt data.

[0026] The specific steps for S3 are as follows: S311: Acquire real-time vehicle speed data and real-time engine speed data, perform short-time Fourier transform on the in-vehicle background noise signal to extract amplitude values ​​of different frequency bands to generate in-vehicle noise spectrum distribution data, and calculate the root mean square amplitude of the in-vehicle background noise signal per unit time to generate in-vehicle acoustic energy data. Based on the real-time vehicle speed data and real-time engine speed data, filter high-frequency wind noise frequency band energy data and low-frequency engine roar frequency band energy data in the in-vehicle noise spectrum distribution data, combine them to generate current environmental noise composition data, and calculate the differences between the driver's speaking volume data, the driver's fundamental frequency data and the driver's spectrum tilt data at the current time node and the previous time node respectively to obtain the driver's volume change deviation, the fundamental frequency change deviation and the spectrum tilt change deviation. Real-time vehicle speed data from the vehicle speed sensor and real-time engine speed data from the engine speed sensor are read via the vehicle's CAN bus network data interface. The in-vehicle background noise signal, obtained from a multi-channel discrete acoustic signal set, is input into a Python software environment. The data is divided into blocks with a fixed window length of 50 milliseconds. A short-time Fourier transform is performed on each block to extract the amplitude values ​​of the components corresponding to different vibration frequencies of the in-vehicle background noise signal, generating in-vehicle noise spectrum distribution data. The total amplitude values ​​of the in-vehicle background noise signal within a 1-second unit time are extracted. In-vehicle acoustic energy data = The system calculates the squared mean of all amplitude values ​​to extract real-time root-mean-square energy parameters and generate in-vehicle acoustic energy data. A vehicle speed threshold of 80 km / h is set, based on aerodynamic testing showing that wind shear noise on the vehicle surface increases exponentially and becomes the main high-frequency noise source when the vehicle speed exceeds this threshold. An engine speed threshold of 2500 rpm is set, based on physical resonance characteristics showing that the low-frequency resonance effect of the engine block begins to significantly transmit to the passenger compartment when this speed is exceeded. The system compares the real-time vehicle speed data with the thresholds. If the real-time vehicle speed is greater than 80 km / h, all amplitude values ​​corresponding to frequencies greater than or equal to 3400 Hz are selected from the in-vehicle noise spectrum distribution data and summed to obtain high-frequency wind noise energy data. If the real-time vehicle speed is less than or equal to 80 km / h, the current wind noise is considered insignificant, and high-frequency wind noise energy data is not extracted. Compare the real-time engine speed data with the defined boundary. If the real-time engine speed data is greater than 2500 rpm, then all amplitude values ​​corresponding to frequencies less than 300 Hz are selected from the in-vehicle noise spectrum distribution data and summed to obtain the low-frequency engine roar frequency band energy data. If the real-time engine speed data is less than or equal to 2500 rpm, then the current engine noise is considered stable, and the low-frequency engine roar frequency band energy data is rejected. The extracted high-frequency wind noise frequency band energy data and low-frequency engine roar frequency band energy data are filled into the same feature matrix and arranged in an array to generate the current environmental noise composition data. Driver volume change deviation = driver's speaking volume data at the current time point - driver's speaking volume data at the previous time point. Radiant fundamental frequency change deviation = driver's radiant fundamental frequency data at the current time point - driver's radiant fundamental frequency data at the previous time point. Spectral tilt change deviation = driver's spectral tilt data at the current time point - driver's spectral tilt data at the previous time point. The driver volume change deviation, radiant fundamental frequency change deviation, and spectral tilt change deviation are obtained respectively.

[0027] S312: Calculate the real-time noise intensity data inside the vehicle based on the in-vehicle acoustic energy data, call the extreme value normalization mapping function to map and transform the real-time noise intensity data inside the vehicle, the driver's volume change deviation, the fundamental frequency change deviation, and the spectral tilt change deviation to the reference range, obtain the normalized mapping values ​​of the in-vehicle noise intensity, the driver's volume change, the fundamental frequency change, and the spectral tilt change, and calculate the weighted combination difference of the normalized mapping values ​​of the in-vehicle noise intensity, the driver's volume change, the fundamental frequency change, and the spectral tilt change according to the preset weight configuration items, and generate the driver's Lombard intensity index; The process of calculating real-time noise intensity data inside the vehicle based on in-vehicle acoustic energy data is as follows: An energy smoothing sliding window is constructed according to a predetermined collection duration. The energy smoothing sliding window is used to extract in-vehicle acoustic energy data at the current time point to generate a local time-segment energy sample sequence. The minimum average pickup energy recorded during the silent calibration phase is set as the decibel conversion reference base. Amplitude characteristic values ​​of multiple consecutive acoustic sampling points within the local time-segment energy sample sequence are extracted. The square mean of the amplitude characteristic values ​​is calculated to extract the real-time root mean square energy parameter. The energy ratio quotient between the real-time root mean square energy parameter and the decibel conversion reference base is calculated. A logarithmic conversion program is called to perform logarithmic calculation processing on the energy ratio quotient to extract initial logarithmic intensity decibel data. The sensitivity attenuation calibration value within the hardware configuration specification file is read to generate a sound pressure level conversion multiplier. The initial logarithmic intensity decibel data and the sound pressure level conversion multiplier are multiplied numerically to generate real-time in-vehicle noise intensity data. An energy smoothing sliding window is constructed in a Python data sequence processing library according to a predetermined acquisition duration of 1 second. This window is used to extract in-vehicle acoustic energy data within a 1-second backward range from the current time point, generating a local time-segment energy sample sequence from all time series points within this interval. During the quiet, sealed calibration phase with the engine off and all doors and windows closed, 10 minutes of ambient noise recordings are continuously acquired, and the mean of the sum of squared amplitudes is calculated. The minimum pickup energy mean of this recording is read and set as the decibel conversion reference base. Amplitude characteristic values ​​corresponding to multiple consecutive acoustic sampling points within the local time-segment energy sample sequence are extracted. Real-time root mean square energy parameter = Energy ratio quotient = Real-time root mean square energy parameter / Decibels conversion reference base. The Python mathematical calculation module is used to perform logarithmic calculations on this energy ratio quotient, with an initial logarithmic intensity decibel data of 10. (Energy ratio quotient). The system reads the preset sensitivity attenuation calibration value from the hardware configuration specification file of the miniature digital microphone array via a hardware interface driver, extracts this calibration value, and directly generates the sound pressure level conversion multiplier. Real-time in-vehicle noise intensity data = initial logarithmic intensity decibel data. Sound pressure level conversion multiplier. Retrieve the historical maximum and minimum records of real-time in-vehicle noise intensity data from the storage area. In-vehicle noise intensity normalized mapping value = (real-time in-vehicle noise intensity data at the current time point - historical minimum record) / (historical maximum record - historical minimum record). Call this extreme value normalization mapping function to map and transform the data to a baseline range of 0 to 1 to obtain the normalized in-vehicle noise intensity mapping value. Using the exact same mathematical subtraction and division extreme value normalization mapping process, read the corresponding historical extreme value ranges for each, calculate and obtain the normalized mapping values ​​for driver volume change, fundamental frequency change, and spectral tilt change. The preset weighting parameters are: in-vehicle noise intensity weighting 0.3, volume change weighting 0.3, fundamental frequency change weighting 0.2, and spectral tilt change weighting 0.2. The basis for these weighting values ​​is as follows: environmental noise is the most direct external stimulus to alter human vocal behavior, hence it is set to the highest level at 0.3; volume change is the most intuitive external manifestation of overcoming the Lombard effect caused by noise, also set to 0.3; and fundamental frequency increase and spectral tilt change are accompanying unconscious physiological adjustment actions, hence they are each set to 0.2. The larger the weighting coefficient, the higher the contribution of that characteristic index to confirming that the driver is currently in a vocal compensation state to overcome noise interference. Driver Lombard Intensity Index = (In-vehicle Noise Intensity Standardized Mapping Value) 0.3) + (Standardized mapping value of driver volume change) 0.3)+(Standardized mapping value of fundamental frequency variation) 0.2)+(Standardized mapping value of spectral tilt change) 0.2), calculate the driver Lombardy intensity index by performing weighted combination difference calculation based on the preset weight configuration items.

[0028] The specific steps of S4 are as follows: S411: Combine the driver's Lombard intensity index, current environmental noise composition data, and real-time in-vehicle noise intensity data into an environmental acoustic feature vector. Input the environmental acoustic feature vector into a preset scene clustering model to calculate the Euclidean distance with each reference scene node. Output the scene category with the smallest Euclidean distance to determine the in-vehicle acoustic scene association state. Based on the in-vehicle acoustic scene association state, generate a multi-channel feature fusion weight vector by allocating the weights of the speech content signal, the in-vehicle background noise signal, and the driver's pronunciation feature signal. Determine the high-frequency attenuation coefficient based on the multi-channel feature fusion weight vector, and use the high-frequency attenuation coefficient to attenuate the high-frequency coefficient values ​​in the speech Mel-frequency cepstral coefficients to obtain the target speech cepstral coefficients. The driver's Lombard intensity index, current ambient noise composition data, and real-time in-vehicle noise intensity data are used as array elements, arranged sequentially and loaded into a list data structure in the Python software environment to form an environmental acoustic feature vector. Audio signals from three typical road conditions—urban roads, highways, and rural roads—are pre-recorded using an onboard acquisition device. The arithmetic mean of a large number of samples along this feature vector dimension is extracted and calculated. These three sets of arithmetic mean records are saved as standard feature vectors for urban roads, highways, and rural roads, respectively, serving as scene reference nodes. The aforementioned loaded environmental acoustic feature vector is then input into a K-Means scene clustering model algorithm program built using the Scikit-learn machine learning library. The Euclidean distance for urban roads is... Similarly, the Euclidean distances for highways and rural roads are calculated. By comparing the specific values ​​of these three Euclidean distances, the scene category with the minimum value is extracted, and the scene category with the smallest Euclidean distance is output and identified as the in-vehicle acoustic scene association state. The current in-vehicle acoustic scene association state category is determined. If the in-vehicle acoustic scene association state is urban road, based on the characteristics of urban roads where there are many horns and complex background noises from people outside the vehicle, which easily cause crosstalk, the weight of the speech content signal is allocated as 0.7, the weight of the in-vehicle background noise signal is 0.1, and the weight of the driver's vocal feature signal is 0.2. If the in-vehicle acoustic scene association state is highway, based on the continuous and stable high-frequency wind noise masking characteristics of highways, the weight of the speech content signal is allocated as 0.5, the weight of the in-vehicle background noise signal is 0.3, and the weight of the driver's vocal feature signal is 0.2. If the in-vehicle acoustic scene association state is rural road, based on the characteristics of rural roads where there are many low-frequency road bump sounds and drivers tend to strain their vocal cords, the weight of the speech content signal is allocated as 0.6, the weight of the in-vehicle background noise signal is 0.1, and the weight of the driver's vocal feature signal is 0.3. The above-mentioned weight values ​​are written into a single-dimensional matrix in sequence to generate a multi-channel feature fusion weight vector. Read the weighting parameters of the in-vehicle background noise signal from the multi-channel feature fusion weight vector. The high-frequency attenuation coefficient = 1 - the weighting parameter of the in-vehicle background noise signal. Target speech cepstral coefficients = the high-frequency coefficient values ​​in the speech Mel-frequency cepstral coefficients. The high-frequency attenuation coefficient is used to multiply and attenuate the high-frequency coefficient values ​​in the Mel-frequency cepstral coefficients of the speech, thereby obtaining the cepstral coefficients of the target speech.

[0029] S412: For the driver's pronunciation fundamental frequency data and driver's speech rate characteristic data, calculate the initial deviation of the data relative to the reference pronunciation data in silent mode, and perform adaptive weighting adjustment on the initial deviation according to the driver's Lombard intensity index to obtain the environmental adaptive pronunciation compensation difference. Subtract the environmental adaptive pronunciation compensation difference from the driver's pronunciation fundamental frequency data and driver's normal pronunciation fundamental frequency data and driver's normal speech rate characteristic data. The process of obtaining the environmental adaptive pronunciation compensation difference is as follows: In silent mode, driver vocal samples are analyzed to extract noise-free fundamental frequency reference parameters and noise-free speech rate reference parameters. The difference between the driver's vocal fundamental frequency data and the noise-free fundamental frequency reference parameters is calculated to obtain the initial fundamental frequency deviation. The difference between the driver's speech rate feature data and the noise-free speech rate reference parameters is calculated to obtain the initial speech rate deviation. Historical Lombard intensity indices from multiple sets of historical vocal samples are extracted as input feature samples. The optimal fundamental frequency adjustment ratio and optimal speech rate adjustment ratio that restore the historical vocal features to normal are extracted as target output labels. A multilayer perceptron network is trained using the backpropagation algorithm to construct a nonlinear mapping model. The current driver Lombard intensity index is input into the trained nonlinear mapping model for forward propagation calculation, and the predicted fundamental frequency adjustment coefficient and speech rate adjustment coefficient are output. The product of the initial fundamental frequency deviation and the fundamental frequency adjustment coefficient is calculated to extract the fundamental frequency compensation difference. The product of the initial speech rate deviation and the speech rate adjustment coefficient is calculated to extract the speech rate compensation difference. The fundamental frequency compensation difference and the speech rate compensation difference are combined to generate an environmentally adaptive vocal compensation difference. In silent mode, specific historical stable speech samples of the driver, stored in the vehicle's local audio database, are retrieved. The fundamental frequency values ​​of ten consecutive audio segments are averaged and extracted as a noise-free fundamental frequency reference parameter. The ratio of the duration of the vocal segment to the duration of the silent segment in the same batch of audio segments is averaged and extracted as a noise-free speech rate reference parameter. Initial fundamental frequency deviation = driver's vocal fundamental frequency data - noise-free fundamental frequency reference parameter. Initial speech rate deviation = driver's speech rate feature data - noise-free speech rate reference parameter. Multiple sets of historical speech samples from 100 noisy environments are retrieved from the local database. The recorded historical Lombard intensity index is extracted as input feature sample data. Simultaneously, the fundamental frequency adjustment parameter and speech rate adjustment parameter that restore the noisy speech recognition rate to the silent state are identified through algorithm comparison. The optimal fundamental frequency adjustment ratio and optimal speech rate adjustment ratio that restore the historical speech characteristics to normal are extracted as target output label data. Samples with missing values ​​are cleaned and removed. A multilayer perceptron network was constructed using the PyTorch deep learning framework in the Python software environment. This network structure consists of one input layer, two hidden layers, and one output layer. The input layer has one neuron, the first hidden layer has 32 neurons, the second hidden layer has 16 neurons, and the output layer has two neurons. Adjacent neurons are connected via fully connected logic. Input feature sample data and target output label data are fed into the network. The first and second hidden layers use the ReLU activation function to perform linear rectification calculations, zeroing out negative values ​​and retaining positive values. The output layer uses the Sigmoid activation function to converge the results to the 0-1 interval. By calculating the mean squared error loss function between the predicted output value and the target output label data, the backpropagation algorithm is iteratively executed to update the weights and bias parameters of the fully connected neurons in each layer, completing the training process of the nonlinear mapping model. The current driver Lombard intensity index is input into the input layer of the trained nonlinear mapping model for forward propagation calculations using matrix multiplication and addition. The output layer nodes output the predicted fundamental frequency adjustment coefficient and speech rate adjustment coefficient. The fundamental frequency compensation difference = initial fundamental frequency deviation. Fundamental frequency adjustment coefficient. Speech rate compensation difference = Initial speech rate deviation. Speech rate adjustment coefficient. The calculated fundamental frequency compensation difference and speech rate compensation difference are combined as floating-point tuples to generate the environmentally adaptive pronunciation compensation difference. Driver's normal pronunciation fundamental frequency data = Driver's pronunciation fundamental frequency data - Fundamental frequency compensation difference. Driver's normal speech rate characteristic data = Driver's speech rate characteristic data - Speech rate compensation difference.

[0030] S413: Calculate the first-order difference rate of change of the driver's normal pronunciation fundamental frequency data along the time axis to separate the driver's dialect tone feature parameters that characterize tone fluctuations, and assign a first enhancement weight coefficient to obtain enhanced dialect tone feature parameters. Call the driver's normal speech rate feature data, extract the ratio of the relative duration of the pronunciation segments to separate the driver's dialect long vowel feature parameters that characterize the prolongation habit, and assign a second enhancement weight coefficient to obtain enhanced dialect long vowel feature parameters. Combine the enhanced dialect tone feature parameters and the enhanced dialect long vowel feature parameters to establish a dialect enhancement compensation feature set. Read the fundamental frequency data of the driver's normal pronunciation from two adjacent time samples, along with the time interval between them. The first-order difference rate of change is calculated as: (Fundamental frequency data of the driver's normal pronunciation at the current time point - Fundamental frequency data of the driver's normal pronunciation at the previous time point) / the time interval between the two time points. Track and obtain the continuous change curve feature sequence of this first-order difference rate of change along the time axis, and extract and separate the driver's dialect tone feature parameters representing tone fluctuations from it. A fixed first enhancement weight coefficient of 1.5 is set. This is based on the fact that the degree of undulation and tortuosity of the tone contour is the most core difference feature in distinguishing dialect accents from standard Mandarin; therefore, a higher weight multiplier is needed to highlight the significance of this feature variable. Enhanced dialect tone feature parameter = Driver's dialect tone feature parameter. The first enhancement weighting coefficient is used. Driver's normal speech rate characteristic data is retrieved, including the recorded continuous duration of pronunciation segments and the total sampling duration. The proportion of relative duration of pronunciation segments = continuous duration of pronunciation segment / total sampling duration. The proportion of relative duration of pronunciation segments is compared to the 60% threshold. This threshold is set based on the fact that standard Mandarin pronunciation has obvious pauses between words and relatively short pronunciation durations for individual words. If the proportion of pronunciation segments exceeds 60%, it indicates a habit of prolonged vowel sounds that consume time. If the proportion of relative duration of pronunciation segments is greater than 60%, the driver's dialect long vowel feature parameter representing this prolonged sound habit is extracted from this data. If the proportion of relative duration of pronunciation segments is less than or equal to 60%, this parameter is set to the default value of 0. A fixed second enhancement weighting coefficient of 1.2 is set, based on the fact that prolonged long vowel sounds are a secondary auxiliary feature of dialect. Enhanced dialect long vowel feature parameter = driver's dialect long vowel feature parameter. The second enhancement weighting coefficient. The enhanced dialect tone feature parameters and the enhanced dialect long vowel feature parameters are loaded into the same memory queue address in a parallel array to establish a dialect enhancement compensation feature set.

[0031] The specific steps of S5 are as follows: S511: Based on the multi-channel feature fusion weight vector, for the cepstral coefficient of the target speech, the current environmental noise composition data and the driver's Lombard intensity index, combined with the enhanced dialect tone feature parameters and enhanced dialect long vowel feature parameters extracted from the dialect enhancement compensation feature set, numerical multiplication operation is performed to obtain a weighted feature parameter array, and the weighted feature parameter array is spliced ​​and connected along the time axis reference dimension to establish a multi-channel comprehensive feature vector; The algorithm calls upon pre-recorded weight coefficient values ​​from the multi-channel feature fusion weight vector, retrieves parameters such as the target speech cepstral coefficients, current environmental noise composition data, and driver Lombard intensity index, and extracts enhanced dialect tone feature parameters and enhanced dialect long vowel feature parameters from the dialect enhancement compensation feature set's storage queue. In the NumPy library of the Python software environment, the speech weighted feature quantity equals the target speech cepstral coefficients. The corresponding speech content signal weighting parameters, background weighted feature quantity = current ambient noise composition data The corresponding in-vehicle background noise signal allocation ratio parameter, the speech weighted feature quantity = driver Lombard intensity index The corresponding driver pronunciation feature weighting parameter, dialect tone weighted feature quantity = enhanced dialect tone feature parameter With a fixed adjustment ratio constant, the dialect vowel weighted feature quantity = enhanced dialect long vowel feature parameter Fixed adjustment constant. After performing the above numerical product operation, all weighted result parameters are entered into an empty matrix to obtain a weighted feature parameter array. Read the timestamps attached to each parameter in the weighted feature parameter array, and according to the chronological flow along the time axis baseline, concatenate the different categories of weighted parameters at the same time point into a one-dimensional long sequence to establish a multi-channel comprehensive feature vector.

[0032] S512: Obtain a local preset syllable template, call the multi-channel integrated feature vector as input to the acoustic matching feature classifier, calculate the difference variable of the Euclidean distance between the multi-channel integrated feature vector and the internal spatial nodes of the preset syllable template to determine the acoustic feature similarity, extract audio segments in the extreme value range of acoustic feature similarity according to the extreme value search comparison rule, extract the optimal matching pronunciation unit of speech, obtain the preset grammar and vocabulary constraint rules, call the optimal matching pronunciation unit of speech as input to the language decoding program, calculate the word temporal jump cost in combination with the preset grammar and vocabulary constraint rules and filter out conflicting branches, coherently reconstruct text units, and generate the target speech recognition sentence.

[0033] The specific process of generating sentences for target speech recognition is as follows: A word sequence transition probability matrix is ​​constructed based on the word co-occurrence frequency of the local corpus and set as a preset grammatical and lexical constraint rule. The predecessor and successor word nodes of the optimal matching pronunciation unit at adjacent time points are extracted, and the acoustic observation probability corresponding to the successor word node is obtained. The conditional probability value of the jump from the predecessor word node to the successor word node is retrieved from the preset grammatical and lexical constraint rule. The logarithmic decay value corresponding to the acoustic observation probability is calculated, and the negative logarithmic transformation value of the conditional probability value is added to the logarithmic decay value to generate the word temporal jump cost. The mean and standard deviation of the total cost data of the historical correct decoding path are extracted, and the sum of the mean and standard deviation is calculated to construct a dynamic pruning baseline value. The cumulative cost sum of all word temporal jump costs within the current decoding path branch is calculated. Decoding path branches with a cumulative cost sum greater than the dynamic pruning baseline value are identified as conflicting branches and removed. The globally optimal path with the smallest cumulative cost sum is extracted from the remaining decoding path branches. The word nodes mapped within the globally optimal path are concatenated sequentially to coherently reconstruct text units and generate the target speech recognition sentence. Retrieve pre-set syllable templates from the local database. This template data was generated before the device left the factory by collecting and extracting multi-dimensional feature parameters from standard pronunciation audio of over a thousand volunteers with different dialect accents, segmenting them into independent syllable fragments, and storing them in the vehicle's SQLite database. The multi-channel integrated feature vector is then input into an acoustic matching feature classifier algorithm running in the Python software environment. The algorithm reads the multi-channel integrated feature vector and the spatial node feature parameters within the pre-set syllable template. Euclidean distance difference variable = Acoustic feature similarity = 1 / Euclidean distance difference variable. Based on the extreme value search comparison rule, the numerical distribution of acoustic feature similarity is retrieved across the entire time series. The corresponding continuous time periods where the acoustic feature similarity value falls within the extreme value interval of 0.9 are identified. The original audio segments within this extreme value interval are extracted as the optimal speech matching pronunciation units. The frequency data of each Chinese character word and its consecutive occurrences in the local historical in-vehicle human-machine interaction dialogue corpus are statistically analyzed. The word sequence transition probability = the combination frequency of two consecutive occurrences of words / the total frequency of the preceding word occurring independently. A word sequence transition probability matrix is ​​constructed based on this frequency and set as a preset grammatical vocabulary constraint rule. The preceding and succeeding word node character texts mapped to the optimal speech matching pronunciation units at adjacent time nodes are extracted, and the acoustic matching feature classifier is simultaneously called to obtain the acoustic observation probability corresponding to the succeeding word node. The conditional probability value of the jump from the preceding word node to the succeeding word node is retrieved from the matrix table recorded by the preset grammatical vocabulary constraint rule. Negative logarithmic transformation value = - (Conditional probability value). Calculate the logarithmic decay value corresponding to the acoustic observation probability. Lexical temporal jump cost = negative logarithmic transformation value + logarithmic decay value, that is, add the negative logarithmic transformation value and the logarithmic decay value to generate the lexical temporal jump cost. Extract the sample sequence of the total historical correct decoding path cost data stored locally, with the mean of the distribution = (Total historical cost data) / Number of historical cost samples, and calculate the standard deviation of the samples. Dynamic pruning baseline value = Distribution mean + Standard deviation. Cumulative cost sum within the current decoding path branch = (Cost of all word transitions under the current branch path). Determine the relationship between the accumulated cost sum and the dynamic pruning baseline. If the accumulated cost sum is greater than the dynamic pruning baseline, the corresponding decoding path branch is identified as a conflicting branch with excessive cost and directly removed from the memory stack. If the accumulated cost sum is less than or equal to the dynamic pruning baseline, the decoding path branch is retained for the next step of selection. Compare the absolute values ​​of the accumulated cost sums among all remaining candidate decoding path branches, and extract the complete path with the smallest accumulated cost sum as the globally optimal path. Consistently reconstruct the text units by combining the Chinese character word nodes mapped and parsed within the globally optimal path according to the chronological flow of the text, and finally output the target speech recognition sentence on the interactive interface.

[0034] Please see Figure 2 An AI-powered intelligent voice device speech recognition system is provided to implement the aforementioned AI-powered intelligent voice device speech recognition method. The system includes: The sound source analysis module collects the driver's simulated pronunciation signal, converts and separates the simulated pronunciation signal to obtain a multi-channel separated acoustic signal set containing speech content signal, in-vehicle background noise signal and driver pronunciation feature signal; The speech representation module extracts the speech Mel-frequency cepstral coefficients, driver's fundamental frequency data, driver's speech rate feature data, driver's speaking volume data, and driver's spectral tilt data from the multi-channel separated acoustic signal set. The driving noise perception module obtains the current environmental noise composition data, real-time in-vehicle noise intensity data, and driver Lombard intensity index based on the driver's voice fundamental frequency data, speech volume data, spectrum tilt data, and in-vehicle background noise signal, combined with the vehicle's real-time speed data and engine's real-time RPM data. The scene compensation module determines the in-vehicle acoustic scene association state based on the driver's Lombard intensity index, current environmental noise composition data, and real-time in-vehicle noise intensity data, and obtains the cepstral coefficients of the target speech and the dialect enhancement compensation feature set. The recognition and recombination module decodes and generates target speech recognition sentences based on the cepstral coefficients of the target speech, the current environmental noise composition data, the driver's Lombard intensity index, and the dialect enhancement compensation feature set.

[0035] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A speech recognition method for an AI intelligent voice device, characterized in that, Includes the following steps: S1: Collect the driver's simulated speech signal, convert and separate the simulated speech signal to obtain a multi-channel separated acoustic signal set containing speech content signal, in-vehicle background noise signal and driver's speech feature signal; S2: Based on the multi-channel separated acoustic signal set, extract the speech Mel frequency cepstral coefficients, driver's pronunciation fundamental frequency data, driver's speech rate characteristic data, driver's speaking volume data, and driver's spectral tilt data; S3: Based on the driver's fundamental frequency data, speaking volume data, spectrum tilt data, and in-vehicle background noise signal, combined with the vehicle's real-time speed data and the engine's real-time rotational speed data, the current environmental noise composition data, in-vehicle real-time noise intensity data, and the driver's Lombard intensity index are obtained. S4: Based on the driver's Lombard intensity index, the current environmental noise composition data, and the real-time noise intensity data inside the vehicle, determine the acoustic scene association state inside the vehicle and obtain the cepstral coefficients of the target speech and the dialect enhancement compensation feature set; S5: Based on the cepstral coefficients of the target speech, the current environmental noise composition data, the driver's Lombard intensity index, and the dialect enhancement compensation feature set, decode and generate the target speech recognition sentence.

2. The AI ​​intelligent voice device speech recognition method according to claim 1, characterized in that, The specific steps of S1 are as follows: S111: While the vehicle is in motion, the driver's voice simulation signal is collected in real time through the in-vehicle miniature digital microphone array. The analog-to-digital conversion program is called to perform continuous waveform sampling operation to extract discrete time signal sequence. Based on the device's preset quantization order, the discrete time signal sequence is subjected to discrete level amplitude quantization processing to extract digital quantization signal features. The digital quantization signal features are combined and encoded to obtain the original mixed sound digital signal stream. S112: Input the original mixed-sound digital signal stream into the digital signal processor, perform frequency band component isolation and screening operation on the original mixed-sound digital signal stream according to the multi-path frequency band decomposition rules built into the digital signal processor, extract independent speech content signals and in-vehicle background noise signals, and compare the acoustic energy distribution change characteristics of the original mixed-sound digital signal stream and the in-vehicle background noise signals to separate and extract the driver's pronunciation feature signals. S113: Perform a timestamp alignment operation under a reference clock on the voice content signal, the in-vehicle background noise signal and the driver's pronunciation feature signal, and perform matrix tensor dimension splicing and encapsulation encoding on the multi-dimensional channel feature data extracted in the timestamp alignment operation stage to establish a multi-channel separated acoustic signal set.

3. The AI ​​intelligent voice device speech recognition method according to claim 1, characterized in that, The specific steps of S2 are as follows: S211: Call the speech content signal from the multi-channel separated acoustic signal set, perform a short-time Fourier transform operation on the speech content signal to extract different frequency components, input the different frequency components into a preset frequency domain filter bank for frequency band energy mapping processing, calculate the discrete cosine transform values ​​of the different frequency components, and generate speech Mel frequency cepstral coefficients. S212: Based on the speech content signal, perform time-domain waveform scanning to extract continuous repeating bands, regard the continuous repeating bands as periodic feature objects of the YIN fundamental frequency estimation algorithm, call the YIN fundamental frequency estimation algorithm to calculate the band repetition frequency of the periodic feature objects, obtain the driver's pronunciation fundamental frequency data, extract the pronunciation segment duration and silence segment duration of the speech content signal, calculate the ratio of the pronunciation segment duration to the silence segment duration, and generate driver speech speed feature data; S213: Extract the driver's vocal feature signal from the multi-channel separated acoustic signal set, calculate the short-time energy mean of the driver's vocal feature signal to generate driver's speaking volume data, extract low-frequency energy data and high-frequency energy data from the driver's vocal feature signal according to the preset frequency band interval rule, calculate the quotient between the low-frequency energy data and the high-frequency energy data, and generate driver's spectral tilt data.

4. The AI ​​intelligent voice device speech recognition method according to claim 1, characterized in that, The specific steps of S3 are as follows: S311: Acquire real-time vehicle speed data and real-time engine speed data; perform short-time Fourier transform on the in-vehicle background noise signal to extract amplitude values ​​of different frequency bands to generate in-vehicle noise spectrum distribution data; calculate the root mean square amplitude of the in-vehicle background noise signal per unit time to generate in-vehicle acoustic energy data; based on the real-time vehicle speed data and the real-time engine speed data, filter high-frequency wind noise frequency band energy data and low-frequency engine roar frequency band energy data within the in-vehicle noise spectrum distribution data, combine them to generate current environmental noise composition data; calculate the differences between the driver's speaking volume data, the driver's fundamental frequency data, and the driver's spectrum tilt data at the current time node and the previous time node respectively, to obtain the driver's volume change deviation, the fundamental frequency change deviation, and the spectrum tilt change deviation. S312: Calculate the real-time noise intensity data inside the vehicle based on the in-vehicle acoustic energy data, call the extreme value normalization mapping function to map and transform the real-time noise intensity data inside the vehicle, the driver's volume change deviation, the fundamental frequency change deviation, and the spectral tilt change deviation to the reference interval, and obtain the normalized mapping values ​​of the in-vehicle noise intensity, the driver's volume change, the fundamental frequency change, and the spectral tilt change. Calculate the weighted combination difference of the normalized mapping values ​​of the in-vehicle noise intensity, the driver's volume change, the fundamental frequency change, and the spectral tilt change according to the preset weight configuration items, and generate the driver's Lombard intensity index.

5. The AI ​​intelligent voice device speech recognition method according to claim 4, characterized in that, The process of calculating the real-time noise intensity data inside the vehicle based on the in-vehicle acoustic energy data is as follows: An energy smoothing sliding window is constructed according to a predetermined collection duration. The energy smoothing sliding window is used to extract the in-vehicle acoustic energy data at the current time node to generate a local time period energy sample sequence. The minimum average pickup energy recorded during the silent calibration phase is set as the decibel conversion reference base; the amplitude characteristic values ​​of multiple continuous acoustic sampling points within the local time period energy sample sequence are extracted; The square mean of the amplitude characteristic values ​​is calculated to extract the real-time root mean square energy parameter; the energy ratio quotient between the real-time root mean square energy parameter and the decibel conversion reference base is calculated; the logarithmic conversion program is called to perform logarithmic calculation processing on the energy ratio quotient to extract the initial logarithmic intensity decibel data; The sensitivity attenuation calibration value inside the hardware configuration specification file is read to generate a sound pressure level conversion multiplier; the initial logarithmic intensity decibel data and the sound pressure level conversion multiplier are multiplied to generate real-time noise intensity data inside the vehicle.

6. The AI ​​intelligent voice device speech recognition method according to claim 1, characterized in that, The specific steps of S4 are as follows: S411: Combine the driver's Lombard intensity index, current environmental noise composition data, and real-time in-vehicle noise intensity data into an environmental acoustic feature vector. Input the environmental acoustic feature vector into a preset scene clustering model to calculate the Euclidean distance with each reference scene node. Output the scene category with the smallest Euclidean distance and determine it as the in-vehicle acoustic scene association state. Generate a multi-channel feature fusion weight vector based on the weight distribution of the in-vehicle acoustic scene association state as speech content signal, in-vehicle background noise signal, and driver's pronunciation feature signal. Determine the high-frequency attenuation coefficient based on the multi-channel feature fusion weight vector. Use the high-frequency attenuation coefficient to attenuate the high-frequency coefficient value in the speech Mel-frequency cepstral coefficient to obtain the target speech cepstral coefficient. S412: For the driver's pronunciation fundamental frequency data and driver's speech rate characteristic data, calculate the initial deviation amount relative to the reference pronunciation data in silent mode, and perform adaptive weighting adjustment on the initial deviation amount according to the driver's Lombard intensity index to obtain the environmental adaptive pronunciation compensation difference. Subtract the environmental adaptive pronunciation compensation difference from the driver's pronunciation fundamental frequency data and the driver's speech rate characteristic data to obtain the driver's normal pronunciation fundamental frequency data and the driver's normal speech rate characteristic data. S413: Calculate the first-order difference rate of change of the driver's normal pronunciation fundamental frequency data along the time axis to separate the driver's dialect tone feature parameters that characterize tone fluctuations, and assign a first enhancement weight coefficient to obtain enhanced dialect tone feature parameters. Call the driver's normal speech rate feature data, extract the ratio of the relative duration of the pronunciation segments to separate the driver's dialect long vowel feature parameters that characterize the habit of prolonging sounds, and assign a second enhancement weight coefficient to obtain enhanced dialect long vowel feature parameters. Combine the enhanced dialect tone feature parameters and the enhanced dialect long vowel feature parameters to establish a dialect enhancement compensation feature set.

7. The AI ​​intelligent voice device speech recognition method according to claim 6, characterized in that, The process for obtaining the environmental adaptive pronunciation compensation difference is as follows: In silent mode, the driver's voice samples were analyzed to extract noise-free fundamental frequency reference parameters and noise-free speech rate reference parameters; The initial fundamental frequency deviation is obtained by calculating the difference between the driver's vocal fundamental frequency data and the noise-free fundamental frequency reference parameter; the initial speech rate deviation is obtained by calculating the difference between the driver's speech rate feature data and the noise-free speech rate reference parameter; historical Lombard intensity indices corresponding to multiple sets of historical vocal samples are extracted as input feature samples, and the optimal fundamental frequency adjustment ratio and optimal speech rate adjustment ratio that restore the historical vocal features to normal are extracted as target output labels. A multilayer perceptron network is trained using the backpropagation algorithm to construct a nonlinear mapping model; the current driver Lombard intensity index is input into the trained nonlinear mapping model for forward propagation calculation, and the predicted fundamental frequency adjustment coefficient and speech rate adjustment coefficient are output. The fundamental frequency compensation difference is extracted by calculating the product of the initial fundamental frequency deviation and the fundamental frequency adjustment coefficient; The speech rate compensation difference is extracted by calculating the product of the initial speech rate deviation and the speech rate adjustment coefficient. The fundamental frequency compensation difference and the speech rate compensation difference are combined to generate an environmentally adaptive pronunciation compensation difference.

8. The AI ​​intelligent voice device speech recognition method according to claim 7, characterized in that, The specific steps of S5 are as follows: S511: Based on the multi-channel feature fusion weight vector, for the target speech cepstral coefficient, the current environmental noise composition data, and the driver's Lombard intensity index, combined with the enhanced dialect tone feature parameters and enhanced dialect long vowel feature parameters extracted from the dialect enhancement compensation feature set, a numerical product operation is performed to obtain a weighted feature parameter array. The weighted feature parameter array is then spliced ​​and connected along the time axis reference dimension to establish a multi-channel comprehensive feature vector. S512: Obtain a local preset syllable template, call the multi-channel integrated feature vector into the acoustic matching feature classifier, calculate the Euclidean distance difference variable between the multi-channel integrated feature vector and the spatial nodes inside the preset syllable template, determine the acoustic feature similarity, extract audio segments in the extreme value range of acoustic feature similarity according to the extreme value search comparison rule, extract the optimal matching pronunciation unit, obtain the preset grammar and vocabulary constraint rules, call the optimal matching pronunciation unit into the language decoding program, calculate the word temporal jump cost in combination with the preset grammar and vocabulary constraint rules and filter out conflicting branches, coherently reconstruct the text unit, and generate the target speech recognition sentence.

9. The AI ​​intelligent voice device speech recognition method according to claim 8, characterized in that, The specific process for generating the target speech recognition sentence is as follows: A word sequence transition probability matrix is ​​constructed based on the word co-occurrence frequency of the local corpus and set as the preset grammatical and lexical constraint rule; the predecessor and successor word nodes of the optimal matching pronunciation unit of the speech are extracted at adjacent time nodes, and the acoustic observation probability corresponding to the successor word node is obtained; the conditional probability value of the jump from the predecessor word node to the successor word node is retrieved in the preset grammatical and lexical constraint rule. Calculate the logarithmic decay value corresponding to the acoustic observation probability, and add the negative logarithmic transformation value of the conditional probability value to the logarithmic decay value to generate the word temporal jump cost; Extract the mean and standard deviation of the total cost data of historical correct decoding paths, and calculate the sum of the mean and standard deviation to construct a dynamic pruning baseline. Calculate the cumulative cost sum of all word temporal jump costs within the current decoding path branch. Decoding path branches with cumulative cost sums greater than the dynamic pruning baseline are identified as conflicting branches and removed. Extract the globally optimal path with the smallest cumulative cost sum from the remaining decoding path branches. Concatenate the word nodes mapped within the globally optimal path in temporal order to coherently reconstruct text units and generate the target speech recognition sentence.

10. A voice recognition system for an AI intelligent voice device, characterized in that, The system is used to implement the AI ​​intelligent voice device speech recognition method as described in any one of claims 1-9, the system comprising: The sound source analysis module collects the driver's simulated pronunciation signal, converts and separates the simulated pronunciation signal to obtain a multi-channel separated acoustic signal set containing speech content signal, in-vehicle background noise signal and driver pronunciation feature signal; The speech representation module extracts the speech Mel-frequency cepstral coefficients, driver's fundamental frequency data, driver's speech rate feature data, driver's speaking volume data, and driver's spectral tilt data based on the multi-channel separated acoustic signal set. The driving noise perception module obtains the current environmental noise composition data, real-time in-vehicle noise intensity data, and driver Lombard intensity index based on the driver's voice fundamental frequency data, speech volume data, spectrum tilt data, and in-vehicle background noise signal, combined with the vehicle's real-time speed data and engine's real-time rotation speed data. The scene compensation module determines the in-vehicle acoustic scene association state based on the driver's Lombard intensity index, the current environmental noise composition data, and the real-time noise intensity data inside the vehicle, and obtains the cepstral coefficients of the target speech and the dialect enhancement compensation feature set. The recognition and recombination module decodes and generates the target speech recognition sentence based on the cepstral coefficients of the target speech, the current environmental noise composition data, the driver's Lombard intensity index, and the dialect enhancement compensation feature set.

Images