A Dynamic Audio Event Online Recognition Method Based on Federated Incremental Learning
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-27
- Publication Date
- 2026-08-14
AI Technical Summary
[0003]然而，在实际复杂行车工况中，现有基于联邦增量学习的动态音频事件在线识别技术，往往更关注音频事件类别识别、语音指令识别或声纹匹配结果本身，而缺乏对短时间、非常态化且具有强声学干扰特征的动态噪声事件进行有效甄别、连续性判断和模型更新保护的机制，尤其是在本地个性化声纹模型进行增量学习、特征写入、样本缓存和联邦学习参数上传过程中，若未能准确识别音频数据是否处于低可信或受污染状态，则容易将异常声学环境下采集到的无效音频样本误作为有效增量学习样本，从而引发声学感知结果与底层模型更新机制之间的冲突
本发明在车辆高速驶入隧道后，通过获取并分析音频数据，评估低频空气动力学轰鸣噪声是否会对作为人类声纹核心特征的共振峰造成持续且稳定的向上掩蔽效应，识别隧道内异常声学环境下动态音频事件，并引入针对本地个性化声纹模型的非对称挂起中断机制，避免在异常声学环境下，采集到无效音频样本误作为有效增量学习样本，导致盲目更新本地个性化学习模型的底层权重参数，造成车辆驶出隧道、回到安静的无隧道路面后本地个性化声纹模型发生了严重的灾难性遗忘和基准漂移，在短时间内根本无法正常识别车主个人声音，进一步地，根据车辆驶入时间、预估穿行时长、挂起触发时间和机制滞后时长生成预警信号，并确定机制解除启动时间点，在车辆驶出隧道前执行针对启动本地个性化声纹模型的非对称挂起中断机制后的后台预解除操作，使本地个性化声纹模型在出隧道后及时恢复至可正常识别车主个人声音状态，避免因系统滞后职场的存在，导致车辆出隧道后出现声纹识别空白期。
Smart Images
Figure CN122575413A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of dynamic audio recognition technology, and more specifically to an online method for dynamic audio event recognition based on federated incremental learning. Background Technology
[0002] With the development of vehicle networking, artificial intelligence, and federated incremental learning technology, dynamic audio event online recognition methods for in-vehicle scenarios are gradually being applied to tasks such as in-vehicle voice interaction, vehicle owner authentication, abnormal acoustic environment monitoring, and local personalized voiceprint model updates. These methods typically collect audio data in real time through in-vehicle microphones, perform online recognition of audio events, voiceprint features, and acoustic environment status locally, and participate the personalized model parameters or feature update results obtained from local incremental learning in the federated learning process to continuously improve the personalized adaptability of in-vehicle audio recognition models while protecting user privacy.
[0003] However, in real-world complex driving conditions, existing online dynamic audio event recognition technologies based on federated incremental learning often focus more on audio event category recognition, voice command recognition, or voiceprint matching results themselves, while lacking mechanisms for effectively identifying, continuously judging, and protecting the model update of short-term, non-normal dynamic noise events with strong acoustic interference characteristics. In particular, during the incremental learning, feature writing, sample caching, and federated learning parameter uploading of local personalized voiceprint models, if it is not accurately identified whether the audio data is in a low-confidence or contaminated state, invalid audio samples collected under abnormal acoustic environments are easily mistaken for valid incremental learning samples, thus causing conflicts between acoustic perception results and the underlying model update mechanism.
[0004] Specifically, when a vehicle passes through a tunnel at a high speed of 80km / h-120km / h, it will generate a strong piston effect due to the compression of the air in front of the tunnel, resulting in a low-frequency aerodynamic boom noise with extremely high sound pressure level (energy is usually concentrated in the range of 20Hz-250Hz). In physical acoustics, the low-frequency aerodynamic boom noise will have a strong upward masking effect on the core features of human voiceprints (such as vowel formants) that are concentrated in the mid-to-high frequency range (energy is usually concentrated in the range of 300Hz-3400Hz). This will cause a large amount of invalid noise to fill the audio collected by the vehicle microphone, resulting in serious distortion of the personal voiceprint features of the vehicle owner to be identified, thus causing a sharp increase in the error rate of the personal voiceprint feature identification of the vehicle owner.
[0005] In this scenario, the local personalized learning model can only perform passive calculations based on audio data filled with a large amount of invalid noise, thus resulting in serious cognitive misjudgments. In order to forcibly eliminate the cognitive error of the car owner's personal voiceprint features caused by low-frequency aerodynamic roaring noise, the underlying local training algorithm blindly updates the underlying weight parameters of the local personalized learning model to adapt to the low-frequency aerodynamic roaring noise. This directly leads to a serious catastrophic forgetting and baseline drift of the local personalized voiceprint model when the vehicle leaves the tunnel and returns to a quiet road surface without tunnels, making it impossible to properly recognize the car owner's personal voice in a short period of time.
[0006] Therefore, this invention provides a dynamic audio event online recognition method based on federated incremental learning. Summary of the Invention
[0007] The purpose of this invention is to provide an online method for dynamic audio event recognition based on federated incremental learning, so as to solve the above-mentioned background problems.
[0008] The objective of this invention can be achieved through the following technical solutions: A dynamic online audio event recognition method based on federated incremental learning includes: When a vehicle enters a tunnel at high speed while driving on the road, the audio data from the vehicle's microphone is acquired and analyzed to assess whether a substantial upward masking effect occurs. When a substantial upward masking effect occurs, the audio frame labeling result corresponding to each windowed audio frame is extracted in real time, and the continuous generation and counting analysis of windowed audio frames in the time dimension is performed to evaluate the stability of the substantial upward masking effect and determine whether to start the asymmetric suspension interruption mechanism for the local personalized voiceprint model. After activating the asymmetric suspension interruption mechanism for the local personalized voiceprint model, the remaining expected time for the vehicle to exit the tunnel is obtained and compared with the mechanism lag time to generate different warning signals. Based on different early warning signals, the mechanism activation time point corresponding to each early warning signal is obtained, and pre-release operations are carried out within the tunnel.
[0009] As a further technical solution of the present invention, the process for evaluating whether a substantial upward masking effect occurs is as follows: After the vehicle enters the tunnel, the continuous raw data stream of PCM time-domain waveform is acquired in real time to obtain the continuously output windowed audio frames. Obtain the cross-frequency energy suppression ratio B1 and formant characteristic ratio B2 corresponding to each windowed audio frame; Under the same windowed audio frame, the corresponding cross-frequency energy suppression ratio B1 is compared with the preset cross-frequency energy suppression threshold, and the corresponding formant feature ratio B2 is compared with the preset formant feature submersion threshold. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is greater than the preset cross-frequency energy suppression threshold and the formant feature ratio B2 is less than the preset formant feature submersion threshold, it is determined that the low-frequency aerodynamic roaring noise has a substantial upward masking effect on the formant, which is the core feature of human voiceprint, and the corresponding windowed audio frame is marked as a masked audio frame.
[0010] As a further technical solution of the present invention, the process of acquiring unmasked audio frames is as follows: Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is greater than the preset cross-frequency energy suppression threshold and the formant feature ratio B2 is greater than or equal to the preset formant feature submersion threshold, the corresponding windowed audio frame will be marked as an unmasked audio frame. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is less than or equal to the preset cross-frequency energy suppression threshold, and the formant feature ratio B2 is less than the preset formant feature submersion threshold, the corresponding windowed audio frame will be marked as an unmasked audio frame. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is less than or equal to the preset cross-frequency energy suppression threshold, and the formant feature ratio B2 is greater than or equal to the preset formant feature submersion threshold, the corresponding windowed audio frame will be marked as an unmasked audio frame.
[0011] As a further technical solution of the present invention, the process for evaluating the stability of the substantial upward masking effect is as follows: Starting from the first windowed audio frame acquired, this process is applied to every subsequent windowed audio frame, including the first windowed audio frame acquired. For each windowed audio frame, the corresponding updated continuous count value is continuously updated, and after each continuous count value is synchronously updated, the corresponding updated continuous count value is compared with the continuous determination frame number threshold. In any windowed audio frame, if the first occurrence of a continuous count value greater than or equal to the duration determination frame threshold, it is determined that the low-frequency aerodynamic roaring noise has a substantial upward masking effect on the resonant peak, which is the core feature of human voiceprint, and generates a steady-state masking signal.
[0012] As a further technical solution of the present invention, the process for evaluating the stability of the substantial upward masking effect is as follows: When a steady-state masking signal is generated, it serves as the basis for triggering the asymmetric suspension interruption mechanism for the local personalized voiceprint model; If the vehicle's passage through the tunnel has reached the maximum duration corresponding to the estimated total travel time interval before the steady-state masking signal is generated, the stability assessment of the substantial upward masking effect will be stopped. At the same time, it will be determined that the triggering conditions of the asymmetric suspension interruption mechanism for the local personalized voiceprint model have not been met throughout the entire passage of the vehicle through the tunnel.
[0013] As a further technical solution of the present invention, the process of generating different early warning signals is as follows: When the vehicle is driving in the tunnel, the first steady-state masking signal is generated and the asymmetric suspension interruption mechanism for the local personalized voiceprint model is activated, the remaining expected duration is obtained. Compare the remaining expected duration with the mechanism lag duration; If the remaining expected duration is greater than or equal to the mechanism lag duration, a sufficient time warning signal will be generated. If the remaining expected duration is less than the mechanism lag duration, an insufficient time warning signal will be generated.
[0014] As a further technical solution of the present invention, the process of obtaining the start time point D1 of the mechanism corresponding to the sufficient time warning signal is as follows: When a sufficient time warning signal is generated, the mechanism lag time is extracted. Based on the expected tunnel exit time, a mechanism lag time is deduced by going back along the historical direction of the time axis to obtain the mechanism deactivation start time point D1 corresponding to the sufficient time warning signal.
[0015] As a further technical solution of the present invention, the pre-release operation within the tunnel corresponding to the ample time warning signal is as follows: When a vehicle is driving in a tunnel, after triggering the asymmetric suspension mechanism for the local personalized voiceprint model, the asymmetric suspension mechanism for the local personalized voiceprint model will continue to be maintained until the corresponding mechanism release start time point D1 is reached. As time progresses, when the corresponding mechanism release start time point D1 is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model will be initiated in the background. When the expected tunnel exit time point is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model will be completed, and the local personalized voiceprint model will be restored from the suspended protection state to the state where the vehicle owner's personal voiceprint characteristics can be identified.
[0016] As a further technical solution of the present invention, the process of obtaining the start time point D2 after the mechanism for responding to the insufficient time warning signal is as follows: When a time-insufficient warning signal is generated, the corresponding mechanism lag time and the remaining expected time are extracted, and the difference between the mechanism lag time and the remaining expected time is calculated to obtain the corresponding warning gap time. Extract the mechanism trigger time point corresponding to the vehicle. Based on the corresponding mechanism trigger time point, backtrack along the historical direction of the time axis by a warning gap duration to obtain the mechanism deactivation start time point D2 corresponding to the insufficient time warning signal.
[0017] As a further technical solution of the present invention, the pre-release operation within the tunnel corresponding to the insufficient time warning signal is as follows: From the moment the vehicle enters the tunnel, as time progresses, when the corresponding mechanism release activation time point D2 is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model is initiated in the background. When the corresponding mechanism trigger time point for the vehicle is reached, the asymmetric suspension interruption mechanism for the local personalized voiceprint model is activated. When the expected tunnel exit time point is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model is completed, and the local personalized voiceprint model is restored from the suspended protection state to the state where the owner's personal voiceprint characteristics can be identified.
[0018] The beneficial effects of this invention are as follows: This invention, after a vehicle enters a tunnel at high speed, acquires and analyzes audio data to assess whether low-frequency aerodynamic booming noise will cause a continuous and stable upward masking effect on the formants, which are core features of human voiceprints. It identifies dynamic audio events under abnormal acoustic conditions within the tunnel and introduces an asymmetric suspension and interruption mechanism for the local personalized voiceprint model. This prevents invalid audio samples from being mistakenly used as valid incremental learning samples under abnormal acoustic conditions, thus avoiding blindly updating the underlying weight parameters of the local personalized learning model. This prevents the local personalized voiceprint model from failing to update after the vehicle exits the tunnel and returns to a quiet, tunnel-free surface. The system experienced severe catastrophic amnesia and baseline drift, making it impossible to properly recognize the vehicle owner's voice in a short period of time. To address this, a warning signal is generated based on the vehicle's entry time, estimated travel time, suspension trigger time, and mechanism lag time. The system also determines the mechanism release activation time and performs a background pre-release operation before the vehicle exits the tunnel, targeting the asymmetric suspension interruption mechanism that activates the local personalized voiceprint model. This ensures that the local personalized voiceprint model recovers to a state where it can properly recognize the vehicle owner's voice after exiting the tunnel, preventing a voiceprint recognition gap after the vehicle exits the tunnel due to system lag. Attached Figure Description
[0019] The invention will now be further described with reference to the accompanying drawings.
[0020] Figure 1 This is a flowchart of a dynamic audio event online recognition method based on federated incremental learning, as described in an embodiment of the present invention. Figure 2 This is a logic judgment diagram of the substantial upward masking effect of a dynamic audio event online recognition method based on federated incremental learning, as described in an embodiment of the present invention. Detailed Implementation
[0021] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.
[0022] Example 1
[0023] Please see Figures 1-2 As shown in the embodiment of the present invention, a dynamic audio event online recognition method based on federated incremental learning is provided. The present invention mainly includes the following steps: Step 1: When the vehicle enters the tunnel at high speed while driving on the road, acquire audio data from the vehicle microphone, analyze the audio data, and assess whether a substantial upward masking effect occurs. In step one, the process of assessing whether a substantial upward masking effect occurs is as follows: It should be noted that since low-frequency aerodynamic roaring noise is usually concentrated in the 20Hz-250Hz frequency band, and the resonance peaks, which are the core characteristics of human voiceprints, are usually concentrated in the 300Hz-3400Hz frequency band, this solution mainly focuses on the analysis of the 20Hz-250Hz and 300Hz-3400Hz frequency bands. In some embodiments, when a vehicle enters a tunnel at a high speed of 80-100 km / h, the vehicle cockpit domain controller acquires a continuous PCM time-domain waveform raw data stream from the vehicle microphone in real time at a preset audio sampling frequency. It should be noted that the preset audio sampling frequency is based on the Nyquist-Shannon sampling theorem. In order to fully preserve the highest frequency of 3400Hz, which is the core feature of human voiceprint, the preset audio sampling frequency is set to at least 6800Hz. When the raw PCM time-domain waveform data stream is acquired, an audio data buffer queue corresponding to the preset time slice length is constructed synchronously, and the audio data buffer queue is used to buffer and package the continuous raw PCM time-domain waveform data stream acquired in real time. It should be noted that: the preset time slice length represents the unit time window length for performing a data truncation operation on the continuous PCM time domain waveform raw data stream to obtain an initial audio frame; Specifically, when the raw PCM time-domain waveform data stream entering the audio data buffer queue reaches the maximum capacity corresponding to the preset time slice, a data truncation action is triggered, packaging all the raw PCM time-domain waveform data streams in the audio data buffer queue into an initial audio frame, and simultaneously clearing the audio data buffer queue to receive subsequent raw PCM time-domain waveform data streams. This process is repeated, and multiple initial audio frames are continuously output on the time axis. It should be noted that the maximum buffer capacity of the audio data buffer queue is determined based on the preset audio sampling frequency, preset time slice length, sampling bit width, and number of channels. For multiple consecutively output initial audio frames, a preset window function is applied to each initial audio frame to perform edge smoothing and convergence processing, resulting in the windowed audio frame corresponding to each initial audio frame. The windowed audio frame is input to the digital signal processing module in the vehicle cockpit domain controller, and a fast Fourier transform with a preset number of points is performed to convert the audio data in the time domain into frequency domain spectrum data. The amplitude square of each discrete frequency point in the frequency domain spectrum data is calculated to obtain the power energy value corresponding to each discrete frequency point in the full frequency band for each windowed audio frame. Extract all corresponding discrete frequency points in the 20Hz-250Hz range for each windowed audio frame from the full frequency band as low frequency points, and extract the power energy value corresponding to each low frequency point as low frequency power energy value; The total low-frequency energy value is obtained by summing all low-frequency power energy values under the same windowed audio frame. Meanwhile, all corresponding discrete frequency points in the 300Hz-3400Hz range of each windowed audio frame are extracted from the full frequency band as mid-high frequency points, and the power energy value corresponding to each mid-high frequency point is extracted as the mid-high frequency power energy value. The total mid-to-high frequency power energy value is obtained by summing all mid-to-high frequency power energy values under the same windowed audio frame. The ratio of the total low-frequency energy to the total mid-to-high-frequency energy under the same windowed audio frame is calculated to obtain the cross-frequency energy suppression ratio B1 corresponding to a single windowed audio frame. Furthermore, all mid-to-high frequency power energy values corresponding to the same windowed audio frame are extracted and averaged to obtain the average mid-to-high frequency power energy value corresponding to a single windowed audio frame. Extract the maximum power energy value from all mid-to-high frequency power energy values corresponding to the same windowed audio frame, and calculate the ratio with the corresponding mid-to-high frequency power energy average to obtain the formant feature ratio B2 corresponding to a single windowed audio frame. It should be noted that the cross-frequency energy suppression ratio B1 is used to quantify the degree of acoustic energy dominance of low-frequency aerodynamic roar noise in the 20Hz-250Hz frequency band relative to the formant peak, which is the core feature of human voiceprint, in the 300Hz-3400Hz frequency band. The larger the cross-frequency energy suppression ratio B1, the higher the degree of acoustic energy dominance of low-frequency aerodynamic roar noise relative to the formant peak, which is the core feature of human voiceprint. The formant feature ratio B2 is used to quantify the prominence of the formant, which is a core feature of human voiceprints, relative to the average acoustic energy in the 300Hz-3400Hz frequency band. The larger the formant feature ratio B2, the more prominent the peak value of the formant, which is a core feature of human voiceprints, is relative to the average acoustic energy in the 300Hz-3400Hz frequency band, and the more obvious the peak-valley structure of the formant. The smaller the formant feature ratio B2, the less prominent the peak value of the formant, which is a core feature of human voiceprints, is relative to the average acoustic energy in the 300Hz-3400Hz frequency band, and the flatter the spectral envelope in the 300Hz-3400Hz frequency band. This means that the formant, which is a core feature of human voiceprints, has degenerated from a staggered peak-valley state to a flat state. It should be noted that the preset cross-frequency energy suppression threshold represents the minimum acoustic energy ratio boundary required to suppress the low-frequency aerodynamic roar noise relative to the frequency band where the resonance peak, which is the core feature of human voiceprint, is concentrated. For example, through actual vehicle tunnel testing (collecting artificial head data in tunnels of different lengths at speeds of 80-100 km / h and subjectively evaluating the degree of masking), the preset cross-frequency energy suppression threshold can be set to 2.5, and the adjustable range for different vehicle models is 2.2-3.0. The preset formant feature submergence threshold represents the minimum proportion boundary at which the peak value of the formant, which is the core feature of human voiceprint in the 300Hz-3400Hz frequency band, can still be considered to have effective prominence relative to the average energy value in the 300Hz-3400Hz frequency band. For example, based on the statistics of car owner voice samples in a noise-free environment (mean 4.2, standard deviation 0.8), the preset formant feature submergence threshold can be set to 2.0, and can be adjusted by ±0.3. Under the same windowed audio frame, the corresponding cross-frequency energy suppression ratio B1 is compared with the preset cross-frequency energy suppression threshold, and the corresponding formant feature ratio B2 is compared with the preset formant feature submersion threshold. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is greater than the preset cross-frequency energy suppression threshold and the formant feature ratio B2 is less than the preset formant feature submersion threshold, it indicates that the low-frequency acoustic energy in the 20Hz-250Hz band has suppressed the mid-to-high frequency acoustic energy in the 300Hz-3400Hz band. At the same time, the formant peak prominence of the core feature of human voiceprint in the 300Hz-3400Hz band is insufficient, and the spectral envelope has degenerated from a peak-valley structure to a flat structure. Based on this, it is determined that the low-frequency aerodynamic roaring noise has a substantial upward masking effect on the formant, which is the core feature of human voiceprint. The corresponding windowed audio frame is marked as a masked audio frame. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is greater than the preset cross-frequency energy suppression threshold, and the formant feature ratio B2 is greater than or equal to the preset formant feature submersion threshold, it indicates that the low-frequency acoustic energy in the 20Hz-250Hz band has suppressed the mid-to-high frequency acoustic energy in the 300Hz-3400Hz band. However, the formant peaks, which are the core features of human voiceprints, are sufficiently prominent in the 300Hz-3400Hz band, and the spectral envelope has not degenerated from a peak-valley structure to a flat structure. Based on this, it is determined that the low-frequency aerodynamic roaring noise has not formed a substantial upward masking effect on the formant peaks, which are the core features of human voiceprints. The corresponding windowed audio frame is marked as an unmasked audio frame. If, under the same windowed audio frame, the cross-frequency energy suppression ratio B1 is less than or equal to the preset cross-frequency energy suppression threshold, and the formant feature ratio B2 is less than the preset formant feature submersion threshold, it indicates that the low-frequency acoustic energy in the 20Hz-250Hz band has not yet formed an energy suppression relative to the mid-high frequency acoustic energy in the 300Hz-3400Hz band, and the formant peak prominence of the core feature of human voiceprint in the 300Hz-3400Hz band is insufficient, and the spectral envelope has degenerated from a peak-valley structure to a flat structure. Based on this, it is determined that the low-frequency aerodynamic roaring noise has not formed a substantial upward masking effect on the formant, which is the core feature of human voiceprint, and the corresponding windowed audio frame is marked as an unmasked audio frame. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is less than or equal to the preset cross-frequency energy suppression threshold, and the formant feature ratio B2 is greater than or equal to the preset formant feature submersion threshold, it indicates that the low-frequency acoustic energy in the 20Hz-250Hz band has not yet formed an energy suppression relative to the mid-high frequency acoustic energy in the 300Hz-3400Hz band, and the formant peaks in the 300Hz-3400Hz band, which are the core features of human voiceprints, are sufficiently prominent, and the spectral envelope has not degenerated from a peak-valley structure to a flat structure. Based on this, it is determined that the low-frequency aerodynamic roaring noise has not formed a substantial upward masking effect on the formant peaks, which are the core features of human voiceprints, and the corresponding windowed audio frame is marked as an unmasked audio frame. It should be noted that when a vehicle enters the tunnel at a speed of 80-100 km / h, that is, when the on-board sensors detect that the front of the vehicle has physically entered the tunnel entrance boundary, the time point when the front of the vehicle physically enters the tunnel entrance boundary is recorded as the entry time point. Simultaneously, the entrance and exit boundary positions of the corresponding tunnel are obtained from the vehicle-mounted high-precision map to determine the tunnel travel distance between the tunnel entrance boundary position and the tunnel exit boundary position; Based on the vehicle's speed when entering the tunnel and the tunnel's travel distance, calculate the estimated total travel time range for the vehicle through the tunnel. The minimum value in the estimated total travel time range is the tunnel travel distance divided by the highest possible speed (e.g., 100 km / h), and the maximum value is the tunnel travel distance divided by the lowest possible speed (e.g., 80 km / h). The estimated total travel time range is used to limit the duration of PCM time-domain waveform raw data stream acquisition and initial audio frame continuous output, so that the vehicle cockpit domain controller can acquire audio data from the vehicle microphone in real time during the tunnel travel period after the vehicle enters the tunnel. When the time it takes for a vehicle to pass through a tunnel reaches the maximum time corresponding to the estimated total travel time interval, it is determined that the vehicle has completed the tunnel travel process, and the on-board cockpit domain controller automatically terminates the acquisition of the PCM time domain waveform raw data stream. Understandably, the purpose of step one is to assess whether low-frequency aerodynamic booming noise in the 20Hz-250Hz band under a single windowed audio frame has a substantial upward masking effect on the resonant peaks in the 300Hz-3400Hz band, which are the core features of human voiceprints. Step 2: When a substantial upward masking effect occurs, extract the audio frame labeling result corresponding to each windowed audio frame in real time, and perform continuous generation counting analysis of windowed audio frames in the time dimension to evaluate the stability of the substantial upward masking effect and determine whether to start the asymmetric suspension interruption mechanism for the local personalized voiceprint model. In step two, the process of evaluating the stability of the substantial upward masking effect is as follows: In some embodiments, for continuously acquired windowed audio frames, if a windowed audio frame is marked as a masked audio frame, a continuous counting mechanism for masked audio frames is synchronously constructed. As multiple windowed audio frames are continuously acquired on the timeline, the masked audio frame or non-masked audio frame marking results corresponding to each windowed audio frame are acquired sequentially according to the order of acquisition time. According to the order of acquisition time, starting from the first windowed audio frame acquired, each subsequent windowed audio frame, including the first windowed audio frame acquired, is processed. If the windowed audio frame acquired at the beginning is marked as a masked audio frame, including the first windowed audio frame acquired, then the counting accumulation instruction of the masked audio frame continuous counting mechanism is triggered, that is, the current continuous count value is incremented by one, and a continuous count value is updated synchronously. If the windowed audio frame acquired at the beginning is marked as an unmasked audio frame, including the first windowed audio frame, then the count zero increment instruction of the masked audio frame continuous counting mechanism is triggered, that is, zero is added to the current continuous count value, and a continuous count value is updated synchronously. Among them, the masked audio frame continuous counting mechanism has a continuous count value, and the initial continuous count value is set to zero; Starting from the first windowed audio frame acquired, each time a masked audio frame is generated, a corresponding count accumulation instruction is triggered to perform a continuous count value synchronization update and obtain the corresponding updated continuous count value. Starting from the first windowed audio frame acquired, each time a masked audio frame is generated, a count zero increment instruction is triggered to perform a continuous count value synchronization update and obtain the corresponding updated continuous count value. Furthermore, after each continuous count value is synchronized and updated, the corresponding updated continuous count value is compared with the continuous determination frame threshold. When continuously acquiring windowed audio frames, if the continuous count value is greater than or equal to the continuous judgment frame threshold for the first time under the current updated continuous count value corresponding to any windowed audio frame, it indicates that the substantial upward masking effect of low-frequency aerodynamic roaring noise on the resonance peak, which is the core feature of human voiceprint, not only exists in the time segment corresponding to a single windowed audio frame, but has also been maintained uninterruptedly on the continuous time axis to reach the preset stable observation duration. Based on this, it is determined that the substantial upward masking effect of low-frequency aerodynamic roaring noise on the resonance peak, which is the core feature of human voiceprint, is continuous and stable, and a steady-state masking signal is generated. When a steady-state masking signal is generated, it serves as the basis for triggering the asymmetric suspension interruption mechanism for the local personalized voiceprint model; If the vehicle's passage through the tunnel has reached the maximum duration corresponding to the estimated total passage time interval before the steady-state masking signal is generated, the stability assessment of the substantial upward masking effect will be stopped, and it will be determined that the triggering conditions of the asymmetric suspension interruption mechanism for the local personalized voiceprint model have not been met throughout the entire passage of the vehicle through the tunnel. It should be noted that the continuous frame count threshold characterizes the stable and continuous substantial upward masking effect of low-frequency aerodynamic roaring noise on the resonance peak, which is the core feature of human voiceprint. The minimum number of consecutive windowed audio frames required to meet this threshold is determined based on the preset stable observation duration and the output step size of the windowed audio frames. The preset stable observation duration is used to limit the shortest continuous time required for low-frequency aerodynamic roaring noise to continuously form a substantial upward masking effect. In some embodiments, the preset stable observation duration can be determined by the vehicle cockpit domain controller based on the frame processing cycle of the vehicle voice acquisition system, the continuous change characteristics of low-frequency aerodynamic noise in the vehicle cockpit, the trigger sensitivity requirements of the voiceprint model anti-pollution control, and preset calibration parameters. For example, by analyzing the low-frequency noise fluctuation cycle in the tunnel (measured at 0.5-1.2 seconds), the preset stable observation duration can be set to 1.0 second. The output step size of the windowed audio frame refers to the starting time interval between two adjacent windowed audio frames on the time axis. For example, if the preset time slice length (frame length) is set to 30ms and a 50% overlapping Hamming window is used, the output step size is 15ms. It should be noted that the asymmetric suspension and interruption mechanism for local personalized voiceprint models is an existing and mature control mechanism in the federated learning personalized audio monitoring technology that combines vehicle-mounted voiceprint recognition. It is usually used to suspend or interrupt the personalized update, feature writing, sample caching, or federated learning parameter uploading processes in the local personalized voiceprint model when there is a risk of contamination, low reliability, or abnormal acoustic environment in the locally collected audio. This does not require the simultaneous interruption of basic functions such as vehicle-mounted speech recognition, basic voiceprint matching, or global model inference. Thus, the asymmetric suspension and interruption mechanism for local personalized voiceprint models can prevent contaminated audio samples from participating in the local personalized model update while preserving the necessary continuity of vehicle-mounted voice services, thereby reducing the risk of the local voiceprint model being contaminated by abnormal acoustic environment. It is understandable that the purpose of step two is to: in the process of continuously acquiring windowed audio frames, based on the labeling results of masked or unmasked audio frames corresponding to each windowed audio frame, perform a time-dimensional continuous generation counting analysis of windowed audio frames, and evaluate the stability of the substantial upward masking effect. Step 3: After activating the asymmetric suspension interruption mechanism for the local personalized voiceprint model, obtain the remaining expected time for the vehicle to exit the tunnel, compare and analyze it with the mechanism lag time, and generate different warning signals; In step three, the process of generating different warning signals is as follows: In some implementations, when a vehicle generates a steady-state masking signal for the first time while driving in a tunnel and initiates an asymmetric suspension interruption mechanism for the local personalized voiceprint model, the absolute timestamp corresponding to the initiation of the asymmetric suspension interruption mechanism for the local personalized voiceprint model is recorded as the mechanism trigger time point. Extract the entry time point and estimated total travel time range corresponding to the vehicle; The duration between the trigger time of the acquisition mechanism and the entry time is taken as the continuous driving time of the vehicle in the tunnel. Extract the minimum estimated total travel time from the estimated total travel time range, calculate the difference between it and the already continuously traveled time, and obtain the remaining expected time of the vehicle at the mechanism trigger time point; The remaining expected duration is used to characterize the estimated available driving time for the vehicle to remain inside the tunnel after the asymmetric suspension interruption mechanism for the local personalized voiceprint model is activated. It should be noted that: Releasing the asymmetric suspension / interruption mechanism for the local personalized voiceprint model refers to the control process of restoring the relevant functional branches of the local personalized voiceprint model that were suspended or interrupted within the asymmetric suspension / interruption mechanism after its initiation. The restored functional branches include: writing local personalized voiceprint features, caching vehicle owner personalized voiceprint samples, updating personalized model parameters, verifying personalized voiceprint templates, and restoring the federated learning parameter upload queue. Since the asymmetric suspension / interruption mechanism does not necessarily interrupt basic functions such as in-vehicle speech recognition, basic voiceprint matching, or global model inference under abnormal acoustic environments, releasing the asymmetric suspension / interruption mechanism for the local personalized voiceprint model does not necessarily interrupt the entire... Instead of restarting the in-vehicle voice system, a conditional recovery is performed on the suspended personalized voiceprint learning, writing, updating, and uploading links. The recovery process includes steps such as isolating contaminated audio samples, cleaning abnormal cache samples, unlocking the personalized model state, restoring the feature writing channel, verifying voiceprint recognition confidence, and confirming the validity of the personalized voiceprint template. Only after the recovery steps are completed is the local personalized voiceprint model considered to have re-entered a state that can be used for effective recognition of the vehicle owner's personal voiceprint features. In this technical solution, the asymmetric suspension interruption mechanism for the local personalized voiceprint model is mainly used to characterize the process by which the in-vehicle terminal restores the local personalized voiceprint model from a suspended protection state to a state where it can perform the recognition of the vehicle owner's personal voiceprint features when the vehicle transitions from an abnormal acoustic environment inside the tunnel to a normal acoustic environment outside the tunnel. It should be noted that: the mechanism lag time represents the estimated recovery time required from the start of the release of the asymmetric suspension interruption mechanism for the local personalized voiceprint model until the local personalized voiceprint model recovers to the state where it can effectively recognize the owner's personal voiceprint features. It is pre-calibrated through vehicle factory testing and tunnel scenario simulation testing. Extract the mechanism lag time and compare the remaining expected time with the mechanism lag time; If the remaining expected duration is greater than or equal to the mechanism lag duration, it indicates that after the corresponding mechanism trigger time, the asymmetric suspension interruption mechanism for the local personalized voiceprint model will be released, thereby restoring the local personalized voiceprint model from the suspension protection state to the state where the vehicle owner's personal voiceprint characteristics can be identified. This process can be completed before the vehicle leaves the tunnel or just as the vehicle leaves the tunnel, generating a warning signal with sufficient time. If the remaining expected duration is less than the mechanism lag duration, it indicates that after the corresponding mechanism triggering time, the asymmetric suspension interruption mechanism for the local personalized voiceprint model is released, and the complete process of restoring the local personalized voiceprint model from the suspension protection state to the state where the vehicle owner's personal voiceprint characteristics can be identified is insufficient to be completed before the vehicle leaves the tunnel, thus generating an insufficient time warning signal. Understandably, the purpose of step three is to analyze whether the asymmetric suspension interruption mechanism for the local personalized voiceprint model can be released before the vehicle leaves the tunnel by comparing the remaining expected duration with the mechanism lag duration, thereby restoring the local personalized voiceprint model from the suspension protection state to the state where the owner's personal voiceprint characteristics can be identified, and generating different warning signals accordingly. Step 4: Based on different warning signals, obtain the mechanism release activation time point corresponding to different warning signals, and carry out pre-release operations in the tunnel; In step four, the process of performing the pre-release operation within the tunnel is as follows: It should be noted that, to avoid a gap in the recognition of the vehicle owner's personal voiceprint after the vehicle exits the tunnel due to the mechanism's lag, the following steps are taken: Extract the vehicle's entry time and the estimated total travel time interval, and use the minimum estimated total travel time within the estimated total travel time interval as the conservative estimate of the tunnel travel time. Based on the entry time, a conservative estimate of the time is advanced along the future direction of the time axis to obtain the absolute time when the vehicle is expected to completely exit the tunnel, which is then used as the expected exit time. In some embodiments, when a time-adequate warning signal is generated, the mechanism lag time is extracted, and a mechanism lag time is deduced backward along the historical direction of the time axis based on the expected tunnel exit time, to obtain the mechanism deactivation start time point D1 corresponding to the time-adequate warning signal; Among them, the mechanism release start time point is used to characterize the latest safe time point before the vehicle starts to execute the asymmetric suspension interruption mechanism release operation for the local personalized voiceprint model before it leaves the tunnel, so that the mechanism lag time required to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model can be covered by the vehicle's corresponding remaining expected time. When a vehicle is driving in a tunnel, after triggering the asymmetric suspension mechanism for the local personalized voiceprint model, the asymmetric suspension mechanism for the local personalized voiceprint model will continue to be maintained until the corresponding mechanism release start time point D1 is reached. As time progresses, when the corresponding mechanism release start time point D1 is reached, the operation of releasing the asymmetric suspension interruption mechanism for the local personalized voiceprint model is started in the background. When the expected tunnel exit time point is reached, the operation of releasing the asymmetric suspension interruption mechanism for the local personalized voiceprint model is completed, and the local personalized voiceprint model is restored from the suspension protection state to the state where the vehicle owner's personal voiceprint characteristics can be identified. In some embodiments, when a time-insufficient warning signal is generated, the corresponding mechanism lag time and the remaining expected time are extracted, and the difference between the mechanism lag time and the remaining expected time is calculated to obtain the corresponding warning gap time. Extract the mechanism trigger time point corresponding to the vehicle, and based on the corresponding mechanism trigger time point, backtrack along the historical direction of the time axis by a warning gap duration to obtain the mechanism release start time point D2 corresponding to the insufficient time warning signal; From the moment the vehicle enters the tunnel, as time progresses, when the corresponding mechanism release activation time point D2 is reached, the operation of releasing the asymmetric suspension interruption mechanism for the local personalized voiceprint model is initiated in the background. When the corresponding mechanism trigger time point for the vehicle is reached, the asymmetric suspension interruption mechanism for the local personalized voiceprint model is activated. When the expected tunnel exit time point is reached, the operation of releasing the asymmetric suspension interruption mechanism for the local personalized voiceprint model is completed, and the local personalized voiceprint model is restored from the suspension protection state to the state where the owner's personal voiceprint characteristics can be identified. It should be noted that in this technical solution, the operation of releasing the asymmetric suspension interruption mechanism for the local personalized voiceprint model is a background pre-release operation. The background pre-release operation is not equivalent to immediately re-integrating the local personalized voiceprint model into the vehicle owner's personal voiceprint feature recognition link, nor is it equivalent to immediately restoring the links for writing local personalized voiceprint features, caching vehicle owner's personalized voiceprint samples, updating local personalized model parameters, or uploading federated learning parameters. After the background pre-release operation is completed, the local personalized voiceprint model enters the waiting link state. As time progresses, when the vehicle reaches the corresponding expected tunnel exit time, the local personalized voiceprint model in the waiting link state will be re-integrated into the vehicle owner's personal voiceprint feature recognition link, and the corresponding links for writing local personalized voiceprint features, caching vehicle owner's personalized voiceprint samples, updating personalized model parameters, or uploading federated learning parameters will be restored. Among them, the pending chain state is used to characterize that the local personalized voiceprint model has completed the background recovery preparation required to switch from the suspended protection state to the state where the owner's personal voiceprint features can be identified, but it has not yet been allowed to receive voice data that may be polluted by low-frequency aerodynamic roaring noise in the tunnel, and has not yet officially resumed writing local personalized voiceprint features, caching of owner's personalized voiceprint samples, updating of local personalized model parameters, or uploading of federated learning parameters. Understandably, the purpose of step four is to avoid a gap in the recognition of the vehicle owner's personal voiceprint after the vehicle exits the tunnel due to the existence of mechanism lag time. Based on different warning signals, the corresponding mechanism release start time point is obtained. After the corresponding mechanism release start time point is reached, the background pre-release operation after the corresponding asymmetric suspension interruption mechanism for the local personalized voiceprint model is performed. When the expected tunnel exit time point is reached, the asymmetric suspension interruption mechanism for the local personalized voiceprint model is released, and the local personalized voiceprint model is restored from the suspension protection state to the state where the vehicle owner's personal voiceprint features can be recognized.
[0024] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.
Claims
1. A dynamic audio event online recognition method based on federated incremental learning, characterized in that: Includes the following steps: When a vehicle enters a tunnel at high speed while driving on the road, the audio data from the vehicle's microphone is acquired and analyzed to assess whether a substantial upward masking effect occurs. When a substantial upward masking effect occurs, the audio frame labeling result corresponding to each windowed audio frame is extracted in real time, and the continuous generation and counting analysis of windowed audio frames in the time dimension is performed to evaluate the stability of the substantial upward masking effect and determine whether to start the asymmetric suspension interruption mechanism for the local personalized voiceprint model. After activating the asymmetric suspension interruption mechanism for the local personalized voiceprint model, the remaining expected time for the vehicle to exit the tunnel is obtained and compared with the mechanism lag time to generate different warning signals. Based on different early warning signals, the mechanism activation time point corresponding to each early warning signal is obtained, and pre-release operations are carried out within the tunnel.
2. The online dynamic audio event recognition method based on federated incremental learning according to claim 1, characterized in that: The process for assessing whether a substantial upward masking effect occurs is as follows: After the vehicle enters the tunnel, the continuous raw data stream of PCM time-domain waveform is acquired in real time to obtain the continuously output windowed audio frames. Obtain the cross-frequency energy suppression ratio B1 and formant characteristic ratio B2 corresponding to each windowed audio frame; Under the same windowed audio frame, the corresponding cross-frequency energy suppression ratio B1 is compared with the preset cross-frequency energy suppression threshold, and the corresponding formant feature ratio B2 is compared with the preset formant feature submersion threshold. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is greater than the preset cross-frequency energy suppression threshold and the formant feature ratio B2 is less than the preset formant feature submersion threshold, it is determined that the low-frequency aerodynamic roaring noise has a substantial upward masking effect on the formant, which is the core feature of human voiceprint, and the corresponding windowed audio frame is marked as a masked audio frame.
3. The online dynamic audio event recognition method based on federated incremental learning according to claim 2, characterized in that: The process of acquiring unmasked audio frames is as follows: Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is greater than the preset cross-frequency energy suppression threshold and the formant feature ratio B2 is greater than or equal to the preset formant feature submersion threshold, the corresponding windowed audio frame will be marked as an unmasked audio frame. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is less than or equal to the preset cross-frequency energy suppression threshold, and the formant feature ratio B2 is less than the preset formant feature submersion threshold, the corresponding windowed audio frame will be marked as an unmasked audio frame. Under the same windowed audio frame, if the cross-frequency energy suppression ratio B1 is less than or equal to the preset cross-frequency energy suppression threshold, and the formant feature ratio B2 is greater than or equal to the preset formant feature submersion threshold, the corresponding windowed audio frame will be marked as an unmasked audio frame.
4. The online dynamic audio event recognition method based on federated incremental learning according to claim 1, characterized in that: The process for assessing the stability of the substantial upward masking effect is as follows: Starting from the first windowed audio frame acquired, this process is applied to every subsequent windowed audio frame, including the first windowed audio frame acquired. For each windowed audio frame, the corresponding updated continuous count value is continuously updated, and after each continuous count value is synchronously updated, the corresponding updated continuous count value is compared with the continuous determination frame number threshold. In any windowed audio frame, if the first occurrence of a continuous count value greater than or equal to the duration determination frame threshold, it is determined that the low-frequency aerodynamic roaring noise has a substantial upward masking effect on the resonant peak, which is the core feature of human voiceprint, and generates a steady-state masking signal.
5. The online dynamic audio event recognition method based on federated incremental learning according to claim 4, characterized in that: The process for assessing the stability of the substantial upward masking effect is as follows: When a steady-state masking signal is generated, it serves as the basis for triggering the asymmetric suspension interruption mechanism for the local personalized voiceprint model; If the vehicle's passage through the tunnel has reached the maximum duration corresponding to the estimated total travel time interval before the steady-state masking signal is generated, the stability assessment of the substantial upward masking effect will be stopped. At the same time, it will be determined that the triggering conditions of the asymmetric suspension interruption mechanism for the local personalized voiceprint model have not been met throughout the entire passage of the vehicle through the tunnel.
6. The online dynamic audio event recognition method based on federated incremental learning according to claim 1, characterized in that: The process of generating different warning signals is as follows: When the vehicle is driving in the tunnel, the first steady-state masking signal is generated and the asymmetric suspension interruption mechanism for the local personalized voiceprint model is activated, the remaining expected duration is obtained. Compare the remaining expected duration with the mechanism lag duration; If the remaining expected duration is greater than or equal to the mechanism lag duration, a sufficient time warning signal will be generated. If the remaining expected duration is less than the mechanism lag duration, an insufficient time warning signal will be generated.
7. The online dynamic audio event recognition method based on federated incremental learning according to claim 1, characterized in that: The process of obtaining the start time point D1 for the mechanism corresponding to the ample time warning signal cancellation is as follows: When a sufficient time warning signal is generated, the mechanism lag time is extracted. Based on the expected tunnel exit time, a mechanism lag time is deduced by going back along the historical direction of the time axis to obtain the mechanism deactivation start time point D1 corresponding to the sufficient time warning signal.
8. The online dynamic audio event recognition method based on federated incremental learning according to claim 7, characterized in that: The pre-release operation within the tunnel corresponding to the ample time warning signal is as follows: When a vehicle is driving in a tunnel, after triggering the asymmetric suspension mechanism for the local personalized voiceprint model, the asymmetric suspension mechanism for the local personalized voiceprint model will continue to be maintained until the corresponding mechanism release start time point D1 is reached. As time progresses, when the corresponding mechanism release start time point D1 is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model will be initiated in the background. When the expected tunnel exit time point is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model will be completed, and the local personalized voiceprint model will be restored from the suspended protection state to the state where the vehicle owner's personal voiceprint characteristics can be identified.
9. The online dynamic audio event recognition method based on federated incremental learning according to claim 1, characterized in that: The mechanism for resolving the insufficient time warning signal and the process for obtaining the start time point D2 are as follows: When a time-insufficient warning signal is generated, the corresponding mechanism lag time and the remaining expected time are extracted, and the difference between the mechanism lag time and the remaining expected time is calculated to obtain the corresponding warning gap time. Extract the mechanism trigger time point corresponding to the vehicle. Based on the corresponding mechanism trigger time point, backtrack along the historical direction of the time axis by a warning gap duration to obtain the mechanism deactivation start time point D2 corresponding to the insufficient time warning signal.
10. The online dynamic audio event recognition method based on federated incremental learning according to claim 9, characterized in that: The pre-release operation within the tunnel corresponding to the insufficient time warning signal is as follows: From the moment the vehicle enters the tunnel, as time progresses, when the corresponding mechanism release activation time point D2 is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model is initiated in the background. When the corresponding mechanism trigger time point for the vehicle is reached, the asymmetric suspension interruption mechanism for the local personalized voiceprint model is activated. When the expected tunnel exit time point is reached, the operation to release the asymmetric suspension interruption mechanism for the local personalized voiceprint model is completed, and the local personalized voiceprint model is restored from the suspended protection state to the state where the owner's personal voiceprint characteristics can be identified.