Automobile Road Noise Treatment Method and Equipment

Abstract

The present application discloses a method and device for processing automotive road noise, relating to the technical field of noise reduction. The method includes: dynamically determining a target filter bank according to the acquired environmental state information; respectively performing sub-band decomposition on a time-domain reference signal and a time-domain error signal according to the target polyphase analysis filter bank; updating sub-band adaptive filter coefficients based on the sub-band reference signal and the sub-band error signal, and performing sub-band synthesis on the updated sub-band adaptive filter coefficients by using the target polyphase synthesis filter bank to obtain updated time-domain adaptive filter coefficients; processing a newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control road noise based on the anti-noise signal. The present application can adapt to environmental changes, avoid the influence of environmental changes on the noise reduction effect, and thus improve the stability of the noise reduction effect.

Description

Technical Field

[0001] The present application relates to the technical field of noise reduction, and particularly to a method and device for processing automotive road noise. Background Art

[0002] With the development of the automotive industry, the comfort and quietness inside the car cockpit have increasingly attracted the attention of the automotive industry and consumers. As a key factor affecting the acoustic quietness and riding experience inside the car, interior noise has gradually become a pain point problem concerned in the industry. Interior noise mainly comes from the power system of the vehicle and road noise during vehicle driving. With the development of new energy vehicles, electric motors have gradually replaced internal combustion engines, thus significantly reducing engine noise. Road Noise Cancellation (RNC) technology has become an important means to improve the acoustic comfort inside the car. The RNC technology combines the reference signal collected by a vibration sensor and the error signal collected by an in-vehicle microphone to generate a sound signal with the opposite characteristics of the noise, which is played through an in-vehicle speaker to cancel the original noise. Although existing RNC systems have been applied in some mass-produced models, in complex actual application environments, there are still problems such as insufficient adaptive ability of the RNC system and poor stability of the noise reduction effect, seriously affecting the driving and riding experience.

[0003] The above content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present application is to provide a method and device for processing automotive road noise, aiming to solve the problems of insufficient adaptive ability of the current RNC system and poor stability of the noise reduction effect.

[0005] To achieve the above object, the present application proposes a method for processing automotive road noise, and the method for processing automotive road noise includes:

[0006] Dynamically determining a target filter bank according to the obtained environmental state information, where the target filter bank includes a target polyphase analysis filter bank and a target polyphase synthesis filter bank, and the environmental state information is information characterizing the environmental state inside and / or outside the vehicle;

[0007] Performing sub-band decomposition on the input first-time-domain reference signal and the input first-time-domain error signal respectively by using the target polyphase analysis filter bank to obtain a first sub-band reference signal and a first sub-band error signal;

[0008] Updating the sub-band adaptive filter coefficients based on the first sub-band reference signal and the first sub-band error signal;

[0009] Using the target polyphase synthesis filter bank to perform subband synthesis on the updated subband adaptive filter coefficients to obtain the updated time-domain adaptive filter coefficients;

[0010] Processing the newly input second time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control road noise based on the anti-noise signal.

[0011] Optionally, different filter banks are preset for different environmental states, and each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank. The filter update delay and / or the spectrum leakage of different filter banks are different;

[0012] The step of dynamically determining the target filter bank according to the obtained environmental state information includes:

[0013] Determining the filter bank preset corresponding to the environmental state represented by the obtained environmental state information as the target filter bank.

[0014] Optionally, when dynamically determining the target filter bank according to the environmental state information, the following rule is followed:

[0015] In the environmental state represented by the environmental state information, the higher the stability of the automotive road noise, the greater the filter update delay and the smaller the spectrum leakage brought by the target filter bank; the lower the stability of the automotive road noise, the smaller the filter update delay and the greater the spectrum leakage brought by the target filter bank.

[0016] Optionally, each filter bank preset for different environmental states is pre-trained with the goal of reducing the signal reconstruction loss and spectrum leakage loss of the filter bank, and the weights of the signal reconstruction loss and frequency leakage loss set during the training of different filter banks are different.

[0017] Optionally, the road noise control method further includes:

[0018] Using a preset prediction model to perform prediction based on the third time-domain reference signal, the third time-domain error signal, the third subband reference signal, and the third subband error signal to obtain the environmental state information;

[0019] Wherein, the prediction model is pre-trained, the third time-domain reference signal and the third time-domain error signal are respectively input before the first time-domain reference signal and the first time-domain error signal, and the third subband reference signal and the third subband error signal are obtained by respectively performing subband decomposition on the third time-domain reference signal and the third time-domain error signal.

[0020] Optionally, the prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module;

[0021] The step of using the preset prediction model to perform prediction based on the third time-domain reference signal, the third time-domain error signal, the third sub-band reference signal, and the third sub-band error signal to obtain the environmental state information includes:

[0022] Input the third sub-band reference signal and the third sub-band error signal into the first feature extraction module for feature extraction to obtain a first feature representation;

[0023] Input the third time-domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation;

[0024] Input the third time-domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation;

[0025] Concatenate the first feature representation, the second feature representation, and the third feature representation and input them into the feature fusion module for feature fusion to obtain a fused feature representation;

[0026] Input the fused feature representation into the prediction module for prediction to obtain the environmental state information.

[0027] Optionally, the first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer; the feature fusion module includes a frequency-axis gated recurrent unit, a time-axis gated recurrent unit, and at least one transposed convolutional layer connected in sequence; the prediction module includes a plurality of task output layers, and each task output layer is respectively used to output one item of the environmental state information.

[0028] Optionally, the environmental state information includes weather condition information and road surface condition information.

[0029] Optionally, the frame indices of the first time-domain reference signal and the first time-domain error signal are l, and the method for processing vehicle road noise further includes:

[0030] Execute the tasks allocated to the N interrupts among the N interrupts in the (l + 1)-th frame, where N represents the frame length, and an interrupt refers to the period from generating an anti-noise signal based on the time-domain reference signal of each sampling point to receiving the time-domain reference signal of the next sampling point. The tasks allocated to the N interrupts include: the task of calculating the updated time-domain adaptive filter coefficients, and / or, the task of inputting the first time-domain reference signal, the first time-domain error signal, the first sub-band reference signal, and the first sub-band error signal into the prediction model to predict the environmental state information.

[0031] In addition, to achieve the above object, the present application also provides an automotive road noise processing device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the automotive road noise processing method as described above.

[0032] One or more technical solutions proposed in the present application have at least the following technical effects:

[0033] In the present application, the target polyphase analysis filter bank and the target polyphase synthesis filter bank are dynamically determined according to the acquired environmental state information. The input time-domain reference signal and the input time-domain error signal are respectively sub-band decomposed by the target polyphase analysis filter bank to obtain the sub-band reference signal and the sub-band error signal. The sub-band adaptive filter coefficients are updated based on the sub-band reference signal and the sub-band error signal. The updated sub-band adaptive filter coefficients are sub-band synthesized by the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients. The newly input time-domain reference signal is processed based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control the road noise based on the anti-noise signal. That is, in the present application, by dynamically determining the target polyphase analysis filter bank and the target polyphase synthesis filter bank for sub-band decomposition and sub-band synthesis according to the acquired environmental state information, during the process of controlling automotive road noise, it can adapt to environmental changes, avoid the influence of environmental changes on the noise reduction effect, and thus improve the stability of the noise reduction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0035] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1 It is a schematic flowchart provided for the first embodiment of the automotive road noise processing method of the present application;

[0037] Figure 2 It is a schematic flowchart of the prediction model training process involved in an embodiment of the present application;

[0038] Figure 3 It is a schematic diagram of the signal processing process involved in an embodiment of the present application;

[0039] Figure 4 Schematic flowchart of an automotive road noise processing method according to an embodiment of the present application;

[0040] Figure 5 Schematic diagram of a model architecture and parameter matching architecture according to an embodiment of the present application;

[0041] Figure 6 Filter coefficient comparison diagram according to an embodiment of the present application;

[0042] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the automotive road noise processing method in the embodiment of the present application.

[0043] The realization of the purpose, functional characteristics, and advantages of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific embodiments

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] For a better understanding of the technical solutions of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific embodiments.

[0046] The RNC technology generates a sound signal opposite to the noise characteristics by combining the reference signal collected by the vibration sensor and the error signal collected by the in-vehicle microphone, and plays it through the in-vehicle speaker to cancel the original noise. Although the existing RNC systems have been applied in some mass-produced vehicles, in complex actual application environments, there are still problems such as insufficient adaptability of the RNC system and poor stability of the noise reduction effect, which seriously affect the driving experience.

[0047] In order to solve the above technical problems, embodiments of the present application propose to dynamically determine a target polyphase analysis filter bank and a target polyphase synthesis filter bank according to the acquired environmental state information, perform subband decomposition on the input time-domain reference signal and the input time-domain error signal respectively using the target polyphase analysis filter bank to obtain a subband reference signal and a subband error signal, update the subband adaptive filter coefficients based on the subband reference signal and the subband error signal, perform subband synthesis on the updated subband adaptive filter coefficients using the target polyphase synthesis filter bank to obtain updated time-domain adaptive filter coefficients, process the newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, and control road noise based on the anti-noise signal. That is, embodiments of the present application can adapt to environmental changes during the process of controlling automotive road noise by dynamically determining the target polyphase analysis filter bank and the target polyphase synthesis filter bank for subband decomposition and subband synthesis, avoiding the impact of environmental changes on the noise reduction effect, and thus improving the stability of the noise reduction effect.

[0048] The following presents the first embodiment of the automotive road noise processing method of the present application. Refer to Figure 1 , Figure 1This is a schematic flowchart of the first embodiment of the method for processing vehicle road noise in this application. In this embodiment, the execution subject of the method for processing vehicle road noise can be a control unit in the vehicle RNC system, but it is not limited to the control unit. For example, it can also be a processing device with data processing and program running functions. In a feasible implementation, the vehicle RNC system may include sensors, a control unit, a noise canceller, and other auxiliary devices set as needed; among them, the sensors include a reference sensor for collecting reference signals and an error sensor for collecting error signals, and multiple reference sensors and error sensors can be set; the reference sensor and the error sensor can be vibration sensors (acceleration sensors) or microphones; the control unit runs adaptive algorithms, such as FxLMS (Filtered-x Least Mean Square), FxNLMS (Filtered-X Normalized Least Mean Square), FxAP (Filter-x Affine Projection) and other adaptive algorithms, updates the adaptive filter coefficients based on the reference signal and the error signal, and calculates the anti-noise signal; the noise canceller is used to output the anti-noise signal to cancel the road noise. In a feasible implementation, the reference sensor can adopt a vibration sensor and can be set at the attachment point of the vehicle body and the chassis, and the error sensor can adopt a microphone and can be set inside the vehicle, such as on the roof lining (near the passenger's head area); the noise canceller can adopt a vibration actuator and / or a speaker. The vibration actuator generates reverse mechanical vibration according to the anti-noise signal to cancel the low-frequency road noise transmitted through the vehicle body, and the speaker emits reverse sound waves according to the anti-noise signal to directly cancel the air-borne noise (such as the high-frequency component of tire noise) entering the passenger compartment.

[0049] In this embodiment, the method for processing vehicle road noise includes steps S10 to S50:

[0050] Step S10, dynamically determine a target filter bank according to the obtained environmental state information, where the target filter bank includes a target polyphase analysis filter bank and a target polyphase synthesis filter bank, and the environmental state information is information characterizing the environmental state inside and / or outside the vehicle.

[0051] In this embodiment, a sub-band adaptive algorithm is adopted, that is, the sub-band adaptive filter coefficients are updated separately and then synthesized into full-band (time-domain) adaptive filter coefficients. Compared with the method of directly updating the full-band (time-domain) adaptive filter coefficients, in this embodiment, the sub-band adaptive filter coefficients are updated separately, and then the updated sub-band adaptive filter coefficients are synthesized into the updated time-domain adaptive filter coefficients, which has the advantages of fast convergence, independence of different sub-bands, the ability to use different adaptive algorithms and filter orders for different sub-bands, and low sensitivity to changes in the input signal.

[0052] The subband adaptive algorithm requires subband decomposition and subband synthesis operations. In this embodiment, a polyphase analysis filter bank is used for subband decomposition, and a polyphase synthesis filter bank is used for subband synthesis. Through this subband decomposition and subband synthesis method, during the process of restoring the filter coefficients to the full band, the synthesis filter bank is directly used, and only an IFFT (inverse fast Fourier transform) operation needs to be performed after adding a synthesis window to the subband domain filter, so that the time-domain filter coefficients almost consistent with the full band can be obtained, ensuring both performance and avoiding additional computational complexity.

[0053] In the specific implementation, a polyphase analysis filter bank based on FFT (hereinafter referred to as the polyphase FFT analysis filter bank) and a polyphase synthesis filter bank based on IFFT (hereinafter referred to as the polyphase IFFT synthesis filter bank) can be used to further improve the computational efficiency. Hereinafter, the analysis filter bank may also be referred to as the analysis window, and the synthesis filter bank may be referred to as the synthesis window.

[0054] The control unit can dynamically determine the filter bank used for subband decomposition and subband synthesis according to the acquired environmental state information. Hereinafter, the determined filter bank is referred to as the target filter bank for distinction. The target filter bank includes a target polyphase analysis filter bank for subband decomposition and a target polyphase synthesis filter bank for subband synthesis. The target polyphase analysis filter bank is used to perform subband decomposition on the time-domain reference signal and the time-domain error signal, and the target polyphase synthesis filter bank is used to perform subband synthesis on the updated subband adaptive filter coefficients. Since changes in the environmental state outside the vehicle or the environmental state inside the vehicle will cause dynamic changes in the acoustic propagation path, which will affect the stability of the vehicle road noise. If a fixed polyphase analysis filter bank and polyphase synthesis filter bank are used for subband decomposition and subband synthesis, it may lead to the inability to adapt to the changing vehicle road noise, resulting in unstable noise reduction effects. Therefore, in this embodiment, the target polyphase analysis filter bank and the target polyphase synthesis filter bank for subband decomposition and subband synthesis are dynamically determined according to the environmental state information to adapt to environmental changes and improve the stability of the noise reduction effect.

[0055] In the specific implementation, the environmental state inside the vehicle may include, for example, the seat adjustment state, the position of the occupants, the window opening and closing state, etc., and the environmental state outside the vehicle may include, for example, the weather condition, the road surface condition, etc., which are not limited in this embodiment.

[0056] In this embodiment, the acquisition method of the environmental state information is not limited. For example, it can be obtained by image analysis of the environmental images collected by a camera.

[0057] The environmental status information can be obtained at regular intervals. The shorter the time interval, the higher the real-time degree. In this embodiment, the acquisition frequency is not limited. For example, in a feasible implementation, the environmental status information can be obtained each time the adaptive filter coefficients are updated.

[0058] Dynamically determining the target filter bank means that after each new environmental status information is obtained, the target filter bank is determined according to the new environmental status information, so that the filter banks used for subband decomposition and subband synthesis change with the change of the environmental status, and the stability of the noise reduction effect is improved by adjusting the filter banks used for subband decomposition and subband synthesis to adapt to the change of the environmental status inside or outside the vehicle.

[0059] In a specific implementation, different filter banks can be preset for different environmental statuses. Each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank. Then, when the control unit needs to perform subband decomposition on the current time-domain reference signal and time-domain error signal, and needs to perform subband synthesis on the current subband adaptive filter coefficients, the filter bank preset corresponding to the environmental status characterized by the obtained environmental status information can be determined as the target filter bank. The filter banks set for different environmental statuses can be filter banks with the best or better noise reduction effect in this environmental status determined in advance through experiments, data statistics, etc.

[0060] Step S20: Use the target polyphase analysis filter bank to perform subband decomposition on the input first time-domain reference signal and the input first time-domain error signal respectively to obtain a first subband reference signal and a first subband error signal.

[0061] When multiple reference sensors are set, the input time-domain reference signal is multi-channel. Similarly, when multiple error sensors are set, the input time-domain error signal is also multi-channel. In this embodiment, the number of channels of the time-domain reference signal and the time-domain error signal is not limited. For example, when applied to a MIMO (Multiple Input Multiple Output) vehicle-mounted RNC system, both the input and output are multi-channel.

[0062] The time-domain reference signal and the time-domain error signal collected by the sensor can be directly input into the control unit, or input into the control unit after preprocessing; the control unit can directly perform subband decomposition on the input time-domain reference signal and time-domain error signal, or can perform preprocessing before subband decomposition. In this embodiment, it is not limited whether to perform preprocessing on the time-domain reference signal and the time-domain error signal, nor what preprocessing operations are performed on the signal. For example, it can include prefiltering, amplitude adjustment, etc. Prefiltering includes high-pass filtering, low-pass filtering, etc., and the preprocessing operations can be set according to needs. In the following embodiments, the preprocessing operation on the signal is not particularly emphasized anymore. That is, even if it is not written that preprocessing is required, it does not necessarily mean that there is no or no need to perform the preprocessing operation.

[0063] The control unit uses a target polyphase analysis filter bank determined according to the environmental state information to perform subband decomposition on the time-domain reference signal to obtain a subband reference signal, and perform subband decomposition on the time-domain error signal to obtain a subband error signal. The prefix "first" is used in step S20 to distinguish from the time-domain reference signal, time-domain error signal, subband reference signal, and subband error signal mentioned in the subsequent steps, indicating that they are input at different times. It can be understood that there are multiple subbands, and there are also multiple subband reference signals and subband error signals obtained through subband decomposition. That is, the subband reference signal and the subband error signal corresponding to each subband are obtained through subband decomposition. In the specific implementation, the subband can be divided from the full frequency band, but in this embodiment, it is not limited to being divided from the full frequency band, and it can be divided from a relatively wide frequency band range, and the number of subbands is not limited either.

[0064] In a feasible implementation, the control unit can perform road noise control in a manner without output delay, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit point by point in real time. The control unit generates an anti-noise signal for a single sampling point of the currently input time-domain reference signal in real time according to the current time-domain adaptive filter coefficients, and then outputs the anti-noise signal in real time to achieve the effect of no output delay; on the other hand, the adaptive filter coefficients are updated frame by frame. Specifically, the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffer in real time. After each frame of time-domain reference signal and each frame of time-domain error signal are stored, subband decomposition is performed on the frame of time-domain reference signal and the frame of time-domain error signal, and subsequent time-domain adaptive filter coefficient update operations are performed. In other embodiments, a method with output delay can also be used, that is, after inputting a plurality of sampling points of the time-domain reference signal, the anti-noise signal starts to be output. In this embodiment, it is not limited whether to use the method without output delay or the method with output delay. In the specific application scenario, the specific implementation method can be selected according to needs.

[0065] Step S30: Update the sub-band adaptive filter coefficients based on the first sub-band reference signal and the first sub-band error signal.

[0066] The sub-band adaptive filter coefficients can be updated according to a preset adaptive algorithm. For example, the adaptive algorithm can adopt FxLMS, FxNLMS, FxAP, etc., which are not limited in this embodiment.

[0067] For example, in a feasible implementation, the adaptive algorithm can adopt the complex-domain FxLMS algorithm. Specifically, the control unit can convolve the first sub-band reference signal with a pre-established secondary path model, and then use the complex-domain LMS algorithm to calculate the updated sub-band adaptive filter coefficients based on the first sub-band error signal, the first sub-band reference signal after convolving with the secondary path model, and the current sub-band adaptive filter coefficients.

[0068] Step S40: Perform sub-band synthesis on the updated sub-band adaptive filter coefficients using the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients.

[0069] To avoid output delay, filtering output needs to be performed in the time domain (full band). Therefore, the sub-band adaptive filter coefficients can be converted to the full band. By performing sub-band synthesis on the updated sub-band adaptive filter coefficients, the updated time-domain adaptive filter coefficients can be obtained.

[0070] The control unit performs sub-band synthesis on the updated sub-band adaptive filter coefficients using the target polyphase synthesis filter bank determined according to the environmental state information to obtain the updated time-domain adaptive filter coefficients.

[0071] Step S50: Process the newly input second time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control road noise based on the anti-noise signal.

[0072] The second time-domain reference signal refers to the time-domain reference signal input after obtaining the updated time-domain adaptive filter coefficients. For example, assuming the frame index of the first time-domain reference signal is l, then the frame index of the second time-domain reference signal may be l + 1.

[0073] After updating the time-domain adaptive filter coefficients, the newly input time-domain reference signal can be processed according to the updated time-domain adaptive filter coefficients to generate an anti-noise signal. After generating the anti-noise signal, road noise can be controlled based on the anti-noise signal. For example, the anti-noise signal can be played through a speaker set inside the vehicle, or the anti-noise signal can be output after some post-processing operations. The post-processing operations can include amplitude limiting, etc., which are not limited here.

[0074] The generated anti-noise signal can be multi-channel. For example, when there are multiple speakers, after generating a multi-channel anti-noise signal, it is output through multiple speakers. In this embodiment, the number of channels of the anti-noise signal is not limited.

[0075] In this embodiment, the target polyphase analysis filter bank and the target polyphase synthesis filter bank are dynamically determined according to the acquired environmental state information. The input time-domain reference signal and the input time-domain error signal are respectively sub-band decomposed by the target polyphase analysis filter bank to obtain the sub-band reference signal and the sub-band error signal. The sub-band adaptive filter coefficients are updated based on the sub-band reference signal and the sub-band error signal. The updated sub-band adaptive filter coefficients are sub-band synthesized by the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients. The newly input time-domain reference signal is processed based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control the road noise based on the anti-noise signal. That is to say, the solution of this embodiment dynamically determines the target polyphase analysis filter bank and the target polyphase synthesis filter bank for sub-band decomposition and sub-band synthesis according to the acquired environmental state information, so that during the process of controlling the vehicle road noise, it can adapt to environmental changes, avoid the influence of environmental changes on the noise reduction effect, and thus improve the stability of the noise reduction effect.

[0076] In a feasible implementation manner, different filter banks are preset for different environmental states. Each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank, and the filter update delay and / or the spectral leakage size brought by different filter banks are different. The step S10 includes: determining the filter bank preset corresponding to the environmental state characterized by the acquired environmental state information as the target filter bank.

[0077] The filter update delay and / or the spectral leakage of each filter bank pre-set for different environmental states are different. The filter update delay and spectral leakage brought by the filter bank refer to the filter update delay and spectral leakage that will occur when sub-band decomposition and sub-band synthesis are performed according to the filter bank. It should be noted that generally, the filter update delay and spectral leakage of the filter bank are negatively correlated. The larger the delay, the smaller the spectral leakage; the smaller the delay, the larger the spectral leakage. And the smaller the delay, the better the robustness and transient response of the system; the smaller the spectral leakage, the better the filtering accuracy and steady-state performance of the system. Since the stability of vehicle road noise is different in different environmental states and the requirements for the robustness, transient response ability, filtering accuracy, and steady-state performance of the system are different, dynamically determining the filter bank to be used according to the environmental state information can balance the delay and spectral leakage, adopt a delay and spectral leakage suitable for the environmental state, thereby balancing the robustness, transient response ability, filtering accuracy, and steady-state performance of the system and ensuring the stability of the noise reduction effect.

[0078] In a feasible implementation, it can be set that when the control unit dynamically determines the target filter bank according to the environmental state information, the following rules need to be met:

[0079] In the environmental state characterized by the environmental state information, the higher the stability of vehicle road noise, the greater the filter update delay and the smaller the spectral leakage brought by the target filter bank; the lower the stability of vehicle road noise, the smaller the filter update delay and the larger the spectral leakage brought by the target filter bank.

[0080] It should be noted that if the stability of vehicle road noise is relatively high in the environmental state characterized by the environmental state information, a filter bank with a larger filter update delay and a smaller spectral leakage can be adopted, focusing on improving the filtering accuracy and steady-state performance to obtain a better noise reduction effect. If the stability of vehicle road noise is relatively low in the environmental state characterized by the environmental state information, a filter bank with a smaller filter update delay and a larger spectral leakage can be adopted, focusing on improving the robustness and transient response of the system to quickly adapt to environmental changes and maintain system stability.

[0081] In a specific implementation, to enable the control unit to meet the above rules when dynamically determining the target filter bank according to the environmental state information, the filter bank corresponding to each environmental state can be set according to this rule.

[0082] In a feasible implementation, the environmental status information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to weather conditions that affect the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information indicating the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0083] In a feasible implementation, when the environmental status information indicates that the environment is good and stable (for example: good road conditions, clear weather), the vehicle road noise is relatively stable. Then, a filter bank with a larger filter update delay and smaller spectral leakage can be adopted, focusing on improving the filtering accuracy and steady-state performance to obtain a better noise reduction effect. When the environmental status information indicates that the environment is unstable (for example: deteriorating road conditions, bad weather), the stability of the vehicle road noise may deteriorate. At this time, a filter bank with a smaller filter update delay and larger spectral leakage can be adopted, focusing on improving the robustness and transient response of the system, quickly adapting to environmental changes and maintaining system stability.

[0084] The following Table 1 gives some examples of determining the filter bank based on the weather condition information and the road surface condition information. Table 1 only gives some examples and does not limit the way of determining the target filter bank in this embodiment. The content in parentheses in Table 1 is an explanatory description of the weather condition information, road surface condition information, etc.

[0085] Table 1

[0086]

[0087] Based on the above first embodiment, a second embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the same or similar content as the above first embodiment can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, each preset filter bank corresponding to different environmental statuses is pre-trained with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank. It should be noted that the method of training with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank is similar to the method of training a neural network model with the goal of reducing the prediction loss of the neural network model. The steps include preparing data for training, designing a loss function, selecting an optimizer for training, etc., and can specifically refer to the training method of the neural network model. Among them, the signal reconstruction loss refers to the error between the signal obtained by sub-band decomposing and sub-band synthesizing the signal through the filter bank and the original signal, and the spectral leakage loss refers to the magnitude of the spectral leakage generated by sub-band decomposing the signal through the filter bank.

[0088] The weights of the signal reconstruction loss and the frequency leakage loss set during the training of different filter banks are different. Specifically, in order to train filter banks with different spectral leakage magnitudes and different filter update delays, different weight combinations of the signal reconstruction loss and the frequency leakage loss can be set during the training. For example, by setting a larger weight for the signal reconstruction loss and a smaller weight for the frequency leakage loss, a filter bank with a smaller delay and a larger leakage can be trained; by setting a smaller weight for the signal reconstruction loss and a larger weight for the frequency leakage loss, a filter bank with a larger delay and a smaller leakage can be trained.

[0089] In a feasible implementation manner, the signal reconstruction loss is calculated based on the error between the reconstructed signal and a preset first type of signal. The reconstructed signal is obtained by performing subband decomposition and subband synthesis on the preset first type of signal using the polyphase analysis filter bank to be trained and the polyphase synthesis filter bank to be trained. Among them, an initialized polyphase analysis filter bank and a polyphase synthesis filter bank can be set in advance, and the coefficients of the polyphase analysis filter bank and the polyphase synthesis filter bank need to be updated during the training process. The preset first type of signal is a signal preset for calculating the signal reconstruction loss, and can be various noise signals, vibration signals, music signals, etc. collected in advance. The preset first type of signal is subjected to subband decomposition using the polyphase analysis filter bank to be trained, and the decomposed result is then subjected to subband synthesis using the polyphase synthesis filter bank to be trained to obtain a reconstructed signal, and the error is calculated with the preset first type of signal to obtain the signal reconstruction loss.

[0090] In a feasible implementation manner, the spectral leakage loss is calculated based on the target subband signal. The target subband signal is obtained by performing subband decomposition on a preset second type of signal using the polyphase analysis filter bank to be trained. The preset second type of signal is a single-frequency signal or a narrowband signal within a preset frequency band range, and the frequency band range of the target subband signal is the range other than the preset frequency band range. Among them, the preset second type of signal is a signal preset for calculating the spectral leakage loss, and can be a single-frequency signal or a narrowband signal within a preset frequency band range collected in advance. The preset frequency band range can be set as needed. The preset second type of signal is subjected to subband decomposition using the polyphase analysis filter bank to be trained, and the spectral leakage loss is calculated based on the target subband signal in the decomposition result. It can be understood that since the preset second type of signal is a single-frequency signal or a narrowband signal within a preset frequency band range, after the subband decomposition of the preset second type of signal, if there is no spectral leakage, then there will be no signal in the range other than the preset frequency band range in the subband decomposition result, and the frequency band range of the target subband signal is the range other than the preset frequency band range, so the magnitude of the target subband signal can represent the spectral leakage magnitude, and the spectral leakage loss can be calculated based on the target subband signal.

[0091] In a feasible implementation, the filter bank can be trained in the following manner:

[0092] Step 1: Data preparation. Two types of data are used for training the analysis filter bank and the synthesis filter bank. The first type is various noise signals, vibration signals, and music signals. The second type is single-frequency signals or narrowband signals uniformly generated according to a known frequency band range (hereinafter represented by S as the set of sub-band indices within this frequency band range). Each of these two types of data accounts for 50% of the total data.

[0093] Step 2: Design of the loss function. The signal reconstruction error and the spectral leakage energy are two criteria for measuring the performance of the analysis window and the synthesis window of the sub-bands. Therefore, these two metrics are used as the loss function for training.

[0094]

[0095]

[0096]

[0097] Among them, L MSE is the reconstruction loss function of the signal, L leakage is the loss function of the spectral leakage of the signal. α and β are weight coefficients used to adjust the influence ratio of the losses of the two parts. For example, α = 0.4 and β = 0.6 are set. z[r] is the preset first type of signal, and z'[r] is the signal after sub-band decomposition and sub-band synthesis of z[r], that is, the reconstructed signal. R is the length of the signal, for example, it can be set as R = 20000. Z[k] is the sub-band signal after sub-band decomposition of the preset second type of signal, k is the index of the sub-band, and S is the set of sub-band indices within the known frequency band range in Step 1.

[0098] Step 3: Select an optimizer for training. Calculate L MSE according to the input preset first type of signal, calculate L leakage according to the input preset second type of signal, and combine these two losses as the final loss L total . Pytorch (an open-source deep learning framework for machine learning and deep learning) can be selected as the training environment, and Adam can be selected as the optimizer to perform training iterations on the analysis window and the synthesis window until the loss function converges, and finally obtain the trained analysis window and synthesis window. The downsampling factor can be set to 3, and the number of points P of the FFT can be set to 128. Therefore, the size of the analysis window and the synthesis window is 128 * 3. During training, a hamming window with a length of 128 * 3 can be used as the initial coefficient.

[0099] Based on the above first and / or second embodiments, the third embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first and / or second embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, it is proposed to predict the environmental state information and / or sensor state information through a prediction model based on the input reference signal and error signal. Specifically, in this embodiment, the automotive road noise processing method further includes step S60: using a preset prediction model to perform prediction based on the third time-domain reference signal, the third time-domain error signal, the third sub-band reference signal, and the third sub-band error signal to obtain the environmental state information; wherein, the prediction model is pre-trained.

[0100] The third time-domain reference signal and the third time-domain error signal are respectively input before the first time-domain reference signal and the first time-domain error signal. For example, assuming that the frame indices of the first time-domain reference signal and the first time-domain error signal are l, then the frame indices of the third time-domain reference signal and the third time-domain error signal may be l - 1. The third sub-band reference signal and the third sub-band error signal are obtained by performing sub-band decomposition on the third time-domain reference signal and the third time-domain error signal respectively.

[0101] In the specific implementation, road noise control can be performed in a manner without output delay, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit point by point in real time. The control unit generates a single-sampling-point anti-noise signal for the currently input single-sampling-point time-domain reference signal according to the current time-domain adaptive filter coefficients in real time, and then outputs the anti-noise signal in real time to achieve the effect of no output delay; on the other hand, the environmental state information is updated frame by frame: the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffer in real time. After each buffer is filled with a frame of time-domain reference signal and a frame of time-domain error signal (assuming the frame index is l), the time-domain reference signal and time-domain error signal of the l-th frame are subjected to sub-band decomposition according to the polyphase analysis filter bank determined according to the environmental state information (predicted by inputting the signal of the (l - 1)-th frame into the prediction model) to obtain the l-th frame sub-band reference signal and the l-th frame sub-band error signal. The l-th frame time-domain reference signal, the l-th frame time-domain error signal, the l-th frame sub-band reference signal, and the l-th frame sub-band error signal are input into the prediction model to predict the new environmental state information, and the filter bank determined according to this environmental state information is used to perform sub-band decomposition on the time-domain reference signal and time-domain error signal of the (l + 1)-th frame.

[0102] In a specific embodiment, the prediction model can be implemented using a neural network model. It can be pre-trained and then deployed in the control unit. When the control unit needs to obtain environmental state information or sensor state information, it calls the prediction model for prediction. In this embodiment, there is no limitation on the implementation manner of the prediction model.

[0103] In a specific embodiment, multiple prediction models can be set up to respectively output different environmental state information or sensor state information, or a single prediction model can be set up to output multiple types of information in a multi-output task mode.

[0104] In this embodiment, since the changes in the environmental state and sensor state will be reflected in the signals collected by the sensor, therefore, the time-domain reference signal, time-domain error signal, sub-band reference signal, and sub-band error signal are used as the input data of the prediction model. Through the pre-trained prediction model, the relationship between these input data and the environmental state information or sensor state information can be learned. Furthermore, the prediction model can be used to predict the environmental state information or sensor state information based on these input data. On the one hand, by obtaining the environmental state information or sensor state information through model prediction, the existing reference signals and error signals can be utilized without the need to additionally set up a data acquisition module. On the other hand, the complex situations of automotive road noise under different environmental states or sensor states can all be reflected in the signals collected by the sensor. Therefore, using the signals collected by the sensor can predict more complex and accurate environmental state information, thereby enabling the dynamic determination of the filter bank for sub-band decomposition and sub-band synthesis based on this information, which can greatly improve the system's ability to handle complex environments. In addition, using both time-domain signals and sub-band signals as the prediction basis takes into account the importance of frequency-domain characteristics and can accurately detect and match more scenarios with small differences in time-domain characteristics.

[0105] In a feasible embodiment, the prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module. The step S60 includes S601 to S605:

[0106] Step S601: Input the third sub-band reference signal and the third sub-band error signal into the first feature extraction module for feature extraction to obtain a first feature representation;

[0107] Step S602: Input the third time-domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation;

[0108] Step S603: Input the third time-domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation;

[0109] Step S604: Concatenate the first feature representation, the second feature representation, and the third feature representation, and input the concatenated result into the feature fusion module for feature fusion to obtain a fused feature representation;

[0110] Step S605: Input the fused feature representation into the prediction module for prediction to obtain the environmental state information and / or the sensor state information.

[0111] It should be noted that, to improve the accuracy of the prediction results and fully extract the information related to the environmental state and sensor state in the signal, in this embodiment, feature extraction modules are respectively set for the subband reference signal, the subband error signal, the time-domain reference signal, and the time-domain error signal to extract the signal features of the three signals. For distinction, they are respectively called the first feature extraction module, the second feature extraction module, and the third feature extraction module; a feature extraction module based on a neural network structure can be used to implement. The results output by the three feature extraction modules, that is, the feature representations, are respectively called the first feature representation, the second feature representation, and the third feature representation for distinction. The feature representation can be in the form of a feature vector or a feature map, which is not limited in this embodiment. The feature fusion module can specifically be a module for simply concatenating the three feature representations, or can also include a module for deeply fusing the concatenated features. The module for performing deep fusion can also be specifically implemented based on a neural network structure, which is not limited here. The prediction module can be designed according to the types of information to be output and the specific forms of each type of information, which is not limited in this embodiment.

[0112] In a feasible embodiment, the first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer, and the convolutional layer is used to extract features from the signal; the feature fusion module includes a frequency-axis gated recurrent unit (FGRU), a time-axis gated recurrent unit (TGRU), and at least one transposed convolutional layer connected in sequence. The frequency-axis gated recurrent unit is used to process data along the frequency axis to capture the correlation in the frequency dimension, the time-axis gated recurrent unit is used to process the output of the frequency-axis gated recurrent unit along the time axis to capture the dynamic features in the time dimension, and the transposed convolutional layer is used to gradually upsample and restore to the original size; the prediction module includes multiple task output layers, and each task output layer is respectively used to output one item of the environmental state information or the sensor state information. Hereinafter, FGRU and TGRU can also be collectively referred to as FT-GRU.

[0113] In a feasible implementation, a prediction model can be set up based on an improved U-Net (a deep learning-based convolutional neural network) architecture, integrating a frequency-axis gated recurrent unit and a time-axis gated recurrent unit, specifically designed to process complex acoustic and vibration signals. The prediction model includes an Encoder and a Decoder, which are specifically designed to process acoustic and vibration signals. The Encoder performs step-by-step downsampling through multiple layers of convolution, batch normalization, and ReLU activation functions to effectively extract features; the Decoder uses multiple layers of transposed convolution and skip connections for step-by-step upsampling to prepare features for the terminal classification task. In addition, FGRU and TGRU are integrated into the network structure to enhance the processing ability of multi-dimensional data, especially in time series and frequency analysis, to adapt to real-time changing environmental conditions. This comprehensive design not only captures complex features spatially but also improves the adaptability and prediction accuracy of the model to complex changes in the vehicle's interior and exterior environments by dynamically adjusting the GRU layer responses. Specifically, the Encoder includes a first feature extraction module, a second feature extraction module, and a third feature extraction module. The first feature extraction module may include three convolutional layers, followed by a max-pooling layer after the first two convolutional layers, and the third convolutional layer is used to adjust the dimension of the output feature representation so that the feature representation output by the first feature extraction module is convenient for splicing with the feature representations output by the other two feature extraction modules. The dimensions of the input and output data of the first feature extraction module and the dimensions of the input and output data of its respective convolutional layers and max-pooling layers can be set as needed and are not limited here. The second feature extraction module and the third feature extraction module are used to extract features from time-domain signals and can be set to include a one-dimensional convolutional layer, a pooling layer, and a convolutional layer for adjusting the dimension of the output feature representation respectively. The dimensions of the input and output data of the second feature extraction module and the third feature extraction module and the dimensions of the input and output data of their respective convolutional layers and pooling layers can be set as needed and are not limited here. After the feature representations output by the three feature extraction modules are spliced, they are input into the frequency-axis gated recurrent unit. The splicing method can be, for example, splicing along the channel dimension, which is not limited here. The role of the frequency-axis gated recurrent unit is to process data along the frequency axis and capture the correlations in the frequency dimension; the spliced feature representations can be divided into multiple time steps and input into the frequency-axis gated recurrent unit sequentially by time step. After cyclic calculation by the frequency-axis gated recurrent unit, the hidden states corresponding to each time step are output and input into the time-axis gated recurrent unit in sequence. The role of the time-axis gated recurrent unit is to process the output of the frequency-axis gated recurrent unit along the time axis and capture the dynamic features in the time dimension; the time-axis gated recurrent unit performs cyclic calculation on the hidden states of each time step input sequentially, and the output result is further used as the input of the transposed convolutional layer. The dimensions of the input and output data of the frequency-axis gated recurrent unit and the time-axis gated recurrent unit are also not limited here.The Decoder after the temporal-axis gated recurrent unit can be set to include two transposed convolutional layers and a convolutional layer for adjusting the dimension of the output data. The dimensions of the specific input and output data are not restricted here. The Decoder is followed by a multi-task output layer, which can include a shared feature extraction layer and output layers separately set for each piece of information. For example, if it is required to output weather condition information, road surface condition information, and sensor status information, an output layer can be separately set for each such piece of information. Among them, the shared feature extraction layer can include one or more convolutional layers, and a pooling layer can be connected after the convolutional layer, whose function is to extract general features and reduce the spatial dimension. The output layers separately set for each piece of information can include a convolutional layer, a global average pooling layer, and a fully connected layer. An activation function is set after the fully connected layer, which is used to output the confidence levels or probability distributions corresponding to the respective value categories of each piece of information, for determining the final values of each piece of information. The dimensions of the input and output data of the multi-task output layer and the dimensions of the input and output data of each layer therein are also not restricted. In this embodiment, not only the independent processing of each task is ensured, but also the overall performance and resource utilization rate of the model are optimized through the sharing of underlying features and the specialization of top-level decisions.

[0114] In a feasible embodiment, a lightweight, real-time, low-parameter-frequency-time-structured U-Net network can be set as the prediction model, which is specifically designed to accurately process and analyze various frequency-domain and time-domain data, ensuring the effective fusion of sub-band domain and time-domain multi-source information, effectively improving the recognition speed and accuracy of complex acoustic environments and road conditions, and improving the robustness of the model to changes in the vehicle interior and exterior environments. Specifically, the dimensions of the input and output data of the prediction model and the input and output data of each layer in the model can all conform to the definition of [batch size, feature dimension, number of channels]. The dimension of the input data of the first feature extraction module can be set to [1, 65, 11], where the batch size is 1, indicating that a single sample is processed each time, the feature dimension is 65, corresponding to the frequency-domain features extracted by sub-band analysis (i.e., 65 sub-bands), and the number of channels is 11, including 9 vibration sensor channels and 2 error microphone channels. The dimension of the input data of the second feature extraction module can be set to [1, 64, 9], the batch size is 1, the time step is 64, indicating the length of the time-domain signal (i.e., the frame length is 64), and the number of channels is 9, corresponding to 9 vibration sensors. The dimension of the input data of the third feature extraction module is set to [1, 64, 2], the batch size is 1, the time step is 64, and the number of channels is 2, corresponding to 2 error microphones. When training the prediction model, a sequence containing T frames can be input, such as [1, T, 65, 11], so that the FT-GRU can learn the inter-frame relationship; during inference, frames can be input one by one, and the FT-GRU realizes inter-frame modeling through state transfer.

[0115] The encoder includes a first feature extraction module, a second feature extraction module, and a third feature extraction module.

[0116] The first feature extraction module may include:

[0117] The first convolutional layer: the convolutional kernel dimension is 3x3, the number of input channels is 11, the number of output channels is 16, the padding is 1 ("same" padding), and the output dimension is [1, 65, 16].

[0118] The max pooling layer: the pooling window is 2x1, the stride is 2, and the output dimension is [1, 32, 16].

[0119] The second convolutional layer: the convolutional kernel dimension is 3x3, the number of input channels is 16, the number of output channels is 32, the padding is 1 ("same" padding), and the output dimension is [1, 32, 32].

[0120] The max pooling layer: the pooling window is 2x1, the stride is 2, and the output dimension is [1, 16, 32].

[0121] The 1D convolutional layer: 1D convolution, the convolutional kernel dimension is 4x1, the number of input channels is 32, the number of output channels is 32, the stride is 2x1, the padding is "valid", and the output dimension is [1, 29, 32].

[0122] The final output dimension of the first feature extraction module: [1, 29, 32].

[0123] The second feature extraction module includes:

[0124] The one-dimensional convolutional layer: the convolutional kernel dimension is 3, the number of input channels is 9, the number of output channels is 16, the padding is 1 ("same" padding), and the output dimension is [1, 64, 16].

[0125] The pooling layer: the pooling window is 2, the stride is 2, and the output dimension is [1, 32, 16].

[0126] The 1D convolutional layer: 1D convolution, the convolutional kernel dimension is 4, the number of input channels is 16, the number of output channels is 32, the stride is 1, the padding is "valid", and the output dimension is [1, 29, 32].

[0127] The third feature extraction module includes:

[0128] The one-dimensional convolutional layer: the convolutional kernel dimension is 3, the number of input channels is 2, the number of output channels is 32, the padding is 1 ("same" padding), and the output dimension is [1, 64, 32].

[0129] The pooling layer: the pooling window is 2, the stride is 2, and the output dimension is [1, 32, 32].

[0130] 1D Convolutional Layer: 1D convolution, with a convolutional kernel dimension of 4, an input channel number of 32, an output channel number of 32, a stride of 1, padding of "valid", and an output dimension of [1, 29, 32].

[0131] Concatenation Operation in the Feature Fusion Stage: The outputs of the three branches are all [1, 29, 32]. They are concatenated along the channel dimension, that is, 32 + 32 + 32 = 96, and the output dimension is [1, 29, 96].

[0132] The FT-GRU layer follows the Encoder. The input dimension of the FT-GRU layer is [1, 29, 96], where 29 represents the feature dimension (or time step), and 96 represents the number of input feature channels. The FT-GRU layer includes:

[0133] The First Layer: Freq GRU (GRU on the frequency axis), Function: Process data along the frequency axis to capture the correlation in the frequency dimension. The input dimension is [1, 29, 96], where 29 represents the time step and 96 represents the feature channel dimension. Processing Method: For each time step (a total of 29), Freq GRU treats the 96-dimensional feature as an input sequence (length 96) and performs cyclic calculations to output the hidden state. Assuming the number of hidden units is H (e.g., H = 32), the output is [1, 29, H].

[0134] The Second Layer: Time GRU (GRU on the time axis), Function: Process the output of Freq GRU along the time axis to capture the dynamic features in the time dimension. The input dimension is [1, 29, H]. Processing Method: Perform cyclic operations over 29 time steps. At each time step, input H-dimensional features and output the final features. Assuming the target output channel number is 32, the output dimension is [1, 29, 32].

[0135] The Decoder after the FT-GRU layer includes two transposed convolutional layers and a convolutional layer for adjusting the output data dimension.

[0136] The input dimension of the first transposed convolutional layer is [1, 29, 32]. The size of the transposed convolutional kernel is 4x1, the stride is 2, the padding is 1, the output channel number is 32, and the output dimension is [1, 58, 32].

[0137] The input dimension of the second transposed convolutional layer is [1, 58, 32]. The size of the transposed convolutional kernel is 8x1, the stride is 1, the padding is 0, the output channel number is 16, and the output dimension is [1, 65, 16].

[0138] The input dimension of the third convolutional layer is [1, 65, 16]. The size of the convolutional kernel is 1x1, the stride is 1, the output channel number is 1, and the output dimension is [1, 65, 1].

[0139] The decoder is followed by a multi-task output layer, which includes a shared feature extraction layer and multiple output layers (pavement condition classification branch, weather condition classification branch, and sensor status monitoring branch).

[0140] The input dimension of the shared feature extraction layer is [1, 65, 1], and the shared feature extraction layer includes:

[0141] Convolution layer: The convolution kernel size is 3x1, the input channels are 1, the output channels are 32, and the padding is 1 ("same" padding), with the output [1, 65, 32]. Max pooling layer: The pooling window is 2x1, the stride is 2, and the output is [1, 32, 32]. Its function is to extract general features and reduce the spatial dimension.

[0142] The pavement condition classification branch, weather condition classification branch, and sensor status monitoring branch connected after the shared feature extraction layer.

[0143] The input dimension of the pavement condition classification branch is [1, 32, 32], and the output dimension is [1, N], which is the probability distribution of N pavement conditions. The pavement condition classification branch includes:

[0144] Convolution layer: 1x1 convolution, with 64 output channels and an output dimension of [1, 32, 64].

[0145] Global average pooling: Reducing the dimension to [1, 64].

[0146] Fully connected layer: Outputting N (the number of pavement condition categories), with the activation function Softmax.

[0147] The input dimension of the weather condition classification branch is [1, 32, 32], and the output dimension is [1, M], which is the probability distribution of M weather conditions. The weather condition classification branch includes:

[0148] Convolution layer: 1x1 convolution, with 64 output channels and an output dimension of [1, 32, 64].

[0149] Global average pooling: Reducing the dimension to [1, 64].

[0150] Fully connected layer: Outputting M (the number of weather condition categories), with the activation function Softmax.

[0151] The input dimension of the sensor status monitoring branch is [1, 32, 32], and the output dimension is [1, 11]. Each value is between 0 and 1, representing the confidence levels of 11 sensors. The sensor status monitoring branch includes:

[0152] Convolution layer: 1x1 convolution, with 128 output channels and an output dimension of [1, 32, 128].

[0153] Global average pooling: Dimension reduction to [1, 128].

[0154] Fully connected layer: Output 11 (corresponding to 11 sensors), activation function Sigmoid.

[0155] Overall, the results output by the prediction model include:

[0156] Road surface condition classification: [1, N], probability distribution of N road surface conditions.

[0157] Weather condition classification: [1, M], probability distribution of M weather conditions.

[0158] Sensor status monitoring: [1, 11], confidence levels (0 - 1) of 11 sensors.

[0159] Based on the above results output by the prediction model, road surface condition information for characterizing the road surface condition, weather condition information for characterizing the weather condition, and sensor status information for characterizing the sensor status can be obtained. For example, in a feasible implementation manner, the category corresponding to the probability distribution with the largest probability among the probability distributions in the road surface condition classification result can be used as the road surface condition information. Also, for example, it can be pre - set that the magnitude of the confidence level in the sensor status monitoring result represents the degree of sensor abnormality. The sensor status information can be binary classification information indicating whether the sensor is in a normal state or an abnormal state, or information indicating the degree of sensor abnormality, which can be set according to needs in specific implementation manners.

[0160] In a feasible implementation manner, referring to Figure 2 , the prediction model can be trained in the following way in advance.

[0161] Step 1: Data collection. Install accelerometers and microphones with sensitivities and frequency responses meeting the test requirements at the positions on the vehicle where accelerometers and microphones need to be installed to comprehensively capture vibration and sound data under different road conditions and speeds.

[0162] Step 2: Data annotation. During the data recording process, the annotation work can be completed synchronously. All data annotation can be directly carried out by annotators during data recording, and the road surface conditions in the data are annotated in real - time and in detail. Specifically, when the microphone and vibration sensors collect data inside and outside the vehicle, the annotator immediately classifies and describes the current road surface condition. For example, these descriptions can include, but are not limited to: road type (such as highway, urban road, rural road), lane markings, road surface texture, pothole conditions, obstacle distribution, and road slipperiness. The annotator will also record (such as rainy or snowy days) and annotate the weather conditions affecting the road surface condition.

[0163] Step 3: Data enhancement. To improve the model's ability to identify abnormal situations, sensor failures and noise can be artificially introduced. First, the robustness of the model is enhanced by simulating various sensor failures and environmental interference. For example, cover the microphone with cloth, plastic, or metal sheets to simulate dust and water blockage, and tap the accelerometer to simulate abnormal shock. In addition, digital signal processing technology can also be used to add various synthetic noises to normal data, such as white noise and natural environmental noise, as well as signals simulating electromagnetic interference. When introducing these failures and interferences, the data can be systematically segmented and categorized by time, such as "microphone occlusion-cloth" or "accelerometer impact interference" to ensure the richness of the data set and the generalization ability of the training model. These refined operations not only increase the model's adaptability to complex real-world situations, but also help maintain the performance and stability of the system in various environments.

[0164] Step 4: Dataset balance. To solve the problem of uneven distribution of collected data, data analysis can be performed to identify which categories or scenarios have too little or too much data, and then data enhancement techniques can be used to increase the number of samples in scarce categories, such as by changing the speed, adding noise, or using synthetic techniques to generate new data. For excessive categories, a downsampling strategy can be used to selectively reduce samples, or representative samples can be selected through clustering analysis. In addition, resampling techniques can also be used to balance the entire dataset to ensure that the data of each category is evenly distributed in the training set, thereby improving the generalization ability and performance of the model under all conditions. This step can also include applying statistical methods to evaluate changes before and after data balancing to verify the effectiveness of the balancing strategy and adjust the balancing parameters as needed.

[0165] Step 5: Model architecture setting: For example, the model architecture in the above implementation can be set.

[0166] Step 6: Model training. Model training can use customized loss functions to cope with different classification tasks. Cross entropy loss can be used for multi-class classification tasks (such as road conditions and weather conditions) to ensure that the model can effectively distinguish different categories; for the binary classification problem of sensor status, binary cross entropy loss can be used to improve the sensitivity of identifying normal and abnormal states. During the training process, you can choose the Adam optimizer (Adaptive Moment Estimation), whose adaptive learning rate mechanism helps to better deal with the problem of gradient sparsity. You can also consider using SGD (stochastic gradient descent) with momentum to stabilize the training process. In addition, early stopping mechanisms and regularization techniques can also be used to prevent overfitting and ensure that the model performs well on unseen data.

[0167] Step 7: Verification and testing. To ensure the effectiveness and reliability of the model in practical applications, the dataset can be divided into a training set, a validation set, and an independent test set. During the model development process, the validation set is used for periodic evaluation to monitor the training progress and adjust the hyperparameters. The test set is used for the final evaluation after the model training is completed to test the performance of the model on completely unknown data. Various performance metrics, such as accuracy, precision, recall, and F1-score, are applied to evaluate the classification task, while the ROC (receiver operating characteristic) curve and AUC (Area Under Curve) value are used to measure the performance of the model for sensor state anomaly detection.

[0168] Based on the above first and / or second embodiments, a third embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first and second embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, a load balancing strategy is proposed, so that the automotive road noise processing algorithm proposed in the above embodiments can be applied to the automotive RNC system with limited computing power conditions. For example, there are a large number of computational peak demands within a single sampling period in a MIMO vehicle RNC system. If the subband adaptive algorithm without output delay is run in real time on an embedded platform such as a DSP (Digital Signal Processor), the system may not be able to run in real time due to insufficient peak computing power. The load balancing strategy proposed in this embodiment aims to ensure that the subband adaptive algorithm without output delay can be run in real time on an embedded platform such as a DSP, solving the problem of insufficient peak computing power and providing an important guarantee for practical applications. Specifically, in this embodiment, the automotive road noise processing method further includes step S70:

[0169] Execute the tasks allocated to the N interrupts among the N interrupts in the (l + 1)-th frame, where N represents the frame length, and the interrupt refers to the period from generating the anti-noise signal based on the time-domain reference signal of each sampling point to receiving the time-domain reference signal of the next sampling point. The tasks allocated to the N interrupts include: the task of calculating the updated time-domain adaptive filter coefficients, and / or the task of inputting the first time-domain reference signal, the first time-domain error signal, the first subband reference signal, and the first subband error signal into the prediction model to predict the environmental state information.

[0170] In this embodiment, the control unit adopts a method without output delay for road noise control, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit point by point in real time. The control unit generates a single-sampling-point anti-noise signal for the currently input single-sampling-point time-domain reference signal in real time according to the current time-domain adaptive filter coefficients, and then outputs the anti-noise signal in real time to achieve the effect of no output delay. On the other hand, the adaptive filter coefficients are updated frame by frame. Specifically, the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffers in real time. After each buffer is filled with one frame of time-domain reference signal and one frame of time-domain error signal, sub-band decomposition is performed on the frame of time-domain reference signal and the frame of time-domain error signal, as well as subsequent time-domain adaptive filter coefficient update operations.

[0171] Let the frame index of the first time-domain reference signal and the first time-domain error signal be l. Then, after updating the time-domain adaptive filter coefficients based on the l-th frame of time-domain reference signal and time-domain error signal, the updated time-domain adaptive filter coefficients after this update will be used to process each point of the time-domain reference signal of N sampling points in the (l + 1)-th frame (i.e., the second time-domain reference signal) to generate an anti-noise signal of N sampling points. Specifically, for each received sampling point of the time-domain reference signal, an anti-noise signal of one sampling point is generated and output according to the updated time-domain adaptive filter coefficients after this update. Then, it can be understood that during the process of processing each point of the time-domain reference signal of N sampling points in the (l + 1)-th frame, it is also necessary to complete the update of the time-domain adaptive filter coefficients based on the time-domain reference signal and time-domain error signal in the (l + 1)-th frame, so as to process the time-domain reference signal of each sampling point in the (l + 2)-th frame to generate an anti-noise signal, and so on.

[0172] It can be understood that within the time period from the start of receiving the first sampling point of the (l + 1)-th frame to the generation of the anti-noise signal by processing the N-th sampling point of the (l + 1)-th frame, it is necessary to complete an update operation of the time-domain adaptive filter coefficients based on the l-th frame of time-domain reference signal and time-domain error signal. In this embodiment, the update task of the adaptive filter coefficients can be distributed to N interrupts for execution, avoiding the execution of the entire adaptive filter coefficient update task in a single interrupt, thereby avoiding the problem of insufficient computing power caused by a sharp increase in computing power.

[0173] During the process of updating the time-domain adaptive filtering coefficients based on the l-th frame time-domain reference signal and the time-domain error signal, the l-th frame time-domain reference signal, the time-domain error signal, and the corresponding subband signals can be input into the prediction model to predict the environmental state information and / or the sensor state information. The filter bank determined based on the environmental state information and / or the sensor state information is used to perform subband decomposition operations on the (l + 2)-th frame time-domain reference signal and the time-domain error signal. Then, within the time period from the start of receiving the first sampling point of the (l + 1)-th frame to the processing of the n-th sampling point of the (l + 1)-th frame to generate the noise cancellation signal, it is also necessary to complete the task of inputting the l-th frame time-domain reference signal, the time-domain error signal, and the corresponding subband reference signals and subband error signals into the prediction model to predict the environmental state information and / or the sensor state information. In this embodiment, the prediction task can be distributed to N interrupts for execution, avoiding the execution of the entire prediction task in a single interrupt, thereby avoiding the problem of insufficient computing power caused by a sharp increase in computing power.

[0174] The load balancing strategy proposed in this embodiment ensures the full utilization of each interrupt, avoids the situation of any single interrupt being overloaded, optimizes the use of computing resources, and guarantees the high-efficiency performance and response speed of the system in real-time processing. This method of distributing computing tasks to multiple interrupts helps the system maintain stable operation in the face of high computing demands, thereby achieving continuous and efficient operation in various driving environments.

[0175] In this embodiment, there is no limitation on the way of distributing the task to each interrupt. For example, the computing operations in the task can be divided into multiple subtasks according to the amount of computation, and the amount of computation of each subtask is the same or approximately the same, and each interrupt processes one subtask.

[0176] In a feasible implementation manner, the task of updating the adaptive filter coefficients can be divided into a subband decomposition task, a subband synthesis task, and multiple subband update tasks. Each subband update task includes the task of updating at least one subband adaptive filter coefficient. The subband decomposition task is executed in the first preset number of interrupts, the subband synthesis task is executed in the subsequent preset second number of interrupts, and one of the subband update tasks is executed in each of the intermediate preset third number of interrupts. The sum of the preset first number, the preset second number, and the preset third number is N.

[0177] In a feasible implementation manner, the prediction task can be divided into multiple subtasks, each subtask including the processing task of at least one layer in the prediction model, and one of the subtasks is executed in each interrupt. For example, let N = 64. The first 32 interrupts process the operations of the Encoder layer, including convolution, activation, pooling, and the FGRU layer. The latter half of the interrupts complete the operations of the TGRU and Decoder layers, and the last interrupt completes the operation of the multi-task output layer.

[0178] It should be noted that when both the adaptive filter coefficient update task and the prediction task need to be distributed to N interrupt processes, since the subband signals are required in the prediction task, the task of subband decomposition can be assigned to the earlier interrupts, and the prediction task can be assigned to the later interrupts. For example, each subtask of the prediction task can be assigned to be executed in each interrupt except for the first predetermined number of interrupts at the front.

[0179] Based on the above first, second, and / or third embodiments, a fourth embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first, second, and third embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, the update step size of the adaptive filter coefficient used in the adaptive algorithm is dynamically adjusted to balance the adaptive convergence speed and system stability, so as to improve the system's adaptive ability and stability and avoid problems such as delayed response or unstable output. Specifically, the control unit can obtain influence factor information and dynamically determine the target update step size corresponding to the subband adaptive filter coefficient according to the influence factor information, where the influence factor information includes environmental state information and / or sensor state information, the environmental state information is information characterizing the environmental state inside and / or outside the vehicle, the sensor state information is information characterizing the working state of the sensor, and the sensor includes a sensor for collecting the time-domain reference signal and / or a sensor for collecting the time-domain error signal.

[0180] It should be noted that the update step size can be determined according to the environmental state information used to determine the synthesis filter bank.

[0181] The influence factor information is information characterizing the specific manifestation of the influence factor, and the influence factor is a factor determining the adjustment strategy of the update step size of the subband adaptive filter coefficient, that is, the specific manifestation of these factors determines different adjustment strategies of the update step size. In this embodiment, the influence factor may at least include one or more of the environmental state inside the vehicle, the environmental state outside the vehicle, the working state of the reference sensor, and the working state of the error sensor. In other embodiments, other influence factors may also be included, such as vehicle speed.

[0182] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause dynamic changes in the acoustic propagation path, which will affect the energy magnitude or stability of the vehicle road noise, and the effects on the energy magnitude or stability of different sub-bands of noise are also different. If the update step size of the sub-band adaptive filter coefficients adopts a fixed step size, when the energy magnitude or stability of the vehicle road noise changes, it may lead to misjudgment of the system or poor acoustic effects due to too large an update step size and too fast update of the sub-band adaptive filter coefficients, or it may also lead to a slow convergence rate of the adaptive filter and poor noise reduction effect due to too small an update step size. Therefore, in the specific implementation, the environmental state outside the vehicle and the environmental state inside the vehicle can be used as influencing factors, and the update step size can be dynamically adjusted based on the environmental state to balance the adaptive convergence rate and system stability, and avoid delayed response or unstable output. When the working states of the reference sensor and the error sensor change, the accuracy of the collected signal will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the estimation of the vehicle road noise in the signal will be incorrect. If the update step size of the sub-band adaptive filter coefficients adopts a fixed step size and the adaptive filter coefficients are still continuously updated, it will cause the generated anti-noise signal to be unable to accurately cancel the real vehicle road noise, resulting in unstable noise reduction effect. Therefore, in the specific implementation, the working states of the reference sensor and the error sensor can be used as influencing factors, and the update step size can be dynamically adjusted based on the working states of the sensors to balance the adaptive convergence rate and system stability, and avoid delayed response or unstable output.

[0183] In this embodiment, the acquisition method of the influencing factor information is not limited. For example, the environmental state information can be obtained by analyzing the environmental images collected by the camera, and the sensor state information can be determined by analyzing the outliers of the signals collected by the sensors.

[0184] The influencing factor information can be obtained every once in a while. The shorter the time interval, the higher the real-time degree. In this embodiment, the acquisition frequency is not limited. For example, in a feasible implementation, the influencing factor information can be obtained every time the adaptive filter coefficients are updated.

[0185] Dynamically determining the update step size of the subband adaptive filter coefficients means that after each new influence factor information is obtained, the update step size is determined according to the new influence factor information, so that the update step size of the adaptive filter coefficients changes according to the specific performance of the influence factors, and the update speed of the adaptive filter coefficients is adjusted by dynamically adjusting the update step size, so as to adapt to the changes in the internal or external environment state of the vehicle, or to adapt to the changes in the working state of the sensor, balance the adaptive convergence speed and system stability, and avoid problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensure the robustness and safety of the system, and avoid delayed response or unstable output.

[0186] The specific method for determining the update step size of the subband adaptive filter coefficients according to the influence factor information is not limited in this embodiment. For example, the mapping relationship between the influence factor information and the update step size of the subband adaptive filter coefficients can be set in advance as needed. After the influence factor information is obtained, the control unit uses the update step size corresponding to the influence factor information as the target update step size according to this mapping relationship. In this embodiment, this mapping relationship is not limited. For example, this mapping relationship can be represented by a mapping table, a relational expression, etc., and this mapping relationship can be determined by experiments, data statistics or other methods.

[0187] It should be noted that since there are multiple subbands, there are multiple subband adaptive filters, and the target update step sizes of the coefficients of each subband adaptive filter may be the same or different, which is not limited in this embodiment.

[0188] In this embodiment, by dynamically determining the update step size of the subband adaptive filter coefficients according to the obtained environmental state information and / or sensor state information, updating the subband adaptive filter coefficients according to the determined update step size, and then converting the subband adaptive filter coefficients into time-domain adaptive filter coefficients, using the time-domain adaptive filter coefficients to process the time-domain reference signal to generate an anti-noise signal, so as to control the road noise based on the anti-noise signal, adapt to the changes in the internal or external environment state of the vehicle, or adapt to the changes in the working state of the sensor, balance the adaptive convergence speed and system stability, and avoid problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensure the robustness and safety of the system, and avoid delayed response or unstable output.

[0189] In a feasible implementation manner, to improve the noise reduction effect of the system and ensure the noise reduction stability, it is set that when the control unit dynamically determines the target update step size according to the influence factor information, it can conform to one or more of the following rules:

[0190] 1. Under the environmental state characterized by the environmental state information, the target update step corresponding to the sub-band with higher automotive road noise energy or higher stability degree is larger than the target update step corresponding to the sub-band with lower automotive road noise energy or lower stability degree.

[0191] Under the same environmental state, the distribution of automotive road noise on each sub-band may be different; for the sub-band with higher noise energy, a larger update step can be adopted to specifically enhance the noise reduction intensity of the sub-band, improve the noise reduction effect, and avoid delayed response; for the sub-band with lower noise energy, a smaller update step can be adopted to ensure the stability of the noise reduction effect and avoid unstable output. Under the same environmental state, the stability degree of automotive road noise on each sub-band may be different; for the sub-band with higher stability degree, a larger update step can be adopted to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response; for the sub-band with lower stability degree, a smaller update step can be adopted to avoid misjudgment of the system or generation of bad acoustic effects, and ensure the robustness and safety of the system.

[0192] It should be noted that the stability degree of automotive road noise refers to the stability degree of the change in noise energy.

[0193] 2. For any target sub-band, under the environmental state characterized by the environmental state information, the higher the energy or the higher the stability degree of the automotive road noise in the target sub-band, the larger the target update step corresponding to the target sub-band; the lower the energy or the lower the stability degree of the automotive road noise in the target sub-band, the smaller the target update step corresponding to the target sub-band.

[0194] For the same sub-band, the energy of vehicle road noise in this sub-band may be different under different environmental conditions. Therefore, for a certain sub-band (referred to as the target sub-band for distinction), if the energy of vehicle road noise in the target sub-band is high under a certain environmental condition, a larger update step size can be adopted for the target sub-band to specifically enhance the noise reduction intensity of this sub-band, improve the noise reduction effect, and avoid delayed response. If the energy of vehicle road noise in the target sub-band is low under a certain environmental condition, a smaller update step size can be adopted for the target sub-band to ensure the stability of the noise reduction effect. For the same sub-band, the stability of vehicle road noise in this sub-band may be different under different environmental conditions. Therefore, for a certain sub-band (referred to as the target sub-band for distinction), if the stability of vehicle road noise in the target sub-band is high under a certain environmental condition, a larger update step size can be adopted for the target sub-band to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response. If the energy of vehicle road noise in the target sub-band is low under a certain environmental condition, a smaller update step size can be adopted for the target sub-band to avoid misjudgment of the system or generation of adverse acoustic effects, and ensure the robustness and safety of the system.

[0195] 3. According to the abnormal degrees of each sensor characterized by the sensor status information, the target update step size corresponding to the sub-band where the sensor with a higher abnormal degree is located is smaller than the target update step size corresponding to the sub-band where the sensor with a lower abnormal degree is located.

[0196] The sub-band where the sensor is located refers to the sub-band in which the signal collected by the sensor is distributed or mainly distributed, and the sub-bands where different sensors are located may be different.

[0197] The sensor status information is used to characterize the working state of the sensor. In the specific implementation manner, the working state of the sensor can be divided into two types, namely normal operation and abnormal operation. The sensor status information can be the information used to indicate whether the sensor belongs to the normal working state or the abnormal working state. Then it can be understood that the abnormal degree of the sensor with the working state of "normal operation" is lower than that of the sensor with the working state of "abnormal operation"; or, the working state of the sensor can also be divided into more than two types. For example, numerical values within 0-1 are used to represent different abnormal degrees, and the sensor status information can be the numerical value indicating the abnormal degree of the sensor.

[0198] Based on the obtained sensor status information, the abnormality degree of each sensor can be determined, and the abnormality degrees of the sensors may be different; for the sensor with a higher abnormality degree, a smaller update step size can be adopted for the subband where the sensor is located to avoid misjudgment of the system and generate poor acoustic effects. For the sensor with a lower abnormality degree, a larger update step size can be adopted for the subband where the sensor is located to ensure the noise reduction effect of the system and avoid delayed response.

[0199] 4. For any target subband, the higher the abnormality degree of the sensor belonging to the target subband, the smaller the target update step size corresponding to the target subband; the lower the abnormality degree of the sensor belonging to the target subband, the larger the target update step size corresponding to the target subband.

[0200] For a certain subband (hereinafter referred to as the target subband for distinction), the sensors belonging to the target subband refer to the sensors whose signals are all or mainly distributed in this subband.

[0201] For the target subband, the obtained sensor status information is different, and the abnormality degrees of the sensors belonging to the target subband may be different; if the abnormality degree of the sensors belonging to the target subband is higher, a smaller update step size can be adopted for the target subband to avoid misjudgment of the system and generate poor acoustic effects; if the abnormality degree of the sensors belonging to the target subband is lower, a larger update step size can be adopted for the target subband to ensure the noise reduction effect of the system and avoid delayed response.

[0202] In a feasible implementation manner, the influencing factor information may further include the real-time vehicle speed of the vehicle. When the vehicle speed changes, it will affect the energy magnitude or stability of the vehicle road noise. When the vehicle speed changes, if a fixed update step size is adopted for the update step size of the subband adaptive filter coefficient, it may cause misjudgment or generate poor acoustic effects when the energy magnitude or stability of the vehicle road noise changes, because the update step size may be too large and the subband adaptive filter coefficient updates too fast. Therefore, the vehicle speed can be used as an influencing factor.

[0203] In a feasible implementation manner, when the influencing factor information further includes the vehicle speed, when dynamically determining the target update step size according to the influencing factor information, it may also conform to one or more of the following rules:

[0204] 5. The higher the vehicle speed stability, the larger the target update step size corresponding to each subband; the lower the vehicle speed stability, the smaller the target update step size corresponding to each subband.

[0205] The vehicle speed stability refers, for example, to the change rate of the vehicle speed. The larger the change rate, the lower the vehicle speed stability; the smaller the change rate, the higher the vehicle speed stability.

[0206] The higher the vehicle speed stability, the higher the stability of the vehicle road noise. At this time, a larger update step size can be adopted to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response. The lower the vehicle speed stability, the lower the stability of the vehicle road noise. At this time, a smaller update step size can be adopted to avoid misjudgment of the system or generation of adverse acoustic effects, and ensure the robustness and safety of the system.

[0207] 6. When the vehicle speed is outside the preset range, the target update step size corresponding to each sub-band is smaller than the target update step size corresponding to each sub-band when the vehicle speed is within the preset range.

[0208] The preset range indicates that the vehicle speed is at a normal level. When the vehicle speed is outside the preset range, it indicates a relatively extreme or abnormal situation, and the vehicle road noise may have relatively extreme or abnormal manifestations. For such extreme or abnormal situations, a smaller update step size can be adopted to avoid misjudgment or generation of adverse acoustic effects.

[0209] In the specific implementation manner, in order to make the control unit conform to one or more of the above rules when dynamically determining the target update step size according to the influence factor information, a mapping relationship that satisfies one or more of the above rules can be set. The control unit determines the update step size corresponding to the influence factor information as the target update step size according to this mapping relationship. The mapping relationship can be represented by a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple above rules, the multiple rules can conform to a certain priority order to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the update step size according to the sensor status information can be higher than the priority of determining the update step size according to the environmental status information.

[0210] In a feasible implementation manner, the environmental status information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to the weather condition that affects the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information characterizing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0211] In a feasible implementation, when the environmental state information and the sensor state information indicate that the environment is good and stable (e.g., good road conditions, clear weather, normal sensors, medium vehicle speed), the vehicle road noise is relatively stable, and there will be no particularly abnormal and prominent noise in each sub-band. At this time, a moderate or larger update step size can be adopted to ensure the adaptive convergence speed of the system and a low steady-state error level. When the environmental state information and the sensor state information indicate that the environment deteriorates or becomes unstable (e.g., deteriorating road conditions, bad weather, abnormal sensors, drastic vehicle speed changes), the stability of the vehicle road noise may deteriorate. At this time, a smaller update step size can be adopted to improve the system stability. When the environmental state information and the sensor state information indicate the existence of a specific type of noise (e.g., waterlogged road surface, brick road surface, wind noise), the step sizes of different frequency sub-bands can be adjusted differentially. A larger update step size is adopted for the sub-band with high noise energy to specifically enhance or weaken the filtering intensity of a specific frequency band and optimize the noise reduction effect. When the environmental state information and the sensor state information indicate an extremely harsh environment or system anomaly (e.g., extremely harsh road conditions, serious sensor failure, extremely low vehicle speed), the update step size can be significantly reduced or even frozen to avoid system misjudgment or adverse acoustic effects and ensure the robustness and safety of the system.

[0212] Table 2 below gives some examples of determining the update step sizes for the low-frequency, medium-frequency, and high-frequency bands based on the weather condition information, road surface condition information, sensor state information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Among them, the low-frequency, medium-frequency, and high-frequency can be divided as needed. It can be understood that each of the three frequency bands contains one or more sub-bands. Table 2 only gives some examples and does not limit the way of determining the target update step size in this embodiment. The content in parentheses in Table 2 is the explanatory description of the weather condition information, road surface condition information, sensor state information, update step size, etc.

[0213] Table 2

[0214]

[0215] Based on the above first, second, third, and / or fourth embodiments, the fifth embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first, second, third, and / or fourth embodiments can be referred to the above introduction and will not be elaborated hereinafter. In this embodiment, the target clipping parameter value can also be dynamically determined according to the acquired environmental state information and / or sensor state information. After generating the anti-noise signal, the target clipping parameter value is used to limit the amplitude of the anti-noise signal, and the anti-noise signal after amplitude limitation is output to control the road noise. That is, post-processing operations can be performed on the anti-noise signal. The post-processing operations include amplitude limitation of the anti-noise signal, and the amplitude limitation is performed according to the target clipping parameter value determined according to the environmental state information and / or sensor state information. The clipping parameter value is a parameter value used to limit the degree of amplitude limitation. What specific parameter it is is not limited in this embodiment. For example, it can be an upper amplitude value, indicating that the amplitude value of the finally output signal is not allowed to exceed this upper amplitude value.

[0216] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause dynamic changes in the acoustic propagation path, which will affect the stability of the vehicle road noise. If a fixed clipping parameter value is used for amplitude limitation of the anti-noise signal, when the stability of the vehicle road noise changes, it may be impossible to fully cancel the noise due to excessive amplitude limitation, or it may be impossible to avoid introducing noise or distortion due to misjudgment or overcompensation due to too small amplitude limitation. Therefore, in the specific implementation, the clipping parameter value can be dynamically adjusted according to the environmental state information, so as to dynamically control the output amplitude of the anti-noise signal, adapt to different environmental states, ensure good noise reduction effect, and avoid introducing noise or distortion due to misjudgment or overcompensation, and ensure system stability and safety.

[0217] When the working state of the reference sensor and the working state of the error sensor change, the accuracy of the collected signal will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the vehicle road noise in the signal will be estimated incorrectly, and the generated anti-noise signal may not only fail to cancel the noise but also introduce noise. Therefore, in the specific implementation, the limit clipping parameter value can be dynamically adjusted according to the sensor state information, so as to dynamically control the output amplitude of the anti-noise signal. For example, when the sensor is abnormal, the output amplitude can be reduced, so as to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0218] The specific method for determining the target amplitude limiting parameter value according to the environmental state information and / or the sensor state information is not limited in this embodiment. For example, the mapping relationship between the environmental state information and / or the sensor state information and the amplitude limiting parameter value can be set in advance as needed. After obtaining the environmental state information and / or the sensor state information, the control unit uses the mapping relationship to take the amplitude limiting parameter value corresponding to the environmental state information and / or the sensor state information as the target amplitude limiting parameter value. In this embodiment, the mapping relationship is not limited. For example, the mapping relationship can be represented in the form of a mapping table, a relational expression, etc., and the mapping relationship can be determined through experiments, data statistics, or other methods.

[0219] In a feasible implementation manner, it can be set that when the control unit determines the target amplitude limiting parameter value according to the influence factor information, it conforms to one or more of the following rules:

[0220] 1. The higher the stability degree of the vehicle road noise in the environmental state represented by the environmental state information, the smaller the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter value; the lower the stability degree of the vehicle road noise in the environmental state represented by the environmental state information, the greater the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter.

[0221] If the stability degree of the vehicle road noise in the environmental state represented by the environmental state information is relatively high, a smaller amplitude limiting parameter value for the anti-noise signal can be adopted, that is, an anti-noise signal with a larger output amplitude is allowed, so as to more fully cancel the noise and ensure a good noise reduction effect. If the stability degree of the vehicle road noise in the environmental state represented by the environmental state information is relatively low, a larger amplitude limiting parameter value for the anti-noise signal can be adopted, that is, the amplitude of the output signal is more restricted, so as to avoid introducing noise or distortion due to system misjudgment or overcompensation and ensure system stability.

[0222] 2. The higher the degree of abnormality of the sensor represented by the sensor state information, the greater the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter; the lower the degree of abnormality of the sensor represented by the sensor state information, the smaller the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter.

[0223] In the case of higher sensor abnormality, the degree of amplitude limitation of the anti-noise signal is greater, so as to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0224] In a feasible implementation manner, on the basis of the environmental state information and / or the sensor state information, the target amplitude limiting parameter value can also be determined according to the vehicle speed. When determining the target amplitude limiting parameter value according to the vehicle speed, it conforms to the following rule:

[0225] 3. The degree of amplitude limitation of the anti-noise signal according to the target amplitude limitation parameter is smaller when the vehicle speed stability is higher, and the degree of amplitude limitation of the anti-noise signal according to the target amplitude limitation parameter is larger when the vehicle speed stability is lower.

[0226] The vehicle speed stability refers, for example, to the change rate of the vehicle speed. The higher the change rate, the lower the vehicle speed stability, and the lower the change rate, the higher the vehicle speed stability.

[0227] When the vehicle speed stability is relatively high, the stability of the vehicle road noise is relatively high. At this time, a limit parameter value with a relatively small degree of amplitude limitation of the anti-noise signal can be adopted, that is, an anti-noise signal with a larger allowable output amplitude is allowed, so as to more fully cancel the noise and ensure a good noise reduction effect. When the vehicle speed stability is relatively low, the stability of the vehicle road noise is relatively low. At this time, a limit parameter value with a relatively large degree of amplitude limitation of the anti-noise signal can be adopted, that is, the amplitude of the output signal is limited to a greater extent, so as to avoid introducing noise or distortion due to system misjudgment or overcompensation and ensure system stability.

[0228] In the specific implementation manner, in order to make the control unit conform to the laws of one or more of the above when determining the target amplitude limitation parameter value, a mapping relationship that satisfies the laws of one or more of the above can be set. The control unit determines the amplitude limitation parameter value corresponding to the environmental state information, sensor state information, and vehicle speed as the target amplitude limitation parameter value according to this mapping relationship. The mapping relationship can be represented by means of a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple laws of the above, the multiple laws can conform to a certain priority order to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the amplitude limitation parameter value according to the sensor state information can be higher than the priority of determining the amplitude limitation parameter value according to the amplitude limitation parameter value.

[0229] In a feasible implementation, when the environmental status information and sensor status information indicate that the environment is good and stable (e.g., good road conditions, clear weather, normal sensors, medium vehicle speed), the vehicle road noise is relatively stable. At this time, a limiter parameter value with a relatively small amplitude limit degree for the anti-noise signal can be adopted, allowing a larger amplitude anti-noise signal to be output, so as to more fully cancel the noise and ensure a good noise reduction effect. When the environmental status information and sensor status information indicate that the environment deteriorates or becomes unstable (e.g., deteriorating road conditions, bad weather, abnormal sensors, drastic vehicle speed changes), the stability of the vehicle road noise may deteriorate. At this time, a limiter parameter value with a relatively large amplitude limit degree for the anti-noise signal can be adopted, that is, the amplitude of the output signal is limited to a greater extent, avoiding introducing noise or distortion due to system misjudgment or overcompensation and ensuring system stability. When the environmental status information and sensor status information indicate an extremely harsh environment or system abnormality (e.g., extremely harsh road conditions, serious sensor failure, extremely low speed), the amplitude of the output signal can be greatly limited to give priority to ensuring system safety.

[0230] The following Table 3 gives some examples of determining the output limit based on the weather condition information, road surface condition information, sensor status information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Table 3 only gives some examples and does not limit the method for determining the target limiter parameter value in this embodiment. The content in parentheses in Table 3 is an explanatory note for the weather condition information, road surface condition information, sensor status information, output limit, etc.

[0231] Table 3

[0232]

[0233] Based on the above first, second, third, fourth, and / or fifth embodiments, a sixth embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first, second, third, fourth, and / or fifth embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, the pre-weighting weights corresponding to each sensor can also be determined according to the obtained environmental status information and / or sensor status information. After the reference signal and the error signal are weighted using the pre-weighting weights corresponding to their respective sensors, they are then used to generate the anti-noise signal and update the coefficients of the adaptive filter. That is, a preprocessing operation is performed on the input reference signal and error signal, and the preprocessing operation includes amplitude adjustment, and the amplitude adjustment is performed according to the pre-weighting weights corresponding to each sensor determined according to the environmental status information and / or sensor status information.

[0234] It should be noted that when the environmental state outside or inside the vehicle changes, it will cause dynamic changes in the acoustic propagation path, which will affect the energy level of the vehicle road noise. The impact on the energy levels of vehicle road noise in different frequency bands may also be different. If a fixed amplitude adjustment strategy is adopted for the reference signal and the error signal, when the energy level of the vehicle road noise changes, it may be impossible to generate a more targeted anti-noise signal due to the weak noise signals in the reference signal and the error signal, resulting in poor noise reduction effect. Therefore, in the specific implementation, the pre-weighting weights of each sensor can be dynamically adjusted according to the environmental state information. Since the frequency bands of the signals collected by different sensors may be different, by adjusting the pre-weighting weights of the sensors, the signals in each frequency band can be enhanced or weakened specifically. For example, the sensors corresponding to the sub-bands with stronger vehicle road noise energy can be set with higher pre-weighting weights, so as to generate a more targeted anti-noise signal and improve the noise reduction effect.

[0235] When the working state of the reference sensor and the working state of the error sensor change, the accuracy of the collected signal will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the vehicle road noise in the signal will be misestimated, and the generated anti-noise signal may not only fail to cancel the noise but also introduce noise. Therefore, in the specific implementation, the pre-weighting weights of each sensor can be dynamically adjusted according to the sensor state information. For example, the pre-weighting weight of the abnormal sensor can be reduced, so as to avoid introducing noise or distortion due to misjudgment or overcompensation, and ensure the safety and robustness of the system.

[0236] The specific method for determining the pre-weighting weights according to the environmental state information and / or the sensor state information is not limited in this embodiment. For example, the mapping relationship between the environmental state information and / or the sensor state information and the pre-weighting weights can be set in advance. After obtaining the environmental state information and / or the sensor state information, the control unit uses this mapping relationship to take the pre-weighting weights corresponding to the environmental state information and / or the sensor state information as the target pre-weighting weights. In this embodiment, the mapping relationship is not limited. For example, the mapping relationship can be represented by a mapping table, a relational expression, etc., and the mapping relationship can be determined through experiments, data statistics or other methods.

[0237] In a feasible implementation, it can be set that when the control unit determines the pre-weighting weights according to the environmental state information and / or the sensor state information, it conforms to one or more of the following rules:

[0238] 1. Under the environmental state characterized by the environmental state information, the pre-weighting weight of the sensor corresponding to the sub-band with higher energy of the vehicle road noise is greater than that of the sensor corresponding to the sub-band with lower energy of the vehicle road noise.

[0239] The sensor corresponding to the sub-band refers to the sensor whose signal is all or mainly distributed in this sub-band.

[0240] Under the same environmental state, the distribution of vehicle road noise in each sub-band may be different; for the sensor corresponding to the sub-band with higher noise energy, a larger pre-weighting weight can be adopted to specifically enhance the noise reduction intensity of this sub-band and improve the noise reduction effect; for the sub-band with lower noise energy, a smaller pre-weighting weight can be adopted to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0241] 2. For any target sub-band, under the environmental state characterized by the environmental state information, the higher the energy of the vehicle road noise in the target sub-band, the greater the pre-weighting weight of the sensor corresponding to the target sub-band, and the lower the energy of the vehicle road noise in the target sub-band under the environmental state characterized by the environmental state information, the smaller the pre-weighting weight of the sensor corresponding to the target sub-band.

[0242] For the same sub-band, under different environmental states, the energy magnitude of the vehicle road noise in this sub-band may be different; so for a certain sub-band (referred to as the target sub-band for distinction), if the energy of the vehicle road noise in the target sub-band is relatively high under a certain environmental state, a larger pre-weighting weight can be adopted for the sensor corresponding to the target sub-band to specifically enhance the noise reduction intensity of this sub-band and improve the noise reduction effect; if the energy of the vehicle road noise in the target sub-band is relatively low under a certain environmental state, a smaller pre-weighting weight can be adopted for the sensor corresponding to the target sub-band to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0243] 3. According to the abnormality degree of each sensor characterized by the sensor state information, the higher the abnormality degree of the sensor, the lower the corresponding pre-weighting weight, and the lower the abnormality degree of the sensor, the higher the corresponding pre-weighting weight.

[0244] According to the obtained sensor state information, the abnormality degree of each sensor can be determined, and the abnormality degrees of each sensor may be different; for the sensor with a higher abnormality degree among them, a smaller pre-weighting weight can be adopted to avoid introducing noise or distortion due to misjudgment or overcompensation, and for the sensor with a lower abnormality degree among them, a larger pre-weighting weight can be adopted to ensure the noise reduction effect of the system.

[0245] In a specific embodiment, in order to enable the control unit to conform to one or more of the above rules when determining the pre-weighting weights of each sensor, a mapping relationship that meets one or more of the above rules can be set. The control unit determines the pre-weighting weight corresponding to the environmental state information and / or sensor state information as the target pre-weighting weight according to this mapping relationship. The mapping relationship can be represented by means of a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple above rules, there can be a certain priority order among the multiple rules to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the pre-weighting weight according to the sensor state information can be higher than that of determining the pre-weighting weight according to the environmental state information.

[0246] In a feasible embodiment, the environmental state information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to the weather condition that affects the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information characterizing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0247] In a feasible embodiment, three frequency bands, namely low frequency, medium frequency, and high frequency, can be divided; when the environmental state information indicates that the low-frequency noise energy is high (such as in wet and waterlogged weather), a higher pre-weighting weight can be adopted for the signals collected by the low-frequency sensors (sensors whose collected signals are mainly distributed in the low-frequency range). For example, the pre-weighting of the signals collected by the low-frequency sensors is increased by 20%, and the remaining sensor channels maintain a 1.0-fold reference; when the environmental state information indicates that the medium-frequency noise energy is high (such as in brick road surface and dry weather), a higher pre-weighting weight can be adopted for the signals collected by the medium-frequency sensors (sensors whose collected signals are mainly distributed in the medium-frequency range). For example, the pre-weighting of the signals collected by the medium-frequency sensors is increased by 20%, and the remaining sensor channels maintain a 1.0-fold reference.

[0248] The following Table 4 gives some examples of determining the pre-weighting strategy based on the weather condition information, road surface condition information, sensor state information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Table 4 only gives some examples and does not limit the way of determining the pre-weighting weight in this embodiment. The content in parentheses in Table 4 is the explanatory description of the weather condition information, road surface condition information, sensor state information, pre-weighting strategy, etc.

[0249] Table 4

[0250]

[0251] To help understand the implementation process of the above embodiments of the automotive road noise processing method, an implementation example is given. Figure 3This is the signal processing flowchart involved in the automotive road noise processing method of this example. Figure 4 Flowchart of the automotive road noise processing method of this example.

[0252] Step 1:

[0253] After pre-filtering, amplitude adjustment and other preprocessing operations on the J-channel time-domain vibration reference signal x(n), convolve with the time-domain adaptive filter coefficient W full (l), where l represents the frame index, indicating that it is updated in units of frames, rather than the sampling time n. The convolved signal is y(n) of the Q-channel, and after post-processing operations such as amplitude limiting, it is sent to Q speakers for output respectively. The amplitude adjustment operation can be performed on the signal x(n) according to the pre-weighting weight determined in Step 3. The amplitude limiting operation can be performed on the signal y(n) according to the target clipping parameter value T(l) determined in Step 3.

[0254] Step 2:

[0255] Put the J-channel time-domain vibration reference signal x(n) and the M-channel time-domain error microphone signal e(n) into two buffers respectively, with lengths of J*N and M*N. N is the frame length. In this example, J = 9, M = 2, Q = 5, and N = 64. Wait until the buffers are full to obtain one frame of the J-channel time-domain vibration reference signal x(l) and one frame of the M-channel time-domain error microphone signal e(l). Use the polyphase analysis filter bank based on FFT to perform sub-band decomposition on the above signals x(l) and e(l) respectively to obtain one frame of the J-channel sub-band vibration reference signal X k (l):

[0256] X k (l)=[X k 1 (l), X k 2 (l)…X k J (l)]

[0257] and one frame of the M-channel sub-band error microphone signal E k (l):

[0258] E k (l)=[E k 1 (l), E k 2 (l)…E k M (l)]

[0259] Among them, k = 1, 2... K, where K is the total number of subbands, and in this example, K = 65. In addition, a polyphase analysis filter bank trained by the training method in the fourth embodiment above can be used as the prototype low-pass filter in subband decomposition:

[0260] H G（l） (d) = [h1(d), h2(d)... h P (d)]

[0261] H G（l） (d) is the target polyphase analysis filter bank matched according to G(l), and G(l) is the identifier of the analysis filter bank and the synthesis filter bank determined according to the environmental state information. The formula for subband decomposition of the time-domain vibration reference signal x(l) is as follows:

[0262] (1)

[0263] (2)

[0264] Among them, D is the downsampling factor, and P is the length of the FFT. In this example, D = 3 and P = 128. In formula (2), v = -2πikp / P. Formula (1) is the process of polyphase filtering the signal x(l). Specifically, it first performs the operation of adding a subband analysis window to the signal x(l), and then accumulates the windowed result to obtain x'(l); formula (2) is to perform DFT (Discrete Fourier Transform) on the signal x'(l) to obtain the subband vibration reference signal X k (l). In actual calculation, FFT can be used to improve the calculation speed. The formula for subband decomposition of the time-domain error microphone signal e(l) will not be elaborated here.

[0265] Step Three:

[0266] Referring to Figure 5 , input a frame of J-channel subband vibration reference signals X1(l)... X K (l), a frame of M-channel subband error microphone signals E1(l)... E K (l), a frame of J-channel time-domain vibration reference signal x(l), and a frame of M-channel time-domain error microphone signal e(l) into a prediction model (U-Net Encoder + FT-GRU + U-Net Decoder + multi-task output layer) based on a neural network structure to predict the road surface condition information, weather condition information, and sensor status information. Corresponding to Figure 5 the function of the parameter matching control module in: dynamically determine the update steps μ1(l)... μ of the subband adaptive filter coefficients based on the road surface condition information, weather condition information, and sensor status information K(l); Dynamically determine the target filter bank based on road surface condition information and weather condition information, and use G(l) to represent the identifier of the analysis filter bank and the synthesis filter bank determined according to road surface condition information and weather condition information; Determine the pre-weighting weights of each sensor and the target clipping parameter value T(l) based on road surface condition information, weather condition information, and sensor status information.

[0267] Step Four:

[0268] Since the subbands decomposed by the polyphase analysis filter bank based on FFT are subbands in the complex domain, the complex-domain FxLMS algorithm is used to update the subband adaptive filter coefficients W k (l), and the size of each W k (l) is C*J*Q, where C is the length of the subband adaptive filter coefficients. In this example, C = 4. In the complex-domain FxLMS algorithm, first convolve the input subband vibration reference signal X k (l) with the secondary path model S(k), and this convolution is a complex convolution:

[0269]

[0270] Then use the complex-domain LMS algorithm to update the subband adaptive filter coefficients W k (l):

[0271]

[0272] where q = 1, 2... Q, Q is the number of speaker signal channels. m = 1, 2... M, M is the number of error microphone signal channels, j = 1, 2... J, J is the number of channels of the vibration reference signal. k = 1, 2... K, K is the number of subbands. μ k (l) is the update step size of the subband adaptive filter coefficients determined according to the influencing factor information, which is used to control the update speed of the filter.

[0273] Step Five: Synthesize the time-domain adaptive filter coefficients from the subband adaptive filter coefficients.

[0274] Use the polyphase synthesis filter bank based on IFFT for subband synthesis. Compared with the weight stacking method with relatively large computational complexity and a large number of FFT and IFFT operations, this method only has one set of IFFT and adding synthesis window operations, which is simple and efficient. Among them, the polyphase synthesis filter bank can adopt the polyphase analysis filter bank trained by the training method in the above-mentioned fourth embodiment, and can be dynamically determined according to the environmental state information. As Figure 6 shown, in this example, the full-band (time-domain) adaptive filter coefficients synthesized based on the subband adaptive filter coefficients can achieve almost the same effect as the target full-band (time-domain) adaptive filter coefficients. Figure 6The abscissa is the index of the length of the filter, and the ordinate is the value of the coefficient corresponding to each tap.

[0275] Step Six: Load balancing strategy. Decompose sub-bands, synthesize sub-bands, and allocate the tasks of updating the coefficients of the sub-band adaptive filter to each interruption, and allocate the neural network model prediction task to each interruption.

[0276] It should be noted that the sequence numbers of the above steps do not constitute a limitation on the sequence between steps. In addition, the above examples are only for understanding the present application and do not constitute a limitation on the method for processing vehicle road noise of the present application. Based on this technical concept, more forms of simple transformation are within the protection scope of the present application.

[0277] The embodiment of the present application provides a vehicle road noise processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the vehicle road noise processing method in the first embodiment above.

[0278] Next, refer to Figure 7 , which shows a schematic structural diagram of a vehicle road noise processing device suitable for implementing the embodiment of the present application. The vehicle road noise processing device in the embodiment of the present application may be a control unit in an RNC system. Figure 7 The shown vehicle road noise processing device is only an example and should not bring any limitation to the functions and usage scope of the embodiment of the present application.

[0279] As Figure 7As shown, the vehicle road noise processing device may include a processing device 1001 (such as a DSP processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1002 or the program loaded from the storage device 1003 into the random access memory 1004. In the random access memory 1004, various programs and data required for the operation of the vehicle road noise processing device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. The input / output interface 1006 is also connected to the bus. Generally, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a microphone, an accelerometer, etc.; an output device 1008 including, for example, a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the vehicle road noise processing device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a vehicle road noise processing device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0280] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0281] Compared with the prior art, the beneficial effects of the vehicle road noise processing device provided by the embodiments of the present application are the same as those of the vehicle road noise processing method provided by the above embodiments, and other technical features in the vehicle road noise processing device are the same as the features disclosed in the method of the previous embodiment, which will not be elaborated here.

[0282] It should be understood that the various parts disclosed in the embodiments of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0283] As described above, this is only the specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims described above.

[0284] An embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the automotive road noise processing method in the above-mentioned embodiment.

[0285] The computer-readable storage medium provided by the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM: Random Access Memory), read-only memory (ROM: Read Only Memory), erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (RadioFrequency: radio frequency), etc., or any suitable combination of the above.

[0286] The above computer-readable storage medium may be included in the automotive road noise processing device; it may also exist alone without being assembled into the automotive road noise processing device.

[0287] The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed by the automotive road noise processing device, the automotive road noise processing device is caused to execute the above-mentioned functions defined in the method of the disclosed embodiment of the present application.

[0288] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (for example, by connecting through the Internet using an Internet service provider).

[0289] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and this module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than marked in the accompanying drawings. For example, two consecutively represented blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0290] The modules involved in the embodiments described in this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0291] The readable storage medium provided by the embodiments of this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned automotive road noise processing method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiments of this application are the same as those of the automotive road noise processing method provided by the above embodiments, and will not be elaborated here.

[0292] The embodiment of the present application also provides a computer program product, including a computer program, which when executed by a processor implements the steps of the above-mentioned method for processing vehicle road noise.

[0293] Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as those of the method for processing vehicle road noise provided by the above embodiment, and will not be elaborated herein.

[0294] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for processing automotive road noise, characterized in that, The method for processing automotive road noise includes: dynamically determining a target filter bank according to the acquired environmental state information, where the target filter bank includes a target polyphase analysis filter bank and a target polyphase synthesis filter bank, and the environmental state information is information characterizing the environmental state inside and / or outside the vehicle; using the target polyphase analysis filter bank to perform sub-band decomposition on the input first time-domain reference signal and the input first time-domain error signal respectively to obtain a first sub-band reference signal and a first sub-band error signal; updating the sub-band adaptive filter coefficients based on the first sub-band reference signal and the first sub-band error signal; using the target polyphase synthesis filter bank to perform sub-band synthesis on the updated sub-band adaptive filter coefficients to obtain updated time-domain adaptive filter coefficients; processing the newly input second time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control the road noise based on the anti-noise signal; wherein, different filter banks are preset for different environmental states, and each filter bank preset for different environmental states is pre-trained with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank, and the weights of the signal reconstruction loss and spectral leakage loss set during the training of different filter banks are different, and the spectral leakage loss refers to the magnitude of spectral leakage generated by performing sub-band decomposition of a signal through a filter bank.

2. The method for processing automobile road noise according to claim 1, characterized in that, Each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank, and the filter update delay magnitude and / or spectral leakage magnitude brought by different filter banks are different; The step of dynamically determining a target filter bank according to the acquired environmental state information includes: determining the filter bank preset corresponding to the environmental state characterized by the acquired environmental state information as the target filter bank.

3. The method for processing automobile road noise according to claim 2, wherein, When dynamically determining the target filter bank according to the environmental state information, the following rule is followed: Under the environmental state characterized by the environmental state information, the higher the stability of the automotive road noise, the greater the filter update delay and the smaller the spectral leakage brought by the target filter bank, and the lower the stability of the automotive road noise, the smaller the filter update delay and the greater the spectral leakage brought by the target filter bank.

4. The method for processing automotive road noise according to claim 1, characterized in that, The method for controlling automotive road noise further includes: performing prediction based on a third time-domain reference signal, a third time-domain error signal, a third sub-band reference signal, and a third sub-band error signal using a preset prediction model to obtain the environmental state information; wherein, the prediction model is pre-trained, the third time-domain reference signal and the third time-domain error signal are respectively input before the first time-domain reference signal and the first time-domain error signal, and the third sub-band reference signal and the third sub-band error signal are obtained by performing sub-band decomposition on the third time-domain reference signal and the third time-domain error signal respectively.

5. The method for processing automobile road noise according to claim 4, characterized in that, The prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module; The step of using a preset prediction model to perform prediction based on a third time-domain reference signal, a third time-domain error signal, a third sub-band reference signal, and a third sub-band error signal to obtain the environmental state information includes: Inputting the third sub-band reference signal and the third sub-band error signal into the first feature extraction module for feature extraction to obtain a first feature representation; Inputting the third time-domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation; Inputting the third time-domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation; Concatenating the first feature representation, the second feature representation, and the third feature representation and inputting the concatenated result into the feature fusion module for feature fusion to obtain a fused feature representation; Inputting the fused feature representation into the prediction module for prediction to obtain the environmental state information.

6. The method for processing automotive road noise according to claim 5, wherein, The first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer; the feature fusion module includes a frequency-axis gated recurrent unit, a time-axis gated recurrent unit, and at least one transposed convolutional layer connected in sequence; the prediction module includes a plurality of task output layers, and each task output layer is respectively used to output one item of the environmental state information.

7. The method for processing automobile road noise according to any one of claims 5 to 6, characterized in that, The environmental state information includes weather condition information and road surface condition information.

8. The method for processing automotive road noise according to any one of claims 5 to 6, characterized in that, The frame indices of the first time-domain reference signal and the first time-domain error signal are l, and the method for processing automotive road noise further includes: Executing tasks allocated to the N interruptions during the N interruptions in the (l + 1)-th frame, where N represents the frame length, and an interruption refers to the period from generating an anti-noise signal based on the time-domain reference signal of each sampling point to receiving the time-domain reference signal of the next sampling point. The tasks allocated to the N interruptions include: the task of calculating the updated time-domain adaptive filter coefficients, and / or, the task of inputting the first time-domain reference signal, the first time-domain error signal, the first sub-band reference signal, and the first sub-band error signal into the prediction model to predict the environmental state information.

9. An automobile road noise treatment device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the method for processing automotive road noise according to any one of claims 1 to 8.

Images