Automobile road noise processing method and device

Abstract

The invention discloses an automobile road noise processing method and device, and relates to the technical field of noise reduction, and the method comprises the steps: dynamically determining a target filter group according to the obtained environment state information; respectively performing sub-band decomposition on the time domain reference signal and the time domain error signal according to the target multiphase analysis filter bank; and updating a sub-band adaptive filter coefficient based on the sub-band reference signal and the sub-band error signal, and performing sub-band synthesis on the updated sub-band adaptive filter coefficient by using a target multi-phase synthesis filter bank to obtain an updated time domain adaptive filter coefficient, and processing a newly input time domain reference signal based on the updated time domain adaptive filter coefficient to obtain an anti-noise signal, and controlling the road noise based on the anti-noise signal. The method and the device can adapt to environment change, avoid influence of the environment change on the noise reduction effect, and improve the stability of the noise reduction effect.

Description

Technical Field

[0001] This application relates to the field of noise reduction technologies, and particularly to a method and device for processing automotive road noise. Background Art

[0002] With the development of the automotive industry, the comfort and quietness inside the vehicle cabin have increasingly attracted the attention of the automotive industry and consumers. As a key factor affecting the acoustic quietness and riding experience inside the vehicle, vehicle interior noise has gradually become a pain point problem that the industry focuses on. Vehicle interior noise mainly comes from the power system of the vehicle and road noise during vehicle driving. With the development of new energy vehicles, electric motors have gradually replaced internal combustion engines, thus significantly reducing engine noise. Road Noise Cancellation (RNC) technology has become an important means to improve the acoustic comfort inside the vehicle. The RNC technology combines the reference signal collected by the vibration sensor and the error signal collected by the in-vehicle microphone to generate a sound signal opposite to the noise characteristics, which is played through the in-vehicle speaker to cancel the original noise. Although existing RNC systems have been applied in some mass-produced vehicle models, in complex actual application environments, there are still problems such as insufficient adaptive ability of the RNC system and poor stability of the noise reduction effect, seriously affecting the driving and riding experience.

[0003] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method and device for processing automotive road noise, aiming to solve the problems of insufficient adaptive ability of the current RNC system and poor stability of the noise reduction effect.

[0005] To achieve the above objective, this application proposes a method for processing automotive road noise, and the method for processing automotive road noise includes: Dynamically determining a target filter bank according to the obtained environmental state information, where the target filter bank includes a target polyphase analysis filter bank and a target polyphase synthesis filter bank, and the environmental state information is information characterizing the environmental state inside and / or outside the vehicle; Performing sub-band decomposition on the input first-time-domain reference signal and the input first-time-domain error signal respectively by using the target polyphase analysis filter bank to obtain a first sub-band reference signal and a first sub-band error signal; Updating the sub-band adaptive filter coefficients based on the first sub-band reference signal and the first sub-band error signal; Performing sub-band synthesis on the updated sub-band adaptive filter coefficients by using the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients; Process the newly input second time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, and control road noise based on the anti-noise signal.

[0006] Optionally, different filter banks are preset for different environmental states. Each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank, and the filter update delay and / or spectral leakage of different filter banks are different; The step of dynamically determining the target filter bank according to the obtained environmental state information includes: Determine the filter bank preset corresponding to the environmental state represented by the obtained environmental state information as the target filter bank.

[0007] Optionally, when dynamically determining the target filter bank according to the environmental state information, the following rules are followed: In the environmental state represented by the environmental state information, the higher the stability of the automotive road noise, the greater the filter update delay and the smaller the spectral leakage brought by the target filter bank, and the lower the stability of the automotive road noise, the smaller the filter update delay and the greater the spectral leakage brought by the target filter bank.

[0008] Optionally, each filter bank preset for different environmental states is pre-trained with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank, and the weights of the signal reconstruction loss and frequency leakage loss set during the training of different filter banks are different.

[0009] Optionally, the road noise control method further includes: Use a preset prediction model to perform prediction based on the third time-domain reference signal, the third time-domain error signal, the third sub-band reference signal, and the third sub-band error signal to obtain the environmental state information; Wherein, the prediction model is pre-trained, the third time-domain reference signal and the third time-domain error signal are respectively input before the first time-domain reference signal and the first time-domain error signal, and the third sub-band reference signal and the third sub-band error signal are obtained by performing sub-band decomposition on the third time-domain reference signal and the third time-domain error signal respectively.

[0010] Optionally, the prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module; The step of using a preset prediction model to perform prediction based on the third time-domain reference signal, the third time-domain error signal, the third sub-band reference signal, and the third sub-band error signal to obtain the environmental state information includes: Input the third sub - band reference signal and the third sub - band error signal into the first feature extraction module for feature extraction to obtain a first feature representation; Input the third time - domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation; Input the third time - domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation; Concatenate the first feature representation, the second feature representation, and the third feature representation and input them into the feature fusion module for feature fusion to obtain a fused feature representation; Input the fused feature representation into the prediction module for prediction to obtain the environmental state information.

[0011] Optionally, the first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer; the feature fusion module includes a frequency - axis gated recurrent unit, a time - axis gated recurrent unit, and at least one transposed convolutional layer connected in sequence; the prediction module includes multiple task output layers, and each task output layer is respectively used to output one item of the environmental state information.

[0012] Optionally, the environmental state information includes weather condition information and road surface condition information.

[0013] Optionally, the frame indices of the first time - domain reference signal and the first time - domain error signal are l, and the method for processing automotive road noise further includes: Execute the tasks allocated to the N interrupts during the N interrupts in the (l + 1)-th frame, where N represents the frame length, and an interrupt refers to the period from generating an anti - noise signal based on the time - domain reference signal of each sampling point to receiving the time - domain reference signal of the next sampling point. The tasks allocated to the N interrupts include: the task of calculating the updated time - domain adaptive filter coefficients, and / or the task of inputting the first time - domain reference signal, the first time - domain error signal, the first sub - band reference signal, and the first sub - band error signal into the prediction model to predict the environmental state information.

[0014] In addition, to achieve the above object, the present application also proposes an automotive road noise processing device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the automotive road noise processing method as described above.

[0015] One or more technical solutions proposed by the present application have at least the following technical effects: In this application, the target polyphase analysis filter bank and the target polyphase synthesis filter bank are dynamically determined according to the acquired environmental state information. The input time-domain reference signal and the input time-domain error signal are respectively subjected to sub-band decomposition by the target polyphase analysis filter bank to obtain the sub-band reference signal and the sub-band error signal. The sub-band adaptive filter coefficients are updated based on the sub-band reference signal and the sub-band error signal. The updated sub-band adaptive filter coefficients are subjected to sub-band synthesis by the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients. The newly input time-domain reference signal is processed based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control the road noise based on the anti-noise signal. That is, in this application, by dynamically determining the target polyphase analysis filter bank and the target polyphase synthesis filter bank for sub-band decomposition and sub-band synthesis according to the acquired environmental state information, during the process of controlling the automotive road noise, it can adapt to environmental changes, avoid the influence of environmental changes on the noise reduction effect, and thus improve the stability of the noise reduction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart provided for the first embodiment of the automotive road noise processing method of this application; Figure 2 It is a schematic flowchart of the prediction model training process involved in an embodiment of this application; Figure 3 It is a schematic diagram of the signal processing process involved in an embodiment of this application; Figure 4 It is a schematic flowchart of the automotive road noise processing method involved in an embodiment of this application; Figure 5 It is a schematic diagram of the model architecture and parameter matching architecture involved in an embodiment of this application; Figure 6 It is a comparison diagram of filter coefficients involved in an embodiment of this application; Figure 7 It is a schematic diagram of the device structure of the hardware operating environment involved in the automotive road noise processing method in the embodiments of this application.

[0019] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with embodiments with reference to the accompanying drawings. Specific embodiments

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0021] In order to better understand the technical solutions of this application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific embodiments.

[0022] The RNC technology generates a sound signal opposite to the noise characteristics by combining the reference signal collected by the vibration sensor and the error signal collected by the in-vehicle microphone, and plays it through the in-vehicle speaker to cancel the original noise. Although the existing RNC systems have been applied in some mass-produced vehicles, in complex actual application environments, there are still problems such as insufficient adaptive ability of the RNC system and poor stability of the noise reduction effect, which seriously affect the driving experience.

[0023] The embodiments of this application propose to dynamically determine the target polyphase analysis filter bank and the target polyphase synthesis filter bank according to the obtained environmental state information, perform subband decomposition on the input time-domain reference signal and the input time-domain error signal respectively according to the target polyphase analysis filter bank to obtain the subband reference signal and the subband error signal, update the subband adaptive filter coefficients based on the subband reference signal and the subband error signal, perform subband synthesis on the updated subband adaptive filter coefficients by using the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients, and process the newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain the anti-noise signal, so as to control the road noise based on the anti-noise signal. That is, the embodiments of this application can adapt to environmental changes during the process of controlling the road noise of the vehicle by dynamically determining the target polyphase analysis filter bank and the target polyphase synthesis filter bank for subband decomposition and subband synthesis, avoiding the influence of environmental changes on the noise reduction effect, thereby improving the stability of the noise reduction effect.

[0024] The following presents the first embodiment of the method for processing vehicle road noise of this application. Refer to Figure 1 , Figure 1It is a schematic flowchart of the first embodiment of the method for processing vehicle road noise in this application. In this embodiment, the execution subject of the method for processing vehicle road noise can be the control unit in the vehicle RNC system, but it is not limited to the control unit. For example, it can also be a processing device with data processing and program running functions. In a feasible implementation, the vehicle RNC system may include sensors, a control unit, a noise canceller, and other auxiliary devices set as needed; among them, the sensors include a reference sensor for collecting reference signals and an error sensor for collecting error signals, and multiple reference sensors and error sensors can be set; the reference sensor and the error sensor can be vibration sensors (acceleration sensors) or microphones; the control unit runs adaptive algorithms, such as FxLMS (Filtered-x Least Mean Square), FxNLMS (Filtered-X Normalized Least Mean Square), FxAP (Filter-x Affine Projection) and other adaptive algorithms, updates the adaptive filter coefficients based on the reference signal and the error signal, and calculates the anti-noise signal; the noise canceller is used to output the anti-noise signal to cancel the road noise. In a feasible implementation, the reference sensor can use a vibration sensor and can be set at the attachment point of the vehicle body and the chassis, and the error sensor can use a microphone and can be set inside the vehicle, such as the roof lining (near the passenger's head area); the noise canceller can use a vibration actuator and / or a speaker. The vibration actuator generates reverse mechanical vibration according to the anti-noise signal to cancel the low-frequency road noise transmitted through the vehicle body, and the speaker emits reverse sound waves according to the anti-noise signal to directly cancel the air-borne noise (such as the high-frequency component of tire noise) entering the cabin.

[0025] In this embodiment, the method for processing vehicle road noise includes steps S10 to S50: Step S10, dynamically determine a target filter bank according to the obtained environmental state information, where the target filter bank includes a target polyphase analysis filter bank and a target polyphase synthesis filter bank, and the environmental state information is information characterizing the environmental state inside and / or outside the vehicle.

[0026] In this embodiment, the sub-band adaptive algorithm is adopted, that is, the sub-band adaptive filter coefficients are updated separately and then synthesized into the full-band (time domain) adaptive filter coefficients. Compared with the method of directly updating the full-band (time domain) adaptive filter coefficients, in this embodiment, the sub-band adaptive filter coefficients are updated separately, and then the updated sub-band adaptive filter coefficients are synthesized into the updated time domain adaptive filter coefficients, which has the advantages of fast convergence, independence of different sub-bands, the ability to use different adaptive algorithms and filter orders for different sub-bands, and low sensitivity to changes in the input signal.

[0027] The subband adaptive algorithm requires subband decomposition and subband synthesis operations. In this embodiment, a polyphase analysis filter bank is used for subband decomposition, and a polyphase synthesis filter bank is used for subband synthesis. Through this subband decomposition and subband synthesis method, during the process of restoring the filter coefficients to the full band, the synthesis filter bank is directly used. Only by adding a synthesis window to the subband-domain filter and then performing the IFFT (Inverse Fast Fourier Transform) operation can the time-domain filter coefficients that are almost the same as those in the full band be obtained, ensuring performance while avoiding additional computational complexity.

[0028] In the specific implementation, a polyphase analysis filter bank based on FFT (hereinafter referred to as the polyphase FFT analysis filter bank) and a polyphase synthesis filter bank based on IFFT (hereinafter referred to as the polyphase IFFT synthesis filter bank) can be used to further improve the computational efficiency. Hereinafter, the analysis filter bank may also be referred to as the analysis window, and the synthesis filter bank may be referred to as the synthesis window.

[0029] The control unit can dynamically determine the filter bank for subband decomposition and subband synthesis according to the acquired environmental state information. Hereinafter, the determined filter bank is referred to as the target filter bank for distinction. The target filter bank includes a target polyphase analysis filter bank for subband decomposition and a target polyphase synthesis filter bank for subband synthesis. The target polyphase analysis filter bank is used to perform subband decomposition on the time-domain reference signal and the time-domain error signal, and the target polyphase synthesis filter bank is used to perform subband synthesis on the updated subband adaptive filter coefficients. Since changes in the environmental state outside the vehicle or inside the vehicle will cause dynamic changes in the acoustic propagation path, which will affect the stability of the vehicle road noise. If a fixed polyphase analysis filter bank and polyphase synthesis filter bank are used for subband decomposition and subband synthesis, it may lead to the inability to adapt to the changing vehicle road noise, resulting in unstable noise reduction effects. Therefore, in this embodiment, the target polyphase analysis filter bank and the target polyphase synthesis filter bank for subband decomposition and subband synthesis are dynamically determined according to the environmental state information to adapt to environmental changes and improve the stability of the noise reduction effect.

[0030] In the specific implementation, the environmental state inside the vehicle may include, for example, the seat adjustment state, the position of the occupants, the window opening and closing state, etc., and the environmental state outside the vehicle may include, for example, the weather condition, the road surface condition, etc., which are not limited in this embodiment.

[0031] In this embodiment, the acquisition method of the environmental state information is not limited. For example, it can be obtained by analyzing the environmental images captured by a camera.

[0032] The environmental status information can be obtained at regular intervals. The shorter the time interval, the higher the real-time degree. The frequency of acquisition is not limited in this embodiment. For example, in a feasible implementation, the environmental status information can be obtained each time the adaptive filter coefficients are updated.

[0033] Dynamically determining the target filter bank means that after each new environmental status information is obtained, the target filter bank is determined according to the new environmental status information, so that the filter banks used for subband decomposition and subband synthesis change with the change of the environmental status, and the change of the internal or external environment of the vehicle is adapted by adjusting the filter banks used for subband decomposition and subband synthesis, thereby improving the stability of the noise reduction effect.

[0034] In a specific implementation, different filter banks can be preset for different environmental statuses. Each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank. Then, when the control unit needs to perform subband decomposition on the current time-domain reference signal and time-domain error signal, and needs to perform subband synthesis on the current subband adaptive filter coefficients, the filter bank preset corresponding to the environmental status represented by the obtained environmental status information can be determined as the target filter bank. The filter banks set for different environmental statuses can be filter banks with the best or better noise reduction effect in this environmental status determined in advance through experiments, data statistics, etc.

[0035] Step S20: Use the target polyphase analysis filter bank to perform subband decomposition on the input first time-domain reference signal and the input first time-domain error signal respectively to obtain a first subband reference signal and a first subband error signal.

[0036] When multiple reference sensors are set, the input time-domain reference signal is multi-channel. Similarly, when multiple error sensors are set, the input time-domain error signal is also multi-channel. In this embodiment, the number of channels of the time-domain reference signal and the time-domain error signal is not limited. For example, when applied to a MIMO (Multiple Input Multiple Output, MIMO) vehicle-mounted RNC system, both the input and output are multi-channel.

[0037] The time-domain reference signal and the time-domain error signal collected by the sensor can be directly input into the control unit, or can be input into the control unit after preprocessing; the control unit can directly perform subband decomposition on the input time-domain reference signal and time-domain error signal, or can also perform preprocessing before subband decomposition. In this embodiment, there is no limitation on whether to perform preprocessing on the time-domain reference signal and the time-domain error signal, nor on what preprocessing operations are performed on the signals. For example, it can include pre-filtering, amplitude adjustment, etc. Pre-filtering includes high-pass filtering, low-pass filtering, etc., and the preprocessing operations can be set according to needs. In the following embodiments, the preprocessing operation on the signal will not be particularly emphasized anymore. That is, even if it is not written that preprocessing is required, it does not necessarily mean that there is no or no need to perform the preprocessing operation.

[0038] The control unit uses a target polyphase analysis filter bank determined according to the environmental state information to perform subband decomposition on the time-domain reference signal to obtain a subband reference signal, and perform subband decomposition on the time-domain error signal to obtain a subband error signal. The prefix "first" is used in step S20 to distinguish from the time-domain reference signal, time-domain error signal, subband reference signal, and subband error signal mentioned in the subsequent steps, indicating that they are input at different times. It can be understood that there are multiple subbands, and there are also multiple subband reference signals and subband error signals obtained through subband decomposition. That is, the subband reference signal and subband error signal corresponding to each subband are obtained through subband decomposition. In the specific implementation, the subband can be divided from the full frequency band, but in this embodiment, there is no limitation that it is divided from the full frequency band. It can be divided from a relatively wide frequency band range, and there is no limitation on the number of subbands.

[0039] In a feasible implementation, the control unit can perform road noise control in a manner without output delay, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit point by point in real time. The control unit generates an anti-noise signal for a single sampling point of the currently input time-domain reference signal in real time according to the current time-domain adaptive filter coefficients, and then outputs the anti-noise signal in real time to achieve the effect of no output delay; on the other hand, the adaptive filter coefficients are updated frame by frame. Specifically, the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffer in real time. After each frame of time-domain reference signal and each frame of time-domain error signal are stored, subband decomposition is performed on the frame of time-domain reference signal and the frame of time-domain error signal, and subsequent time-domain adaptive filter coefficient update operations are performed. In other embodiments, a method with output delay can also be adopted, that is, after inputting the time-domain reference signal of multiple sampling points, the anti-noise signal starts to be output. In this embodiment, there is no limitation on whether to adopt the method without output delay or the method with output delay. In the specific application scenario, the specific implementation method can be selected according to needs.

[0040] Step S30: Update the subband adaptive filter coefficients based on the first subband reference signal and the first subband error signal.

[0041] The subband adaptive filter coefficients can be updated according to a preset adaptive algorithm. For example, the adaptive algorithm can adopt FxLMS, FxNLMS, FxAP, etc., which are not limited in this embodiment.

[0042] For example, in a feasible implementation, the adaptive algorithm can adopt the complex-domain FxLMS algorithm. Specifically, the control unit can convolve the first subband reference signal with a pre-established secondary path model, and then use the complex-domain LMS algorithm to calculate the updated subband adaptive filter coefficients based on the first subband error signal, the first subband reference signal after convolving with the secondary path model, and the current subband adaptive filter coefficients.

[0043] Step S40: Perform subband synthesis on the updated subband adaptive filter coefficients using the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients.

[0044] To avoid output delay, filtering output needs to be performed in the time domain (full band). Therefore, the subband adaptive filter coefficients can be converted to the full band. By performing subband synthesis on the updated subband adaptive filter coefficients, the updated time-domain adaptive filter coefficients can be obtained.

[0045] The control unit performs subband synthesis on the updated subband adaptive filter coefficients using the target polyphase synthesis filter bank determined according to the environmental state information to obtain the updated time-domain adaptive filter coefficients.

[0046] Step S50: Process the newly input second time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control road noise based on the anti-noise signal.

[0047] The second time-domain reference signal refers to the time-domain reference signal input after obtaining the updated time-domain adaptive filter coefficients. For example, assuming the frame index of the first time-domain reference signal is l, then the frame index of the second time-domain reference signal may be l + 1.

[0048] After updating the time-domain adaptive filter coefficients, the newly input time-domain reference signal can be processed according to the updated time-domain adaptive filter coefficients to generate an anti-noise signal. After generating the anti-noise signal, road noise can be controlled based on the anti-noise signal. For example, the anti-noise signal can be played through a speaker installed inside the vehicle, or the anti-noise signal can be output after some post-processing operations. The post-processing operations can include, for example, amplitude limiting, etc., which are not limited here.

[0049] The generated anti-noise signal can be multi-channel. For example, when there are multiple speakers, a multi-channel anti-noise signal is generated and output through multiple speakers. In this embodiment, the number of channels of the anti-noise signal is not limited.

[0050] In this embodiment, according to the acquired environmental state information, the target polyphase analysis filter bank and the target polyphase synthesis filter bank are dynamically determined. The input time-domain reference signal and the input time-domain error signal are respectively sub-band decomposed by the target polyphase analysis filter bank to obtain the sub-band reference signal and the sub-band error signal. The sub-band adaptive filter coefficients are updated based on the sub-band reference signal and the sub-band error signal. The updated sub-band adaptive filter coefficients are sub-band synthesized by the target polyphase synthesis filter bank to obtain the updated time-domain adaptive filter coefficients. The newly input time-domain reference signal is processed based on the updated time-domain adaptive filter coefficients to obtain the anti-noise signal, so as to control the road noise based on the anti-noise signal. That is to say, the solution of this embodiment dynamically determines the target polyphase analysis filter bank and the target polyphase synthesis filter bank for sub-band decomposition and sub-band synthesis according to the acquired environmental state information, so that during the process of controlling the automotive road noise, it can adapt to environmental changes and avoid the impact of environmental changes on the noise reduction effect, thereby improving the stability of the noise reduction effect.

[0051] In a feasible implementation manner, different filter banks are preset for different environmental states. Each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank, and the filter update delay size and / or the spectrum leakage size brought by different filter banks are different. The step S10 includes: determining the filter bank preset corresponding to the environmental state characterized by the acquired environmental state information as the target filter bank.

[0052] The filter update delay and / or spectral leakage sizes brought by each filter bank pre-set for different environmental states are different. The filter update delay and spectral leakage brought by the filter bank refer to the filter update delay and spectral leakage that will occur when sub-band decomposition and sub-band synthesis are performed according to the filter bank. It should be noted that generally, the filter update delay size and spectral leakage size brought by the filter bank are negatively correlated. The larger the delay, the smaller the spectral leakage; the smaller the delay, the larger the spectral leakage. And the smaller the delay, the better the robustness and transient response of the system; the smaller the spectral leakage, the better the filtering accuracy and steady-state performance of the system. Since the stability of vehicle road noise is different under different environmental states and the requirements for the robustness, transient response ability, filtering accuracy, and steady-state performance of the system are different, therefore, dynamically determining the filter bank to be used according to the environmental state information can balance the delay size and spectral leakage size, adopt the delay and spectral leakage suitable for the environmental state, so as to balance the robustness, transient response ability, filtering accuracy, and steady-state performance of the system and ensure the stability of the noise reduction effect.

[0053] In a feasible implementation manner, it can be set that when the control unit dynamically determines the target filter bank according to the environmental state information, the following rules need to be met: In the environmental state characterized by the environmental state information, the higher the stability of the vehicle road noise, the greater the filter update delay and the smaller the spectral leakage brought by the target filter bank; the lower the stability of the vehicle road noise, the smaller the filter update delay and the larger the spectral leakage brought by the target filter bank.

[0054] It should be noted that if the stability of the vehicle road noise is relatively high in the environmental state characterized by the environmental state information, a filter bank with a larger filter update delay and a smaller spectral leakage can be adopted to focus on improving the filtering accuracy and steady-state performance and obtain a better noise reduction effect. If the stability of the vehicle road noise is relatively low in the environmental state characterized by the environmental state information, a filter bank with a smaller filter update delay and a larger spectral leakage can be adopted to focus on improving the robustness and transient response of the system, quickly adapt to environmental changes, and maintain the stability of the system.

[0055] In a specific implementation manner, in order to enable the control unit to meet the above rules when dynamically determining the target filter bank according to the environmental state information, the filter bank corresponding to each environmental state can be set according to this rule.

[0056] In a feasible implementation manner, the environmental state information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to weather conditions that affect the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information characterizing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0057] In a feasible implementation manner, when the environmental state information indicates that the environment is good and stable (for example: good road conditions, clear weather), the automotive road noise is relatively stable. Then, a filter bank with a larger filter update delay and smaller spectral leakage can be adopted, focusing on improving the filtering accuracy and steady-state performance to obtain a better noise reduction effect. When the environmental state information indicates that the environment is unstable (for example: deteriorating road conditions, bad weather), the stability of the automotive road noise may deteriorate. At this time, a filter bank with a smaller filter update delay and larger spectral leakage can be adopted, focusing on improving the robustness and transient response of the system to quickly adapt to environmental changes and maintain system stability.

[0058] The following Table 1 gives some examples of determining the filter bank based on the weather condition information and the road surface condition information. Table 1 only gives some examples and does not limit the way of determining the target filter bank in this embodiment. The content in parentheses in Table 1 is the explanatory description of the weather condition information, road surface condition information, etc.

[0059] Table 1

[0060] Based on the above first embodiment, a second embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the same or similar content as that in the above first embodiment can be referred to the above introduction and will not be elaborated hereinafter. In this embodiment, each preset filter bank corresponding to different environmental states is pre-trained with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank. It should be noted that the method of training with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank is similar to the method of training a neural network model with the goal of reducing the prediction loss of the neural network model. The steps include preparing data for training, designing a loss function, selecting an optimizer for training, etc., and can specifically refer to the training method of the neural network model. Among them, the signal reconstruction loss refers to the error between the signal obtained by subband decomposition and subband synthesis of the signal through the filter bank and the original signal, and the spectral leakage loss refers to the magnitude of the spectral leakage generated by subband decomposition of the signal through the filter bank.

[0061] The weights of the signal reconstruction loss and the frequency leakage loss set during the training of different filter banks are different. Specifically, to train filter banks with different degrees of spectral leakage and different filter update delays, different weight combinations of the signal reconstruction loss and the frequency leakage loss can be set during the training. For example, by setting a larger weight for the signal reconstruction loss and a smaller weight for the frequency leakage loss, a filter bank with a smaller delay and a larger leakage can be trained; by setting a smaller weight for the signal reconstruction loss and a larger weight for the frequency leakage loss, a filter bank with a larger delay and a smaller leakage can be trained.

[0062] In a feasible implementation manner, the signal reconstruction loss is calculated based on the error between the reconstructed signal and a preset first type of signal. The reconstructed signal is obtained by performing sub-band decomposition and sub-band synthesis on the preset first type of signal using the polyphase analysis filter bank to be trained and the polyphase synthesis filter bank to be trained. Among them, an initialized polyphase analysis filter bank and a polyphase synthesis filter bank can be set in advance, and the coefficients of the polyphase analysis filter bank and the polyphase synthesis filter bank need to be updated during the training process. The preset first type of signal is a signal preset for calculating the signal reconstruction loss, and can be various noise signals, vibration signals, music signals, etc. collected in advance. The preset first type of signal is decomposed into sub-bands using the polyphase analysis filter bank to be trained, and the decomposed result is then synthesized into sub-bands using the polyphase synthesis filter bank to be trained to obtain a reconstructed signal, and the error is calculated with the preset first type of signal to obtain the signal reconstruction loss.

[0063] In a feasible implementation manner, the spectral leakage loss is calculated based on the target sub-band signal. The target sub-band signal is obtained by performing sub-band decomposition on a preset second type of signal using the polyphase analysis filter bank to be trained. The preset second type of signal is a single-frequency signal or a narrow-band signal within a preset frequency band range, and the frequency band range of the target sub-band signal is the range other than the preset frequency band range. Among them, the preset second type of signal is a signal preset for calculating the spectral leakage loss, and can be a single-frequency signal or a narrow-band signal within a preset frequency band range collected in advance. The preset frequency band range can be set as needed. The preset second type of signal is decomposed into sub-bands using the polyphase analysis filter bank to be trained, and the spectral leakage loss is calculated according to the target sub-band signal in the decomposition result. It can be understood that since the preset second type of signal is a single-frequency signal or a narrow-band signal within a preset frequency band range, after the preset second type of signal is decomposed into sub-bands, if there is no spectral leakage, then there will be no signal in the range other than the preset frequency band range in the sub-band decomposition result, and the frequency band range of the target sub-band signal is the range other than the preset frequency band range, so the magnitude of the target sub-band signal can represent the magnitude of the spectral leakage, and the spectral leakage loss can be calculated based on the target sub-band signal.

[0064] In a feasible implementation, the filter bank can be trained in the following way: Step 1: Data preparation. Two types of data are used for training the analysis filter bank and the synthesis filter bank. The first type is various noise signals, vibration signals, and music signals. The second type is single-frequency signals or narrowband signals uniformly generated according to a known frequency band range (hereinafter represented by S as the set of sub-band indices within this frequency band range). Each of these two types of data accounts for 50% of the total data.

[0065] Step 2: Design of the loss function. The signal reconstruction error and the spectral leakage energy are two criteria for measuring the performance of the analysis window and the synthesis window of the sub-bands. Therefore, these two metrics are used as the loss functions for training.

[0066]

[0067]

[0068]

[0069] Among them, L MSE is the reconstruction loss function of the signal, L leakage is the loss function of the spectral leakage of the signal, and α and β are weight coefficients used to adjust the influence ratio of the losses of the two parts, for example, set as α = 0.4 and β = 0.6. z[r] is the preset first type of signal, and z'[r] is the signal after sub-band decomposition and sub-band synthesis of z[r], that is, the reconstructed signal. R is the length of the signal, for example, it can be set as R = 20000. Z[k] is the sub-band signal after sub-band decomposition of the preset second type of signal, k is the index of the sub-band, and S is the set of sub-band indices within the known frequency band range in Step 1.

[0070] Step 3: Select an optimizer for training. Calculate L MSE according to the input preset first type of signal, calculate L leakage according to the input preset second type of signal, and combine these two losses as the final loss L total . Pytorch (an open-source deep learning framework for machine learning and deep learning) can be selected as the training environment, and Adam can be selected as the optimizer to perform training iterations on the analysis window and the synthesis window until the loss function converges, and finally obtain the trained analysis window and synthesis window. The downsampling factor can be set to 3, and the number of points P of the FFT can be set to 128. Therefore, the sizes of the analysis window and the synthesis window are 128 * 3. During training, a hamming window with a length of 128 * 3 can be used as the initial coefficient.

[0071] Based on the above first and / or second embodiments, a third embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, for the same or similar content as the above first and / or second embodiments, reference can be made to the above introduction and will not be elaborated hereinafter. In this embodiment, it is proposed to predict environmental state information and / or sensor state information through a prediction model based on the input reference signal and error signal. Specifically, in this embodiment, the automotive road noise processing method further includes step S60: using a preset prediction model to perform prediction based on a third time-domain reference signal, a third time-domain error signal, a third sub-band reference signal, and a third sub-band error signal to obtain the environmental state information; wherein, the prediction model is pre-trained.

[0072] The third time-domain reference signal and the third time-domain error signal are respectively input before the first time-domain reference signal and the first time-domain error signal. For example, assuming that the frame indices of the first time-domain reference signal and the first time-domain error signal are l, then the frame indices of the third time-domain reference signal and the third time-domain error signal may be l - 1. The third sub-band reference signal and the third sub-band error signal are obtained by respectively performing sub-band decomposition on the third time-domain reference signal and the third time-domain error signal.

[0073] In the specific implementation manner, road noise control can be performed in a manner without output delay, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit point by point in real time. The control unit generates a single-sampling-point anti-noise signal for the currently input single-sampling-point time-domain reference signal according to the current time-domain adaptive filter coefficients in real time, and then outputs the anti-noise signal in real time to achieve the effect of no output delay; on the other hand, the environmental state information is updated frame by frame: the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffer in real time. After each buffer is filled with one frame of time-domain reference signal and one frame of time-domain error signal (assuming the frame index is l), the multi-phase analysis filter bank determined according to the environmental state information (predicted by inputting the signals of the (l - 1)-th frame into the prediction model) is used to perform sub-band decomposition on the l-th frame time-domain reference signal and the l-th frame time-domain error signal to obtain the l-th frame sub-band reference signal and the l-th frame sub-band error signal. The l-th frame time-domain reference signal, the l-th frame time-domain error signal, the l-th frame sub-band reference signal, and the l-th frame sub-band error signal are input into the prediction model to predict new environmental state information, and the filter bank determined according to this environmental state information is used to perform sub-band decomposition on the time-domain reference signal and the time-domain error signal of the (l + 1)-th frame.

[0074] In a specific embodiment, the prediction model can be implemented using a neural network model, which can be pre-trained and then deployed in the control unit. The control unit calls the prediction model for prediction when it needs to obtain environmental state information or sensor state information. In this embodiment, the implementation manner of the prediction model is not limited.

[0075] In a specific embodiment, multiple prediction models can be set up to output different environmental state information or sensor state information respectively, or a single prediction model can be set up to output multiple types of information in a multi-output task mode.

[0076] In this embodiment, since changes in the environmental state and sensor state are reflected in the signals collected by the sensor, time-domain reference signals, time-domain error signals, sub-band reference signals, and sub-band error signals are used as input data for the prediction model. Through the pre-trained prediction model, the relationship between these input data and environmental state information or sensor state information can be learned. Furthermore, the prediction model can be used to predict environmental state information or sensor state information based on these input data. On the one hand, by obtaining environmental state information or sensor state information through model prediction, existing reference signals and error signals can be utilized without the need to additionally set up a data acquisition module. On the other hand, the complex situation of automotive road noise under different environmental states or sensor states can be reflected in the signals collected by the sensor. Therefore, using the signals collected by the sensor can predict more complex and accurate environmental state information, thereby enabling the dynamic determination of the filter bank for sub-band decomposition and sub-band synthesis based on this information, which can greatly improve the system's ability to handle complex environments. Additionally, using both time-domain signals and sub-band signals as the prediction basis takes into account the importance of frequency-domain characteristics and can accurately detect and match more scenarios with small differences in time-domain characteristics.

[0077] In a feasible embodiment, the prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module. The step S60 includes S601 to S605: Step S601: Input the third sub-band reference signal and the third sub-band error signal into the first feature extraction module for feature extraction to obtain a first feature representation. Step S602: Input the third time-domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation. Step S603: Input the third time-domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation. Step S604: Concatenate the first feature representation, the second feature representation, and the third feature representation, and input the concatenated result into the feature fusion module for feature fusion to obtain a fused feature representation. Step S605: Input the fused feature representation into the prediction module for prediction to obtain the environmental state information and / or the sensor state information.

[0078] It should be noted that, to improve the accuracy of the prediction results and fully extract the information related to the environmental state and the sensor state in the signal, in this embodiment, feature extraction modules are respectively set for the sub-band reference signal, the sub-band error signal, the time-domain reference signal, and the time-domain error signal to extract the signal features of the three signals. For the sake of distinction, they are respectively called the first feature extraction module, the second feature extraction module, and the third feature extraction module; a feature extraction module based on a neural network structure can be used to implement them. The results output by the three feature extraction modules, namely the feature representations, are respectively called the first feature representation, the second feature representation, and the third feature representation for the sake of distinction. The feature representation can be in the form of a feature vector or a feature map, which is not limited in this embodiment. The feature fusion module can specifically be a module for simply concatenating the three feature representations, or can also include a module for deeply fusing the concatenated features. The module for deep fusion can specifically also be implemented based on a neural network structure, which is not limited here. The prediction module can be designed according to the types of information to be output and the specific forms of each type of information, which is not limited in this embodiment.

[0079] In a feasible embodiment, the first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer to extract features from the signal through the convolutional layer; the feature fusion module includes a frequency-axis gated recurrent unit (FGRU), a time-axis gated recurrent unit (TGRU), and at least one transposed convolutional layer connected in sequence. The frequency-axis gated recurrent unit is used to process data along the frequency axis to capture the correlation in the frequency dimension, the time-axis gated recurrent unit is used to process the output of the frequency-axis gated recurrent unit along the time axis to capture the dynamic features in the time dimension, and the transposed convolutional layer is used to gradually upsample and restore to the original size; the prediction module includes multiple task output layers, and each task output layer is respectively used to output one item of the environmental state information or the sensor state information. Hereinafter, FGRU and TGRU can also be collectively referred to as FT-GRU.

[0080] In a feasible implementation, a prediction model can be set up based on an improved U-Net (a deep learning-based convolutional neural network) architecture, integrating a frequency-axis gated recurrent unit and a time-axis gated recurrent unit, specifically designed to process complex acoustic and vibration signals. The prediction model includes an Encoder and a Decoder, which are specifically designed to process acoustic and vibration signals. The Encoder performs progressive downsampling through multiple layers of convolution, batch normalization, and ReLU activation functions to effectively extract features; the Decoder uses multiple layers of transposed convolution and skip connections for progressive upsampling to prepare features for the terminal classification task. In addition, FGRU and TGRU are integrated into the network structure to enhance the processing ability of multi-dimensional data, especially in time series and frequency analysis, to adapt to real-time changing environmental conditions. This comprehensive design not only captures complex features spatially but also improves the adaptability and prediction accuracy of the model to complex changes in the vehicle's interior and exterior environments by dynamically adjusting the GRU layer responses. Specifically, the Encoder includes a first feature extraction module, a second feature extraction module, and a third feature extraction module. The first feature extraction module may include three convolutional layers, followed by a max-pooling layer after the first two convolutional layers, and the third convolutional layer is used to adjust the dimension of the output feature representation so that the feature representation output by the first feature extraction module is convenient for concatenation with the feature representations output by the other two feature extraction modules. The dimensions of the input and output data of the first feature extraction module and the input and output data of its respective convolutional layers and max-pooling layers can be set as needed and are not limited here. The second feature extraction module and the third feature extraction module are used for feature extraction of time-domain signals and can be set to include a one-dimensional convolutional layer, a pooling layer, and a convolutional layer for adjusting the dimension of the output feature representation respectively. The dimensions of the input and output data of the second feature extraction module and the third feature extraction module and the input and output data of their respective convolutional layers and pooling layers can be set as needed and are not limited here. After the feature representations output by the three feature extraction modules are concatenated, they are input into the frequency-axis gated recurrent unit. The concatenation method can be, for example, concatenation along the channel dimension, which is not limited here. The role of the frequency-axis gated recurrent unit is to process data along the frequency axis and capture the correlations in the frequency dimension; the concatenated feature representation can be divided into multiple time steps and input into the frequency-axis gated recurrent unit sequentially by time step. After cyclic calculation by the frequency-axis gated recurrent unit, the hidden states corresponding to each time step are output and input into the time-axis gated recurrent unit in sequence. The role of the time-axis gated recurrent unit is to process the output of the frequency-axis gated recurrent unit along the time axis and capture the dynamic features in the time dimension; the time-axis gated recurrent unit performs cyclic calculation on the hidden states of each time step input sequentially, and the output result is further used as the input of the transposed convolutional layer. The dimensions of the input and output data of the frequency-axis gated recurrent unit and the time-axis gated recurrent unit are also not limited here.The Decoder after the time-axis gated recurrent unit can be set to include two transposed convolutional layers and a convolutional layer for adjusting the dimension of the output data. The dimensions of the specific input and output data are not restricted here. The Decoder is followed by a multi-task output layer, which can include a shared feature extraction layer and output layers separately set for each piece of information. For example, if it is necessary to output weather condition information, road surface condition information, and sensor status information, an output layer can be separately set for this kind of information. Among them, the shared feature extraction layer can include one or more convolutional layers, and a pooling layer can be connected after the convolutional layer, whose function is to extract general features and reduce the spatial dimension. The output layers separately set for each piece of information can include a convolutional layer, a global average pooling layer, and a fully connected layer. An activation function is set after the fully connected layer, which is used to output the confidence or probability distribution corresponding to each value category of each piece of information, for determining the final value of each piece of information. The dimensions of the input and output data of the multi-task output layer and the dimensions of the input and output data of each layer therein are also not restricted. In this embodiment, not only is the independent processing of each task ensured, but also the overall performance and resource utilization rate of the model are optimized through the sharing of underlying features and the specialization of top-level decisions.

[0081] In a feasible embodiment, a lightweight, real-time, low-parameter frequency-time structured U-Net network can be set as the prediction model, which is specifically designed to accurately process and analyze various frequency-domain and time-domain data, ensuring the effective fusion of sub-band domain and time-domain multi-source information, effectively improving the recognition speed and accuracy of complex acoustic environments and road conditions, and improving the robustness of the model to changes in the vehicle interior and exterior environments. Specifically, the dimensions of the input and output data of the prediction model and the input and output data of each layer in the model can all conform to the definition of [batch size, feature dimension, number of channels]. The dimension of the input data of the first feature extraction module can be set to [1, 65, 11], where the batch size is 1, indicating that a single sample is processed each time, the feature dimension is 65, corresponding to the frequency-domain features extracted by sub-band analysis (i.e., 65 sub-bands), and the number of channels is 11, including 9 vibration sensor channels and 2 error microphone channels. The dimension of the input data of the second feature extraction module can be set to [1, 64, 9], the batch size is 1, the time step is 64, indicating the length of the time-domain signal (i.e., the frame length is 64), and the number of channels is 9, corresponding to 9 vibration sensors. The dimension of the input data of the third feature extraction module is set to [1, 64, 2], the batch size is 1, the time step is 64, and the number of channels is 2, corresponding to 2 error microphones. When training the prediction model, a sequence containing T frames can be input, such as [1, T, 65, 11], so that the FT-GRU can learn the inter-frame relationship; during inference, it can be input frame by frame, and the FT-GRU realizes inter-frame modeling through state transfer.

[0082] The Encoder includes a first feature extraction module, a second feature extraction module, and a third feature extraction module.

[0083] The first feature extraction module may include: The first convolutional layer: The convolutional kernel dimension is 3x3, the number of input channels is 11, the number of output channels is 16, the padding is 1 ("same" padding), and the output dimension is [1, 65, 16].

[0084] The max pooling layer: The pooling window is 2x1, the stride is 2, and the output dimension is [1, 32, 16].

[0085] The second convolutional layer: The convolutional kernel dimension is 3x3, the number of input channels is 16, the number of output channels is 32, the padding is 1 ("same" padding), and the output dimension is [1, 32, 32].

[0086] The max pooling layer: The pooling window is 2x1, the stride is 2, and the output dimension is [1, 16, 32].

[0087] The 1D convolutional layer: 1D convolution, the convolutional kernel dimension is 4x1, the number of input channels is 32, the number of output channels is 32, the stride is 2x1, the padding is "valid", and the output dimension is [1, 29, 32].

[0088] The final output dimension of the first feature extraction module: [1, 29, 32].

[0089] The second feature extraction module includes: The one-dimensional convolutional layer: The convolutional kernel dimension is 3, the number of input channels is 9, the number of output channels is 16, the padding is 1 ("same" padding), and the output dimension is [1, 64, 16].

[0090] The pooling layer: The pooling window is 2, the stride is 2, and the output dimension is [1, 32, 16].

[0091] The 1D convolutional layer: 1D convolution, the convolutional kernel dimension is 4, the number of input channels is 16, the number of output channels is 32, the stride is 1, the padding is "valid", and the output dimension is [1, 29, 32].

[0092] The third feature extraction module includes: The one-dimensional convolutional layer: The convolutional kernel dimension is 3, the number of input channels is 2, the number of output channels is 32, the padding is 1 ("same" padding), and the output dimension is [1, 64, 32].

[0093] The pooling layer: The pooling window is 2, the stride is 2, and the output dimension is [1, 32, 32].

[0094] 1D Convolutional Layer: 1D convolution, with a convolutional kernel dimension of 4, an input channel number of 32, an output channel number of 32, a stride of 1, padding of "valid", and an output dimension of [1, 29, 32].

[0095] Concatenation Operation in the Feature Fusion Stage: The outputs of the three branches are all [1, 29, 32]. They are concatenated along the channel dimension, i.e., 32 + 32 + 32 = 96, and the output dimension is [1, 29, 96].

[0096] The FT-GRU layer follows the Encoder. The input dimension of the FT-GRU layer is [1, 29, 96], where 29 represents the feature dimension (or time step), and 96 represents the number of input feature channels. The FT-GRU layer includes: The First Layer: Freq GRU (GRU on the frequency axis), Function: Process data along the frequency axis to capture the correlations in the frequency dimension. Input dimension: [1, 29, 96], where 29 represents the time step and 96 represents the feature channel dimension. Processing Method: For each time step (a total of 29), Freq GRU treats the 96-dimensional features as an input sequence (length 96) and performs cyclic calculations to output the hidden state. Assuming the number of hidden units is H (e.g., H = 32), the output is [1, 29, H].

[0097] The Second Layer: Time GRU (GRU on the time axis), Function: Process the output of Freq GRU along the time axis to capture the dynamic features in the time dimension. Input dimension: [1, 29, H]. Processing Method: Perform cyclic operations over 29 time steps. At each time step, input H-dimensional features and output the final features. Assuming the target output channel number is 32, the output dimension is [1, 29, 32].

[0098] The Decoder after the FT-GRU layer includes two transposed convolutional layers and a convolutional layer for adjusting the output data dimension.

[0099] The input dimension of the first transposed convolutional layer is [1, 29, 32]. The size of the transposed convolutional kernel is 4x1, the stride is 2, the padding is 1, the output channel number is 32, and the output dimension is [1, 58, 32].

[0100] The input dimension of the second transposed convolutional layer is [1, 58, 32]. The size of the transposed convolutional kernel is 8x1, the stride is 1, the padding is 0, the output channel number is 16, and the output dimension is [1, 65, 16].

[0101] The input dimension of the third convolutional layer is [1, 65, 16]. The size of the convolutional kernel is 1x1, the stride is 1, the output channel number is 1, and the output dimension is [1, 65, 1].

[0102] The decoder is followed by a multi-task output layer, which includes a shared feature extraction layer and multiple output layers (pavement condition classification branch, weather condition classification branch, and sensor status monitoring branch).

[0103] The input dimension of the shared feature extraction layer is [1, 65, 1], and the shared feature extraction layer includes: Convolutional layer: The convolutional kernel size is 3x1, the input channels are 1, the output channels are 32, the padding is 1 ("same" padding), and the output is [1, 65, 32]. Max pooling layer: The pooling window is 2x1, the stride is 2, and the output is [1, 32, 32]. Its function is to extract general features and reduce the spatial dimension.

[0104] The pavement condition classification branch, weather condition classification branch, and sensor status monitoring branch connected after the shared feature extraction layer.

[0105] The input dimension of the pavement condition classification branch is [1, 32, 32], and the output dimension is [1, N], which is the probability distribution of N pavement conditions. The pavement condition classification branch includes: Convolutional layer: 1x1 convolution, the output channels are 64, and the output dimension is [1, 32, 64].

[0106] Global average pooling: Reducing the dimension to [1, 64].

[0107] Fully connected layer: Output N (the number of pavement condition categories), and the activation function is Softmax.

[0108] The input dimension of the weather condition classification branch is [1, 32, 32], and the output dimension is [1, M], which is the probability distribution of M weather conditions. The weather condition classification branch includes: Convolutional layer: 1x1 convolution, the output channels are 64, and the output dimension is [1, 32, 64].

[0109] Global average pooling: Reducing the dimension to [1, 64].

[0110] Fully connected layer: Output M (the number of weather condition categories), and the activation function is Softmax.

[0111] The input dimension of the sensor status monitoring branch is [1, 32, 32], and the output dimension is [1, 11]. Each value is between 0 and 1, representing the confidence of 11 sensors. The sensor status monitoring branch includes: Convolutional layer: 1x1 convolution, the output channels are 128, and the output dimension is [1, 32, 128].

[0112] Global average pooling: Reducing the dimension to [1, 128].

[0113] Fully connected layer: Output is 11 (corresponding to 11 sensors), and the activation function is Sigmoid.

[0114] Overall, the results output by the prediction model include: Road surface condition classification: [1, N], probability distribution of N road surface conditions.

[0115] Weather condition classification: [1, M], probability distribution of M weather conditions.

[0116] Sensor status monitoring: [1, 11], confidence levels (0 - 1) of 11 sensors.

[0117] Based on the above results output by the prediction model, road surface condition information for characterizing the road surface condition, weather condition information for characterizing the weather condition, and sensor status information for characterizing the sensor status can be obtained. For example, in a feasible implementation manner, the category corresponding to the largest probability distribution among the various probability distributions in the road surface condition classification result can be used as the road surface condition information. Also, for example, it can be pre - set that the magnitude of the confidence level in the sensor status monitoring result represents the degree of sensor abnormality. The sensor status information can be binary classification information indicating whether the sensor is in a normal state or an abnormal state, or information indicating the degree of sensor abnormality, which can be set according to needs in specific implementation manners.

[0118] In a feasible implementation manner, referring to Figure 2 , the prediction model can be trained in the following way in advance.

[0119] Step 1: Data collection. Install accelerometers and microphones with sensitivities and frequency responses meeting the test requirements at the positions where acceleration sensors and microphones need to be installed on the vehicle to comprehensively capture vibration and sound data under different road conditions and speeds.

[0120] Step 2: Data annotation. During the data recording process, the annotation work can be completed synchronously. All data annotation can be directly carried out by annotators during data recording, and the road surface conditions in the data are annotated in real - time and in detail. Specifically, when the microphones and vibration sensors collect data inside and outside the vehicle, the annotators immediately classify and describe the current road surface conditions in detail. For example, these descriptions can include, but are not limited to: road type (such as highway, urban road, rural road), lane markings, road surface texture, pothole conditions, obstacle distribution, and road slipperiness. The annotators will also record (such as rainy or snowy days) and annotate the weather conditions affecting the road surface conditions.

[0121] Step 3: Data augmentation. To improve the model's ability to recognize abnormal situations, sensor faults and noises can be artificially introduced. First, the robustness of the model can be enhanced by simulating various sensor faults and environmental interferences. For example, covering the microphone with cloth, plastic or metal sheets to simulate the blockage of dust and water, and simulating abnormal impacts by tapping the accelerometer. In addition, digital signal processing techniques can also be used to add various synthetic noises to the normal data, such as white noise, natural environmental noise, and signals simulating electromagnetic interference. When introducing these faults and interferences, the data can be systematically segmented by time and classified and labeled, such as "microphone occlusion - cloth" or "accelerometer impact interference", to ensure the richness of the dataset and the generalization ability of the trained model. These refined operations not only increase the model's adaptability to complex real-world situations, but also help maintain the performance and stability of the system in various environments.

[0122] Step 4: Dataset balancing. To address the problem of uneven data distribution in the collected data, data analysis can be performed to identify which categories or scenarios have too little or too much data. Then, data augmentation techniques can be used to increase the number of samples in the scarce categories, such as generating new data by varying speed, adding noise, or using synthetic techniques. For the excessive categories, a downsampling strategy can be adopted to selectively reduce the samples, or representative samples can be selected through cluster analysis. In addition, resampling techniques can also be used to balance the entire dataset to ensure that the data of each category is evenly distributed in the training set, thereby improving the generalization ability and performance of the model under all conditions. This step can also include applying statistical methods to evaluate the changes before and after data balancing to verify the effectiveness of the balancing strategy and adjust the balancing parameters as needed.

[0123] Step 5: Model architecture setting. For example, the model architecture in the above embodiments can be set.

[0124] Step 6: Model training. Customized loss functions can be used for model training to handle different classification tasks. Cross-entropy loss can be used for multi-class classification tasks (such as road surface conditions and weather conditions) to ensure that the model can effectively distinguish different categories; for the binary classification problem of sensor status, binary cross-entropy loss can be adopted to improve the sensitivity of recognizing normal and abnormal states. During the training process, the Adam optimizer (Adaptive Moment Estimation) can be selected, and its adaptive learning rate mechanism helps to better handle the problem of gradient sparsity. At the same time, SGD (Stochastic Gradient Descent) with momentum can also be considered to stabilize the training process. In addition, early stopping mechanisms and regularization techniques can also be adopted to prevent overfitting and ensure that the model can perform well on unseen data.

[0125] Step 7: Verification and testing. To ensure the effectiveness and reliability of the model in practical applications, the dataset can be divided into a training set, a validation set, and an independent test set. During the model development process, the validation set is used for periodic evaluation to monitor the training progress and adjust hyperparameters. The test set is used for the final evaluation after the model training is completed to test the performance of the model on completely unknown data. Various performance metrics, such as accuracy, precision, recall, and F1 score, are applied to evaluate the classification task, and the ROC (receiver operating characteristic) curve and AUC (Area Under Curve) value are used to measure the performance of the model for sensor status anomaly detection.

[0126] Based on the above first and / or second embodiments, a third embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first and second embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, a load balancing strategy is proposed, so that the automotive road noise processing algorithm proposed in the above embodiments can be applied to the automotive RNC system with limited computing power conditions. For example, in a MIMO vehicle-mounted RNC system, there are a large number of computational peak demands within a single sampling period. If the subband adaptive algorithm without output delay is run in real time on an embedded platform such as a DSP (Digital Signal Processor), the system may not be able to run in real time due to insufficient peak computing power. The load balancing strategy proposed in this embodiment aims to ensure that the subband adaptive algorithm without output delay can be run in real time on an embedded platform such as a DSP, solves the problem of insufficient peak computing power, and provides an important guarantee for practical applications. Specifically, in this embodiment, the automotive road noise processing method further includes step S70: Execute the tasks allocated to the N interrupts among the N interrupts in the (l + 1)-th frame, where N represents the frame length, and the interrupt refers to the period from generating the anti-noise signal based on the time-domain reference signal of each sampling point to receiving the time-domain reference signal of the next sampling point. The tasks allocated to the N interrupts include: the task of calculating the updated time-domain adaptive filter coefficients, and / or the task of inputting the first time-domain reference signal, the first time-domain error signal, the first subband reference signal, and the first subband error signal into the prediction model to predict the environmental state information.

[0127] In this embodiment, the control unit performs road noise control in a manner without output delay, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit in real time point by point. The control unit generates a single-sampling-point anti-noise signal for the currently input single-sampling-point time-domain reference signal in real time according to the current time-domain adaptive filter coefficients, and then outputs the anti-noise signal in real time to achieve the effect of no output delay. On the other hand, the adaptive filter coefficients are updated frame by frame. Specifically, the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffer in real time. After each frame of time-domain reference signal and each frame of time-domain error signal are filled, sub-band decomposition is performed on the frame of time-domain reference signal and the frame of time-domain error signal, and subsequent time-domain adaptive filter coefficient update operations are carried out.

[0128] Let the frame index of the first time-domain reference signal and the first time-domain error signal be l. Then, after updating the time-domain adaptive filter coefficients based on the l-th frame of time-domain reference signal and time-domain error signal, the updated time-domain adaptive filter coefficients after this update will be used to process each point of the time-domain reference signal (i.e., the second time-domain reference signal) of the (l + 1)-th frame with N sampling points to generate an anti-noise signal with N sampling points. Specifically, for each received sampling point of the time-domain reference signal, an anti-noise signal with one sampling point is generated and output according to the updated time-domain adaptive filter coefficients after this update. Then, it can be understood that during the process of processing each point of the time-domain reference signal with N sampling points of the (l + 1)-th frame, it is also necessary to complete the update of the time-domain adaptive filter coefficients based on the time-domain reference signal and time-domain error signal of the (l + 1)-th frame to be used for processing the time-domain reference signal of each sampling point of the (l + 2)-th frame to generate an anti-noise signal, and so on.

[0129] It can be understood that during the period from the start of receiving the first sampling point of the (l + 1)-th frame to the generation of the anti-noise signal by processing the N-th sampling point of the (l + 1)-th frame, it is necessary to complete an update operation of the time-domain adaptive filter coefficients based on the time-domain reference signal and time-domain error signal of the l-th frame. In this embodiment, the update task of the adaptive filter coefficients can be distributed to N interrupts for execution, avoiding the execution of the entire adaptive filter coefficient update task in a single interrupt, thereby avoiding the problem of insufficient computing power caused by a sharp increase in computing power.

[0130] During the process of updating the time-domain adaptive filter coefficients based on the l-th frame time-domain reference signal and the time-domain error signal, the l-th frame time-domain reference signal, the time-domain error signal, and the corresponding subband signals can be input into the prediction model to predict the environmental state information and / or the sensor state information. The filter bank determined based on the environmental state information and / or the sensor state information is used to perform subband decomposition operations on the (l + 2)-th frame time-domain reference signal and the time-domain error signal. Then, within the time period from the start of receiving the first sampling point of the (l + 1)-th frame to the processing of the n-th sampling point of the (l + 1)-th frame to generate the noise-canceling signal, it is also necessary to complete the task of inputting the l-th frame time-domain reference signal, the time-domain error signal, and the corresponding subband reference signals and subband error signals into the prediction model to predict the environmental state information and / or the sensor state information. In this embodiment, the prediction task can be distributed among N interrupts for execution, avoiding the execution of the entire prediction task in a single interrupt, thereby avoiding the problem of insufficient computing power caused by a sharp increase in computing power.

[0131] The load balancing strategy proposed in this embodiment ensures the full utilization of each interrupt, avoids the situation of any single interrupt being overloaded, optimizes the use of computing resources, and guarantees the high efficiency and response speed of the system in real-time processing. This method of distributing computing tasks to multiple interrupts helps the system maintain stable operation when facing high computing demands, thereby achieving continuous and efficient operation in various driving environments.

[0132] In this embodiment, there is no limitation on the way of distributing tasks to each interrupt. For example, the computing operations in the task can be divided into multiple subtasks according to the amount of computation, and the amount of computation of each subtask is the same or approximately the same, and each interrupt processes one subtask.

[0133] In a feasible implementation manner, the task of updating the adaptive filter coefficients can be divided into a subband decomposition task, a subband synthesis task, and multiple subband update tasks. Each subband update task includes the task of updating at least one subband adaptive filter coefficient. The subband decomposition task is executed in the first preset number of interrupts, the subband synthesis task is executed in the subsequent preset second number of interrupts, and one of the subband update tasks is executed in each of the intermediate preset third number of interrupts. The sum of the preset first number, the preset second number, and the preset third number is N.

[0134] In a feasible implementation manner, the prediction task can be divided into multiple subtasks, and each subtask includes the processing task of at least one layer in the prediction model. One of the subtasks is executed in each interrupt. For example, let N = 64. The first 32 interrupts process the operations of the Encoder layer, including convolution, activation, pooling, and the FGRU layer. The latter half of the interrupts complete the operations of the TGRU and Decoder layers, and the last interrupt completes the operations of the multi-task output layer.

[0135] It should be noted that when both the adaptive filter coefficient update task and the prediction task need to be allocated to N interrupt processes, since the sub-band signals are required in the prediction task, the task of sub-band decomposition can be allocated to the earlier interrupts, and the prediction task can be allocated to the later interrupts. For example, each sub-task of the prediction task can be allocated to be executed in each interrupt except for the first preset number of interrupts at the front.

[0136] Based on the above first, second, and / or third embodiments, a fourth embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first, second, and third embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, the update step size of the adaptive filter coefficients used in the adaptive algorithm is dynamically adjusted to balance the adaptive convergence speed and system stability, so as to improve the system's adaptive ability and stability and avoid problems such as delayed response or unstable output. Specifically, the control unit can obtain the influence factor information and dynamically determine the target update step size corresponding to the sub-band adaptive filter coefficients according to the influence factor information, where the influence factor information includes environmental state information and / or sensor state information, the environmental state information is information characterizing the environmental state inside and / or outside the vehicle, the sensor state information is information characterizing the working state of the sensor, and the sensor includes a sensor for collecting the time-domain reference signal and / or a sensor for collecting the time-domain error signal.

[0137] It should be noted that the update step size can be determined according to the environmental state information used to determine the synthesis filter bank.

[0138] The influence factor information is information characterizing the specific manifestation of the influencing factors, and the influencing factors are the factors determining the adjustment strategy of the update step size of the sub-band adaptive filter coefficients, that is, the specific manifestation of these factors determines different adjustment strategies of the update step size. In this embodiment, the influencing factors can at least include one or more of the environmental state inside the vehicle, the environmental state outside the vehicle, the working state of the reference sensor, and the working state of the error sensor. In other embodiments, other influencing factors can also be included, such as vehicle speed.

[0139] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause dynamic changes in the acoustic propagation path, which will affect the energy magnitude or stability of the vehicle road noise. The effects on the energy magnitude or stability of different sub-bands of noise are also different. If the update step size of the sub-band adaptive filter coefficients adopts a fixed step size, when the energy magnitude or stability of the vehicle road noise changes, it may lead to misjudgment of the system or poor acoustic effects due to too large an update step size and too fast update of the sub-band adaptive filter coefficients. It may also lead to a slow convergence speed of the adaptive filter and poor noise reduction effect due to too small an update step size. Therefore, in the specific implementation, the environmental state outside the vehicle and the environmental state inside the vehicle can be used as influencing factors to dynamically adjust the update step size based on the environmental state, balance the adaptive convergence speed and system stability, and avoid delayed response or unstable output. When the working states of the reference sensor and the error sensor change, the accuracy of the collected signal will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the estimation of the vehicle road noise in the signal will be incorrect. If the update step size of the sub-band adaptive filter coefficients adopts a fixed step size and the adaptive filter coefficients are still continuously updated, it will cause the generated anti-noise signal to be unable to accurately cancel the real vehicle road noise, resulting in unstable noise reduction effect. Therefore, in the specific implementation, the working states of the reference sensor and the error sensor can be used as influencing factors to dynamically adjust the update step size based on the working states of the sensors, balance the adaptive convergence speed and system stability, and avoid delayed response or unstable output.

[0140] In this embodiment, there is no limitation on the way to obtain the influencing factor information. For example, the environmental state information can be obtained by image analysis of the environmental images collected by the camera, and the sensor state information can be determined by outlier analysis of the signals collected by the sensors.

[0141] The influencing factor information can be obtained every once in a while. The shorter the time interval, the higher the real-time degree. In this embodiment, there is no limitation on the acquisition frequency. For example, in a feasible implementation, the influencing factor information can be obtained every time the adaptive filter coefficients are updated.

[0142] Dynamically determining the update step size of the sub-band adaptive filter coefficients means that after each new influencing factor information is obtained, the update step size is determined according to the new influencing factor information, so that the update step size of the adaptive filter coefficients changes according to the specific performance of the influencing factors, and the update speed of the adaptive filter coefficients is adjusted by dynamically adjusting the update step size, so as to adapt to the changes in the internal or external environment state of the vehicle, or adapt to the changes in the working state of the sensor, balance the adaptive convergence speed and system stability, and avoid problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensure the robustness and safety of the system, and avoid delayed response or unstable output.

[0143] The specific method of determining the update step size of the sub-band adaptive filter coefficients according to the influencing factor information is not limited in this embodiment. For example, the mapping relationship between the influencing factor information and the update step size of the sub-band adaptive filter coefficients can be set in advance as needed. After the influencing factor information is obtained, the control unit uses the update step size corresponding to the influencing factor information as the target update step size according to this mapping relationship. In this embodiment, the mapping relationship is not limited. For example, the mapping relationship can be represented by a mapping table, a relational expression, etc., and the mapping relationship can be determined by experiments, data statistics or other methods.

[0144] It should be noted that since there are multiple sub-bands, there are multiple sub-band adaptive filters, and the target update step sizes of the coefficients of each sub-band adaptive filter may be the same or different, which is not limited in this embodiment.

[0145] In this embodiment, by dynamically determining the update step size of the sub-band adaptive filter coefficients according to the obtained environmental state information and / or sensor state information, updating the sub-band adaptive filter coefficients according to the determined update step size, and then converting the sub-band adaptive filter coefficients into time-domain adaptive filter coefficients, using the time-domain adaptive filter coefficients to process the time-domain reference signal to generate an anti-noise signal, so as to control the road noise based on the anti-noise signal, realize adapting to the changes in the internal or external environment state of the vehicle, or adapting to the changes in the working state of the sensor, balance the adaptive convergence speed and system stability, and avoid problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensure the robustness and safety of the system, and avoid delayed response or unstable output.

[0146] In a feasible implementation manner, to improve the noise reduction effect of the system and ensure the noise reduction stability, it is set that when the control unit dynamically determines the target update step size according to the influencing factor information, it can conform to one or more of the following rules: 1. Under the environmental state characterized by the environmental state information, the target update step corresponding to the sub-band with higher automotive road noise energy or higher stability degree is larger than the target update step corresponding to the sub-band with lower automotive road noise energy or lower stability degree.

[0147] Under the same environmental state, the distribution of automotive road noise on each sub-band may be different; for the sub-band with higher noise energy, a larger update step can be adopted to specifically enhance the noise reduction intensity of this sub-band, improve the noise reduction effect, and avoid delayed response; for the sub-band with lower noise energy, a smaller update step can be adopted to ensure the stability of the noise reduction effect and avoid unstable output. Under the same environmental state, the stability degree of automotive road noise on each sub-band may be different; for the sub-band with higher stability degree, a larger update step can be adopted to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response; for the sub-band with lower stability degree, a smaller update step can be adopted to avoid misjudgment of the system or generating adverse acoustic effects, and ensure the robustness and safety of the system.

[0148] It should be noted that the stability degree of automotive road noise refers to the stability degree of the change in noise energy.

[0149] 2. For any target sub-band, under the environmental state characterized by the environmental state information, the higher the energy or the higher the stability degree of the automotive road noise in the target sub-band, the larger the target update step corresponding to the target sub-band; the lower the energy or the lower the stability degree of the automotive road noise in the target sub-band, the smaller the target update step corresponding to the target sub-band.

[0150] For the same sub-band, the energy of vehicle road noise in this sub-band may be different under different environmental conditions. Therefore, for a certain sub-band (referred to as the target sub-band for distinction), if the energy of vehicle road noise in the target sub-band is high under a certain environmental condition, a larger update step size can be adopted for the target sub-band to specifically enhance the noise reduction intensity of this sub-band, improve the noise reduction effect, and avoid delayed response. If the energy of vehicle road noise in the target sub-band is low under a certain environmental condition, a smaller update step size can be adopted for the target sub-band to ensure the stability of the noise reduction effect. For the same sub-band, the stability degree of vehicle road noise in this sub-band may be different under different environmental conditions. Therefore, for a certain sub-band (referred to as the target sub-band for distinction), if the stability degree of vehicle road noise in the target sub-band is high under a certain environmental condition, a larger update step size can be adopted for the target sub-band to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response. If the energy of vehicle road noise in the target sub-band is low under a certain environmental condition, a smaller update step size can be adopted for the target sub-band to avoid misjudgment of the system or generation of adverse acoustic effects, and ensure the robustness and safety of the system.

[0151] 3. According to the abnormal degrees of each sensor characterized by the sensor status information, the target update step size corresponding to the sub-band where the sensor with a higher abnormal degree is located is smaller than the target update step size corresponding to the sub-band where the sensor with a lower abnormal degree is located.

[0152] The sub-band where the sensor is located refers to the sub-band in which the signal collected by the sensor is distributed or mainly distributed, and the sub-bands where different sensors are located may be different.

[0153] The sensor status information is used to characterize the working status of the sensor. In the specific implementation manner, the working status of the sensor can be divided into two types, namely normal operation and abnormal operation. The sensor status information can be the information used to indicate whether the sensor belongs to the normal working state or the abnormal working state. Then it can be understood that the abnormal degree of the sensor with the working status of "normal operation" is lower than that of the sensor with the working status of "abnormal operation". Or, the working status of the sensor can also be divided into more than two types. For example, different abnormal degrees are represented by values within 0-1, and the sensor status information can be the value indicating the abnormal degree of the sensor.

[0154] Based on the acquired sensor status information, the abnormality degree of each sensor can be determined, and the abnormality degrees of different sensors may be different. For the sensor with a higher abnormality degree, a smaller update step size can be adopted for the sub-band where the sensor is located to avoid misjudgment of the system and produce poor acoustic effects. For the sensor with a lower abnormality degree, a larger update step size can be adopted for the sub-band where the sensor is located to ensure the noise reduction effect of the system and avoid delayed response.

[0155] 4. For any target sub-band, the higher the abnormality degree of the sensors belonging to the target sub-band, the smaller the target update step size corresponding to the target sub-band; the lower the abnormality degree of the sensors belonging to the target sub-band, the larger the target update step size corresponding to the target sub-band.

[0156] For a certain sub-band (hereinafter referred to as the target sub-band for distinction), the sensors belonging to the target sub-band refer to the sensors whose signals are all or mainly distributed in this sub-band.

[0157] For the target sub-band, different sensor status information is obtained, and the abnormality degrees of the sensors belonging to the target sub-band may be different. If the abnormality degree of the sensors belonging to the target sub-band is higher, a smaller update step size can be adopted for the target sub-band to avoid misjudgment of the system and produce poor acoustic effects. If the abnormality degree of the sensors belonging to the target sub-band is lower, a larger update step size can be adopted for the target sub-band to ensure the noise reduction effect of the system and avoid delayed response.

[0158] In a feasible implementation manner, the influencing factor information may further include the real-time vehicle speed of the vehicle. When the vehicle speed changes, it will affect the energy magnitude or stability of the vehicle road noise. If a fixed update step size is adopted for the update step size of the sub-band adaptive filter coefficient when the vehicle speed changes, it may lead to misjudgment or poor acoustic effects due to too large an update step size and too fast update of the sub-band adaptive filter coefficient when the energy magnitude or stability of the vehicle road noise changes. Therefore, the vehicle speed can be used as an influencing factor.

[0159] In a feasible implementation manner, when the influencing factor information further includes the vehicle speed, when dynamically determining the target update step size according to the influencing factor information, the following rule or rules may also be satisfied: 5. The higher the vehicle speed stability, the larger the target update step size corresponding to each sub-band; the lower the vehicle speed stability, the smaller the target update step size corresponding to each sub-band.

[0160] The vehicle speed stability refers to, for example, the change rate of the vehicle speed. The larger the change rate, the lower the vehicle speed stability; the smaller the change rate, the higher the vehicle speed stability.

[0161] The higher the vehicle speed stability, the higher the stability of the vehicle road noise. At this time, a larger update step size can be adopted to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response. The lower the vehicle speed stability, the lower the stability of the vehicle road noise. At this time, a smaller update step size can be adopted to avoid misjudgment of the system or the generation of adverse acoustic effects, and ensure the robustness and safety of the system.

[0162] 6. When the vehicle speed is outside the preset range, the target update step size corresponding to each sub-band is smaller than the target update step size corresponding to each sub-band when the vehicle speed is within the preset range.

[0163] The preset range indicates that the vehicle speed is at a normal level. When the vehicle speed is outside the preset range, it indicates a relatively extreme or abnormal situation, and the vehicle road noise may have relatively extreme or abnormal manifestations. For such extreme or abnormal situations, a smaller update step size can be adopted to avoid misjudgment or the generation of adverse acoustic effects.

[0164] In a specific implementation manner, in order to enable the control unit to conform to one or more of the above rules when dynamically determining the target update step size according to the influence factor information, a mapping relationship that satisfies one or more of the above rules can be set. The control unit determines the update step size corresponding to the influence factor information as the target update step size according to this mapping relationship. The mapping relationship can be represented by a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple of the above rules, the multiple rules can conform to a certain priority order to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the update step size according to the sensor status information can be higher than the priority of determining the update step size according to the environmental status information.

[0165] In a feasible implementation manner, the environmental status information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to the weather condition that affects the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information characterizing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0166] In a feasible implementation, when the environmental status information and the sensor status information indicate that the environment is good and stable (for example: good road conditions, clear weather, normal sensors, medium vehicle speed), the vehicle road noise is relatively stable, and there will be no particularly abnormal and prominent noise in each sub-band. At this time, a moderate or large update step size can be adopted to ensure the adaptive convergence speed of the system and a low steady-state error level. When the environmental status information and the sensor status information indicate that the environment deteriorates or becomes unstable (for example: deteriorating road conditions, bad weather, abnormal sensors, drastic changes in vehicle speed), the stability of the vehicle road noise may deteriorate. At this time, a smaller update step size can be adopted to improve the system stability. When the environmental status information and the sensor status information indicate the existence of a specific type of noise (for example: waterlogged road surface, brick road surface, wind noise), the step sizes of different frequency sub-bands can be adjusted differentially. A larger update step size is adopted for the sub-band with high noise energy to specifically enhance or weaken the filtering intensity of a specific frequency band and optimize the noise reduction effect. When the environmental status information and the sensor status information indicate an extremely harsh environment or system abnormality (for example: extremely harsh road conditions, serious sensor failure, extremely low vehicle speed), the update step size can be greatly reduced or even frozen to avoid system misjudgment or adverse acoustic effects and ensure the robustness and safety of the system.

[0167] Table 2 below gives some examples of determining the update step sizes for the low-frequency, medium-frequency, and high-frequency bands based on the weather condition information, road surface condition information, sensor status information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Among them, the low-frequency, medium-frequency, and high-frequency bands can be divided as needed. It can be understood that each of the three frequency bands contains one or more sub-bands. Table 2 only gives some examples and does not limit the method of determining the target update step size in this embodiment. The content in parentheses in Table 2 is the explanatory description of the weather condition information, road surface condition information, sensor status information, update step size, etc.

[0168] Table 2

[0169] Based on the above first, second, third, and / or fourth embodiments, the fifth embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the same or similar content as the above first, second, third, and / or fourth embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, the target limiting parameter value can also be dynamically determined according to the acquired environmental state information and / or sensor state information. After generating the anti-noise signal, the target limiting parameter value is used to limit the amplitude of the anti-noise signal, and the anti-noise signal after amplitude limitation is output to control the road noise. That is, post-processing operations can be performed on the anti-noise signal. The post-processing operations include amplitude limitation of the anti-noise signal, and the amplitude limitation is performed according to the target limiting parameter value determined according to the environmental state information and / or sensor state information. The limiting parameter value is a parameter value used to limit the degree of amplitude limitation. What specific parameter is not limited in this embodiment. For example, it can be an upper limit amplitude value, indicating that the amplitude value of the finally output signal is not allowed to exceed this upper limit amplitude value.

[0170] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause a dynamic change in the acoustic propagation path, which will affect the stability of the vehicle road noise. If a fixed limiting parameter value is used for the amplitude limitation of the anti-noise signal, when the stability of the vehicle road noise changes, it may be due to excessive amplitude limitation that the noise cannot be fully cancelled, or it may be due to too small amplitude limitation that noise or distortion may be introduced due to misjudgment or overcompensation. Therefore, in the specific implementation, the limiting parameter value can be dynamically adjusted according to the environmental state information, so as to dynamically control the output amplitude of the anti-noise signal, adapt to different environmental states, ensure a good noise reduction effect, and avoid introducing noise or distortion due to misjudgment or overcompensation, and ensure the stability and safety of the system.

[0171] When the working state of the reference sensor or the working state of the error sensor changes, the accuracy of the collected signal will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the vehicle road noise in the signal will be estimated incorrectly, and the generated anti-noise signal may not only not cancel the noise but also introduce noise. Therefore, in the specific implementation, the limiting parameter value can be dynamically adjusted according to the sensor state information, so as to dynamically control the output amplitude of the anti-noise signal. For example, when the sensor is abnormal, the output amplitude can be reduced, so as to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0172] The specific method for determining the target amplitude limiting parameter value according to the environmental state information and / or sensor state information is not limited in this embodiment. For example, a mapping relationship between the environmental state information and / or sensor state information and the amplitude limiting parameter value can be set in advance. After obtaining the environmental state information and / or sensor state information, the control unit uses the mapping relationship to take the amplitude limiting parameter value corresponding to the environmental state information and / or sensor state information as the target amplitude limiting parameter value. In this embodiment, the mapping relationship is not limited. For example, the mapping relationship can be represented by a mapping table, a relational expression, etc., and the mapping relationship can be determined through experiments, data statistics, or other methods.

[0173] In a feasible implementation manner, it can be set that when the control unit determines the target amplitude limiting parameter value according to the influence factor information, it conforms to one or more of the following rules: 1. The higher the stability degree of the vehicle road noise in the environmental state characterized by the environmental state information, the smaller the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter value; the lower the stability degree of the vehicle road noise in the environmental state characterized by the environmental state information, the greater the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter.

[0174] If the stability degree of the vehicle road noise is relatively high in the environmental state characterized by the environmental state information, an amplitude limiting parameter value with a relatively small degree of amplitude limitation of the anti-noise signal can be adopted, that is, an anti-noise signal with a larger output amplitude is allowed, so as to more fully cancel the noise and ensure a good noise reduction effect. If the stability degree of the vehicle road noise is relatively low in the environmental state characterized by the environmental state information, an amplitude limiting parameter value with a relatively large degree of amplitude limitation of the anti-noise signal can be adopted, that is, the amplitude of the output signal is limited to a greater extent, so as to avoid introducing noise or distortion due to system misjudgment or overcompensation and ensure system stability.

[0175] 2. The higher the degree of abnormality of the sensor characterized by the sensor state information, the greater the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter; the lower the degree of abnormality of the sensor characterized by the sensor state information, the smaller the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter.

[0176] In the case of higher sensor abnormality, the degree of amplitude limitation of the anti-noise signal is greater, so as to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0177] In a feasible implementation manner, based on the environmental state information and / or sensor state information, the target amplitude limiting parameter value can also be determined according to the vehicle speed. When determining the target amplitude limiting parameter value according to the vehicle speed, it conforms to the following rule: 3. The degree of amplitude limitation of the anti-noise signal according to the target amplitude limitation parameter is smaller when the vehicle speed stability is higher, and the degree of amplitude limitation of the anti-noise signal according to the target amplitude limitation parameter is larger when the vehicle speed stability is lower.

[0178] The vehicle speed stability refers, for example, to the change rate of the vehicle speed. The higher the change rate, the lower the vehicle speed stability, and the lower the change rate, the higher the vehicle speed stability.

[0179] When the vehicle speed stability is relatively high, the stability of the vehicle road noise is relatively high. At this time, a limit parameter value with a smaller degree of amplitude limitation for the anti-noise signal can be adopted, that is, an anti-noise signal with a larger allowable output amplitude can be allowed, so as to more fully cancel the noise and ensure a good noise reduction effect. When the vehicle speed stability is relatively low, the stability of the vehicle road noise is relatively low. At this time, a limit parameter value with a larger degree of amplitude limitation for the anti-noise signal can be adopted, that is, the amplitude of the output signal is limited to a greater extent, so as to avoid introducing noise or distortion due to system misjudgment or overcompensation and ensure system stability.

[0180] In the specific implementation manner, in order to make the control unit determine the target amplitude limitation parameter value in accordance with one or more of the above laws, a mapping relationship that satisfies one or more of the above laws can be set. The control unit determines the amplitude limitation parameter value corresponding to the environmental state information, the sensor state information, and the vehicle speed as the target amplitude limitation parameter value according to this mapping relationship. The mapping relationship can be represented by means of a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple of the above laws, there can be a certain priority order between multiple laws to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the amplitude limitation parameter value according to the sensor state information can be higher than the priority of determining the amplitude limitation parameter value according to the amplitude limitation parameter value.

[0181] In a feasible implementation, when the environmental status information and the sensor status information indicate that the environment is good and stable (for example: good road conditions, clear weather, normal sensors, medium vehicle speed), the road noise of the vehicle is relatively stable. At this time, a clipping parameter value with a relatively small amplitude limit for the anti-noise signal can be adopted, allowing a larger amplitude anti-noise signal to be output, so as to more fully cancel the noise and ensure a good noise reduction effect. When the environmental status information and the sensor status information indicate that the environment deteriorates or becomes unstable (for example: deteriorated road conditions, bad weather, abnormal sensors, drastic vehicle speed changes), the stability of the vehicle road noise may deteriorate. At this time, a clipping parameter value with a relatively large amplitude limit for the anti-noise signal can be adopted, that is, the amplitude of the output signal is limited to a greater extent, to avoid introducing noise or distortion due to system misjudgment or overcompensation and ensure system stability. When the environmental status information and the sensor status information indicate an extremely harsh environment or system abnormality (for example: extremely harsh road conditions, serious sensor failure, extremely low speed), the amplitude of the output signal can be greatly limited to give priority to ensuring system safety.

[0182] The following Table 3 gives some examples of determining the output clipping based on the weather condition information, road surface condition information, sensor status information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Table 3 only gives some examples and does not limit the way of determining the target clipping parameter value in this embodiment. The content in the brackets in Table 3 is an explanatory description of the weather condition information, road surface condition information, sensor status information, output clipping, etc.

[0183] Table 3

[0184] Based on the above first, second, third, fourth, and / or fifth embodiments, a sixth embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first, second, third, fourth, and / or fifth embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, the pre-weighting weights corresponding to each sensor can also be determined according to the obtained environmental status information and / or sensor status information. After the reference signal and the error signal are weighted using the pre-weighting weights corresponding to their respective sensors, they are then used to generate the anti-noise signal and update the coefficients of the adaptive filter. That is, a preprocessing operation is performed on the input reference signal and error signal. The preprocessing operation includes amplitude adjustment, and the amplitude adjustment is performed according to the pre-weighting weights corresponding to each sensor determined according to the environmental status information and / or sensor status information.

[0185] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause a dynamic change in the acoustic propagation path, which will affect the energy magnitude of the vehicle road noise, and the influence on the energy magnitude of the vehicle road noise in different frequency bands may also be different. If a fixed amplitude adjustment strategy is adopted for the reference signal and the error signal, when the energy magnitude of the vehicle road noise changes, it may be impossible to generate a more targeted anti-noise signal because the noise signals in the reference signal and the error signal are weak, resulting in poor noise reduction effect. Therefore, in the specific implementation, the pre-weighting weights of each sensor can be dynamically adjusted according to the environmental state information. Since the frequency bands to which the signals collected by different sensors are distributed may be different, by adjusting the pre-weighting weights of the sensors, the signals in each frequency band can be enhanced or weakened specifically. For example, the sensors corresponding to the sub-bands with stronger vehicle road noise energy can be set with higher pre-weighting weights, so that a more targeted anti-noise signal can be generated and the noise reduction effect can be improved.

[0186] When the working state of the reference sensor and the working state of the error sensor change, the accuracy of the collected signal will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the vehicle road noise in the signal will be estimated incorrectly, and the generated anti-noise signal may not only fail to cancel the noise but also introduce noise. Therefore, in the specific implementation, the pre-weighting weights of each sensor can be dynamically adjusted according to the sensor state information. For example, the pre-weighting weight of the abnormal sensor can be reduced, so as to avoid introducing noise or distortion due to misjudgment or over-compensation, and ensure the safety and robustness of the system.

[0187] The specific method for determining the pre-weighting weight according to the environmental state information and / or the sensor state information is not limited in this embodiment. For example, the mapping relationship between the environmental state information and / or the sensor state information and the pre-weighting weight can be set in advance. After obtaining the environmental state information and / or the sensor state information, the control unit uses the pre-weighting weight corresponding to the environmental state information and / or the sensor state information as the target pre-weighting weight according to this mapping relationship. In this embodiment, this mapping relationship is not limited. For example, this mapping relationship can be represented by a mapping table, a relational expression, etc., and this mapping relationship can be determined by experiments, data statistics or other methods.

[0188] In a feasible implementation, it can be set that when the control unit determines the pre-weighting weight according to the environmental state information and / or the sensor state information, it conforms to one or more of the following rules: 1. Under the environmental state characterized by the environmental state information, the pre-weighting weight of the sensor corresponding to the sub-band with higher energy of the vehicle road noise is greater than that of the sensor corresponding to the sub-band with lower energy of the vehicle road noise.

[0189] The sensor corresponding to the sub-band refers to the sensor whose signal is all or mainly distributed in this sub-band.

[0190] Under the same environmental state, the distribution of vehicle road noise in each sub-band may be different; for the sensor corresponding to the sub-band with higher noise energy, a larger pre-weighting weight can be adopted to specifically enhance the noise reduction intensity of this sub-band and improve the noise reduction effect; for the sub-band with lower noise energy, a smaller pre-weighting weight can be adopted to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0191] 2. For any target sub-band, under the environmental state characterized by the environmental state information, the higher the energy of the vehicle road noise in the target sub-band, the greater the pre-weighting weight of the sensor corresponding to the target sub-band, and the lower the energy of the vehicle road noise in the target sub-band under the environmental state characterized by the environmental state information, the smaller the pre-weighting weight of the sensor corresponding to the target sub-band.

[0192] For the same sub-band, under different environmental states, the energy level of vehicle road noise in this sub-band may be different; so for a certain sub-band (referred to as the target sub-band for distinction), if the energy of vehicle road noise in the target sub-band is higher under a certain environmental state, a larger pre-weighting weight can be adopted for the sensor corresponding to the target sub-band to specifically enhance the noise reduction intensity of this sub-band and improve the noise reduction effect; if the energy of vehicle road noise in the target sub-band is lower under a certain environmental state, a smaller pre-weighting weight can be adopted for the sensor corresponding to the target sub-band to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0193] 3. According to the degree of abnormality of each sensor characterized by the sensor state information, the higher the degree of abnormality of the sensor, the lower the corresponding pre-weighting weight, and the lower the degree of abnormality of the sensor, the higher the corresponding pre-weighting weight.

[0194] According to the obtained sensor state information, the degree of abnormality of each sensor can be determined, and the degree of abnormality of each sensor may be different; for the sensor with a higher degree of abnormality, a smaller pre-weighting weight can be adopted to avoid introducing noise or distortion due to misjudgment or overcompensation, and for the sensor with a lower degree of abnormality, a larger pre-weighting weight can be adopted to ensure the noise reduction effect of the system.

[0195] In a specific implementation manner, in order for the control unit to conform to one or more of the above rules when determining the pre-weighting weights of the sensors, a mapping relationship that conforms to one or more of the above rules can be set. The control unit determines the pre-weighting weight corresponding to the environmental state information and / or sensor state information as the target pre-weighting weight according to this mapping relationship. The mapping relationship can be represented by means of a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple above rules, there can be a certain priority order among the multiple rules to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the pre-weighting weight according to the sensor state information can be higher than the priority of determining the pre-weighting weight according to the environmental state information.

[0196] In a feasible implementation manner, the environmental state information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to the weather condition that affects the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information characterizing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0197] In a feasible implementation manner, three frequency bands, namely low frequency, medium frequency, and high frequency, can be divided; when the environmental state information indicates that the low-frequency noise energy is high (such as in wet and waterlogged weather), a higher pre-weighting weight can be adopted for the signals collected by the low-frequency sensors (sensors whose collected signals are mainly distributed in the low-frequency range). For example, the pre-weighting of the signals collected by the low-frequency sensors is increased by 20%, and the remaining sensor channels maintain a 1.0-fold reference; when the environmental state information indicates that the medium-frequency noise energy is high (such as in brick road surface, dry weather), a higher pre-weighting weight can be adopted for the signals collected by the medium-frequency sensors (sensors whose collected signals are mainly distributed in the medium-frequency range). For example, the pre-weighting of the signals collected by the medium-frequency sensors is increased by 20%, and the remaining sensor channels maintain a 1.0-fold reference.

[0198] The following Table 4 gives some examples of determining the pre-weighting strategy based on the weather condition information, road surface condition information, sensor state information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Table 4 only gives some examples and does not limit the way of determining the pre-weighting weight in this embodiment. The content in parentheses in Table 4 is the explanatory description of the weather condition information, road surface condition information, sensor state information, pre-weighting strategy, etc.

[0199] Table 4

[0200] To help understand the implementation process of the above-mentioned vehicle road noise processing methods in each embodiment, an implementation example is given. Figure 3This is the signal processing flowchart involved in the method for processing automotive road noise in this example. Figure 4 Flowchart of the method for processing automotive road noise in this example.

[0201] Step 1: After pre-filtering, amplitude adjustment and other preprocessing operations on the J-channel time-domain vibration reference signal x(n), convolve with the time-domain adaptive filter coefficient W full (l), where l represents the frame index, indicating that it is updated in units of frames rather than the sampling time n. The convolved signal is y(n) of the Q channel, and after post-processing operations such as amplitude limiting, it is sent to Q speakers for output respectively. The amplitude adjustment operation can be performed on the signal x(n) according to the pre-weighting weight determined in Step 3. The amplitude limiting operation can be performed on the signal y(n) according to the target clipping parameter value T(l) determined in Step 3.

[0202] Step 2: Put the J-channel time-domain vibration reference signal x(n) and the M-channel time-domain error microphone signal e(n) into two buffers respectively, with lengths of J*N and M*N. N is the frame length. In this example, J = 9, M = 2, Q = 5, and N = 64. Wait until the buffers are full to obtain one frame of the J-channel time-domain vibration reference signal x(l) and one frame of the M-channel time-domain error microphone signal e(l). Use the polyphase analysis filter bank based on FFT to perform sub-band decomposition on the above signals x(l) and e(l) respectively to obtain one frame of the J-channel sub-band vibration reference signal X k (l): X k (l)=[X k 1 (l), X k 2 (l)…X k J (l)] and one frame of the M-channel sub-band error microphone signal E k (l): E k (l)=[E k 1 (l), E k 2 (l)…E k M (l)] where k = 1, 2…K, and K is the total number of sub-bands. In this example, K = 65. In addition, the polyphase analysis filter bank trained by the training method in the above fourth embodiment can be used as the prototype low-pass filter in the sub-band decomposition: H G（l） (d)=[h 1(d), h 2 (d)…h P (d)] H G（l） (d) is the target polyphase analysis filter bank matched according to G(l), where G(l) is the identifier of the analysis filter bank and the synthesis filter bank determined according to the environmental state information. The formula for subband decomposition of the time-domain vibration reference signal x(l) is as follows: (1)

[0203] (2)

[0204] where D is the decimation factor and P is the length of the FFT. In this example, D = 3 and P = 128. In formula (2), v = -2πikp / P. Formula (1) is the process of polyphase filtering of the signal x(l). Specifically, it first performs the operation of adding a subband analysis window to the signal x(l), and then accumulates the windowed result to obtain x'(l); formula (2) is to perform DFT (Discrete Fourier Transform) on the signal x'(l) to obtain the subband vibration reference signal X k (l). In actual calculation, FFT can be used to improve the calculation speed. The formula for subband decomposition of the time-domain error microphone signal e(l) will not be elaborated here.

[0205] Step 3: Refer to Figure 5 , and input a frame of subband vibration reference signals X 1 (l)…X K (l), a frame of subband error microphone signals E 1 (l)…E K (l), a frame of time-domain vibration reference signal x(l) of the J channel, and a frame of time-domain error microphone signal e(l) of the M channel into a prediction model (U-Net Encoder + FT-GRU + U-Net Decoder + multi-task output layer) based on a neural network structure to predict the road surface condition information, weather condition information, and sensor status information. Corresponding to Figure 5 the function of the parameter matching control module in: dynamically determining the update step size μ 1 (l)…μ K (l) of the subband adaptive filter coefficients based on the road surface condition information, weather condition information, and sensor status information; dynamically determining the target filter bank based on the road surface condition information and weather condition information, and using G(l) to represent the identifier of the analysis filter bank and the synthesis filter bank determined according to the road surface condition information and weather condition information; determining the pre-weighting weights of each sensor and the target clipping parameter value T(l) based on the road surface condition information, weather condition information, and sensor status information.

[0206] Step Four: Since the subbands decomposed by the polyphase analysis filter bank based on FFT are subbands in the complex domain, the complex-domain FxLMS algorithm is used to update the subband adaptive filter coefficients W k (l), and the size of each W k (l) is C*J*Q, where C is the length of the subband adaptive filter coefficients. In this example, C = 4. In the complex-domain FxLMS algorithm, first, the input subband vibration reference signal X k (l) is convolved with the secondary path model S(k), and this convolution is a complex convolution:

[0207] Then, the complex-domain LMS algorithm is used to update the subband adaptive filter coefficients W k (l):

[0208] where q = 1, 2…Q, Q is the number of speaker signal channels. m = 1, 2…M, M is the number of error microphone signal channels, j = 1, 2…J, J is the number of channels of the vibration reference signal. k = 1, 2…K, K is the number of subbands. μ k (l) is the update step size of the subband adaptive filter coefficients determined according to the influencing factor information, which is used to control the update speed of the filter.

[0209] Step Five: Synthesize the time-domain adaptive filter coefficients from the subband adaptive filter coefficients.

[0210] Use the polyphase synthesis filter bank based on IFFT for subband synthesis. Compared with the weight stacking method with relatively large computational complexity and many sets of FFT and IFFT operations, this method only has one set of IFFT and adding a synthesis window operation, which is simple and efficient. Among them, the polyphase synthesis filter bank can adopt the polyphase analysis filter bank trained by the training method in the above-mentioned fourth embodiment, and can be dynamically determined according to the environmental state information. As Figure 6 shown, in this example, the full-band (time-domain) adaptive filter coefficients synthesized based on the subband adaptive filter coefficients can achieve almost the same effect as the target full-band (time-domain) adaptive filter coefficients. Figure 6 In the figure, the abscissa is the index of the filter length, and the ordinate is the value of the coefficient corresponding to each tap.

[0211] Step Six: Load balancing strategy. Split the tasks of subband decomposition, subband synthesis, and updating of subband adaptive filter coefficients among each interrupt, and split the neural network model prediction task among each interrupt.

[0212] It should be noted that the sequence numbers of the above steps do not constitute a limitation on the sequence between the steps. In addition, the above examples are only used to understand the present application and do not constitute a limitation on the method for processing automotive road noise of the present application. Based on this technical concept, more forms of simple transformation are within the protection scope of the present application.

[0213] An embodiment of the present application provides an automotive road noise processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the automotive road noise processing method in the first embodiment above.

[0214] Reference is made below Figure 7 , which shows a schematic structural diagram of an automotive road noise processing device suitable for implementing the embodiments of the present application. The automotive road noise processing device in the embodiments of the present application may be a control unit in an RNC system. Figure 7 The automotive road noise processing device shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0215] As Figure 7 shown, the automotive road noise processing device may include a processing device 1001 (such as a DSP processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1002 or the program loaded from the storage device 1003 into the random access memory 1004. In the random access memory 1004, various programs and data required for the operation of the automotive road noise processing device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a microphone, an accelerometer, etc.; an output device 1008 including, for example, a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the automotive road noise processing device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an automotive road noise processing device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0216] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0217] Compared with the prior art, the beneficial effects of the vehicle road noise processing device provided by the embodiments of the present application are the same as those of the vehicle road noise processing method provided by the above embodiments, and other technical features in the vehicle road noise processing device are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated herein.

[0218] It should be understood that the various parts disclosed in the embodiments of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0219] As described above, the above are only the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0220] The embodiments of the present application provide a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the vehicle road noise processing method in the above embodiments.

[0221] The computer-readable storage medium provided by the embodiments of the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0222] The above computer-readable storage medium can be included in the automotive road noise processing device; it can also exist independently without being assembled into the automotive road noise processing device.

[0223] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the automotive road noise processing device, the automotive road noise processing device is caused to perform the above functions defined in the method of the disclosed embodiments of the present application.

[0224] Computer program code for performing the operations of the present application can be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; they also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0225] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0226] The modules described in the embodiments of the present application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0227] The readable storage medium provided in the embodiments of the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned method for processing automotive road noise. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiments of the present application are the same as those of the method for processing automotive road noise provided in the above embodiments, and will not be elaborated herein.

[0228] The embodiments of the present application also provide a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method for processing automotive road noise as described above.

[0229] Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of the present application are the same as those of the method for processing automotive road noise provided in the above embodiments, and will not be elaborated herein.

[0230] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for processing automobile road noise, characterized in that: The automobile road noise processing method comprises: Dynamically determine a target filter group according to the acquired environmental state information, the target filter group includes a target polyphase analysis filter group and a target polyphase synthesis filter group, and the environmental state information is information characterizing the environmental state inside and / or outside the vehicle; Using the target polyphase analysis filter bank to perform sub-band decomposition on the input first time domain reference signal and the input first time domain error signal, respectively, to obtain a first sub-band reference signal and a first sub-band error signal; updating subband adaptive filter coefficients based on the first subband reference signal and the first subband error signal; Using the target polyphase synthesis filter bank to perform sub-band synthesis on the updated sub-band adaptive filter coefficients to obtain updated time-domain adaptive filter coefficients; The newly input second time domain reference signal is processed based on the updated time domain adaptive filter coefficient to obtain an anti-noise signal, so as to control the road noise based on the anti-noise signal.

2. The method for processing automobile road noise according to claim 1, characterized in that: Different filter groups are preset for different environmental states, each filter group includes a polyphase analysis filter group and a polyphase synthesis filter group, and different filter groups bring different filter update delays and / or spectrum leakages; The step of dynamically determining the target filter group according to the acquired environmental state information comprises: A preset filter group corresponding to the environmental state represented by the acquired environmental state information is determined as a target filter group.

3. The method for processing automobile road noise according to claim 2, characterized in that: The following rules are met when dynamically determining the target filter group according to the environmental state information: Under the environmental state represented by the environmental state information, the higher the stability of the automobile road noise, the greater the filter update delay and the smaller the spectrum leakage brought by the target filter group; the lower the stability of the automobile road noise, the smaller the filter update delay and the greater the spectrum leakage brought by the target filter group.

4. The method for processing automobile road noise according to claim 2, characterized in that: The preset filter groups corresponding to different environmental conditions are pre-trained with the goal of reducing the signal reconstruction loss and spectrum leakage loss of the filter groups. Different filter groups have different weights of signal reconstruction loss and frequency leakage loss set during training.

5. The method for processing automobile road noise according to claim 1, characterized in that: The automobile road noise control method further comprises: Using a preset prediction model to perform prediction based on a third time domain reference signal, a third time domain error signal, a third subband reference signal, and a third subband error signal to obtain the environmental state information; In which, the prediction model is pre-trained, the third time domain reference signal and the third time domain error signal are respectively input before the first time domain reference signal and the first time domain error signal, and the third subband reference signal and the third subband error signal are obtained by respectively performing subband decomposition on the third time domain reference signal and the third time domain error signal.

6. The method for processing automobile road noise according to claim 5, characterized in that: The prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module and a prediction module; The step of using a preset prediction model to perform prediction based on a third time domain reference signal, a third time domain error signal, a third subband reference signal and a third subband error signal to obtain the environmental state information comprises: Inputting the third sub-band reference signal and the third sub-band error signal into the first feature extraction module for feature extraction to obtain a first feature representation; Inputting the third time domain reference signal into the second feature extraction module to perform feature extraction to obtain a second feature representation; Inputting the third time domain error signal into the third feature extraction module to perform feature extraction to obtain a third feature representation; splicing the first feature representation, the second feature representation, and the third feature representation, and inputting the concatenated features into the feature fusion module for feature fusion to obtain a fused feature representation; The fused feature representation is input into the prediction module for prediction to obtain the environmental state information.

7. The method for processing automobile road noise according to claim 6, characterized in that: The first feature extraction module, the second feature extraction module and the third feature extraction module respectively include at least one convolution layer; the feature fusion module includes a frequency axis gated recurrent unit, a time axis gated recurrent unit and at least one transposed convolution layer connected in sequence; the prediction module includes multiple task output layers, and each task output layer is used to output one item of the environmental state information.

8. The method for processing automobile road noise according to any one of claims 5 to 7, characterized in that: The environmental status information includes weather condition information and road condition information.

9. The method for processing automobile road noise according to any one of claims 5 to 7, characterized in that: The frame index of the first time domain reference signal and the first time domain error signal is 1, and the automobile road noise processing method further includes: In N interruptions of the l+1th frame, tasks allocated to the N interruptions are executed, wherein N represents the frame length, an interruption refers to a period after an anti-noise signal is generated based on the time domain reference signal of each sampling point and before a time domain reference signal of a next sampling point is received, and the tasks allocated to the N interruptions include: a task of calculating the updated time domain adaptive filter coefficients, and / or a task of inputting the first time domain reference signal, the first time domain error signal, the first sub-band reference signal and the first sub-band error signal into the prediction model to predict environmental state information.

10. An automobile road noise processing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the automobile road noise processing method according to any one of claims 1 to 9.

Images