Automobile Road Noise Processing Method, Device, Equipment, Storage Medium and Product

Abstract

The present application discloses a method, device, equipment, storage medium and product for processing vehicle road noise, relating to the technical field of noise reduction. The method includes: respectively performing sub-band decomposition on the input time-domain reference signal and the input time-domain error signal; dynamically determining a target update step size according to the obtained influencing factor information, where the influencing factor information includes environmental state information and / or sensor state information; updating the sub-band adaptive filter coefficients based on the sub-band reference signal and the sub-band error signal according to the target update step size; performing sub-band synthesis on the updated sub-band adaptive filter coefficients to obtain the updated time-domain adaptive filter coefficients; processing the newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control road noise based on the anti-noise signal. The present application realizes guaranteeing the robustness and safety of the vehicle road noise control system and avoiding delayed response or unstable output.

Description

Technical Field

[0001] This application relates to the field of noise reduction technology, and particularly to a method, device, equipment, storage medium and product for processing vehicle road noise. Background Art

[0002] With the development of the automotive industry, the comfort and quietness inside the vehicle cockpit have increasingly attracted the attention of the automotive industry and consumers. As a key factor affecting the acoustic quietness and riding experience inside the vehicle, interior noise has gradually become a pain point issue of concern in the industry. Interior noise mainly comes from the vehicle's power system and road noise during vehicle driving. With the development of new energy vehicles, electric motors have gradually replaced internal combustion engines, thus significantly reducing engine noise. Road Noise Cancellation (RNC) technology has become an important means to improve the acoustic comfort inside the vehicle. The RNC technology combines the reference signal collected by a vibration sensor and the error signal collected by an in-vehicle microphone to generate a sound signal opposite to the noise characteristics, which is played through an in-vehicle speaker to cancel the original noise. Although existing RNC systems have been applied to some mass-produced models, in complex actual application environments, existing RNC systems have problems such as insufficient adaptive ability or poor stability, which can lead to delayed response or unstable output, seriously affecting the driving and riding experience.

[0003] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment, storage medium and product for processing vehicle road noise, aiming to solve the problem that existing RNC systems have insufficient adaptive ability or poor stability, which can lead to delayed response or unstable output.

[0005] To achieve the above object, this application proposes a method for processing vehicle road noise, including:

[0006] Performing subband decomposition on the input time-domain reference signal and the input time-domain error signal respectively to obtain a subband reference signal and a subband error signal;

[0007] Dynamically determining a target update step size corresponding to the subband adaptive filter coefficients according to the obtained influence factor information, where the influence factor information includes environmental state information and / or sensor state information, the environmental state information is information characterizing the environmental state inside and / or outside the vehicle, the sensor state information is information characterizing the working state of the sensor, and the sensor includes a sensor for collecting the time-domain reference signal and / or a sensor for collecting the time-domain error signal;

[0008] Update the sub - band adaptive filter coefficients based on the sub - band reference signal and the sub - band error signal according to the target update step size;

[0009] Perform sub - band synthesis on the updated sub - band adaptive filter coefficients to obtain the updated time - domain adaptive filter coefficients;

[0010] Process the newly input time - domain reference signal based on the updated time - domain adaptive filter coefficients to obtain an anti - noise signal, and control the road noise based on the anti - noise signal.

[0011] Optionally, when dynamically determining the target update step size according to the influence factor information, it conforms to one or more of the following rules:

[0012] In the environmental state characterized by the environmental state information, the target update step size corresponding to the sub - band with higher automotive road noise energy or higher stability degree is larger than the target update step size corresponding to the sub - band with lower automotive road noise energy or lower stability degree;

[0013] For any target sub - band, in the environmental state characterized by the environmental state information, the higher the energy or the higher the stability degree of the automotive road noise in the target sub - band, the larger the target update step size corresponding to the target sub - band, and the lower the energy or the lower the stability degree of the automotive road noise in the target sub - band, the smaller the target update step size corresponding to the target sub - band;

[0014] According to the abnormal degree of each sensor characterized by the sensor state information, the target update step size corresponding to the sub - band where the sensor with a higher abnormal degree is located is smaller than the target update step size corresponding to the sub - band where the sensor with a lower abnormal degree is located;

[0015] For any target sub - band, the higher the abnormal degree of the sensor belonging to the target sub - band, the smaller the target update step size corresponding to the target sub - band, and the lower the abnormal degree of the sensor belonging to the target sub - band, the larger the target update step size corresponding to the target sub - band.

[0016] Optionally, the step of dynamically determining the target update step size corresponding to the sub - band adaptive filter coefficients according to the obtained influence factor information includes:

[0017] Determine the update step size corresponding to the influence factor information as the target update step size according to a pre - set mapping relationship, where the mapping relationship satisfies one or more of the above rules.

[0018] Optionally, the influence factor information further includes the vehicle speed. When dynamically determining the target update step size according to the influence factor information, it conforms to one or more of the following rules:

[0019] The higher the vehicle speed stability is, the larger the target update step corresponding to each sub - band is, and the lower the vehicle speed stability is, the smaller the target update step corresponding to each sub - band is;

[0020] When the vehicle speed is outside the preset range, the target update step corresponding to each sub - band is smaller than that when the vehicle speed is within the preset range.

[0021] Optionally, the vehicle road noise processing method further includes:

[0022] Using a preset prediction model to perform prediction based on the input time - domain reference signal, the input time - domain error signal, the sub - band reference signal, and the sub - band error signal to obtain the environmental state information and / or the sensor state information, where the prediction model is pre - trained.

[0023] Optionally, the prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module;

[0024] The step of using a preset prediction model to perform prediction based on the input time - domain reference signal, the input time - domain error signal, the sub - band reference signal, and the sub - band error signal to obtain the environmental state information and / or the sensor state information includes:

[0025] Inputting the sub - band reference signal and the sub - band error signal into the first feature extraction module for feature extraction to obtain a first feature representation;

[0026] Inputting the input time - domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation;

[0027] Inputting the input time - domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation;

[0028] Concatenating the first feature representation, the second feature representation, and the third feature representation and inputting them into the feature fusion module for feature fusion to obtain a fused feature representation;

[0029] Inputting the fused feature representation into the prediction module for prediction to obtain the environmental state information and / or the sensor state information.

[0030] Optionally, the first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer; the feature fusion module includes a frequency-axis gated recurrent unit, a time-axis gated recurrent unit, and at least one transposed convolutional layer connected in sequence; the prediction module includes a plurality of task output layers, and each task output layer is respectively used to output one item of the environmental state information or the sensor state information.

[0031] Optionally, the prediction module further includes a shared feature extraction layer disposed before the plurality of task output layers, and the shared feature extraction layer includes one or more convolutional layers.

[0032] Optionally, the environmental state information includes weather condition information and road surface condition information.

[0033] Optionally, the frame index of the input time-domain reference signal and the input time-domain error signal is l, and the method for processing automotive road noise further includes:

[0034] Executing tasks allocated to the N interrupts during the N interrupts in the (l + 1)-th frame, where N represents the frame length, and an interrupt refers to the period from generating an anti-noise signal based on the time-domain reference signal of each sampling point to receiving the next time-domain reference signal of the sampling point. The tasks allocated to the N interrupts include: the task of calculating the updated time-domain adaptive filter coefficients, and / or the task of predicting the environmental state information and / or the sensor state information using the prediction model.

[0035] Optionally, the task of calculating the updated time-domain adaptive filter coefficients is divided into a subband decomposition task, a subband synthesis task, and a plurality of subband update tasks. Each subband update task includes the task of updating at least one subband adaptive filter coefficient. The subband decomposition task is allocated to the first preset number of interrupts at the front for execution, the subband synthesis task is allocated to the second preset number of interrupts at the back for execution, and the plurality of subband update tasks are allocated to the third preset number of interrupts in the middle for execution. One subband update task is allocated to each interrupt, and the sum of the preset first number, the preset second number, and the preset third number is N.

[0036] Optionally, the step of updating the subband adaptive filter coefficients according to the target update step size based on the subband reference signal and the subband error signal includes:

[0037] Convolving the subband reference signal with a pre-established secondary path model;

[0038] Using the complex domain LMS algorithm, calculate the updated subband adaptive filter coefficients based on the subband error signal, the subband reference signal after the convolutional secondary path model, and the current subband adaptive filter coefficients.

[0039] Optionally, the sensor for collecting the time-domain reference signal includes at least one vibration sensor, and the sensor for collecting the time-domain error signal includes at least one microphone.

[0040] Optionally, the influence factor information further includes the signal characteristics of the vibration sensor, and the signal characteristics include acceleration amplitude, energy, and sound pressure signal amplitude.

[0041] Optionally, the step of respectively performing subband decomposition on the input time-domain reference signal and the input time-domain error signal to obtain a subband reference signal and a subband error signal includes:

[0042] Preprocess the input time-domain reference signal and the input time-domain error signal, and respectively perform subband decomposition on the preprocessed time-domain reference signal and time-domain error signal to obtain a subband reference signal and a subband error signal, where the preprocessing includes pre-filtering processing and / or amplitude adjustment processing.

[0043] Optionally, the step of processing the newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, and controlling road noise based on the anti-noise signal includes:

[0044] Process the newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal;

[0045] Play the anti-noise signal through a speaker provided inside the vehicle, or play the anti-noise signal through a speaker provided inside the vehicle after performing post-processing operations on the anti-noise signal.

[0046] In addition, to achieve the above object, the present application further proposes an automotive road noise processing device, and the automotive road noise processing device includes:

[0047] A subband decomposition module, configured to respectively perform subband decomposition on the input time-domain reference signal and the input time-domain error signal to obtain a subband reference signal and a subband error signal;

[0048] A step-size update module, configured to dynamically determine a target update step-size corresponding to the sub-band adaptive filter coefficients according to the obtained influence factor information, where the influence factor information includes environmental state information and / or sensor state information, the environmental state information is information characterizing the environmental state inside and / or outside the vehicle, the sensor state information is information characterizing the operating state of the sensor, and the sensor includes a sensor for collecting the time-domain reference signal and / or a sensor for collecting the time-domain error signal;

[0049] A filter update module, configured to update the sub-band adaptive filter coefficients based on the sub-band reference signal and the sub-band error signal according to the target update step-size;

[0050] A sub-band synthesis module, configured to perform sub-band synthesis on the updated sub-band adaptive filter coefficients to obtain updated time-domain adaptive filter coefficients;

[0051] A noise cancellation module, configured to process a newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain a noise-cancelled signal, so as to control road noise based on the noise-cancelled signal.

[0052] In addition, to achieve the above object, the present application further provides a vehicle road noise processing device, where the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the vehicle road noise processing method as described above.

[0053] In addition, to achieve the above object, the present application further provides a storage medium, where the storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the vehicle road noise processing method as described above are implemented.

[0054] In addition, to achieve the above object, the present application further provides a computer program product, where the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the vehicle road noise processing method as described above are implemented.

[0055] One or more technical solutions proposed by the present application have at least the following technical effects:

[0056] In this application, the update step size of the sub-band adaptive filter coefficients is dynamically determined according to the acquired environmental state information and / or sensor state information. The sub-band adaptive filter coefficients are updated according to the determined update step size, and then the sub-band adaptive filter coefficients are converted into time-domain adaptive filter coefficients. The time-domain adaptive filter coefficients are used to process the time-domain reference signal to generate an anti-noise signal, so as to control the road noise based on the anti-noise signal, achieve adaptation to changes in the environmental state inside or outside the vehicle, or adaptation to changes in the working state of the sensor, balance the adaptive convergence speed and system stability, and avoid problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensure the robustness and safety of the system, and avoid delayed response or unstable output. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0058] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0059] Figure 1 It is a schematic flowchart provided for the first embodiment of the method for processing automotive road noise in this application;

[0060] Figure 2 It is a schematic flowchart of the prediction model training process involved in an embodiment of this application;

[0061] Figure 3 It is a schematic diagram of the signal processing process involved in an embodiment of this application;

[0062] Figure 4 It is a schematic flowchart of the method for processing automotive road noise involved in an embodiment of this application;

[0063] Figure 5 It is a schematic diagram of the model architecture and parameter matching architecture involved in an embodiment of this application;

[0064] Figure 6 It is a comparison diagram of filter coefficients involved in an embodiment of this application;

[0065] Figure 7 It is a schematic diagram of the device structure of the hardware operating environment involved in the method for processing automotive road noise in the embodiments of this application.

[0066] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with embodiments with reference to the accompanying drawings. Specific embodiments

[0067] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0068] In order to better understand the technical solutions of this application, the following will be described in detail in conjunction with the drawings of the specification and specific embodiments.

[0069] Although existing RNC systems have been applied in some mass-produced vehicles, in complex actual application environments, the existing RNC systems have problems such as insufficient adaptive ability or poor stability, which can lead to delayed response or unstable output, seriously affecting the driving experience.

[0070] The embodiments of this application solve the above technical problems by dynamically determining the update step size of the subband adaptive filter coefficients according to the acquired environmental state information and / or sensor state information, updating the subband adaptive filter coefficients according to the determined update step size, and then converting the subband adaptive filter coefficients into time-domain adaptive filter coefficients, and using the time-domain adaptive filter coefficients to process the time-domain reference signal to generate an anti-noise signal, so as to control the road noise based on the anti-noise signal, realize adaptation to changes in the environmental state inside or outside the vehicle, or adaptation to changes in the working state of the sensor, balance the adaptive convergence speed and system stability, and avoid problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensure the robustness and safety of the system, and avoid delayed response or unstable output.

[0071] The first embodiment of the automotive road noise processing method of this application is proposed below. Refer to Figure 1 , Figure 1This is a schematic flowchart of the first embodiment of the automotive road noise processing method of this application. In this embodiment, the execution subject of the automotive road noise processing method may be the control unit in the automotive RNC system, but it is not limited to the control unit. For example, it may also be a processing device with data processing and program running functions. In a feasible implementation, the automotive RNC system may include sensors, a control unit, a noise canceller, and other auxiliary devices set as needed; among them, the sensors include a reference sensor for collecting reference signals and an error sensor for collecting error signals, and multiple reference sensors and error sensors may be set; the reference sensor and the error sensor may be vibration sensors (acceleration sensors) or microphones; the control unit runs adaptive algorithms, such as FxLMS (Filtered-x Least Mean Square), FxNLMS (Filtered-X Normalized Least Mean Square), FxAP (Filter-x Affine Projection), etc. adaptive algorithms, update the adaptive filter coefficients based on the reference signal and the error signal and calculate the anti-noise signal. In this embodiment of the automotive road noise processing method, the update step size of the adaptive filter coefficients used in the adaptive algorithm is dynamically adjusted to balance the adaptive convergence speed and system stability, so as to improve the system's adaptive ability and stability and avoid problems such as delayed response or unstable output; the noise canceller is used to output the anti-noise signal to cancel the road noise. In a feasible implementation, the reference sensor may use a vibration sensor and may be set at the attachment point of the vehicle body and the chassis, and the error sensor may use a microphone and may be set inside the vehicle, such as the roof lining (near the passenger's head area); the noise canceller may use a vibration actuator and / or a speaker. The vibration actuator generates reverse mechanical vibration according to the anti-noise signal to cancel the low-frequency road noise transmitted through the vehicle body, and the speaker emits reverse sound waves according to the anti-noise signal to directly cancel the air-borne noise (such as the high-frequency component of tire noise) entering the cabin.

[0072] In this embodiment, the automotive road noise processing method includes steps S10 to S50:

[0073] Step S10, perform sub-band decomposition on the input time-domain reference signal and the input time-domain error signal respectively to obtain the sub-band reference signal and the sub-band error signal.

[0074] When multiple reference sensors are set, the input time-domain reference signal is multi-channel. Similarly, when multiple error sensors are set, the input time-domain error signal is also multi-channel. In this embodiment, the number of channels of the time-domain reference signal and the time-domain error signal is not limited. For example, when applied to a MIMO (Multiple Input Multiple Output) vehicle RNC system, both the input and output are multi-channel.

[0075] The time-domain reference signal and the time-domain error signal collected by the sensor can be directly input into the control unit or input into the control unit after preprocessing; the control unit can directly perform subband decomposition on the input time-domain reference signal and time-domain error signal, or can perform preprocessing before subband decomposition. In this embodiment, whether to perform preprocessing on the time-domain reference signal and the time-domain error signal is not limited, nor is it limited what preprocessing is performed on the signal. For example, it can include pre-filtering, amplitude adjustment, etc. Pre-filtering includes high-pass filtering, low-pass filtering, etc., and the preprocessing operation can be set according to needs. In the following embodiments, the preprocessing operation on the signal will not be particularly emphasized. That is, even if it is not written that preprocessing is required, it does not necessarily mean that there is no or no need to perform the preprocessing operation.

[0076] The control unit performs subband decomposition on the time-domain reference signal to obtain a subband reference signal, and performs subband decomposition on the time-domain error signal to obtain a subband error signal. It can be understood that there are multiple subbands, and the subband reference signals and subband error signals obtained through subband decomposition are also multiple respectively. That is, the subband reference signal and the subband error signal corresponding to each subband are obtained through subband decomposition. In the specific implementation, the subband can be divided from the full frequency band, but in this embodiment, it is not limited to being divided from the full frequency band, and can be divided from a relatively wide frequency band range, and the number of subbands is not limited either.

[0077] In a feasible implementation, the road noise control can be carried out in the control unit without output delay, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit point by point in real time. The control unit generates a single-sampling-point anti-noise signal for the currently input single-sampling-point time-domain reference signal in real time according to the current time-domain adaptive filter coefficients, and then outputs the anti-noise signal in real time to achieve the effect of no output delay. On the other hand, the adaptive filter coefficients are updated frame by frame. Specifically, the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffers in real time. After each buffer is filled with a frame of time-domain reference signal and a frame of time-domain error signal, subband decomposition is performed on the frame of time-domain reference signal and the frame of time-domain error signal, and subsequent operations for updating the time-domain adaptive filter coefficients are carried out. In other implementations, a method with output delay can also be adopted, that is, after inputting the time-domain reference signal of multiple sampling points, the anti-noise signal starts to be output. In this embodiment, neither the method without output delay nor the method with output delay is restricted. In a specific application scenario, a specific implementation method can be selected according to needs.

[0078] Step S20: Dynamically determine the target update step size corresponding to the subband adaptive filter coefficients according to the obtained influencing factor information, where the influencing factor information includes environmental state information and / or sensor state information, the environmental state information is information characterizing the environmental state inside and / or outside the vehicle, the sensor state information is information characterizing the working state of the sensor, and the sensor includes a sensor for collecting the time-domain reference signal and / or a sensor for collecting the time-domain error signal.

[0079] The influencing factor information is information characterizing the specific manifestation of the influencing factors, and the influencing factors are factors determining the adjustment strategy of the update step size of the subband adaptive filter coefficients, that is, the specific manifestations of these factors determine different adjustment strategies of the update step size. In this embodiment, the influencing factors can at least include one or more of the environmental state inside the vehicle, the environmental state outside the vehicle, the working state of the reference sensor, and the working state of the error sensor. In other implementations, other influencing factors can also be included, such as vehicle speed, signal characteristics of vibration sensors, etc. The signal characteristics can include acceleration amplitude, energy, and sound pressure signal amplitude.

[0080] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause dynamic changes in the acoustic propagation path, which will affect the energy magnitude or stability of the vehicle road noise. The effects on the energy magnitude or stability of the noise in different sub-bands are also different. If a fixed step size is used for updating the sub-band adaptive filter coefficients, when the energy magnitude or stability of the vehicle road noise changes, it may be due to too large an update step size, resulting in too fast an update of the sub-band adaptive filter coefficients, leading to system misjudgment or poor acoustic effects. It may also be due to too small an update step size, resulting in a slow convergence speed of the adaptive filter and poor noise reduction effect. Therefore, in the specific implementation, the environmental state outside the vehicle and the environmental state inside the vehicle can be used as influencing factors, and the update step size can be dynamically adjusted based on the environmental state to balance the adaptive convergence speed and system stability, and avoid delayed response or unstable output. When the working state of the reference sensor or the working state of the error sensor changes, it will affect the accuracy of the collected signal. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the vehicle road noise in the signal will be estimated incorrectly. If a fixed step size is used for updating the sub-band adaptive filter coefficients and the adaptive filter coefficients are still continuously updated, it will cause the generated anti-noise signal to be unable to accurately cancel the real vehicle road noise, resulting in unstable noise reduction effect. Therefore, in the specific implementation, the working state of the reference sensor and the working state of the error sensor can be used as influencing factors, and the update step size can be dynamically adjusted based on the working state of the sensor to balance the adaptive convergence speed and system stability, and avoid delayed response or unstable output.

[0081] In the specific implementation, the environmental state inside the vehicle can include, for example, the seat adjustment state, the position of the occupants, the window opening and closing state, etc. The environmental state outside the vehicle can include, for example, the weather condition, the road surface condition, etc. These are not limited in this embodiment.

[0082] In this embodiment, the acquisition method of the influencing factor information is not limited. For example, the environmental state information can be obtained by image analysis of the environmental images collected by the camera, and the sensor state information can be determined by outlier analysis of the signals collected by the sensor.

[0083] The influencing factor information can be acquired once every certain period of time. The shorter the time interval, the higher the real-time degree. The acquisition frequency is not limited in this embodiment. For example, in a feasible implementation, the influencing factor information can be acquired every time the adaptive filter coefficients are updated.

[0084] Dynamically determining the update step size of the subband adaptive filter coefficients means that after obtaining new influencing factor information each time, the update step size is determined according to the new influencing factor information, so that the update step size of the adaptive filter coefficients changes according to the specific performance of the influencing factors. By dynamically adjusting the update step size, the update speed of the adaptive filter coefficients is adjusted to adapt to the changes in the internal or external environment state of the vehicle, or to adapt to the changes in the working state of the sensor, balancing the adaptive convergence speed and system stability, and avoiding problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensuring the robustness and safety of the system, and avoiding delayed response or unstable output.

[0085] The specific method of determining the update step size of the subband adaptive filter coefficients according to the influencing factor information is not limited in this embodiment. For example, the mapping relationship between the influencing factor information and the update step size of the subband adaptive filter coefficients can be set in advance as needed. After obtaining the influencing factor information, the control unit uses this mapping relationship to take the update step size corresponding to the influencing factor information as the target update step size. In this embodiment, the mapping relationship is not limited. For example, the mapping relationship can be represented by a mapping table, a relational expression, etc., and the mapping relationship can be determined by experiments, data statistics, or other methods.

[0086] It should be noted that since there are multiple subbands, there are multiple subband adaptive filters, and the target update step sizes of the coefficients of each subband adaptive filter may be the same or different, which is not limited in this embodiment.

[0087] Step S30, updating the subband adaptive filter coefficients based on the subband reference signal and the subband error signal according to the target update step size.

[0088] After determining the target update step size, the subband adaptive filter coefficients can be updated according to a pre-set adaptive algorithm. For example, the adaptive algorithm can adopt FxLMS, FxNLMS, or FxAP, etc., which is not limited in this embodiment.

[0089] For example, in a feasible implementation manner, the adaptive algorithm can adopt the complex-domain FxLMS algorithm. Specifically, the control unit can convolve the subband reference signal with a pre-established secondary path model, and then use the complex-domain LMS algorithm to calculate the updated subband adaptive filter coefficients based on the subband error signal, the subband reference signal after convolving the secondary path model, and the current subband adaptive filter coefficients.

[0090] Step S40, performing subband synthesis on the updated subband adaptive filter coefficients to obtain the updated time-domain adaptive filter coefficients.

[0091] To avoid output delay, filtering output should be performed in the time domain (full frequency band). Therefore, the sub-band adaptive filter coefficients can be converted to the full frequency band. By performing sub-band synthesis on the updated sub-band adaptive filter coefficients, the updated time-domain adaptive filter coefficients can be obtained.

[0092] It should be noted that the sub-band adaptive algorithm is adopted in this embodiment. That is, the sub-band adaptive filter coefficients are updated separately and then synthesized into the full frequency band (time domain) adaptive filter coefficients. Compared with the method of directly updating the full frequency band (time domain) adaptive filter coefficients, in this embodiment, each sub-band adaptive filter coefficient is updated according to its corresponding target update step size, and then the updated sub-band adaptive filter coefficients are synthesized into the updated time-domain adaptive filter coefficients, which has the advantages of fast convergence, independence of different sub-bands, and the ability to use different adaptive algorithms and filter orders for different sub-bands. At the same time, it is less sensitive to changes in the input signal.

[0093] In this embodiment, the algorithms used for sub-band decomposition and sub-band synthesis are not limited and can be selected according to needs. For example, in a feasible implementation, the UDFTM (Uniform Discrete Fourier Transform Model) method can be used to perform sub-band decomposition on the time-domain reference signal and the time-domain error signal respectively, and the sub-band filter coefficients are restored to the full frequency band through the weight stacking technique. Another example is that in a feasible implementation, a polyphase analysis filter bank can be used for sub-band decomposition, and a polyphase synthesis filter bank can be used for sub-band synthesis.

[0094] Step S50: Process the newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control road noise based on the anti-noise signal.

[0095] After updating the time-domain adaptive filter coefficients, the newly input time-domain reference signal can be processed according to the updated time-domain adaptive filter coefficients to generate an anti-noise signal. After generating the anti-noise signal, road noise can be controlled based on the anti-noise signal. For example, the anti-noise signal can be played through the speakers installed inside the car, or the anti-noise signal can be output after some post-processing operations. The post-processing operations can include, for example, amplitude limiting, etc., which are not limited here.

[0096] The generated anti-noise signal can be multi-channel. For example, when there are multiple speakers, a multi-channel anti-noise signal is generated and output through multiple speakers. In this embodiment, the number of channels of the anti-noise signal is not limited.

[0097] In this embodiment, by dynamically determining the update step size of the subband adaptive filter coefficients according to the acquired environmental state information and / or sensor state information, updating the subband adaptive filter coefficients according to the determined update step size, and then converting the subband adaptive filter coefficients into time-domain adaptive filter coefficients, the time-domain adaptive filter coefficients are used to process the time-domain reference signal to generate an anti-noise signal, so as to control the road noise based on the anti-noise signal, achieve adaptation to changes in the environmental state inside or outside the vehicle, or adaptation to changes in the working state of the sensor, balance the adaptive convergence speed and system stability, and avoid problems such as misjudgment, poor acoustic effects, and unstable noise reduction effects caused by too fast or too slow update speed of the adaptive filter coefficients, ensure the robustness and safety of the system, and avoid delayed response or unstable output.

[0098] In a feasible implementation manner, to improve the noise reduction effect of the system and ensure the stability of noise reduction, it is set that when the control unit dynamically determines the target update step size according to the influence factor information, it can conform to one or more of the following rules:

[0099] 1. In the environmental state characterized by the environmental state information, the target update step size corresponding to the subband with higher or more stable automotive road noise energy is larger than the target update step size corresponding to the subband with lower or less stable automotive road noise energy.

[0100] In the same environmental state, the distribution of automotive road noise on each subband may be different; for subbands with higher noise energy, a larger update step size can be used to specifically enhance the noise reduction intensity of the subband, improve the noise reduction effect, and avoid delayed response; for subbands with lower noise energy, a smaller update step size can be used to ensure the stability of the noise reduction effect and avoid unstable output. In the same environmental state, the stability of automotive road noise on each subband may be different; for subbands with higher stability, a larger update step size can be used to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response; for subbands with lower stability, a smaller update step size can be used to avoid system misjudgment or poor acoustic effects, and ensure the robustness and safety of the system.

[0101] It should be noted that the stability of automotive road noise refers to the stability of the change in noise energy.

[0102] 2. For any target subband, in the environmental state characterized by the environmental state information, the higher the energy or the higher the stability of the automotive road noise in the target subband, the larger the target update step size corresponding to the target subband, and the lower the energy or the lower the stability of the automotive road noise in the target subband in the environmental state characterized by the environmental state information, the smaller the target update step size corresponding to the target subband.

[0103] For the same sub-band, the energy level of vehicle road noise in this sub-band may be different under different environmental conditions. Therefore, for a certain sub-band (referred to as the target sub-band for distinction), if the energy of vehicle road noise in the target sub-band is high under a certain environmental condition, a larger update step size can be adopted for the target sub-band to specifically enhance the noise reduction intensity of this sub-band, improve the noise reduction effect, and avoid delayed response. If the energy of vehicle road noise in the target sub-band is low under a certain environmental condition, a smaller update step size can be adopted for the target sub-band to ensure the stability of the noise reduction effect. For the same sub-band, the stability of vehicle road noise in this sub-band may be different under different environmental conditions. Therefore, for a certain sub-band (referred to as the target sub-band for distinction), if the stability of vehicle road noise in the target sub-band is high under a certain environmental condition, a larger update step size can be adopted for the target sub-band to ensure the adaptive convergence speed of the system and a low steady-state error level, and avoid delayed response. If the energy of vehicle road noise in the target sub-band is low under a certain environmental condition, a smaller update step size can be adopted for the target sub-band to avoid misjudgment of the system or generating adverse acoustic effects, and ensure the robustness and safety of the system.

[0104] 3. According to the abnormal degree of each sensor characterized by the sensor state information, the target update step size corresponding to the sub-band where the sensor with a higher abnormal degree is located is smaller than the target update step size corresponding to the sub-band where the sensor with a lower abnormal degree is located.

[0105] The sub-band where the sensor is located refers to the sub-band in which the signal collected by the sensor is distributed or mainly distributed, and the sub-bands where different sensors are located may vary.

[0106] The sensor state information is used to characterize the working state of the sensor. In the specific implementation manner, the working state of the sensor can be divided into two types, namely normal operation and abnormal operation. The sensor state information can be the information used to indicate whether the sensor belongs to the normal working state or the abnormal working state. Then it can be understood that the abnormal degree of the sensor with the working state of "normal operation" is lower than that of the sensor with the working state of "abnormal operation". Or, the working state of the sensor can also be divided into more than two types. For example, numerical values within 0-1 are used to represent different abnormal degrees, and the sensor state information can be the numerical value indicating the abnormal degree of the sensor.

[0107] Based on the acquired sensor status information, the abnormality degree of each sensor can be determined, and the abnormality degrees of different sensors may be different. For the sensor with a higher abnormality degree, a smaller update step size can be adopted for the sub-band where the sensor is located to avoid misjudgment of the system and generate poor acoustic effects. For the sensor with a lower abnormality degree, a larger update step size can be adopted for the sub-band where the sensor is located to ensure the noise reduction effect of the system and avoid delayed response.

[0108] 4. For any target sub-band, the higher the abnormality degree of the sensors belonging to the target sub-band, the smaller the target update step size corresponding to the target sub-band; the lower the abnormality degree of the sensors belonging to the target sub-band, the larger the target update step size corresponding to the target sub-band.

[0109] For a certain sub-band (hereinafter referred to as the target sub-band for distinction), the sensors belonging to the target sub-band refer to the sensors whose signals are all or mainly distributed in this sub-band.

[0110] For the target sub-band, different acquired sensor status information may result in different abnormality degrees of the sensors belonging to the target sub-band. If the abnormality degree of the sensors belonging to the target sub-band is higher, a smaller update step size can be adopted for the target sub-band to avoid misjudgment of the system and generate poor acoustic effects. If the abnormality degree of the sensors belonging to the target sub-band is lower, a larger update step size can be adopted for the target sub-band to ensure the noise reduction effect of the system and avoid delayed response.

[0111] In a feasible implementation manner, the influencing factor information may further include the real-time vehicle speed of the vehicle. When the vehicle speed changes, it will affect the energy magnitude or stability of the vehicle road noise. If a fixed update step size is adopted for the update step size of the sub-band adaptive filter coefficient when the vehicle speed changes, it may lead to misjudgment or poor acoustic effects because the update step size is too large and the sub-band adaptive filter coefficient is updated too fast when the energy magnitude or stability of the vehicle road noise changes. Therefore, the vehicle speed can be used as an influencing factor.

[0112] In a feasible implementation manner, when the influencing factor information further includes the vehicle speed, when dynamically determining the target update step size according to the influencing factor information, it may also conform to one or more of the following rules:

[0113] 5. The higher the vehicle speed stability, the larger the target update step size corresponding to each sub-band; the lower the vehicle speed stability, the smaller the target update step size corresponding to each sub-band.

[0114] The vehicle speed stability refers, for example, to the change rate of the vehicle speed. The larger the change rate, the lower the vehicle speed stability; the smaller the change rate, the higher the vehicle speed stability.

[0115] The higher the vehicle speed stability, the higher the stability of the vehicle road noise. At this time, a larger update step size can be adopted to ensure the adaptive convergence speed of the system and a lower steady-state error level, and avoid delayed response; the lower the vehicle speed stability, the lower the stability of the vehicle road noise. At this time, a smaller update step size can be adopted to avoid misjudgment of the system or generation of adverse acoustic effects, and ensure the robustness and safety of the system.

[0116] 6. When the vehicle speed is outside the preset range, the target update step size corresponding to each sub-band is smaller than the target update step size corresponding to each sub-band when the vehicle speed is within the preset range.

[0117] The preset range indicates that the vehicle speed is at a normal level. When the vehicle speed is outside the preset range, it represents a relatively extreme or abnormal situation, and the vehicle road noise may have relatively extreme or abnormal manifestations. For such extreme or abnormal situations, a smaller update step size can be adopted to avoid misjudgment or generation of adverse acoustic effects.

[0118] In a feasible implementation manner, step S20 includes: determining the update step size corresponding to the influencing factor information as the target update step size according to a pre-set mapping relationship, where the mapping relationship satisfies one or more of the above rules.

[0119] In a specific implementation manner, in order to enable the control unit to conform to one or more of the above rules when dynamically determining the target update step size according to the influencing factor information, a mapping relationship that satisfies one or more of the above rules can be set. The control unit determines the update step size corresponding to the influencing factor information as the target update step size according to this mapping relationship. The mapping relationship can be represented by a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple of the above rules, there can be a certain priority order between multiple rules to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the update step size according to the sensor status information can be higher than the priority of determining the update step size according to the environmental status information.

[0120] In a feasible implementation manner, the environmental status information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to the weather condition that affects the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information representing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0121] In a feasible implementation, when the environmental status information and sensor status information indicate that the environment is good and stable (e.g., good road conditions, clear weather, normal sensors, medium vehicle speed), the vehicle road noise is relatively stable, and there will be no particularly abnormal and prominent noise in each sub-band. At this time, a moderate or large update step size can be adopted to ensure the adaptive convergence speed of the system and a low steady-state error level. When the environmental status information and sensor status information indicate that the environment deteriorates or becomes unstable (e.g., deteriorating road conditions, bad weather, abnormal sensors, drastic vehicle speed changes), the stability of the vehicle road noise may deteriorate. At this time, a smaller update step size can be adopted to improve the system stability. When the environmental status information and sensor status information indicate the existence of specific types of noise (e.g., waterlogged road surface, brick road surface, wind noise), the step sizes of different frequency sub-bands can be adjusted differentially. A larger update step size is adopted for the sub-bands with high noise energy to specifically enhance or weaken the filtering intensity of specific frequency bands and optimize the noise reduction effect. When the environmental status information and sensor status information indicate an extremely harsh environment or system anomaly (e.g., extremely harsh road conditions, severe sensor failure, extremely low vehicle speed), the update step size can be significantly reduced or even frozen to avoid system misjudgment or adverse acoustic effects and ensure the robustness and safety of the system.

[0122] Table 1 below gives some examples of determining the update step sizes for the low-frequency, medium-frequency, and high-frequency bands based on weather condition information, road surface condition information, sensor status information, and the signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of a vibration sensor. Among them, the low-frequency, medium-frequency, and high-frequency can be divided as needed. It can be understood that each of the three frequency bands contains one or more sub-bands. Table 1 only gives some examples and does not limit the method of determining the target update step size in this embodiment. The content in parentheses in the table is an explanatory note on the weather condition information, road surface condition information, sensor status information, update step size, etc.

[0123] Table 1 Examples of Update Step Size Selection

[0124]

[0125] Based on the above first embodiment, the second embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the same or similar content as the above first embodiment can be referred to the above introduction and will not be elaborated hereinafter. In this embodiment, it is proposed to predict environmental state information and / or sensor state information based on the input reference signal and error signal through a prediction model. Specifically, in this embodiment, the vehicle road noise processing method further includes step S60: using a preset prediction model to perform prediction based on the input time-domain reference signal, the input time-domain error signal, the sub-band reference signal, and the sub-band error signal to obtain the environmental state information and / or the sensor state information, where the prediction model is pre-trained.

[0126] In the specific implementation, the prediction model can be implemented by a neural network model, which can be pre-trained and then deployed in the control unit. The control unit calls the prediction model for prediction when it needs to obtain environmental state information or sensor state information. In this embodiment, the implementation manner of the prediction model is not limited.

[0127] In the specific implementation, multiple prediction models can be set to output different environmental state information or sensor state information respectively, or one prediction model can be set to output multiple types of information in the mode of multi-output tasks.

[0128] In this embodiment, since the changes in the environmental state and the sensor state will be reflected in the signals collected by the sensor, therefore, using the time-domain reference signal, the time-domain error signal, the sub-band reference signal, and the sub-band error signal as the input data of the prediction model, the pre-trained prediction model can learn the relationship between these input data and the environmental state information or the sensor state information. Furthermore, the prediction model can be used to predict the environmental state information or the sensor state information based on these input data. On the one hand, by obtaining the environmental state information or the sensor state information through model prediction, the existing reference signal and error signal can be utilized without the need to additionally set up a data acquisition module; on the other hand, the complex situations of vehicle road noise under different environmental states or sensor states can be reflected in the signals collected by the sensor. Therefore, using the prediction model, the signals collected by the sensor can predict more complex and accurate environmental state information and sensor state information, thereby enabling the dynamic determination of the update step size of the sub-band adaptive filter based on this information, which can greatly improve the system's ability to cope with complex environments; in addition, using the time-domain signal and the sub-band signal together as the prediction basis takes into account the importance of the frequency-domain characteristics and can accurately detect and match more scenarios with small differences in time-domain characteristics.

[0129] In a feasible implementation manner, the prediction model may include a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module. The step S60 includes S601 to S605:

[0130] Step S601: Input the sub-band reference signal and the sub-band error signal into the first feature extraction module for feature extraction to obtain a first feature representation.

[0131] Step S602: Input the input time-domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation.

[0132] Step S603: Input the input time-domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation.

[0133] Step S604: Concatenate the first feature representation, the second feature representation, and the third feature representation and input them into the feature fusion module for feature fusion to obtain a fused feature representation.

[0134] Step S605: Input the fused feature representation into the prediction module for prediction to obtain the environmental state information and / or the sensor state information.

[0135] It should be noted that, in order to improve the accuracy of the prediction result and fully extract the information related to the environmental state and the sensor state in the signal, in this implementation manner, feature extraction modules are respectively set for the sub-band reference signal, the sub-band error signal, the time-domain reference signal, and the time-domain error signal to extract the signal features of the three signals. For distinction, they are respectively called the first feature extraction module, the second feature extraction module, and the third feature extraction module; a feature extraction module based on a neural network structure can be used to implement it. The results output by the three feature extraction modules, namely the feature representations, are respectively called the first feature representation, the second feature representation, and the third feature representation for distinction. The feature representation can be in the form of a feature vector or a feature map, which is not limited in this implementation manner. The feature fusion module can specifically be a module for simply concatenating the three feature representations, or can also include a module for deeply fusing the concatenated features. The module for deep fusion can specifically also be implemented based on a neural network structure, which is not limited here. The prediction module can be designed according to the type of information to be output and the specific form of each piece of information, which is not limited in this implementation manner.

[0136] In a feasible implementation manner, the first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer, and the convolutional layer is used to extract features from the signal; the feature fusion module includes a frequency-axis gated recurrent unit (FGRU), a time-axis gated recurrent unit (TGRU), and at least one transposed convolutional layer connected in sequence. The frequency-axis gated recurrent unit is used to process data along the frequency axis to capture the correlation in the frequency dimension, the time-axis gated recurrent unit is used to process the output of the frequency-axis gated recurrent unit along the time axis to capture the dynamic features in the time dimension, and the transposed convolutional layer is used to gradually upsample and restore to the original size; the prediction module includes a plurality of task output layers, and each task output layer is respectively used to output one item of the environmental state information or the sensor state information. Hereinafter, FGRU and TGRU may also be collectively referred to as FT-GRU.

[0137] In a feasible implementation, a prediction model can be set up based on an improved U-Net (a convolutional neural network based on deep learning) architecture, integrating a frequency-axis gated recurrent unit and a time-axis gated recurrent unit, specifically designed to process complex acoustic and vibration signals. The prediction model includes an Encoder and a Decoder, which are specifically designed to process acoustic and vibration signals. The Encoder performs progressive downsampling through multiple layers of convolution, batch normalization, and ReLU activation functions to effectively extract features; the Decoder uses multiple layers of transposed convolution and skip connections for progressive upsampling to prepare features for the terminal classification task. In addition, FGRU and TGRU are integrated into the network structure to enhance the processing ability of multi-dimensional data, especially in time series and frequency analysis, to adapt to real-time changing environmental conditions. This comprehensive design not only captures complex features spatially but also improves the adaptability and prediction accuracy of the model to complex changes in the vehicle's interior and exterior environments by dynamically adjusting the GRU layer responses. Specifically, the Encoder includes a first feature extraction module, a second feature extraction module, and a third feature extraction module. The first feature extraction module may include three convolutional layers, followed by a max-pooling layer after the first two convolutional layers, and the third convolutional layer is used to adjust the dimension of the output feature representation so that the feature representation output by the first feature extraction module is convenient for splicing with the feature representations output by the other two feature extraction modules. The dimensions of the input and output data of the first feature extraction module and the input and output data of its respective convolutional layers and max-pooling layers can be set as needed and are not limited here. The second feature extraction module and the third feature extraction module perform feature extraction on time-domain signals and can be set to include a one-dimensional convolutional layer, a pooling layer, and a convolutional layer for adjusting the dimension of the output feature representation respectively. The dimensions of the input and output data of the second feature extraction module and the third feature extraction module and the input and output data of their respective convolutional layers and pooling layers can be set as needed and are not limited here. After the feature representations output by the three feature extraction modules are spliced, they are input into the frequency-axis gated recurrent unit. The splicing method can be, for example, splicing along the channel dimension, which is not limited here. The role of the frequency-axis gated recurrent unit is to process data along the frequency axis and capture the correlations in the frequency dimension; the spliced feature representations can be divided into multiple time steps and are sequentially input into the frequency-axis gated recurrent unit. After cyclic calculation by the frequency-axis gated recurrent unit, the hidden states corresponding to each time step are output and sequentially input into the time-axis gated recurrent unit. The role of the time-axis gated recurrent unit is to process the output of the frequency-axis gated recurrent unit along the time axis and capture the dynamic features in the time dimension; the time-axis gated recurrent unit performs cyclic calculation on the hidden states of each sequentially input time step, and the output result is further used as the input of the transposed convolutional layer. The dimensions of the input and output data of the frequency-axis gated recurrent unit and the time-axis gated recurrent unit are also not limited here.The Decoder after the time-axis gated recurrent unit can be set to include two transposed convolutional layers and a convolutional layer for adjusting the dimension of the output data. The dimensions of the specific input and output data are not restricted here. The Decoder is followed by a multi-task output layer, which can include a shared feature extraction layer and output layers separately set for each piece of information. For example, if it is necessary to output weather condition information, road surface condition information, and sensor status information, an output layer can be separately set for each such piece of information. Among them, the shared feature extraction layer can include one or more convolutional layers, and a pooling layer can be connected after the convolutional layer, whose function is to extract general features and reduce the spatial dimension. The output layers separately set for each piece of information can include a convolutional layer, a global average pooling layer, and a fully connected layer. An activation function is set after the fully connected layer, which is used to output the confidence levels or probability distributions corresponding to the respective value categories of each piece of information, for determining the final values of each piece of information. The dimensions of the input and output data of the multi-task output layer and the dimensions of the input and output data of each layer therein are also not restricted. In this embodiment, not only is the independent processing of each task ensured, but also the overall performance and resource utilization rate of the model are optimized through the sharing of underlying features and the specialization of top-level decisions.

[0138] In a feasible embodiment, a lightweight, real-time, low-parameter-frequency-time-structured U-Net network can be set as the prediction model, which is specifically designed to accurately process and analyze various frequency-domain and time-domain data, ensuring the effective fusion of sub-band domain and time-domain multi-source information, effectively improving the recognition speed and accuracy of complex acoustic environments and road conditions, and improving the robustness of the model to changes in the vehicle interior and exterior environments. Specifically, the dimensions of the input and output data of the prediction model and the input and output data of each layer in the model can all conform to the definition of [batch size, feature dimension, number of channels]. The dimension of the input data of the first feature extraction module can be set to [1, 65, 11], where the batch size is 1, indicating that a single sample is processed each time, the feature dimension is 65, corresponding to the frequency-domain features extracted by sub-band analysis (i.e., 65 sub-bands), and the number of channels is 11, including 9 vibration sensor channels and 2 error microphone channels. The dimension of the input data of the second feature extraction module can be set to [1, 64, 9], the batch size is 1, the time step is 64, indicating the length of the time-domain signal (i.e., the frame length is 64), and the number of channels is 9, corresponding to 9 vibration sensors. The dimension of the input data of the third feature extraction module is set to [1, 64, 2], the batch size is 1, the time step is 64, and the number of channels is 2, corresponding to 2 error microphones. When training the prediction model, a sequence containing T frames can be input, such as [1, T, 65, 11], so that the FT-GRU can learn the inter-frame relationship; during inference, frames can be input one by one, and the FT-GRU realizes inter-frame modeling through state transfer.

[0139] The encoder includes a first feature extraction module, a second feature extraction module, and a third feature extraction module.

[0140] The first feature extraction module may include:

[0141] The first convolutional layer: the convolutional kernel dimension is 3x3, the number of input channels is 11, the number of output channels is 16, the padding is 1 ("same" padding), and the output dimension is [1, 65, 16].

[0142] The max pooling layer: the pooling window is 2x1, the stride is 2, and the output dimension is [1, 32, 16].

[0143] The second convolutional layer: the convolutional kernel dimension is 3x3, the number of input channels is 16, the number of output channels is 32, the padding is 1 ("same" padding), and the output dimension is [1, 32, 32].

[0144] The max pooling layer: the pooling window is 2x1, the stride is 2, and the output dimension is [1, 16, 32].

[0145] The 1D convolutional layer: 1D convolution, the convolutional kernel dimension is 4x1, the number of input channels is 32, the number of output channels is 32, the stride is 2x1, the padding is "valid", and the output dimension is [1, 29, 32].

[0146] The final output dimension of the first feature extraction module: [1, 29, 32].

[0147] The second feature extraction module includes:

[0148] The one-dimensional convolutional layer: the convolutional kernel dimension is 3, the number of input channels is 9, the number of output channels is 16, the padding is 1 ("same" padding), and the output dimension is [1, 64, 16].

[0149] The pooling layer: the pooling window is 2, the stride is 2, and the output dimension is [1, 32, 16].

[0150] The 1D convolutional layer: 1D convolution, the convolutional kernel dimension is 4, the number of input channels is 16, the number of output channels is 32, the stride is 1, the padding is "valid", and the output dimension is [1, 29, 32].

[0151] The third feature extraction module includes:

[0152] The one-dimensional convolutional layer: the convolutional kernel dimension is 3, the number of input channels is 2, the number of output channels is 32, the padding is 1 ("same" padding), and the output dimension is [1, 64, 32].

[0153] The pooling layer: the pooling window is 2, the stride is 2, and the output dimension is [1, 32, 32].

[0154] 1D Convolutional Layer: 1D convolution with a kernel dimension of 4, an input channel number of 32, an output channel number of 32, a stride of 1, padding of "valid", and an output dimension of [1, 29, 32].

[0155] Concatenation Operation in the Feature Fusion Stage: The outputs of the three branches are all [1, 29, 32]. They are concatenated along the channel dimension, i.e., 32 + 32 + 32 = 96, and the output dimension is [1, 29, 96].

[0156] The FT-GRU layer follows the Encoder. The input dimension of the FT-GRU layer is [1, 29, 96], where 29 represents the feature dimension (or time step), and 96 represents the number of input feature channels. The FT-GRU layer includes:

[0157] The First Layer: Freq GRU (GRU on the frequency axis), Function: Process data along the frequency axis to capture correlations in the frequency dimension. The input dimension is [1, 29, 96], where 29 represents the time step and 96 represents the feature channel dimension. Processing Method: For each time step (a total of 29), Freq GRU treats the 96-dimensional feature as an input sequence (length 96) and performs cyclic calculations to output the hidden state. Assuming the number of hidden units is H (e.g., H = 32), the output is [1, 29, H].

[0158] The Second Layer: Time GRU (GRU on the time axis), Function: Process the output of Freq GRU along the time axis to capture dynamic features in the time dimension. The input dimension is [1, 29, H]. Processing Method: Perform cyclic operations over 29 time steps, with each time step inputting H-dimensional features and outputting the final features. Assuming the target output channel number is 32, the output dimension is [1, 29, 32].

[0159] The Decoder after the FT-GRU layer includes two transposed convolutional layers and a convolutional layer for adjusting the output data dimension.

[0160] The input dimension of the first transposed convolutional layer is [1, 29, 32], the size of the transposed convolutional kernel is 4x1, the stride is 2, the padding is 1, the output channel number is 32, and the output dimension is [1, 58, 32].

[0161] The input dimension of the second transposed convolutional layer is [1, 58, 32], the size of the transposed convolutional kernel is 8x1, the stride is 1, the padding is 0, the output channel number is 16, and the output dimension is [1, 65, 16].

[0162] The input dimension of the third convolutional layer is [1, 65, 16], the size of the convolutional kernel is 1x1, the stride is 1, the output channel number is 1, and the output dimension is [1, 65, 1].

[0163] The decoder is followed by a multi-task output layer, which includes a shared feature extraction layer and multiple output layers (pavement condition classification branch, weather condition classification branch, and sensor status monitoring branch).

[0164] The input dimension of the shared feature extraction layer is [1, 65, 1], and the shared feature extraction layer includes:

[0165] Convolutional layer: The convolutional kernel size is 3x1, the input channels are 1, the output channels are 32, and the padding is 1 ("same" padding), with the output being [1, 65, 32]. Max pooling layer: The pooling window is 2x1, the stride is 2, and the output is [1, 32, 32]. Its function is to extract general features and reduce the spatial dimension.

[0166] The pavement condition classification branch, weather condition classification branch, and sensor status monitoring branch connected after the shared feature extraction layer.

[0167] The input dimension of the pavement condition classification branch is [1, 32, 32], and the output dimension is [1, N], which is the probability distribution of N pavement conditions. The pavement condition classification branch includes:

[0168] Convolutional layer: 1x1 convolution, with 64 output channels and an output dimension of [1, 32, 64].

[0169] Global average pooling: Reducing the dimension to [1, 64].

[0170] Fully connected layer: Outputting N (the number of pavement condition categories), with the activation function Softmax.

[0171] The input dimension of the weather condition classification branch is [1, 32, 32], and the output dimension is [1, M], which is the probability distribution of M weather conditions. The weather condition classification branch includes:

[0172] Convolutional layer: 1x1 convolution, with 64 output channels and an output dimension of [1, 32, 64].

[0173] Global average pooling: Reducing the dimension to [1, 64].

[0174] Fully connected layer: Outputting M (the number of weather condition categories), with the activation function Softmax.

[0175] The input dimension of the sensor status monitoring branch is [1, 32, 32], and the output dimension is [1, 11]. Each value is between 0 and 1, representing the confidence levels of 11 sensors. The sensor status monitoring branch includes:

[0176] Convolutional layer: 1x1 convolution, with 128 output channels and an output dimension of [1, 32, 128].

[0177] Global average pooling: The dimension is reduced to [1, 128].

[0178] Fully connected layer: Output 11 (corresponding to 11 sensors), and the activation function is Sigmoid.

[0179] Overall, the results output by the prediction model include:

[0180] Road surface condition classification: [1, N], the probability distribution of N road surface conditions.

[0181] Weather condition classification: [1, M], the probability distribution of M weather conditions.

[0182] Sensor status monitoring: [1, 11], the confidence levels (0 - 1) of 11 sensors.

[0183] Based on the above results output by the prediction model, road surface condition information for characterizing the road surface condition, weather condition information for characterizing the weather condition, and sensor status information for characterizing the sensor status can be obtained. For example, in a feasible implementation manner, the category corresponding to the largest probability distribution among the various probability distributions in the road surface condition classification result can be used as the road surface condition information. Also, for example, it can be preset that the magnitude of the confidence level in the sensor status monitoring result represents the degree of sensor abnormality. The sensor status information can be this binary classification information indicating whether the sensor is in a normal state or an abnormal state, or it can be information indicating the degree of sensor abnormality, which can be set according to needs in specific implementation manners.

[0184] In a feasible implementation manner, referring to Figure 2 , the prediction model can be trained in the following way in advance.

[0185] Step 1: Data collection. Install accelerometers and microphones with sensitivities and frequency responses meeting the test requirements at the positions where acceleration sensors and microphones need to be installed on the vehicle to comprehensively capture vibration and sound data under different road conditions and speeds.

[0186] Step 2: Data annotation. During the data recording process, the annotation work can be completed synchronously. All data annotations can be directly carried out by annotators during data recording to perform real-time and detailed annotation on the road surface conditions in the data. Specifically, when the microphone and vibration sensor collect data inside and outside the vehicle, the annotator immediately classifies and describes the current road surface condition. For example, these descriptions can include, but are not limited to: road type (such as highway, urban road, rural road), lane markings, road surface texture, pothole conditions, obstacle distribution, and road wetness. The annotator will also record (such as rainy or snowy days) and annotate the weather conditions affecting the road surface condition.

[0187] Step 3: Data augmentation. To improve the model's ability to recognize abnormal situations, sensor faults and noises can be artificially introduced. First, the robustness of the model can be enhanced by simulating various sensor faults and environmental interferences. For example, covering the microphone with cloth, plastic or metal sheets to simulate the blockage of dust and water, and simulating abnormal impacts by tapping the accelerometer. In addition, digital signal processing techniques can also be used to add various synthetic noises to the normal data, such as white noise and natural environmental noise, as well as signals simulating electromagnetic interference. When introducing these faults and interferences, the data can be systematically segmented by time and classified and labeled, such as "microphone occlusion - cloth" or "accelerometer impact interference", to ensure the richness of the dataset and the generalization ability of the trained model. These refined operations not only increase the model's adaptability to complex real-world situations, but also help maintain the performance and stability of the system in various environments.

[0188] Step 4: Dataset balancing. To address the problem of uneven data distribution in the collected data, data analysis can be performed to identify which categories or scenarios have too little or too much data. Then, data augmentation techniques can be used to increase the number of samples in the scarce categories, such as generating new data by varying the speed, adding noise or using synthetic techniques. For the overabundant categories, a downsampling strategy can be adopted to selectively reduce the samples, or representative samples can be selected through clustering analysis. In addition, resampling techniques can also be used to balance the entire dataset, ensuring that the data of each category is evenly distributed in the training set, thereby improving the generalization ability and performance of the model under all conditions. This step can also include applying statistical methods to evaluate the changes before and after data balancing to verify the effectiveness of the balancing strategy and adjust the balancing parameters as needed.

[0189] Step 5: Model architecture setting. For example, the model architecture in the above embodiments can be set.

[0190] Step 6: Model training. Customized loss functions can be used for model training to handle different classification tasks. For multi-class classification tasks (such as road surface conditions and weather conditions), cross-entropy loss can be used to ensure that the model can effectively distinguish different categories; while for the binary classification problem of sensor status, binary cross-entropy loss can be adopted to improve the sensitivity of recognizing normal and abnormal states. During the training process, the Adam optimizer (Adaptive Moment Estimation) can be selected, and its adaptive learning rate mechanism helps to better handle the problem of gradient sparsity. At the same time, SGD (Stochastic Gradient Descent) with momentum can also be considered to stabilize the training process. In addition, early stopping mechanisms and regularization techniques can also be adopted to prevent overfitting and ensure that the model can perform well on unseen data.

[0191] Step Seven: Verification and Testing. To ensure the effectiveness and reliability of the model in practical applications, the dataset can be divided into a training set, a validation set, and an independent test set. During the model development process, the validation set is used for periodic evaluation to monitor the training progress and adjust hyperparameters. The test set is used for the final evaluation after the model training is completed to test the performance of the model on completely unknown data. Various performance metrics, such as accuracy, precision, recall, and F1-score, are applied to evaluate the classification task, and the ROC (Receiver Operating Characteristic) curve and AUC (Area Under Curve) value are used to measure the performance of the model for sensor status anomaly detection.

[0192] Based on the above first and / or second embodiments, the third embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the same or similar content as the above first and second embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, a load balancing strategy is proposed, so that the automotive road noise processing algorithm proposed in the above embodiments can be applied to the automotive RNC system with limited computing power conditions. For example, there are a large number of computational peak demands within a single sampling period in a MIMO vehicle-mounted RNC system. If the subband adaptive algorithm without output delay is run in real time on an embedded platform such as a DSP (Digital Signal Processor), the system may not be able to run in real time due to insufficient peak computing power. The load balancing strategy proposed in this embodiment aims to ensure that the subband adaptive algorithm without output delay can be run in real time on an embedded platform such as a DSP, solves the problem of insufficient peak computing power, and provides an important guarantee for practical applications. Specifically, in this embodiment, the automotive road noise processing method further includes step S70:

[0193] Execute the tasks allocated to the N interrupts among the N interrupts in the (l + 1)-th frame, where N represents the frame length, and the interrupt refers to the period from generating the anti-noise signal based on the time-domain reference signal of each sampling point to receiving the time-domain reference signal of the next sampling point. The tasks allocated to the N interrupts include: the task of calculating the updated time-domain adaptive filter coefficients, and / or the task of predicting the environmental state information and / or the sensor state information using the prediction model.

[0194] In this embodiment, the control unit adopts a method without output delay for road noise control, that is: the time-domain reference signal collected by the reference sensor and the time-domain error signal collected by the error sensor are input into the control unit in real time point by point. The control unit generates a single-sampling-point anti-noise signal for the currently input single-sampling-point time-domain reference signal according to the current time-domain adaptive filter coefficients in real time, and then outputs the anti-noise signal in real time, achieving the effect of no output delay; on the other hand, the adaptive filter coefficients are updated frame by frame. Specifically, the control unit puts the input time-domain reference signal and time-domain error signal into the corresponding buffer in real time. After each buffer is filled with a frame of time-domain reference signal and a frame of time-domain error signal, subband decomposition of the frame of time-domain reference signal and the frame of time-domain error signal is performed, as well as subsequent operations for updating the time-domain adaptive filter coefficients.

[0195] Then, after updating the time-domain adaptive filter coefficients based on the l-th frame of time-domain reference signal and time-domain error signal, the updated time-domain adaptive filter coefficients will be used to process each point of the l+1-th frame of N sampling points of the time-domain reference signal to generate N sampling points of anti-noise signals. Specifically, for each received sampling point of the time-domain reference signal, an anti-noise signal for one sampling point is generated and output according to the updated time-domain adaptive filter coefficients. Then, it can be understood that during the process of processing each point of the l+1-th frame of N sampling points of the time-domain reference signal, it is also necessary to complete the update of the time-domain adaptive filter coefficients based on the l+1-th frame of time-domain reference signal and the l+1-th frame of time-domain error signal for processing each sampling point of the l+2-th frame of the time-domain reference signal to generate anti-noise signals, and so on.

[0196] It can be understood that within the time period from the start of receiving the first sampling point of the l+1-th frame to the generation of anti-noise signals by processing the N-th sampling point of the l+1-th frame, it is necessary to complete an update operation of the time-domain adaptive filter coefficients based on the l-th frame of time-domain reference signal and time-domain error signal. In this embodiment, the update task of the adaptive filter coefficients can be distributed to N interrupts for execution, avoiding the execution of the entire adaptive filter coefficient update task in a single interrupt, thereby avoiding the problem of insufficient computing power caused by a sharp increase in computing power.

[0197] Before updating the time-domain adaptive filter coefficients based on the l-th frame time-domain reference signal and the time-domain error signal, the l-th frame time-domain reference signal, the time-domain error signal, and the corresponding subband reference signal and subband error signal can be input into a prediction model to predict the environmental state information and / or sensor state information, so as to determine the update step size required for updating the filter coefficients this time according to the environmental state information and / or sensor state information. Then, during the period from the start of receiving the first sampling point of the (l + 1)-th frame to the processing of the n-th sampling point of the (l + 1)-th frame to generate the anti-noise signal, it is also necessary to complete the task of inputting the l-th frame time-domain reference signal, the time-domain error signal, and the corresponding subband reference signal and subband error signal into the prediction model to predict the environmental state information and / or sensor state information. In this embodiment, the prediction task can be distributed to N interrupts for execution, avoiding the execution of the entire prediction task in a single interrupt, thereby avoiding the problem of insufficient computing power caused by a sharp increase in computing power.

[0198] The load balancing strategy proposed in this embodiment ensures the full utilization of each interrupt, avoids the situation of any single interrupt being overloaded, optimizes the use of computing resources, and ensures the high-efficiency performance and response speed of the system in real-time processing. This method of distributing computing tasks to multiple interrupts helps the system to maintain stable operation in the face of high computing requirements, thereby achieving continuous and efficient operation in various driving environments.

[0199] In this embodiment, there is no limitation on the way of distributing tasks to each interrupt. For example, the computing operations in the task can be divided into multiple subtasks according to the amount of computation, and the amount of computation of each subtask is the same or approximately the same, and each interrupt processes one subtask.

[0200] In a feasible implementation manner, the update task of the adaptive filter coefficients can be divided into a subband decomposition task, a subband synthesis task, and multiple subband update tasks. Each subband update task includes the update task of at least one subband adaptive filter coefficient. The subband decomposition task is executed in the first preset number of interrupts, the subband synthesis task is executed in the second preset number of interrupts, and one of the subband update tasks is executed in each of the middle preset number of interrupts. The sum of the preset first number, the preset second number, and the preset third number is N.

[0201] In a feasible implementation manner, the prediction task can be divided into multiple subtasks, each subtask includes the processing task of at least one layer in the prediction model, and one of the subtasks is executed in each interrupt. For example, assume N = 64. The first 32 interrupts process the operations of the Encoder layer, including convolution, activation, pooling, and the FGRU layer. The latter half of the interrupts complete the operations of the TGRU and Decoder layers, and the last interrupt completes the multi-task output layer operation.

[0202] It should be noted that when both the adaptive filter coefficient update task and the prediction task need to be allocated to N interrupt processes, since the subband signals are required in the prediction task, the task of subband decomposition can be allocated to the earlier interrupts, and the prediction task can be allocated to the later interrupts. For example, each subtask of the prediction task can be allocated to be executed in each interrupt except for the first preset number of interrupts at the front.

[0203] Based on the above first, second, and / or third embodiments, a fourth embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first, second, and third embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, a polyphase analysis filter bank can be used for subband decomposition, and a polyphase synthesis filter bank can be used for subband synthesis. Through this way of subband decomposition and subband synthesis, during the process of restoring the filter coefficients to the full band, the synthesis filter bank is directly used, and only the subband domain filter needs to be added with a synthesis window and then an IFFT (Inverse Fast Fourier Transform) operation is performed to obtain the time-domain filter coefficients that are almost the same as those in the full band, which not only ensures the performance but also avoids additional computational amount. In the specific implementation manner, a polyphase analysis filter bank based on FFT (hereinafter can be referred to as a polyphase FFT analysis filter bank) and a polyphase synthesis filter bank based on IFFT (hereinafter can be referred to as a polyphase IFFT synthesis filter bank) can be adopted to further improve the computational efficiency. Hereinafter, the analysis filter bank can also be referred to as an analysis window, and the synthesis filter bank can be referred to as a synthesis window.

[0204] In a feasible implementation manner, the control unit can dynamically determine a filter bank for sub-band decomposition and sub-band synthesis according to the acquired environmental state information. Hereinafter, the determined filter bank is referred to as a target filter bank for distinction. The environmental state information can be the environmental state information for dynamically determining the update step of the sub-band adaptive filter coefficients. The target filter bank can be determined by using the latest acquired environmental state information. The target filter bank includes a target polyphase analysis filter bank for sub-band decomposition and a target polyphase synthesis filter bank for sub-band synthesis. The target polyphase analysis filter bank is used to perform sub-band decomposition on the time-domain reference signal and the time-domain error signal, and the target polyphase synthesis filter bank is used to perform sub-band synthesis on the updated sub-band adaptive filter coefficients. Since the dynamic change of the acoustic propagation path will occur when the environmental state outside the vehicle or the environmental state inside the vehicle changes, which will affect the stability of the vehicle road noise. If a fixed polyphase analysis filter bank and polyphase synthesis filter bank are used for sub-band decomposition and sub-band synthesis, it may lead to the inability to adapt to the changing vehicle road noise and result in unstable noise reduction effect. Therefore, in this implementation manner, the target polyphase analysis filter bank and the target polyphase synthesis filter bank for sub-band decomposition and sub-band synthesis are dynamically determined according to the environmental state information to adapt to environmental changes and improve the stability of the noise reduction effect.

[0205] In a specific implementation manner, different filter banks can be preset for different environmental states. Each filter bank includes a polyphase analysis filter bank and a polyphase synthesis filter bank. Then, when the control unit needs to perform sub-band decomposition on the current time-domain reference signal and time-domain error signal, and needs to perform sub-band synthesis on the current sub-band adaptive filter coefficients, the filter bank preset corresponding to the environmental state represented by the acquired environmental state information can be determined as the target filter bank. The filter banks set for different environmental states can be the filter banks with the best or better noise reduction effect in this environmental state determined in advance through experiments, data statistics, etc.

[0206] For example, in a feasible implementation, the filter update delay and / or the spectral leakage of each filter bank pre-set for different environmental states are different. The filter update delay and spectral leakage brought by the filter bank refer to the filter update delay and spectral leakage that will occur when sub-band decomposition and sub-band synthesis are performed according to the filter bank. It should be noted that generally, the filter update delay and spectral leakage brought by the filter bank are negatively correlated. The larger the delay, the smaller the spectral leakage; the smaller the delay, the larger the spectral leakage. And the smaller the delay, the better the robustness and transient response of the system; the smaller the spectral leakage, the better the filtering accuracy and steady-state performance of the system. Since the stability of automotive road noise is different in different environmental states and the requirements for the robustness, transient response ability, filtering accuracy, and steady-state performance of the system are different, dynamically determining the filter bank to be used according to the environmental state information can balance the delay and spectral leakage, and adopt a delay and spectral leakage suitable for the environmental state, so as to balance the robustness, transient response ability, filtering accuracy, and steady-state performance of the system and ensure the stability of the noise reduction effect.

[0207] In a feasible implementation, it can be set that when the control unit dynamically determines the target filter bank according to the environmental state information, the following rules need to be met: in the environmental state characterized by the environmental state information, the higher the stability of the automotive road noise, the greater the filter update delay and the smaller the spectral leakage brought by the target filter bank; the lower the stability of the automotive road noise, the smaller the filter update delay and the larger the spectral leakage brought by the target filter bank.

[0208] It should be noted that if the stability of the automotive road noise is relatively high in the environmental state characterized by the environmental state information, a filter bank with a larger filter update delay and a smaller spectral leakage can be adopted, focusing on improving the filtering accuracy and steady-state performance to obtain a better noise reduction effect. If the stability of the automotive road noise is relatively low in the environmental state characterized by the environmental state information, a filter bank with a smaller filter update delay and a larger spectral leakage can be adopted, focusing on improving the robustness and transient response of the system to quickly adapt to environmental changes and maintain system stability.

[0209] In a specific implementation, in order to make the control unit meet the above rules when dynamically determining the target filter bank according to the environmental state information, the filter bank corresponding to each environmental state can be set according to this rule.

[0210] In a feasible implementation, the environmental state information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to the weather conditions that affect the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information characterizing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0211] In a feasible implementation, when the environmental state information indicates that the environment is good and stable (for example: good road conditions, clear weather), the road noise of the vehicle is relatively stable. Then, a filter bank with a larger filter update delay and smaller spectral leakage can be adopted, focusing on improving the filtering accuracy and steady-state performance to obtain a better noise reduction effect. When the environmental state information indicates that the environment is unstable (for example: deteriorating road conditions, bad weather), the stability of the road noise of the vehicle may become worse. At this time, a filter bank with a smaller filter update delay and larger spectral leakage can be adopted, focusing on improving the robustness and transient response of the system, quickly adapting to environmental changes and maintaining system stability.

[0212] The following Table 2 gives some examples of determining the filter bank based on the weather condition information and the road surface condition information. Table 2 only gives some examples and does not limit the way of determining the target filter bank in this embodiment. The content in parentheses in the table is the explanatory description of the weather condition information, road surface condition information, etc.

[0213] Table 2 Examples of selection of analysis window and synthesis window

[0214]

[0215] In a feasible implementation, each filter bank preset for different environmental states can be obtained by pre-training with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank. It should be noted that the method of training with the goal of reducing the signal reconstruction loss and spectral leakage loss of the filter bank is similar to the method of training a neural network model with the goal of reducing the prediction loss of the neural network model. The steps include preparing data for training, designing the loss function, selecting an optimizer for training, etc., which can specifically refer to the training method of the neural network model. Among them, the signal reconstruction loss refers to the error between the signal obtained by sub-band decomposition and sub-band synthesis of the signal through the filter bank and the original signal, and the spectral leakage loss refers to the magnitude of the spectral leakage generated by sub-band decomposition of the signal through the filter bank.

[0216] To train filter banks with different levels of spectral leakage and different filter update delays, different weight combinations of signal reconstruction loss and frequency leakage loss can be set during training. For example, by setting a larger weight for signal reconstruction loss and a smaller weight for frequency leakage loss, a filter bank with smaller delay and larger leakage can be trained. By setting a smaller weight for signal reconstruction loss and a larger weight for frequency leakage loss, a filter bank with larger delay and smaller leakage can be trained.

[0217] In a feasible implementation, the signal reconstruction loss is calculated based on the error between the reconstructed signal and a preset first type of signal. The reconstructed signal is obtained by performing subband decomposition and subband synthesis on the preset first type of signal using the polyphase analysis filter bank to be trained and the polyphase synthesis filter bank to be trained. Among them, an initialized polyphase analysis filter bank and polyphase synthesis filter bank can be set in advance, and the coefficients of the polyphase analysis filter bank and polyphase synthesis filter bank need to be updated during the training process. The preset first type of signal is a signal preset for calculating the signal reconstruction loss, which can be various noise signals, vibration signals, music signals, etc. collected in advance. The preset first type of signal is decomposed into subbands using the polyphase analysis filter bank to be trained, and the decomposed result is then synthesized into subbands using the polyphase synthesis filter bank to be trained to obtain the reconstructed signal, and the error is calculated with the preset first type of signal to obtain the signal reconstruction loss.

[0218] In a feasible implementation, the spectral leakage loss is calculated based on the target subband signal. The target subband signal is obtained by performing subband decomposition on a preset second type of signal using the polyphase analysis filter bank to be trained. The preset second type of signal is a single-frequency signal or narrowband signal within a preset frequency band range, and the frequency band range of the target subband signal is the range outside the preset frequency band range. Among them, the preset second type of signal is a signal preset for calculating the spectral leakage loss, which can be a single-frequency signal or narrowband signal within a preset frequency band range collected in advance, and the preset frequency band range can be set as needed. The preset second type of signal is decomposed into subbands using the polyphase analysis filter bank to be trained, and the spectral leakage loss is calculated based on the target subband signal in the decomposition result. It can be understood that since the preset second type of signal is a single-frequency signal or narrowband signal within a preset frequency band range, after decomposing the preset second type of signal into subbands, if there is no spectral leakage, then there will be no signal in the range outside the preset frequency band range in the subband decomposition result, and the frequency band range of the target subband signal is the range outside the preset frequency band range, so the magnitude of the target subband signal can represent the magnitude of spectral leakage, and the spectral leakage loss can be calculated based on the target subband signal.

[0219] In a feasible implementation, the filter bank can be trained in the following manner:

[0220] Step 1: Data preparation. Two types of data are used for training the analysis filter bank and the synthesis filter bank. The first type is various noise signals, vibration signals, and music signals. The second type is single-frequency signals or narrowband signals uniformly generated according to a known frequency band range (hereinafter, the set of sub-band indices within this frequency band range is denoted by S). Each of these two types of data accounts for 50% of the total data.

[0221] Step 2: Design of the loss function. The signal reconstruction error and the spectral leakage energy are two criteria for measuring the performance of the analysis window and the synthesis window of the sub-bands. Therefore, these two metrics are used as the loss function for training.

[0222]

[0223]

[0224]

[0225] Among them, is the reconstruction loss function of the signal, is the loss function of the spectral leakage of the signal. α and β are weight coefficients used to adjust the influence ratio of the losses of the two parts. For example, they are set as α = 0.4 and β = 0.6. z[r] is the preset first type of signal, and z'[r] is the signal after sub-band decomposition and sub-band synthesis of z[r], that is, the reconstructed signal. R is the length of the signal, for example, it can be set as R = 20000. Z[k] is the sub-band signal after sub-band decomposition of the preset second type of signal, k is the index of the sub-band, and S is the set of sub-band indices within the known frequency band range in Step 1.

[0226] Step 3: Select an optimizer for training. Calculate according to the input preset first type of signal, and calculate according to the input preset second type of signal. Combine these two losses as the final loss . Pytorch (an open-source deep learning framework for machine learning and deep learning) can be selected as the training environment, and Adam can be selected as the optimizer to perform training iterations on the analysis window and the synthesis window until the loss function converges, and finally obtain the trained analysis window and synthesis window. The downsampling factor can be set to 3, and the number of points P of the FFT can be set to 128. Therefore, the sizes of the analysis window and the synthesis window are 128 * 3. During training, a hamming window with a length of 128 * 3 can be used as the initial coefficient.

[0227] Based on the above first, second, third, and / or fourth embodiments, the fifth embodiment of the automotive road noise processing method of the present application is proposed. In this embodiment, the same or similar content as that in the above first, second, third, and / or fourth embodiments can be referred to the above introduction and will not be repeated hereinafter. In this embodiment, the target clipping parameter value can also be dynamically determined according to the obtained environmental state information and / or sensor state information. After generating the anti-noise signal, the target clipping parameter value is used to limit the amplitude of the anti-noise signal, and the anti-noise signal after amplitude limitation is output to control the road noise. That is, post-processing operations can be performed on the anti-noise signal. The post-processing operations include amplitude limitation of the anti-noise signal, and the amplitude limitation is performed according to the target clipping parameter value determined according to the environmental state information and / or sensor state information. The clipping parameter value is a parameter value used to limit the degree of amplitude limitation. What specific parameter it is is not limited in this embodiment. For example, it can be an upper amplitude value, indicating that the amplitude value of the finally output signal is not allowed to exceed this upper amplitude value.

[0228] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause a dynamic change in the acoustic propagation path, which will affect the stability of the automotive road noise. If a fixed clipping parameter value is used for amplitude limitation of the anti-noise signal, when the stability of the automotive road noise changes, it may be impossible to fully cancel the noise due to excessive amplitude limitation, or it may be impossible to avoid introducing noise or distortion due to misjudgment or overcompensation due to too small amplitude limitation. Therefore, in the specific implementation, the clipping parameter value can be dynamically adjusted according to the environmental state information, so as to dynamically control the output amplitude of the anti-noise signal, adapt to different environmental states, ensure good noise reduction effect, and avoid introducing noise or distortion due to misjudgment or overcompensation, and ensure the stability and safety of the system.

[0229] When the working state of the reference sensor and the working state of the error sensor change, the accuracy of the collected signal will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signal will decrease, and then the estimation of the automotive road noise in the signal will be incorrect, and the generated anti-noise signal may not only fail to cancel the noise but also introduce noise. Therefore, in the specific implementation, the limit clipping parameter value can be dynamically adjusted according to the sensor state information, so as to dynamically control the output amplitude of the anti-noise signal. For example, when the sensor is abnormal, the output amplitude can be reduced, so as to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0230] The specific method for determining the target amplitude limiting parameter value based on the environmental state information and / or sensor state information is not limited in this embodiment. For example, the mapping relationship between the environmental state information and / or sensor state information and the amplitude limiting parameter value can be set in advance as needed. After obtaining the environmental state information and / or sensor state information, the control unit uses this mapping relationship to take the amplitude limiting parameter value corresponding to the environmental state information and / or sensor state information as the target amplitude limiting parameter value. In this embodiment, the mapping relationship is not limited. For example, the mapping relationship can be represented in the form of a mapping table, a relational expression, etc., and the mapping relationship can be determined through experiments, data statistics, or other methods.

[0231] In a feasible implementation manner, it can be set that when the control unit determines the target amplitude limiting parameter value according to the influence factor information, it conforms to one or more of the following rules:

[0232] 1. The higher the stability degree of the vehicle road noise in the environmental state characterized by the environmental state information, the smaller the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter value; the lower the stability degree of the vehicle road noise in the environmental state characterized by the environmental state information, the greater the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter.

[0233] If the stability degree of the vehicle road noise in the environmental state characterized by the environmental state information is relatively high, an amplitude limiting parameter value with a relatively small degree of amplitude limitation for the anti-noise signal can be adopted, that is, an anti-noise signal with a larger output amplitude is allowed, so as to more fully cancel the noise and ensure a good noise reduction effect. If the stability degree of the vehicle road noise in the environmental state characterized by the environmental state information is relatively low, an amplitude limiting parameter value with a relatively large degree of amplitude limitation for the anti-noise signal can be adopted, that is, the amplitude of the output signal is more severely limited, so as to avoid introducing noise or distortion due to system misjudgment or overcompensation and ensure system stability.

[0234] 2. The higher the degree of abnormality of the sensor characterized by the sensor state information, the greater the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter; the lower the degree of abnormality of the sensor characterized by the sensor state information, the smaller the degree of amplitude limitation of the anti-noise signal according to the target amplitude limiting parameter.

[0235] In the case of a higher sensor abnormality, the degree of amplitude limitation of the anti-noise signal is greater, so as to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0236] In a feasible implementation manner, on the basis of the environmental state information and / or sensor state information, the target amplitude limiting parameter value can also be determined according to the vehicle speed. When determining the target amplitude limiting parameter value according to the vehicle speed, it conforms to the following rule:

[0237] 3. When the vehicle speed stability is higher, the degree of amplitude limitation of the anti-noise signal according to the target amplitude limitation parameter is smaller; when the vehicle speed stability is lower, the degree of amplitude limitation of the anti-noise signal according to the target amplitude limitation parameter is larger.

[0238] The vehicle speed stability refers, for example, to the change rate of the vehicle speed. The larger the change rate, the lower the vehicle speed stability; the smaller the change rate, the higher the vehicle speed stability.

[0239] When the vehicle speed stability is relatively high, the stability of the vehicle road noise is relatively high. At this time, a limit parameter value with a relatively small degree of amplitude limitation of the anti-noise signal can be adopted, that is, an anti-noise signal with a larger allowable output amplitude is allowed, so as to more fully cancel the noise and ensure a good noise reduction effect. When the vehicle speed stability is relatively low, the stability of the vehicle road noise is relatively low. At this time, a limit parameter value with a relatively large degree of amplitude limitation of the anti-noise signal can be adopted, that is, the amplitude of the output signal is limited to a greater extent, so as to avoid introducing noise or distortion due to system misjudgment or overcompensation and ensure the system stability.

[0240] In the specific implementation manner, in order to make the control unit determine the target amplitude limitation parameter value in accordance with one or more of the above rules, a mapping relationship that satisfies one or more of the above rules can be set. The control unit determines the amplitude limitation parameter value corresponding to the environmental state information, the sensor state information, and the vehicle speed as the target amplitude limitation parameter value according to this mapping relationship. The mapping relationship can be represented by a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple above rules, the multiple rules can conform to a certain priority order to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the amplitude limitation parameter value according to the sensor state information can be higher than the priority of determining the amplitude limitation parameter value according to the environmental state information.

[0241] In a feasible implementation, when the environmental status information and the sensor status information indicate that the environment is good and stable (e.g., good road conditions, clear weather, normal sensors, medium vehicle speed), the vehicle road noise is relatively stable. At this time, a clipping parameter value with a smaller degree of amplitude limitation for the anti-noise signal can be adopted, allowing a larger amplitude anti-noise signal to be output, so as to more fully cancel the noise and ensure a good noise reduction effect. When the environmental status information and the sensor status information indicate that the environment deteriorates or becomes unstable (e.g., deteriorating road conditions, bad weather, abnormal sensors, drastic changes in vehicle speed), the stability of the vehicle road noise may deteriorate. At this time, a clipping parameter value with a larger degree of amplitude limitation for the anti-noise signal can be adopted, that is, the amplitude of the output signal is limited to a greater extent, to avoid introducing noise or distortion due to system misjudgment or overcompensation, and ensure system stability. When the environmental status information and the sensor status information indicate an extremely harsh environment or system abnormality (e.g., extremely harsh road conditions, serious sensor failure, extremely low speed), the amplitude of the output signal can be greatly limited to give priority to ensuring system safety.

[0242] Table 3 below gives some examples of determining the output clipping based on the weather condition information, road surface condition information, sensor status information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Table 3 only gives some examples and does not limit the way of determining the target clipping parameter value in this embodiment. The content in parentheses in the table is the explanatory description of the weather condition information, road surface condition information, sensor status information, output clipping, etc.

[0243] Table 3 Examples of Output Clipping Selection

[0244]

[0245] Based on the above first, second, third, fourth, and / or fifth embodiments, the sixth embodiment of the vehicle road noise processing method of the present application is proposed. In this embodiment, the content that is the same as or similar to the above first, second, third, fourth, and / or fifth embodiments can be referred to the above introduction and will not be elaborated hereinafter. In this embodiment, the pre-weighting weights corresponding to each sensor can also be determined according to the obtained environmental status information and / or sensor status information. After the reference signal and the error signal are weighted using the pre-weighting weights corresponding to their respective sensors, they are then used to generate the anti-noise signal and update the coefficients of the adaptive filter. That is, a preprocessing operation is performed on the input reference signal and error signal, and the preprocessing operation includes amplitude adjustment, and the amplitude adjustment is performed according to the pre-weighting weights corresponding to each sensor determined according to the environmental status information and / or sensor status information.

[0246] It should be noted that when the environmental state outside the vehicle or the environmental state inside the vehicle changes, it will cause a dynamic change in the acoustic propagation path, which will affect the energy level of the vehicle road noise, and the influence on the energy levels of vehicle road noise in different frequency bands may also be different. If a fixed amplitude adjustment strategy is adopted for the reference signal and the error signal, when the energy level of the vehicle road noise changes, it may be impossible to generate a more targeted anti-noise signal because the noise signals in the reference signal and the error signal are weak, resulting in poor noise reduction effect. Therefore, in the specific implementation, the pre-weighting weights of each sensor can be dynamically adjusted according to the environmental state information. Since the frequency bands of the signals collected by different sensors may be different, by adjusting the pre-weighting weights of the sensors, the signals in each frequency band can be enhanced or weakened specifically. For example, the sensors corresponding to the sub-bands with stronger vehicle road noise energy can be set with higher pre-weighting weights, so that a more targeted anti-noise signal can be generated and the noise reduction effect can be improved.

[0247] When the working states of the reference sensor and the error sensor change, the accuracy of the collected signals will be affected. For example, when there is an abnormality in the reference sensor or the error sensor, the accuracy of the collected signals will decrease, and then the vehicle road noise in the signals will be estimated incorrectly, and the generated anti-noise signal may not only fail to cancel the noise but also introduce noise. Therefore, in the specific implementation, the pre-weighting weights of each sensor can be dynamically adjusted according to the sensor state information. For example, the pre-weighting weight of the abnormal sensor can be reduced, so as to avoid introducing noise or distortion due to misjudgment or overcompensation, and ensure the safety and robustness of the system.

[0248] The specific method for determining the pre-weighting weights according to the environmental state information and / or the sensor state information is not limited in this embodiment. For example, the mapping relationship between the environmental state information and / or the sensor state information and the pre-weighting weights can be set in advance. After obtaining the environmental state information and / or the sensor state information, the control unit uses the pre-weighting weights corresponding to the environmental state information and / or the sensor state information as the target pre-weighting weights according to this mapping relationship. In this embodiment, the mapping relationship is not limited. For example, this mapping relationship can be represented by a mapping table, a relational expression, etc., and this mapping relationship can be determined through experiments, data statistics or other methods.

[0249] In a feasible implementation, it can be set that when the control unit determines the pre-weighting weights according to the environmental state information and / or the sensor state information, it conforms to one or more of the following rules:

[0250] 1. Under the environmental state characterized by the environmental state information, the pre-weighting weight of the sensor corresponding to the sub-band with higher energy of vehicle road noise is greater than that of the sensor corresponding to the sub-band with lower energy of vehicle road noise.

[0251] The sensor corresponding to the sub-band refers to the sensor whose signal is all or mainly distributed in this sub-band.

[0252] Under the same environmental state, the distribution of vehicle road noise in each sub-band may be different; for the sensor corresponding to the sub-band with higher noise energy, a larger pre-weighting weight can be adopted to specifically enhance the noise reduction intensity of this sub-band and improve the noise reduction effect; for the sub-band with lower noise energy, a smaller pre-weighting weight can be adopted to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0253] 2. For any target sub-band, under the environmental state characterized by the environmental state information, the higher the energy of vehicle road noise in the target sub-band, the greater the pre-weighting weight of the sensor corresponding to the target sub-band, and the lower the energy of vehicle road noise in the target sub-band under the environmental state characterized by the environmental state information, the smaller the pre-weighting weight of the sensor corresponding to the target sub-band.

[0254] For the same sub-band, under different environmental states, the energy level of vehicle road noise in this sub-band may be different; so for a certain sub-band (referred to as the target sub-band for distinction), if the energy of vehicle road noise in the target sub-band is higher under a certain environmental state, a larger pre-weighting weight can be adopted for the sensor corresponding to the target sub-band to specifically enhance the noise reduction intensity of this sub-band and improve the noise reduction effect; if the energy of vehicle road noise in the target sub-band is lower under a certain environmental state, a smaller pre-weighting weight can be adopted for the sensor corresponding to the target sub-band to avoid introducing noise or distortion due to misjudgment or overcompensation.

[0255] 3. According to the degree of abnormality of each sensor characterized by the sensor state information, the higher the degree of abnormality of the sensor, the lower the corresponding pre-weighting weight, and the lower the degree of abnormality of the sensor, the higher the corresponding pre-weighting weight.

[0256] According to the obtained sensor state information, the degree of abnormality of each sensor can be determined, and the degree of abnormality of each sensor may be different; for the sensor with a higher degree of abnormality, a smaller pre-weighting weight can be adopted to avoid introducing noise or distortion due to misjudgment or overcompensation, and for the sensor with a lower degree of abnormality, a larger pre-weighting weight can be adopted to ensure the noise reduction effect of the system.

[0257] In a specific implementation manner, in order to enable the control unit to conform to one or more of the above rules when determining the pre-weighting weights of each sensor, a mapping relationship that satisfies one or more of the above rules can be set. The control unit determines the pre-weighting weight corresponding to the environmental state information and / or sensor state information as the target pre-weighting weight according to this mapping relationship. The mapping relationship can be represented by means of a mapping table, a relational expression, etc. It should be noted that when setting to conform to multiple above rules, there can be a certain priority order among the multiple rules to avoid conflicts. The priority order can be specifically set according to needs and is not limited here. For example, the priority of determining the pre-weighting weight according to the sensor state information can be higher than the priority of determining the pre-weighting weight according to the environmental state information.

[0258] In a feasible implementation manner, the environmental state information includes weather condition information and road surface condition information. Among them, the weather condition information can specifically be information related to the weather condition that affects the road surface, such as the dryness of the road surface, rainfall, humidity, etc. The road surface condition information can specifically be information representing the type of the road surface, such as asphalt road surface, potholed road surface, brick road surface, etc.

[0259] In a feasible implementation manner, three frequency bands, namely low frequency, medium frequency, and high frequency, can be divided; when the environmental state information indicates that the low-frequency noise energy is high (such as in wet and waterlogged weather), a higher pre-weighting weight can be adopted for the signals collected by the low-frequency sensors (sensors whose collected signals are mainly distributed in the low-frequency range). For example, the pre-weighting of the signals collected by the low-frequency sensors is increased by 20%, and the remaining sensor channels maintain a reference of 1.0 times. When the environmental state information indicates that the medium-frequency noise energy is high (such as on a brick road surface, dry weather), a higher pre-weighting weight can be adopted for the signals collected by the medium-frequency sensors (sensors whose collected signals are mainly distributed in the medium-frequency range). For example, the pre-weighting of the signals collected by the medium-frequency sensors is increased by 20%, and the remaining sensor channels maintain a reference of 1.0 times.

[0260] The following Table 4 gives some examples of determining the pre-weighting strategy based on the weather condition information, road surface condition information, sensor state information, and signal characteristics (acceleration amplitude, energy, sound pressure signal amplitude) of the vibration sensor. Table 4 only gives some examples and does not limit the way of determining the pre-weighting weight in this embodiment. The content in parentheses in the table is the explanatory description of the weather condition information, road surface condition information, sensor state information, pre-weighting strategy, etc.

[0261] Table 4 Examples of Selecting Pre-weighting Strategy

[0262]

[0263] To help understand the implementation process of the above-mentioned vehicle road noise processing methods in each embodiment, an implementation example is given.Figure 3 This is the signal processing flowchart involved in the method for processing road noise of the example vehicle. Figure 4 Schematic flowchart of the method for processing road noise of the example vehicle.

[0264] Step 1:

[0265] After pre-filtering, amplitude adjustment and other preprocessing operations are performed on the J-channel time-domain vibration reference signal x(n), the time-domain adaptive filter coefficients are convolved. Here, l represents the frame index, indicating that it is updated in units of frames, rather than the sampling time n. The convolved signal is y(n) of the Q-channel, and after post-processing operations such as amplitude limiting, it is sent to Q speakers for output respectively. The amplitude adjustment operation can be performed on the signal x(n) according to the pre-weighting weights determined in Step 3. The amplitude limiting operation can be performed on the signal y(n) according to the target clipping parameter value T(l) determined in Step 3.

[0266] Step 2:

[0267] The J-channel time-domain vibration reference signal x(n) and the M-channel time-domain error microphone signal e(n) are respectively put into two buffers, with lengths of J*N and M*N. N is the frame length. In this example, J = 9, M = 2, Q = 5, and N = 64. When the buffers are full, a frame of J-channel time-domain vibration reference signal x(l) and a frame of M-channel time-domain error microphone signal e(l) are obtained. The above signals x(l) and e(l) are respectively sub-band decomposed using a polyphase analysis filter bank based on FFT, and a frame of J-channel sub-band vibration reference signal is obtained. :

[0268]

[0269] And a frame of M-channel sub-band error microphone signal :

[0270]

[0271] where k = 1, 2...K, and K is the total number of sub-bands. In this example, K = 65. In addition, a polyphase analysis filter bank trained by the training method in the above fourth embodiment can be used as the prototype low-pass filter in the sub-band decomposition:

[0272]

[0273] is the target polyphase analysis filter bank matched according to G(l), and G(l) is the identifier of the analysis filter bank and the synthesis filter bank determined according to the environmental state information. The formula for sub-band decomposing the time-domain vibration reference signal x(l) is as follows:

[0274] (1)

[0275] (2)

[0276] Among them, D is the downsampling factor, and P is the length of the FFT. In this example, D = 3 and P = 128. In formula (2), v = -2πikp / P, and i is the imaginary unit.

[0277] Formula (1) is the process of performing polyphase filtering on the signal x(l). Specifically, first perform the operation of adding a subband analysis window to the signal x(l), and then accumulate the results after windowing to obtain x'(l); formula (2) is to perform DFT (Discrete Fourier Transform) on the signal x'(l) to obtain the subband vibration reference signal , and the FFT can be used in actual calculations to improve the calculation speed. The formula for subband decomposition of the time-domain error microphone signal e(l) will not be elaborated here.

[0278] Step Three:

[0279] Refer to Figure 5 , input a frame of subband vibration reference signal of the J channel … , a frame of subband error microphone signal of the M channel … , a frame of time-domain vibration reference signal x(l) of the J channel and a frame of time-domain error microphone signal e(l) of the M channel into a prediction model (U-Net Encoder + FT-GRU + U-Net Decoder + multi-task output layer) based on a neural network structure, and predict the road surface condition information, weather condition information, and sensor status information. Corresponding to Figure 5 the function of the parameter matching control module in: dynamically determine the update step size of the subband adaptive filter coefficients based on the road surface condition information, weather condition information, and sensor status information … ; dynamically determine the target filter bank based on the road surface condition information and weather condition information, and use G(l) to represent the identification of the analysis filter bank and synthesis filter bank determined according to the road surface condition information and weather condition information; determine the pre-weighting weights of each sensor and the target clipping parameter value T(l) based on the road surface condition information, weather condition information, and sensor status information.

[0280] Step Four:

[0281] Since the subbands decomposed by the polyphase analysis filter bank based on the FFT are complex-domain subbands, the complex-domain FxLMS algorithm is used to update the subband adaptive filter coefficients , each is of size C*J*Q, where C is the length of the subband adaptive filter coefficients, and in this example C = 4. In the complex-domain FxLMS algorithm, first, for the input subband vibration reference signal convolve with the secondary path model S(k), and this convolution is a complex convolution:

[0282]

[0283] Then, the complex-domain LMS algorithm is used to update the subband adaptive filter coefficients as follows:

[0284]

[0285] where q = 1, 2…Q, Q is the number of speaker signal channels. m = 1, 2…M, M is the number of error microphone signal channels, j = 1, 2…J, J is the number of channels of the vibration reference signal. k = 1, 2…K, K is the number of subbands. is the update step size of the subband adaptive filter coefficients determined according to the influencing factor information, and is used to control the update speed of the filter.

[0286] Step Five: Synthesize the time-domain adaptive filter coefficients from the subband adaptive filter coefficients.

[0287] Use the polyphase synthesis filter bank based on IFFT for subband synthesis. Compared with the weight stacking method with relatively large computational complexity and multiple groups of FFT and IFFT operations, this method only has one group of IFFT and adding synthesis window operations, which is simple and efficient. Among them, the polyphase synthesis filter bank can adopt the polyphase analysis filter bank trained by the training method in the above-mentioned fourth embodiment, and can be dynamically determined according to the environmental state information. As Figure 6 shown, in this example, the full-band (time-domain) adaptive filter coefficients synthesized based on the subband adaptive filter coefficients can achieve almost the same effect as the target full-band (time-domain) adaptive filter coefficients. Figure 6 In it, the abscissa is the index of the filter length, and the ordinate is the value of the coefficients of each tap.

[0288] Step Six: Load balancing strategy. Split the tasks of subband decomposition, subband synthesis, and updating of subband adaptive filter coefficients among each interrupt, and split the neural network model prediction task among each interrupt.

[0289] It should be noted that the sequence numbers of the above steps do not constitute a limitation on the sequence order between the steps. In addition, the above example is only for understanding this application and does not constitute a limitation on the automotive road noise processing method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0290] The embodiments of the present application further provide an automotive road noise processing device, which includes:

[0291] A sub-band decomposition module, configured to perform sub-band decomposition on the input time-domain reference signal and the input time-domain error signal respectively, to obtain a sub-band reference signal and a sub-band error signal;

[0292] A step-size update module, configured to dynamically determine a target update step size corresponding to the sub-band adaptive filter coefficients according to the obtained influence factor information, where the influence factor information includes environmental state information and / or sensor state information, the environmental state information is information characterizing the environmental state inside and / or outside the vehicle, and the sensor state information is information characterizing the working state of the sensor, and the sensor includes a sensor for collecting the time-domain reference signal and / or a sensor for collecting the time-domain error signal;

[0293] A filter update module, configured to update the sub-band adaptive filter coefficients based on the sub-band reference signal and the sub-band error signal according to the target update step size;

[0294] A sub-band synthesis module, configured to perform sub-band synthesis on the updated sub-band adaptive filter coefficients to obtain updated time-domain adaptive filter coefficients;

[0295] A noise reduction module, configured to process a newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain a noise reduction signal, so as to control road noise based on the noise reduction signal.

[0296] Compared with the prior art, the beneficial effects of the automotive road noise processing device provided by the embodiments of the present application are the same as those of the automotive road noise processing method provided by the above embodiments, and other technical features in the automotive road noise processing device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0297] The embodiments of the present application provide an automotive road noise processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the automotive road noise processing method in the first embodiment above.

[0298] Next, refer to Figure 7 , which shows a schematic structural diagram of an automotive road noise processing device suitable for implementing the embodiments of the present application. The automotive road noise processing device in the embodiments of the present application may be a control unit in an RNC system. Figure 7The shown vehicle road noise processing device is merely an example and shall not impose any limitation on the functions and scope of use of the embodiments of the present application.

[0299] As Figure 7 shown, the vehicle road noise processing device may include a processing device 1001 (such as a DSP processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1002 or the program loaded from the storage device 1003 into the random access memory 1004. In the random access memory 1004, various programs and data required for the operation of the vehicle road noise processing device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. The input / output interface 1006 is also connected to the bus. Generally, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a microphone, an accelerometer, etc.; an output device 1008 including, for example, a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the vehicle road noise processing device to communicate with other devices wirelessly or wireline to exchange data. Although the figure shows a vehicle road noise processing device with various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be alternatively implemented or had.

[0300] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0301] Compared with the prior art, the beneficial effects of the vehicle road noise processing device provided by the embodiments of the present application are the same as those of the vehicle road noise processing method provided by the above embodiments, and other technical features in the vehicle road noise processing device are the same as the features disclosed in the method of the previous embodiment, which will not be elaborated herein.

[0302] It should be understood that each part disclosed in the embodiments of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0303] As described above, it is only the specific implementation manner of the present application. However, the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the said claims.

[0304] The embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the vehicle road noise processing method in the above embodiment.

[0305] The computer-readable storage medium provided by the embodiment of the present application can be, for example, a USB flash drive, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0306] The above computer-readable storage medium can be included in the vehicle road noise processing device; or it can exist separately and not be assembled into the vehicle road noise processing device.

[0307] The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the vehicle road noise processing device, the vehicle road noise processing device is caused to execute the above functions defined in the method of the disclosed embodiment of the present application.

[0308] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0309] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and this module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutively represented blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0310] The modules involved in the embodiments described in this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0311] The readable storage medium provided in the embodiments of this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned automotive road noise processing method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiments of this application are the same as those of the automotive road noise processing method provided in the above embodiments, and will not be elaborated here.

[0312] An embodiment of the present application also provides a computer program product, including a computer program, which when executed by a processor implements the steps of the above-mentioned method for processing automotive road noise.

[0313] Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as those of the method for processing automotive road noise provided by the above embodiment, and will not be elaborated here.

[0314] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for processing automotive road noise, characterized in that, The described method for processing vehicle road noise includes: Performing sub-band decomposition on the input time-domain reference signal and the input time-domain error signal respectively to obtain a sub-band reference signal and a sub-band error signal; Dynamically determining a target update step size corresponding to the sub-band adaptive filter coefficients according to the obtained influence factor information, where the influence factor information includes environmental state information and sensor state information. The environmental state information is information characterizing the environmental state inside and / or outside the vehicle, and the sensor state information is information characterizing the working state of the sensor. The sensor includes a sensor for collecting the time-domain reference signal and / or a sensor for collecting the time-domain error signal. Different adjustment strategies are adopted for the target update step sizes corresponding to different sub-bands according to the environmental state information and the sensor state information; Updating the sub-band adaptive filter coefficients based on the sub-band reference signal and the sub-band error signal according to the target update step size; Performing sub-band synthesis on the updated sub-band adaptive filter coefficients to obtain updated time-domain adaptive filter coefficients; Processing a newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, so as to control road noise based on the anti-noise signal.

2. The method for processing automotive road noise according to claim 1, characterized in that When dynamically determining the target update step size according to the influence factor information, it conforms to one or more of the following rules: Under the environmental state characterized by the environmental state information, the target update step size corresponding to the sub-band with higher vehicle road noise energy or higher stability degree is larger than the target update step size corresponding to the sub-band with lower vehicle road noise energy or lower stability degree; For any target sub-band, under the environmental state characterized by the environmental state information, the higher the energy or the higher the stability degree of the vehicle road noise in the target sub-band, the larger the target update step size corresponding to the target sub-band, and the lower the energy or the lower the stability degree of the vehicle road noise in the target sub-band under the environmental state characterized by the environmental state information, the smaller the target update step size corresponding to the target sub-band; According to the abnormal degree of each sensor characterized by the sensor state information, the target update step size corresponding to the sub-band where the sensor with a higher abnormal degree is located is smaller than the target update step size corresponding to the sub-band where the sensor with a lower abnormal degree is located; For any target sub-band, the higher the abnormal degree of the sensor belonging to the target sub-band, the smaller the target update step size corresponding to the target sub-band, and the lower the abnormal degree of the sensor belonging to the target sub-band, the larger the target update step size corresponding to the target sub-band.

3. The method for processing automotive road noise according to claim 2, wherein, The step of dynamically determining a target update step size corresponding to the sub-band adaptive filter coefficients according to the obtained influence factor information includes: Determining the update step size corresponding to the influence factor information as the target update step size according to a pre-set mapping relationship, where the mapping relationship satisfies one or more of the above rules.

4. The method for processing automotive road noise according to claim 1, characterized in that, The influence factor information further includes vehicle speed. When dynamically determining the target update step size according to the influence factor information, it conforms to one or more of the following rules: The greater the vehicle speed stability, the larger the target update step size corresponding to each sub-band; the lower the vehicle speed stability, the smaller the target update step size corresponding to each sub-band. When the vehicle speed is outside the preset range, the target update step size corresponding to each sub-band is smaller than that when the vehicle speed is within the preset range.

5. The method for processing automotive road noise according to claim 1, characterized in that The method for processing vehicle road noise further includes: Using a preset prediction model to perform prediction based on the input time-domain reference signal, the input time-domain error signal, the sub-band reference signal, and the sub-band error signal to obtain the environmental state information and / or the sensor state information, where the prediction model is pre-trained.

6. The method for processing automotive road noise according to claim 5, characterized in that, The prediction model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module, and a prediction module; The step of using a preset prediction model to perform prediction based on the input time-domain reference signal, the input time-domain error signal, the sub-band reference signal, and the sub-band error signal to obtain the environmental state information and / or the sensor state information includes: Inputting the sub-band reference signal and the sub-band error signal into the first feature extraction module for feature extraction to obtain a first feature representation; Inputting the input time-domain reference signal into the second feature extraction module for feature extraction to obtain a second feature representation; Inputting the input time-domain error signal into the third feature extraction module for feature extraction to obtain a third feature representation; Concatenating the first feature representation, the second feature representation, and the third feature representation and inputting them into the feature fusion module for feature fusion to obtain a fused feature representation; Inputting the fused feature representation into the prediction module for prediction to obtain the environmental state information and / or the sensor state information.

7. The method for processing automotive road noise according to claim 6, wherein, The first feature extraction module, the second feature extraction module, and the third feature extraction module each include at least one convolutional layer; the feature fusion module includes a frequency-axis gated recurrent unit, a time-axis gated recurrent unit, and at least one transposed convolutional layer connected in sequence; the prediction module includes a plurality of task output layers, and each task output layer is respectively used to output one piece of the environmental state information or the sensor state information.

8. The method for processing automotive road noise according to claim 7, wherein, The prediction module further includes a shared feature extraction layer arranged before the plurality of task output layers, and the shared feature extraction layer includes one or more convolutional layers.

9. The method for processing automotive road noise according to claim 7, characterized in that The environmental state information includes weather condition information and road surface condition information.

10. The method for processing automotive road noise according to any one of claims 5 to 9, characterized in that, The frame index of the input time-domain reference signal and the input time-domain error signal is l, and the method for processing vehicle road noise further includes: Executing the tasks allocated to the N interruptions during the N interruptions in the (l + 1)-th frame, where N represents the frame length, and an interruption refers to the period from generating an anti-noise signal based on the time-domain reference signal of each sampling point to receiving the time-domain reference signal of the next sampling point. The tasks allocated to the N interruptions include: the task of calculating the updated time-domain adaptive filter coefficients, and / or the task of predicting the environmental state information and / or the sensor state information using the prediction model.

11. The method for processing automotive road noise according to claim 10, wherein The task of calculating the updated time-domain adaptive filter coefficients is divided into a sub-band decomposition task, a sub-band synthesis task, and multiple sub-band update tasks. Each of the sub-band update tasks includes an update task for at least one of the sub-band adaptive filter coefficients. The sub-band decomposition task is assigned to be executed in the first preset number of interrupts, the sub-band synthesis task is assigned to be executed in the subsequent second preset number of interrupts, and the multiple sub-band update tasks are assigned to be executed in the intermediate third preset number of interrupts. One sub-band update task is assigned to each interrupt. The sum of the preset first number, the preset second number, and the preset third number is N.

12. The method for processing automotive road noise according to claim 1, wherein The step of updating the sub-band adaptive filter coefficients based on the sub-band reference signal and the sub-band error signal according to the target update step size includes: Convolving the sub-band reference signal with a pre-established secondary path model; Using the complex-domain LMS algorithm to calculate the updated sub-band adaptive filter coefficients based on the sub-band error signal, the sub-band reference signal after convolving the secondary path model, and the current sub-band adaptive filter coefficients.

13. The method for processing automotive road noise according to claim 1, characterized in that, The sensor for collecting the time-domain reference signal includes at least one vibration sensor, and the sensor for collecting the time-domain error signal includes at least one microphone.

14. The method for processing automotive road noise according to claim 13, wherein The influence factor information further includes the signal characteristics of the vibration sensor, and the signal characteristics include acceleration amplitude, energy, and sound pressure signal amplitude.

15. The method for processing automotive road noise according to claim 1, characterized in that, The step of respectively performing sub-band decomposition on the input time-domain reference signal and the input time-domain error signal to obtain a sub-band reference signal and a sub-band error signal includes: Preprocessing the input time-domain reference signal and the input time-domain error signal, and respectively performing sub-band decomposition on the preprocessed time-domain reference signal and time-domain error signal to obtain a sub-band reference signal and a sub-band error signal, where the preprocessing includes pre-filtering processing and / or amplitude adjustment processing.

16. The method for processing automotive road noise according to claim 1, characterized in that, The step of processing a newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal, and controlling road noise based on the anti-noise signal includes: Processing a newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain an anti-noise signal; Playing the anti-noise signal through a speaker provided inside the vehicle, or playing the anti-noise signal through a speaker provided inside the vehicle after performing a post-processing operation on the anti-noise signal.

17. An automobile road noise processing device, characterized in that, The vehicle road noise processing device includes: A sub-band decomposition module, configured to respectively perform sub-band decomposition on the input time-domain reference signal and the input time-domain error signal to obtain a sub-band reference signal and a sub-band error signal; A step-size update module, configured to dynamically determine a target update step-size corresponding to sub-band adaptive filter coefficients according to the acquired influencing factor information, where the influencing factor information includes environmental state information and sensor state information, the environmental state information is information characterizing the environmental state inside and / or outside the vehicle, the sensor state information is information characterizing the working state of the sensors, the sensors include sensors for collecting the time-domain reference signal and / or sensors for collecting the time-domain error signal, and different adjustment strategies are adopted for the target update step-sizes corresponding to different sub-bands according to the environmental state information and the sensor state information; A filter update module, configured to update the sub-band adaptive filter coefficients based on the sub-band reference signal and the sub-band error signal according to the target update step-size; A sub-band synthesis module, configured to perform sub-band synthesis on the updated sub-band adaptive filter coefficients to obtain updated time-domain adaptive filter coefficients; A noise reduction module, configured to process a newly input time-domain reference signal based on the updated time-domain adaptive filter coefficients to obtain a noise reduction signal, so as to control road noise based on the noise reduction signal.

18. An automobile road noise treatment device, characterized in that The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the vehicle road noise processing method according to any one of claims 1 to 16.

19. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the steps of the vehicle road noise processing method according to any one of claims 1 to 16.

20. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the vehicle road noise processing method according to any one of claims 1 to 16.

Images