Method and apparatus for predicting vehicle engine noise

Abstract

The invention provides a method and a device for predicting engine noise of a vehicle. The engine noise separated from the vehicle noise is predicted. The method may include: acquiring engine noise data not including vehicle noise; acquiring vehicle noise data not including engine noise; generating training data by mixing the engine noise data and the vehicle noise data; providing a deep learning model differently constructed for each vehicle through transfer learning based on a pre-training model pre-trained by the training data; and predicting engine noise of each vehicle by using the deep learning model.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to Korean Patent Application No. 10-2024-0062273, filed with the Korean Intellectual Property Office on May 13, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to a method and apparatus for predicting vehicle engine (motor) noise. Background Technology

[0004] In the automotive industry, vehicle components can be diagnosed by capturing and analyzing noise data specific to each part. For example, in the case of a vehicle engine, analyzing the generated noise allows for assessment of the engine's condition and prediction of potential problems. Abnormal engine noise may indicate various defects such as bearing damage, rotor imbalance, stator-rotor friction, or coil short circuits, which can be identified by examining specific noise patterns and frequencies. Regular noise monitoring also allows for tracking engine wear and estimating remaining engine life. Furthermore, noise data can aid in performance optimization by identifying factors affecting engine efficiency and adjusting operating conditions accordingly. To improve the accuracy and effectiveness of diagnostics, noise data from individual components must be captured, excluding conventional vehicle noise. Summary of the Invention

[0005] This disclosure relates to a method and apparatus for predicting vehicle engine noise by isolating and capturing engine-specific noise, which is separated from conventional vehicle noise.

[0006] According to one aspect of this disclosure, a method for predicting vehicle engine noise may include: acquiring engine noise data excluding vehicle noise; acquiring vehicle noise data excluding engine noise; generating training data by mixing engine noise data and vehicle noise data; providing a deep learning model constructed differently for each vehicle through transfer learning based on a pre-trained model, the pre-trained model being pre-trained using the training data; and predicting engine noise for each vehicle by using the deep learning model.

[0007] In some implementations, generating training data may include mixing engine noise data and vehicle noise data by using signal-to-noise ratio (SNR) values ​​calculated for engine noise and vehicle noise.

[0008] In some implementations, mixing may include: obtaining amplified engine noise data by multiplying engine noise data by an SNR value; and mixing amplified engine noise data with vehicle noise data.

[0009] In some implementations, the pre-trained model may include a time-domain-based audio source separation model.

[0010] In some implementations, the pre-trained model may include: an encoder including one-dimensional (1-D) convolutional layers; a separation network including deep convolutional layers; and a decoder including 1-D transposed convolutional layers.

[0011] In some implementations, the loss function of a deep learning model may include scale-invariant signal distortion ratio (SI-SDR).

[0012] In some implementations, the loss function can be defined by the following Equation 1:

[0013] (Equation 1)

[0014]

[0015] Among them, S target For the target value, and This represents the output value of the deep learning model.

[0016] In some implementations, the performance metrics of a deep learning model may include scale-invariant signal-distortion ratio improvement (SI-SDRi).

[0017] In some implementations, providing a deep learning model may include providing a first deep learning model via transfer learning based on a pre-trained model, the first deep learning model having a first new layer specifically for the first vehicle type added; and predicting engine noise may include using the first deep learning model to predict the engine noise of a vehicle corresponding to the first vehicle type.

[0018] In some implementations, providing a deep learning model may further include providing a second deep learning model via transfer learning based on a pre-trained model, the second deep learning model having a second new layer specifically for a second vehicle type different from the first vehicle type, and predicting engine noise may further include using the second deep learning model to predict the engine noise of a vehicle corresponding to the second vehicle type.

[0019] According to another aspect of this disclosure, an apparatus for predicting vehicle engine noise is provided. This apparatus executes program code loaded on one or more memory devices via one or more processors and predicts engine noise separate from vehicle noise. The program code can be executed to: acquire engine noise data excluding vehicle noise; acquire vehicle noise data excluding engine noise; generate training data by mixing engine noise data and vehicle noise data; provide a deep learning model, differently constructed for each vehicle, through transfer learning based on a pre-trained model, which is pre-trained using training data; and predict engine noise for each vehicle by utilizing the deep learning model.

[0020] In some implementations, generating training data may include mixing engine noise data and vehicle noise data by using signal-to-noise ratio (SNR) values ​​calculated for engine noise and vehicle noise.

[0021] In some implementations, mixing may include: obtaining amplified engine noise data by multiplying engine noise data by an SNR value; and mixing amplified engine noise data with vehicle noise data.

[0022] In some implementations, the pre-trained model may include a time-domain-based audio source separation model.

[0023] In some implementations, the pre-trained model may include: an encoder including one-dimensional (1-D) convolutional layers; a separation network including deep convolutional layers; and a decoder including 1-D transposed convolutional layers.

[0024] In some implementations, the loss function of a deep learning model may include scale-invariant signal distortion ratio (SI-SDR).

[0025] In some implementations, the loss function can be defined by the following Equation 1:

[0026] (Equation 1)

[0027]

[0028] Among them, S target For the target value, and This represents the output value of the deep learning model.

[0029] In some implementations, the performance metrics of a deep learning model may include scale-invariant signal-distortion ratio improvement (SI-SDRi).

[0030] In some implementations, providing a deep learning model may include providing a first deep learning model via transfer learning based on a pre-trained model, the first deep learning model having a first new layer specifically for the first vehicle type added; and predicting engine noise may include using the first deep learning model to predict the engine noise of a vehicle corresponding to the first vehicle type.

[0031] In some implementations, providing a deep learning model may further include providing a second deep learning model via transfer learning based on a pre-trained model, the second deep learning model having a second new layer specifically for a second vehicle type different from the first vehicle type; and predicting engine noise may further include using the second deep learning model to predict the engine noise of a vehicle corresponding to the second vehicle type. Attached Figure Description

[0032] Figure 1 This is a block diagram illustrating an example of a device for predicting vehicle engine noise.

[0033] Figure 2 This is a flowchart illustrating an example of a method for predicting vehicle engine noise.

[0034] Figure 3 This is a diagram illustrating an example of a device and method for predicting vehicle engine noise.

[0035] Figures 4 to 6 This is a diagram illustrating an example of a device and method for predicting vehicle engine noise.

[0036] Figure 7 This is a diagram illustrating an example of a device and method for predicting vehicle engine noise.

[0037] Figure 8 This is a diagram illustrating an example of a computing device. Detailed Implementation

[0038] Figure 1 This is a block diagram illustrating an example of a device for predicting vehicle engine noise.

[0039] Reference Figure 1 The device 10 for predicting vehicle engine noise can run program code loaded on one or more memory devices via one or more processors. For example, the device 10 for predicting vehicle engine noise can be implemented as described below. Figure 8The computing device 50. In some embodiments, one or more processors may correspond to processor 510 of the computing device 50, and one or more memory devices may correspond to memory 530 of the computing device 50. The program code may be executed by one or more processors to acquire only engine-specific noise that is separated from conventional vehicle noise.

[0040] The device 10 for predicting vehicle engine noise may include a data acquisition module 110, a training data generation module 120, a deep learning model providing module 130, and an engine noise prediction module 140, thereby predicting engine-specific noise that is separate from conventional vehicle noise.

[0041] The data acquisition module 110 can acquire engine-specific noise data that excludes conventional vehicle noise. For example, the data acquisition module 110 can acquire engine-specific noise data separately to assign labels corresponding to precise correct answers when learning an artificial intelligence model. In some embodiments, the engine-specific noise data can be acquired from an engine operating alone in an anechoic chamber or by providing separate power to the engine without starting the vehicle.

[0042] In some implementations, the data acquisition module 110 can acquire conventional vehicle noise data that excludes engine-specific noise. For example, the data acquisition module 110 can acquire conventional vehicle noise data separately to obtain accurate values ​​beyond the labels used when learning an artificial intelligence model. In some implementations, conventional vehicle noise data can be acquired from a vehicle operating with its engine shielded in an anechoic chamber.

[0043] The training data generation module 120 can generate training data by combining engine-specific noise data acquired by the data acquisition module 110 with conventional vehicle noise data. In some embodiments, the training data generation module 120 can calculate the signal-to-noise ratio (SNR) value for the engine-specific noise data and conventional vehicle noise data acquired by the data acquisition module 110, and combine the engine-specific noise data with the conventional vehicle noise data using the calculated SNR value. The SNR can be calculated as follows.

[0044]

[0045] Here, P signal It can refer to the power of the engine noise signal, and P noise The SNR (Signal Noise Ratio) can refer to the power of a vehicle noise signal. Engine noise signal power can refer to the average power of engine noise signal transmission over a specific time period. Vehicle noise signal power can refer to the average power of unwanted or irrelevant signals generated during engine noise signal transmission. A higher SNR value indicates better quality of the engine noise signal.

[0046] The training data generation module 120 can acquire amplified engine noise data, which corresponds to a signal amplified by multiplying the engine noise data acquired by the data acquisition module 110 by the SNR value. Subsequently, the training data generation module 120 can combine the amplified engine noise data and the vehicle noise data. As described above, SNR-based combination can be performed to prevent the overfitting problem—where high accuracy is achieved with training data but performance degrades with new data—and improve the accuracy of engine noise prediction.

[0047] The deep learning model providing module 130 can provide a different deep learning model for each vehicle through transfer learning based on a pre-trained model, which is pre-trained using training data generated by the training data generation module 120.

[0048] Transfer learning is a methodology that utilizes a model trained on one domain or task and applies that model to another related or similar domain or task, thereby using a previously pre-trained model. Here, a pre-trained model can refer to a pre-trained model that has been pre-trained using training data generated by the training data generation module 120.

[0049] In some implementations, the pre-trained model may include a time-domain-based audio source separation model. The pre-trained model can process audio signals directly in the time domain rather than the frequency domain, learn patterns from complex audio signals, and separate sources using a deep convolutional neural network. For example, the structure of the pre-trained model may include an encoder, a separation network, and a decoder. The encoder may include one-dimensional (1-D) convolutional layers, and the separation network may include deep convolutional layers. In some implementations, the decoder may include a 1-D transposed convolutional layer. A transposed convolutional layer, also known as deconvolution, can be used to extend the spatial dimension in a convolutional neural network. For this purpose, for example, a method such as inserting values ​​like 0 between elements of the input data in space can be employed.

[0050] In some implementations, the loss function of the deep learning model may include Scale-Invariant Signal Distortion Ratio (SI-SDR). SI-SDR can be a loss function used in the audio and audio processing domain to evaluate signal quality, and it can be used to evaluate the performance of engine noise separation tasks by measuring the ratio between the original signal and the estimated signal. SI-SDR is scale-invariant, meaning that it can accurately measure the degree of signal distortion even when the scale of the estimated signal differs from that of the original signal, thus handling signals of various volumes. The loss function can be defined by Equation 1 below.

[0051] (Equation 1)

[0052]

[0053] Here, S target It can refer to the target value, and It can also refer to the output value of a deep learning model. Therefore, This can correspond to errors. Errors can correspond to interference signals e that may occur during the processing. interf Background noise e noise And human distortion e artif The sum of these can be minimized, thus improving quality.

[0054] The engine noise prediction module 140 can predict the engine noise of each vehicle by using the deep learning model provided by the deep learning model provided by the deep learning model provision module 130.

[0055] In some implementations, the performance metrics of a deep learning model may include Scale Invariant Signal Distortion Ratio Improvement (SI-SDRi). SI-SDRi can indicate the degree of quality improvement of the output signal compared to the original signal after signal processing related to the separation of engine-specific noise. For example, SI-SDRi can be calculated by subtracting the SDR value of the input signal before processing from the SDR value of the processed output signal, and it is understood that the larger the value, the better the quality improvement compared to the original signal.

[0056] In some implementations, only engine-specific noise that is completely separated from vehicle noise can be acquired, unlike traditional systems where reliability regarding the accuracy of Fast Fourier Transform (FFT) peaks is lacking due to the overlap between vehicle and engine noise, and where it is difficult to apply other analytical techniques besides checking the FFT peaks. Therefore, unlike traditional systems that analyze various types of noise overlap in the frequency domain, this approach not only analyzes representative noise values ​​in the time domain (e.g., root mean square (RMS))—which is difficult to achieve in traditional systems—but also analyzes clear peaks separated without being masked by other noise in the frequency domain.

[0057] In some implementations, the deep learning model providing module 130 can provide deep learning models specifically for different vehicle types through transfer learning based on pre-trained models, and the engine noise prediction module 140 can predict engine-specific noise for each vehicle type by using each specialized deep learning model.

[0058] For example, the deep learning model providing module 130 can provide a first deep learning model through transfer learning based on a pre-trained model, which adds a first new layer specifically for a first vehicle type. Here, the first new layer may refer to a layer used to extract features unique to the first vehicle type. Additionally, the deep learning model providing module 130 can provide a second deep learning model through transfer learning based on a pre-trained model, which adds a second new layer specifically for a second vehicle type different from the first vehicle type. Here, the second new layer may refer to a layer used to extract features unique to the second vehicle type. The engine noise prediction module 140 can predict engine-specific noise of a vehicle corresponding to the first vehicle type using the first deep learning model, and predict engine-specific noise of a vehicle corresponding to the second vehicle type using the second deep learning model.

[0059] In some implementations, the manpower, time and cost required to perform engine noise analysis for each vehicle type can be reduced by implementing an objective and reliable platform that obtains only engine-specific noise from a reference vehicle model, distributing the model to and applying it to various derivative vehicles.

[0060] Figure 2 This is a flowchart illustrating an example of a method for predicting vehicle engine noise.

[0061] Methods for predicting vehicle engine noise may include: acquiring engine noise data excluding vehicle noise (S201); acquiring vehicle noise data excluding engine noise (S202); generating training data by combining engine noise data and vehicle noise data (S203); providing a deep learning model with different constructions for each vehicle through transfer learning based on a pre-trained model, the pre-trained model being pre-trained with training data (S204); and predicting engine-specific noise for each vehicle by using a deep learning model (S205).

[0062] Figure 3 This is a diagram illustrating an example of a device and method for predicting vehicle engine noise.

[0063] Reference Figure 3 In an example embodiment of the apparatus and method for predicting vehicle engine noise, training data 26 can be generated by combining engine noise data 20 and vehicle noise data 22. To do this, an SNR value can be calculated for the engine noise data 20 and vehicle noise data 22, and engine noise amplification data 24 is obtained by multiplying the calculated SNR value by the engine noise data 20. Subsequently, training data 26 can be generated by combining the engine noise amplification data 24 and vehicle noise data 22. For example, engine noise data 20 can be obtained from an engine operating alone in an anechoic chamber or by providing power to the engine alone without starting the vehicle. In some embodiments, vehicle noise data 22 can be obtained from a vehicle operating with the engine shielded in an anechoic chamber.

[0064] Figures 4 to 6 This is a diagram illustrating an example of a device and method for predicting vehicle engine noise.

[0065] Reference Figures 4 to 6 As a non-limiting example that can be used as a pre-trained model for a temporal-domain audio source separation model, the architecture of a convolutional temporal audio separation network (Conv-TasNet) is shown. In some implementations, refer to... Figure 1 The deep learning model providing module 130 can provide a different deep learning model for each vehicle through transfer learning based on Conv-TasNet, which is pre-trained using training data generated by the training data generation module 120.

[0066] The pre-trained model 30 may include an encoder 32, a separation network 34, and a decoder 36. A waveform combining vehicle noise and engine noise signals can be input to the encoder 32, and the decoder 36 can output predicted data that separates the vehicle noise and engine noise signals. The encoder 32 and decoder 36 may each include 1-D convolutional layers, and the separation network 34 may stack 1-D convolutional layers with different dilation rates, using PReLU as the activation function.

[0067] Figure 7 This is a diagram illustrating an example of a device and method for predicting vehicle engine noise.

[0068] Reference Figure 7 In an example of an apparatus and method for predicting vehicle engine noise, transfer learning of a pre-trained model can be used to predict engine-specific noise for each vehicle type by employing a deep learning model specifically designed for different vehicle types. For instance, a deep learning model implemented in a reference vehicle (parent vehicle) A can be provided to a first derived vehicle B and a second derived vehicle C1. At this time, a deep learning model implemented in the reference vehicle A with added layers specifically for the first derived vehicle B can be provided to the first derived vehicle B, and a deep learning model implemented in the reference vehicle A with added layers specifically for the second derived vehicle C1 can be provided to the second derived vehicle C1. Furthermore, a deep learning model implemented in the second derived vehicle C1 can be provided to a third derived vehicle C2. At this time, a deep learning model implemented in the second derived vehicle C1 with added layers specifically for the third derived vehicle C2 can be provided to the third derived vehicle C2. Therefore, by implementing an objective and reliable platform that obtains only engine-specific noise from reference vehicle A, the model can be distributed to and applied to the first through third derivative vehicles B, C1, and C2, thereby reducing the labor, time, and cost required for engine noise analysis for each vehicle type.

[0069] Figure 8 This is a diagram illustrating an example of a computing device.

[0070] Reference Figure 8 The method and apparatus for predicting vehicle engine noise can be implemented using the computing device 50.

[0071] The computing device 50 may include at least one of the following: a processor 510, a memory 530, a user interface input device 540, a user interface output device 550, and a storage device 560, which communicate with each other via a bus 520. The computing device 50 may also include a network interface 570 electrically connected to the network 40. The network interface 570 can send signals to or receive signals from other entities through the network 40.

[0072] Processor 510 can be implemented in various types, such as a microcontroller unit (MCU), application processor (AP), central processing unit (CPU), graphics processing unit (GPU), neural processing unit (NPU), and quantum processing unit (QPU), and can be any semiconductor device that executes commands stored in memory 530 or storage device 560. Processor 510 can be configured to implement the above-mentioned... Figures 1 to 7 The functions and methods described.

[0073] The memory 530 and storage device 560 may include various types of volatile or non-volatile storage media. For example, the memory 530 may include read-only memory (ROM) 531 and random access memory (RAM) 532. In some embodiments, the memory 530 may be located inside or outside the processor 510, and the memory 530 may be connected to the processor 510 in various known ways.

[0074] In some embodiments, at least some components or functions of the method and apparatus for predicting vehicle engine noise may be implemented in a program or software run by a computing device 50, and the program or software may be stored in a computer-readable medium. Specifically, the computer-readable medium may contain a program recorded on a computer for running the steps included in the method and apparatus for predicting vehicle engine noise, the computer including a processor 510 that runs programs or commands stored in a memory 530 or a storage device 560.

[0075] In some implementations, at least some components or functions of the method and apparatus for predicting vehicle engine noise can be implemented using the hardware or circuitry of the computing device 50, or can be implemented as separate hardware or circuitry that can be electrically connected to the computing device 50.

[0076] As mentioned above, it is possible to acquire only engine noise that is completely separate from vehicle noise, unlike traditional systems where the overlap between vehicle and engine noise leads to a lack of reliability regarding the accuracy of Fast Fourier Transform (FFT) peaks, and makes it difficult to apply other analytical techniques beyond checking FFT peaks. Therefore, unlike traditional systems that analyze various types of noise overlap in the frequency domain, this method allows analysis not only of representative noise values ​​(e.g., RMS) in the time domain—which is difficult to achieve in traditional systems—but also of clear peaks separated without being masked by other noise in the frequency domain. Furthermore, by implementing an objective and reliable platform that acquires only engine-specific noise from a reference vehicle, the model can be distributed and applied to various derived vehicles, thereby reducing the labor, time, and cost required for engine noise analysis for each vehicle type.

Claims

1. A method for predicting vehicle engine noise, comprising: Acquire engine noise data, which includes only engine-specific noise and excludes noise from other vehicle components; Acquire vehicle noise data, which includes noise from the other vehicle components but excludes engine-specific noise; Training data is generated by combining the engine noise data with the vehicle noise data; By using transfer learning based on a pre-trained model, a deep learning model with different constructions for each vehicle is provided, wherein the pre-trained model is pre-trained using the training data. as well as The deep learning model is used to predict engine-specific noise for each vehicle.

2. The method according to claim 1, wherein, Generating the training data includes: The engine noise data is combined with the vehicle noise data using a signal-to-noise ratio (SNR) value, which is calculated for the engine-specific noise and noise from other vehicle components.

3. The method according to claim 2, wherein, Combining the engine noise data with the vehicle noise data includes: Engine noise amplification data is obtained by multiplying the engine noise data by the SNR value, and The engine noise amplification data is combined with the vehicle noise data.

4. The method according to claim 1, wherein, The pre-trained model includes a time-domain-based audio source separation model.

5. The method according to claim 1, wherein, The pre-trained model includes: The encoder includes one-dimensional convolutional layers, i.e., 1-D convolutional layers; Separate networks, including deep convolutional layers; and The decoder includes 1-D transposed convolutional layers.

6. The method according to claim 5, wherein, The loss function of the deep learning model includes the scale-invariant signal-to-distortion ratio, i.e., SI-SDR.

7. The method according to claim 6, wherein, The loss function is defined by the following equation: Among them, S target For the target value, and This is the output value of the deep learning model.

8. The method according to claim 5, wherein, The performance metrics of the deep learning model include scale-invariant signal-to-distortion ratio improvement, i.e., SI-SDRi.

9. The method according to claim 1, wherein, Providing the deep learning model includes providing a first deep learning model through transfer learning based on the pre-trained model, the first deep learning model having a first new layer specifically for the first vehicle type added, and Predicting the engine-specific noise includes predicting the engine-specific noise of a vehicle corresponding to the first vehicle type based on the first deep learning model.

10. The method according to claim 9, wherein, Providing the deep learning model further includes providing a second deep learning model through transfer learning based on the pre-trained model, the second deep learning model having a second new layer specifically for a second vehicle type, which is different from the first vehicle type, and Predicting the engine-specific noise further includes predicting the engine-specific noise of a vehicle corresponding to the second vehicle type based on the second deep learning model.

11. A device for predicting vehicle engine noise, comprising: One or more memory devices that store instructions; as well as One or more processors execute the instructions to perform operations including the following: Acquire engine noise data, which includes only engine-specific noise and excludes noise from other vehicle components; Acquire vehicle noise data, which includes noise from the other vehicle components but excludes engine-specific noise; Training data is generated by combining the engine noise data with the vehicle noise data; By using transfer learning based on a pre-trained model, a deep learning model with different constructions for each vehicle is provided, wherein the pre-trained model is pre-trained using the training data; as well as The deep learning model is used to predict engine-specific noise for each vehicle.

12. The apparatus according to claim 11, wherein, Generating the training data includes: The engine noise data and the vehicle noise data are combined using a signal-to-noise ratio (SNR) value, which is calculated for the engine-specific noise and noise from other vehicle components.

13. The apparatus according to claim 12, wherein, Combining the engine noise data with the vehicle noise data includes: Engine noise amplification data is obtained by multiplying the engine noise data by the SNR value, and The engine noise amplification data is combined with the vehicle noise data.

14. The apparatus according to claim 11, wherein, The pre-trained model includes a time-domain-based audio source separation model.

15. The apparatus according to claim 11, wherein, The pre-trained model includes: The encoder includes one-dimensional convolutional layers, i.e., 1-D convolutional layers; Separate networks, including deep convolutional layers; and The decoder includes 1-D transposed convolutional layers.

16. The apparatus according to claim 15, wherein, The loss function of the deep learning model includes the scale-invariant signal-to-distortion ratio, i.e., SI-SDR.

17. The apparatus of claim 16, wherein, The loss function is defined by the following equation: Among them, S target For the target value, and This is the output value of the deep learning model.

18. The apparatus according to claim 15, wherein, The performance metrics of the deep learning model include scale-invariant signal-to-distortion ratio improvement, i.e., SI-SDRi.

19. The apparatus according to claim 11, wherein, Providing the deep learning model includes providing a first deep learning model through transfer learning based on the pre-trained model, the first deep learning model having a first new layer specifically for the first vehicle type added, and Predicting the engine-specific noise includes predicting the engine-specific noise of a vehicle corresponding to the first vehicle type based on the first deep learning model.

20. The apparatus according to claim 19, wherein, Providing the deep learning model further includes providing a second deep learning model through transfer learning based on the pre-trained model, the second deep learning model having a second new layer specifically for a second vehicle type, which is different from the first vehicle type, and Predicting the engine-specific noise further includes predicting the engine-specific noise of a vehicle corresponding to the second vehicle type based on the second deep learning model.

Images