Vehicle door closing sound signal quality evaluation method, system and equipment based on deep learning

Abstract

The invention relates to the technical field of automobile manufacturing and user experience optimization, in particular to a vehicle door closing sound signal quality evaluation method, system and device based on deep learning, and the method comprises the steps: collecting a vehicle door closing sound signal; extracting at least one target feature from the vehicle door closing sound signal, and performing data enhancement on the vehicle door closing sound signal to obtain an enhanced vehicle door closing sound signal; fusing at least one target feature and the enhanced vehicle door closing sound signal to obtain high-dimensional feature data fused with various acoustic features; and inputting the high-dimensional feature data fused with various acoustic features into a multi-task deep learning network model, and outputting a multi-dimensional scoring result and a fault diagnosis result of the door closing sound signal of the vehicle door. Therefore, the problems that a traditional machine evaluation system is single in extracted feature, poor in model generalization ability and single in evaluation dimension are solved, evaluation objectivity, accuracy and evaluation efficiency are improved, meanwhile, multi-dimensional evaluation results can be provided, and the model generalization ability is improved.

Description

Technical Field

[0001] This application relates to the fields of automotive manufacturing and user experience optimization technology, and in particular to a method, system and device for evaluating the quality of car door closing sound signals based on deep learning. Background Technology

[0002] In the fields of automotive manufacturing and user experience optimization, the quality of the door closing sound signal is a key indicator affecting users' perception of the car's texture and safety. Currently, the evaluation of door closing sound signal quality mainly relies on two types of technical solutions, but both have significant drawbacks. The limitations of subjective human evaluation: High subjectivity: Evaluation results are significantly affected by the evaluator's physiological state (e.g., auditory fatigue) and subjective preferences (e.g., differences in the definition of "crisp sound"). Inconsistency in evaluation results among different personnel at different times makes it difficult to establish a unified standard. Low efficiency: A single evaluation requires multiple personnel to repeatedly listen and score, with an average evaluation time exceeding 5 minutes per sample, which cannot meet the batch testing needs of automotive production lines (which need to process at least 2-3 door samples per minute). High cost: Professional evaluators require long-term training, and the team maintenance cost is high, making it difficult for small and medium-sized car companies to afford.

[0003] Limitations of traditional machine learning evaluation: Limited features and missing information: Existing technologies only extract spectral features (such as MFCC and Mel spectrum), ignoring physical quantity features directly related to "quality perception" in the door closing sound signal (such as loudness, sharpness, and low-frequency duration). For example, a user's perception of "weight" is not only related to the spectral distribution but also strongly correlated with the duration of the low-frequency components of the door closing sound; existing technologies suffer from low evaluation accuracy due to the lack of this feature. Poor model generalization ability: Traditional machine learning algorithms have weak fitting ability to nonlinear features of audio data. When the type of door (such as sedan door, SUV door) or the testing environment (such as workshop noise, temperature changes) changes, the model evaluation error increases significantly, requiring re-labeling a large number of samples for model iteration. Limited evaluation dimensions: Existing technologies mostly output only binary classification results of "qualified / unqualified," unable to output multi-dimensional scores such as "weight," "sportiness," and "crispness" like human evaluation, making it difficult to meet the refined optimization needs of automakers for door sound quality.

[0004] In summary, existing technologies cannot provide an automated car door closing sound signal evaluation solution that can both quantify subjective human auditory perception and automatically diagnose mechanical faults. Summary of the Invention

[0005] This application provides a method, system, and device for evaluating the quality of car door closing sound signals based on deep learning, in order to solve the problems of traditional machine evaluation systems having single feature extraction, poor model generalization ability, and single evaluation dimensions.

[0006] The first aspect of this application provides a method for evaluating the quality of a car door closing sound signal based on deep learning, comprising the following steps: acquiring a car door closing sound signal; extracting at least one target feature from the car door closing sound signal, and performing data augmentation on the car door closing sound signal to obtain an enhanced car door closing sound signal; fusing the at least one target feature and the enhanced car door closing sound signal to obtain high-dimensional feature data fusing multiple acoustic features; and inputting the high-dimensional feature data fusing multiple acoustic features into a preset multi-task deep learning network model to output a multi-dimensional scoring result and a fault diagnosis result of the car door closing sound signal.

[0007] Optionally, the step of enhancing the door closing sound signal to obtain the enhanced door closing sound signal includes: adding random noise, changing volume gain, time offsetting, and pitch adjusting the door closing sound signal to obtain the enhanced door closing sound signal.

[0008] Optionally, the preset multi-task deep learning network model includes a shared bottom-level feature extraction layer and multiple dedicated output layers. The shared bottom-level feature extraction layer is composed of a multi-layer one-dimensional convolutional neural network and a long short-term memory network. The dedicated output layers include a sound quality evaluation task output head and a fault diagnosis task output head. The sound quality evaluation task output head includes a comprehensive scoring output layer and a perception dimension scoring output layer. The fault diagnosis task output head is composed of multiple parallel binary classification output layers, and each binary classification output layer outputs the corresponding fault diagnosis result.

[0009] Optionally, before the high-dimensional feature data based on the fusion of multiple acoustic features is input into the preset multi-task deep learning network model to output the multi-dimensional scoring results and fault diagnosis results of the car door closing sound signal, the process includes: constructing an initial multi-task deep learning network model; dividing the model into training, testing, and validation sets based on expert scoring annotation data; performing end-to-end training of the multi-task deep learning network model using the training set and calculating the training loss of the model using a composite loss function; updating the model parameters using an adaptive learning rate optimizer and integrating a preset callback function for adjustment to obtain the trained model; calculating the training loss of the trained model using the validation set, stopping training when the training loss of the trained model meets a preset stopping condition to obtain an optimized model; testing the optimized model using the testing set to obtain the test accuracy of the optimized model, and if the test accuracy is greater than a preset threshold, then the preset multi-task deep learning network model is obtained.

[0010] Optionally, the fault diagnosis result of the door closing sound signal output by the high-dimensional feature data based on the fusion of multiple acoustic features is input to the preset multi-task deep learning network model, including determining that the door is fault-free if the output value of the preset multi-task deep learning network model is 1, and determining that the door is faulty if the output value of the preset multi-task deep learning network model is 0.

[0011] Optionally, the multi-dimensional rating result includes a rating value for at least one perception dimension and a comprehensive rating value.

[0012] Optionally, after determining that the door is faulty, the process includes: generating a door faulty reminder message to the target terminal.

[0013] A second aspect of this application provides a deep learning-based system for evaluating the quality of a car door closing sound signal, comprising: a data acquisition module for acquiring car door closing sound signals; a feature extraction module for extracting target features from the car door closing sound signals and performing data enhancement on the car door closing sound signals to obtain enhanced car door closing sound signals; a feature fusion module for fusing the target features and the enhanced car door closing sound signals to obtain high-dimensional feature data fusing multiple acoustic features; and an evaluation result output module for inputting the high-dimensional feature data fusing multiple acoustic features into a preset multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results for the car door closing sound signals.

[0014] Optionally, the feature extraction module is used to: add random noise, change volume gain, time offset, and adjust pitch to the door closing sound signal to obtain the enhanced door closing sound signal.

[0015] Optionally, the preset multi-task deep learning network model includes a shared bottom-level feature extraction layer and multiple dedicated output layers. The shared bottom-level feature extraction layer is composed of a multi-layer one-dimensional convolutional neural network and a long short-term memory network. The dedicated output layers include a sound quality evaluation task output head and a fault diagnosis task output head. The sound quality evaluation task output head includes a comprehensive scoring output layer and a perception dimension scoring output layer. The fault diagnosis task output head is composed of multiple parallel binary classification output layers, and each binary classification output layer outputs the corresponding fault diagnosis result.

[0016] Optionally, before the high-dimensional feature data based on the fusion of multiple acoustic features is input into the preset multi-task deep learning network model to output the multi-dimensional scoring results and fault diagnosis results of the car door closing sound signal, the evaluation result output module is further used to: construct an initial multi-task deep learning network model; divide the training set, test set, and validation set based on expert scoring annotation data; perform end-to-end training of the multi-task deep learning network model using the training set, and calculate the training loss of the model using a composite loss function; update the model parameters using an adaptive learning rate optimizer, and adjust them by integrating a preset callback function to obtain the trained model; calculate the training loss of the trained model using the validation set, and stop training when the training loss of the trained model meets a preset stopping condition to obtain an optimized model; test the optimized model using the test set to obtain the test accuracy of the optimized model, and if the test accuracy is greater than a preset threshold, then the preset multi-task deep learning network model is obtained.

[0017] Optionally, the evaluation result output module is further configured to: determine that the car door is fault-free if the output value of the preset multi-task deep learning network model is 1, and determine that the car door is faulty if the output value of the preset multi-task deep learning network model is 0.

[0018] Optionally, the multi-dimensional rating result includes a rating value for at least one perception dimension and a comprehensive rating value.

[0019] Optionally, after determining that the car door is faulty, the evaluation result output module is further configured to: generate a reminder message that the car door is faulty and send it to the target terminal.

[0020] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the deep learning-based method for evaluating the quality of a car door closing sound signal as described in the above embodiments.

[0021] The fourth aspect of this application provides a computer program product having a computer program stored thereon, which is executed by a processor to implement the deep learning-based method for evaluating the quality of a car door closing sound signal as described in the above embodiments.

[0022] In the above implementation, the sound signal of a car door closing is collected; at least one target feature is extracted from the sound signal, and the sound signal is augmented to obtain an enhanced sound signal; the at least one target feature and the enhanced sound signal are fused to obtain high-dimensional feature data that integrates multiple acoustic features; based on the high-dimensional feature data integrating multiple acoustic features, it is input into a preset multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results for the sound signal of the car door closing. This solves the problems of traditional machine evaluation systems, such as single feature extraction, poor model generalization ability, and single evaluation dimension, improving the objectivity, accuracy, and efficiency of the evaluation, while providing multi-dimensional evaluation results and enhancing the model's generalization ability.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a deep learning-based method for evaluating the quality of a car door closing sound signal, according to an embodiment of this application. Figure 2 This is a flowchart of a deep learning-based method for evaluating the quality of a car door closing sound signal according to an embodiment of this application; Figure 3 This is a schematic diagram showing the location of a sound acquisition device according to an embodiment of this application; Figure 4 A Mel spectrum diagram according to an embodiment of this application; Figure 5 This is an example diagram of a deep learning-based vehicle door closing sound signal quality evaluation system according to an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0026] The following describes a method, system, and device for evaluating the quality of car door closing sound signals based on deep learning, with reference to the accompanying drawings. Addressing the problems mentioned in the background art regarding traditional machine learning evaluation systems, such as single feature extraction, poor model generalization ability, and limited evaluation dimensions, this application provides a method for evaluating the quality of car door closing sound signals based on deep learning. In this method, car door closing sound signals are collected; at least one target feature is extracted from the car door closing sound signal, and the enhanced car door closing sound signal is obtained after data augmentation; the at least one target feature and the enhanced car door closing sound signal are fused to obtain high-dimensional feature data fusing multiple acoustic features; based on the high-dimensional feature data fusing multiple acoustic features, it is input into a preset multi-task deep learning network model, which outputs multi-dimensional scoring results and fault diagnosis results for the car door closing sound signal. This solves the problems of single feature extraction, poor model generalization ability, and limited evaluation dimensions in traditional machine learning evaluation systems, improving the objectivity, accuracy, and efficiency of the evaluation, while providing multi-dimensional evaluation results and enhancing the model's generalization ability.

[0027] Specifically, Figure 1 This is a flowchart illustrating a deep learning-based method for evaluating the quality of a car door closing sound signal, as provided in an embodiment of this application.

[0028] like Figure 1 As shown, the deep learning-based method for evaluating the quality of car door closing sound signals includes the following steps: In step S101, the door closing sound signal is collected.

[0029] Specifically, the process of the deep learning-based method for evaluating the quality of car door closing sound signals is as follows: Figure 2 As shown, during data acquisition, the artificial head was placed 60cm vertically from the car door, at a height of 175cm (ear height). The artificial head was used to record the sound signal at the moment the car door closed. The artificial head was positioned as follows. Figure 3 As shown.

[0030] The original WAV audio file of the car door closing sound is further loaded through the audio loading and normalization unit, and then uniformly resampled to the same sampling rate (e.g., 22050 Hz), and truncated or padded to a fixed duration (e.g., 3 seconds) to ensure that the input data scale is consistent.

[0031] In step S102, at least one target feature is extracted from the door closing sound signal, and the door closing sound signal is augmented to obtain an augmented door closing sound signal. At least one target feature includes loudness, sharpness, roughness, Mel frequency cepstral coefficient, chromaticity feature, spectral contrast, root mean square energy, and zero-crossing rate.

[0032] Optionally, in some embodiments, the enhanced door closing sound signal is obtained by performing data enhancement on the door closing sound signal, including: adding random noise, changing volume gain, time offsetting, and pitch adjustment on the door closing sound signal to obtain the enhanced door closing sound signal.

[0033] Specifically, after collecting the car door closing sound signal, the signal is randomly transformed, and the dataset is manually expanded to improve the model's generalization ability. Enhancement methods include: adding random noise to simulate interference in the real environment; changing volume gain to simulate the sound under different closing forces; time offset to simulate subtle differences in the timing of the closing action; and pitch adjustment to simulate pitch changes that may be caused by different materials or structures, thereby obtaining an enhanced car door closing sound signal.

[0034] In step S103, at least one target feature and the enhanced door closing sound signal are fused to obtain high-dimensional feature data that fuses multiple acoustic features. The multi-dimensional feature fusion extraction unit fuses at least one target feature and the enhanced car door closing sound signal to extract high-dimensional feature data that integrates multiple acoustic features. This high-dimensional feature data includes: Mel-frequency cepstral coefficients (MFCCs): used to describe the short-time power spectrum of sound, simulating the characteristics of human hearing; Chroma features: focusing on the pitch and harmony information of sound; and Mel-spectrograms: reflecting the energy distribution of sound at the Mel scale, such as... Figure 4 As shown. Spectral Contrast: Describes the difference between peaks and valleys in the spectrum. Root Mean Square Energy (RMS Energy): Characterizes the loudness of the sound. Zero-Crossing Rate: Reflects a rough estimate of the signal frequency. This unit aligns and stitches all the above features on the time axis to form a feature matrix that comprehensively characterizes the sound properties.

[0035] In step S104, based on high-dimensional feature data that integrates multiple acoustic features, the data is input into a preset multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results of the door closing sound signal.

[0036] In some embodiments, the multi-dimensional rating results include a rating value for at least one perception dimension and a comprehensive rating value (1-10 points).

[0037] The rating for at least one sensory dimension includes “weightiness” rating (1-5 points), “motionality” rating (1-5 points), and “crispness” rating (1-5 points).

[0038] Optionally, in some embodiments, the preset multi-task deep learning network model includes a shared bottom-level feature extraction layer and multiple dedicated output layers. The shared bottom-level feature extraction layer is composed of multiple one-dimensional convolutional neural networks and long short-term memory networks. The dedicated output layers include a sound quality evaluation task output head and a fault diagnosis task output head. The sound quality evaluation task output head includes a comprehensive scoring output layer and a perception dimension scoring output layer. The fault diagnosis task output head is composed of multiple parallel binary classification output layers, and each binary classification output layer outputs the corresponding fault diagnosis result.

[0039] Optionally, in some embodiments, before the high-dimensional feature data based on the fusion of multiple acoustic features is input into the preset multi-task deep learning network model to output the multi-dimensional scoring results and fault diagnosis results of the car door closing sound signal, the process includes: constructing an initial multi-task deep learning network model; dividing the model into training, testing, and validation sets based on expert scoring annotation data; performing end-to-end training of the multi-task deep learning network model using the training set and calculating the training loss of the model using a composite loss function; updating the model's parameters using an adaptive learning rate optimizer and integrating a preset callback function for adjustment to obtain the trained model; calculating the training loss of the trained model using the validation set, stopping training when the training loss of the trained model meets a preset stopping condition to obtain an optimized model; testing the optimized model using the testing set to obtain the test accuracy of the optimized model, and if the test accuracy is greater than a preset threshold, then the preset multi-task deep learning network model is obtained.

[0040] Optionally, in some embodiments, based on high-dimensional feature data that integrates multiple acoustic features, the fault diagnosis result of the door closing sound signal output by the preset multi-task deep learning network model is input to the preset multi-task deep learning network model. This includes determining that the door is fault-free if the output value of the preset multi-task deep learning network model is 1, and determining that the door is faulty if the output value of the preset multi-task deep learning network model is 0.

[0041] Optionally, in some embodiments, after determining that the door is faulty, the process includes: generating a reminder message indicating that the door is faulty and sending it to the target terminal.

[0042] Specifically, the pre-defined multi-task deep learning network model adopts a shared bottom feature extraction layer and multiple dedicated output layers. The shared bottom feature extraction layer is composed of a multi-layer one-dimensional convolutional neural network (1D-CNN) and a long short-term memory network (LSTM).

[0043] The CNN layer is responsible for extracting local patterns from audio features, while the LSTM layer is responsible for capturing long-term temporal dependencies in the sound signal. This shared encoder simultaneously learns a rich common feature representation for both sound quality assessment and fault diagnosis tasks. Sound quality assessment task output head: Comprehensive score output layer: One linearly activated neuron outputs a comprehensive quality score of 1-5. Perception dimension score output layer: Three linearly activated neurons output scores of 1-5 for "weight," "motion," and "crispness," respectively. Fault diagnosis task output head: Consists of multiple parallel binary classification output layers, each corresponding to a specific fault (such as sheet metal problems, blowout valve noise, etc.), using the Sigmoid activation function to output a probability of 0 (no fault) or 1 (fault present).

[0044] To extract the temporal and frequency domain correlation information from spectral features, a combined structure of "1D convolution + LSTM" is used. First convolutional block: 1D convolutional layer (64 3×1 convolutional kernels, ReLU activation) → batch normalization layer → max pooling layer (2×1 pooling kernel), output feature map shape is (64, 64). Second convolutional block: 1D convolutional layer (128 3×1 convolutional kernels, ReLU activation) → batch normalization layer → max pooling layer (2×1 pooling kernel), output feature map shape is (32, 128). The third convolutional block consists of a 1D convolutional layer (256 3×1 convolutional kernels, ReLU activation) → batch normalization layer → max pooling layer (2×1 pooling kernel), with the output feature map shape being (16, 256). LSTM layer: 128 hidden units, does not return intermediate time steps (return_sequences=False), output shape is (128,); Dropout layer: Dropout ratio 0.5, used to suppress overfitting, output shape preserved (128,).

[0045] The pre-defined training steps for a multi-task deep learning network model are as follows: The module uses labeled data from expert ratings to train the network end-to-end. The outputs of the two branches are fused using feature concatenation to obtain a fused feature vector with the shape (128+32) = (160,). This is then further processed by a fully connected layer (256 neurons, ReLU activation) and a dropout layer (scale 0.6) to enhance the non-linear expressive power of the features. The composite loss function is a weighted sum of the loss from the sound quality regression task (e.g., Mean Absolute Error, MAE) and the fault diagnosis classification task (e.g., Binary Cross-Entropy). The weights are adjusted to balance the learning priorities of the two tasks. The optimization strategy employs an adaptive learning rate optimizer (e.g., Adam) and integrates callback functions such as Early Stopping and ReduceLROnPlateau to prevent overfitting and ensure the optimal model is obtained.

[0046] High-dimensional feature data integrating multiple acoustic features is input into a pre-set multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results of the car door closing sound signal. It also automatically generates visualization charts such as confusion matrix, accuracy curve, and scatter plot of predicted and true values ​​for model performance analysis and result verification.

[0047] If the fault diagnosis result output by the preset multi-task deep learning network model is 0 for sheet metal abnormality and 1 for door reaction force abnormality, it indicates that the door reaction force is abnormal. At this time, a door reaction force abnormality reminder message is generated and sent to the target terminal to remind the tester.

[0048] It should be noted that the hardware and software involved in this application are as follows: Hardware environment: This system can be deployed on one or more high-performance computing servers or workstations. Recommended configuration includes: multi-core CPU (such as Intel Xeon series), high-performance GPU (such as NVIDIA RTX A5000 or V100, for accelerating model training and prediction), at least 32GB of memory, and a large-capacity solid-state drive for storing large amounts of audio data and model files.

[0049] Software environment: The operating system can be Linux or Windows. The software framework is primarily based on the Python language and depends on the following core libraries: TensorFlow 2.x or PyTorch: As deep learning frameworks, they are used to build, train, and deploy multi-task deep learning models.

[0050] Librosa: Used for audio file loading, normalization, and advanced acoustic feature extraction.

[0051] Scikit-learn: Used for data partitioning and performance metric calculation (such as accuracy and confusion matrix).

[0052] NumPy and Pandas are used for efficient numerical computation and data processing.

[0053] Matplotlib and Seaborn are used to generate various visualization charts, such as loss curves, confusion matrices, scatter plots, etc.

[0054] In summary, the beneficial effects of this application are as follows: 1. Improved objectivity and accuracy By fusing multiple features (spectral features + physical quantity features) with a deep learning model, the evaluation results are quantitatively output. The evaluation error (mean absolute error MAE) is less than 0.3 points (score range 1-5 points), which is significantly better than traditional machine evaluation (MAE of about 0.8 points). Moreover, the consistency of the results (intragroup correlation coefficient ICC) is higher than 0.9, which is close to the consistency level of evaluation by professional human teams.

[0055] 2. Evaluation efficiency has been greatly improved. The evaluation time for a single sample has been reduced from 5 minutes for manual evaluation to less than 0.5 seconds. It can process more than 700 samples per hour, fully meeting the batch testing needs of automotive production lines. At the same time, it saves the training and maintenance costs of a professional evaluation team, reducing the sound quality evaluation cost for car companies by more than 30%.

[0056] 3. Refinement of evaluation dimensions It can simultaneously output an "overall score" (1-5 points) and three dimensions of feeling scores: "weight," "sportiness," and "crispness," which are completely matched with human evaluation dimensions, providing automakers with precise directions for optimizing door structures (such as guiding the adjustment of door sealing strip thickness by the correlation between "200Hz low frequency duration" and "weight").

[0057] 4. Strong generalization ability By expanding the training samples using data augmentation techniques (noise addition, volume adjustment, time shift, etc.), the model's evaluation error fluctuation is less than 0.1 points under different car door types (sedans, SUVs, MPVs) and test environments (noise intensity 0-60dB), eliminating the need for frequent retraining and adapting to the testing needs of multiple car production lines.

[0058] The method for evaluating the quality of car door closing sound signals based on deep learning, as proposed in this application, involves: collecting car door closing sound signals; extracting at least one target feature from the sound signals and performing data augmentation on the sound signals to obtain augmented car door closing sound signals; fusing the at least one target feature and the augmented car door closing sound signals to obtain high-dimensional feature data that integrates multiple acoustic features; and inputting the high-dimensional feature data into a preset multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results for the car door closing sound signals. This solves the problems of traditional machine evaluation systems, such as single feature extraction, poor model generalization ability, and single evaluation dimensions, improving the objectivity, accuracy, and efficiency of the evaluation, while providing multi-dimensional evaluation results and enhancing the model's generalization ability.

[0059] Next, referring to the accompanying drawings, a deep learning-based system for evaluating the quality of a car door closing sound signal is described according to an embodiment of this application.

[0060] Figure 5 This is a block diagram of a deep learning-based car door closing sound signal quality evaluation system according to an embodiment of this application.

[0061] like Figure 5 As shown, the deep learning-based car door closing sound signal quality evaluation system 10 includes: a data acquisition module 100, a feature extraction module 200, a feature fusion module 300, and an evaluation result output module 400.

[0062] The system includes a data acquisition module 100 for acquiring door closing sound signals; a feature extraction module 200 for extracting target features from the door closing sound signals and performing data enhancement on the door closing sound signals to obtain enhanced door closing sound signals; a feature fusion module 300 for fusing the target features and the enhanced door closing sound signals to obtain high-dimensional feature data that integrates multiple acoustic features; and an evaluation result output module 400 for inputting the high-dimensional feature data that integrates multiple acoustic features into a preset multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results for the door closing sound signals.

[0063] Optionally, in some embodiments, the feature extraction module 200 is used to: add random noise to the door closing sound signal, change the volume gain, perform time offset processing and pitch adjustment processing to obtain an enhanced door closing sound signal.

[0064] Optionally, in some embodiments, the preset multi-task deep learning network model includes a shared bottom-level feature extraction layer and multiple dedicated output layers. The shared bottom-level feature extraction layer is composed of multiple one-dimensional convolutional neural networks and long short-term memory networks. The dedicated output layers include a sound quality evaluation task output head and a fault diagnosis task output head. The sound quality evaluation task output head includes a comprehensive scoring output layer and a perception dimension scoring output layer. The fault diagnosis task output head is composed of multiple parallel binary classification output layers, and each binary classification output layer outputs the corresponding fault diagnosis result.

[0065] Optionally, before the high-dimensional feature data based on the fusion of multiple acoustic features is input into the preset multi-task deep learning network model to output the multi-dimensional scoring results and fault diagnosis results of the car door closing sound signal, the evaluation result output module 400 is also used for: constructing an initial multi-task deep learning network model; dividing the training set, test set, and validation set based on expert scoring annotation data; performing end-to-end training of the multi-task deep learning network model using the training set, and calculating the training loss of the model using a composite loss function; updating the model parameters using an adaptive learning rate optimizer, and adjusting them by integrating a preset callback function to obtain the trained model; calculating the training loss of the trained model using the validation set, stopping training when the training loss of the trained model meets a preset stopping condition, and obtaining the optimized model; testing the optimized model using the test set to obtain the test accuracy of the optimized model, and if the test accuracy is greater than a preset threshold, then the preset multi-task deep learning network model is obtained.

[0066] Optionally, in some embodiments, the evaluation result output module 400 is further configured to: determine that the car door is fault-free if the output value of the preset multi-task deep learning network model is 1, and determine that the car door is faulty if the output value of the preset multi-task deep learning network model is 0.

[0067] Optionally, in some embodiments, the multi-dimensional rating results include a rating value for at least one perception dimension and a comprehensive rating value.

[0068] Optionally, in some embodiments, after determining that the car door is faulty, the evaluation result output module 400 is further configured to: generate a reminder message that the car door is faulty and send it to the target terminal.

[0069] It should be noted that the foregoing explanation of the embodiment of the deep learning-based method for evaluating the quality of car door closing sound signals also applies to the deep learning-based car door closing sound signal quality evaluation system of this embodiment, and will not be repeated here.

[0070] The deep learning-based vehicle door closing sound signal quality evaluation system proposed in this application collects vehicle door closing sound signals; extracts at least one target feature from the sound signals and performs data augmentation on the sound signals to obtain augmented vehicle door closing sound signals; fuses the at least one target feature and the augmented vehicle door closing sound signals to obtain high-dimensional feature data that integrates multiple acoustic features; based on the high-dimensional feature data integrating multiple acoustic features, it is input into a preset multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results for the vehicle door closing sound signals. This solves the problems of traditional machine evaluation systems, such as single feature extraction, poor model generalization ability, and single evaluation dimensions, improving the objectivity, accuracy, and efficiency of the evaluation, while providing multi-dimensional evaluation results and enhancing the model's generalization ability.

[0071] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0072] When the processor 602 executes the program, it implements the deep learning-based method for evaluating the quality of car door closing sound signals provided in the above embodiments.

[0073] Furthermore, electronic devices also include: Communication interface 603 is used for communication between memory 601 and processor 602.

[0074] The memory 601 is used to store computer programs that can run on the processor 602.

[0075] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0076] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0077] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0078] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0079] This application also provides a computer program product on which a computer program is stored. When the program is executed by a processor, it implements the above-mentioned deep learning-based method for evaluating the quality of car door closing sound signals.

[0080] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0081] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0082] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0083] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be specifically implemented in any computer program product for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer program product" can be any means that can contain, store, communicate, propagate, or transmit a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples of computer program products (a non-exhaustive list) include the following: an electrical connection having one or N wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, the computer program product can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0084] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0085] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer program product, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0086] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer program product.

[0087] The computer program product mentioned above may be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for evaluating the quality of a car door closing sound signal based on deep learning, characterized in that, Includes the following steps: Collect the sound signal of the car door closing; At least one target feature is extracted from the door closing sound signal, and the door closing sound signal is augmented to obtain an augmented door closing sound signal. By fusing at least one target feature and the enhanced door closing sound signal, high-dimensional feature data incorporating multiple acoustic features is obtained. Based on the high-dimensional feature data that integrates multiple acoustic features, the data is input into a preset multi-task deep learning network model, which outputs multi-dimensional scoring results and fault diagnosis results for the door closing sound signal.

2. The method according to claim 1, characterized in that, The process of enhancing the door closing sound signal to obtain the enhanced door closing sound signal includes: The enhanced door closing sound signal is obtained by adding random noise, changing volume gain, time offsetting, and adjusting pitch.

3. The method according to claim 1, characterized in that, The preset multi-task deep learning network model includes a shared bottom-level feature extraction layer and multiple dedicated output layers. The shared bottom-level feature extraction layer is composed of a multi-layer one-dimensional convolutional neural network and a long short-term memory network. The dedicated output layers include a sound quality evaluation task output head and a fault diagnosis task output head. The sound quality evaluation task output head includes a comprehensive scoring output layer and a perception dimension scoring output layer. The fault diagnosis task output head is composed of multiple parallel binary classification output layers, and each binary classification output layer outputs the corresponding fault diagnosis result.

4. The method according to claim 3, characterized in that, Before the high-dimensional feature data based on the fusion of multiple acoustic features is input into the preset multi-task deep learning network model to output the multi-dimensional scoring results and fault diagnosis results of the car door closing sound signal, the following steps are included: Construct an initial multi-task deep learning network model; The training set, test set, and validation set were divided based on expert rating and annotation data. The training set is used to train the multi-task deep learning network model end-to-end, and the training loss of the model is calculated using a composite loss function. An adaptive learning rate optimizer is used to update the model's parameters, and a preset callback function is integrated for adjustment to obtain the trained model. The training loss of the trained model is calculated using the validation set. Training is stopped when the training loss of the trained model meets a preset stopping condition, and the optimized model is obtained. The optimized model is tested using the test set to obtain the test accuracy of the optimized model. If the test accuracy is greater than a preset threshold, the preset multi-task deep learning network model is obtained.

5. The method according to claim 4, characterized in that, The high-dimensional feature data based on the fusion of multiple acoustic features is input into a preset multi-task deep learning network model, which outputs the fault diagnosis result of the car door closing sound signal, including... If the output value of the preset multi-task deep learning network model is 1, the car door is determined to be fault-free; if the output value of the preset multi-task deep learning network model is 0, the car door is determined to be faulty.

6. The method according to claim 1, characterized in that, The multi-dimensional rating results include a rating value for at least one perception dimension and a comprehensive rating value.

7. The method according to claim 5, characterized in that, After determining that there is a malfunction in the car door, including: Generate a notification message indicating a door malfunction to the target terminal.

8. A deep learning-based system for evaluating the quality of a car door closing sound signal, characterized in that, include: The data acquisition module is used to collect the sound signal of the car door closing; The feature extraction module is used to extract target features from the door closing sound signal and perform data enhancement on the door closing sound signal to obtain an enhanced door closing sound signal. The feature fusion module is used to fuse the target features and the enhanced door closing sound signal to obtain high-dimensional feature data that fuses multiple acoustic features. The evaluation result output module is used to input the high-dimensional feature data that integrates multiple acoustic features into a preset multi-task deep learning network model to output multi-dimensional scoring results and fault diagnosis results of the door closing sound signal.

9. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deep learning-based method for evaluating the quality of a car door closing sound signal as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based method for evaluating the quality of car door closing sound signals as described in any one of claims 1-7.

Images