Vehicle-mounted sound abnormity diagnosis method and system and related equipment

Abstract

The invention discloses a vehicle-mounted sound abnormity diagnosis method and system and related equipment, and the method specifically comprises the steps: carrying out the real-time sound signal detection at a vehicle-mounted terminal through a light-weight first model, and uploading an obtained abnormal signal and vehicle data corresponding to the abnormal signal to a cloud server when the abnormal signal is detected, meanwhile, sound features in the abnormal signals are input into a second model, and a specific vehicle fault is determined; the cloud server receives the maintenance result of the vehicle fault and combines the maintenance result with the uploaded data to form an incremental training sample; generating a model increment parameter of a second model based on the increment training sample; and directionally issuing the model increment parameter to a related vehicle, and completing updating of the second model under a safety condition. According to the vehicle-mounted sound abnormity diagnosis method provided by the invention, maintenance result optimization can be utilized, and the adaptive capacity and accuracy of vehicle fault diagnosis are improved.

Description

Technical Field

[0001] This application relates to the field of automotive fault diagnosis and repair technology, and in particular to methods, systems and related equipment for diagnosing abnormal vehicle sounds. Background Technology

[0002] With the rapid development of automotive electronics and intelligence, vehicle health management and early fault warning have become core requirements for improving driving safety and reducing maintenance costs.

[0003] Current vehicle diagnostic technologies still face core bottlenecks: existing systems mostly use static models, which cannot evolve once deployed, making it difficult to cope with new fault modes and noise environments that constantly emerge in real-world scenarios, resulting in a decline in diagnostic accuracy over time; model optimization relies on high-quality labeled data, but current solutions only rely on general user feedback and fail to automatically correlate specific repair results with precise sound data at the time of the fault, thus failing to form an efficient data loop to continuously generate effective training samples; model update mechanisms are generally cumbersome and inefficient, relying on full model replacement, which poses safety and performance risks such as large update packages, long processing times, and lack of personalized adaptation.

[0004] Therefore, there is an urgent need for an intelligent diagnostic solution that can automatically utilize real maintenance data, achieve continuous model evolution in a lightweight and safe manner, and maintain knowledge stability. Summary of the Invention

[0005] This application provides a method, system, and related equipment for diagnosing in-vehicle audio anomalies. The method is used to comprehensively analyze multiple audio fault information from a vehicle to accurately determine the vehicle's fault. The specific technical solution is as follows: Firstly, a method for diagnosing abnormal in-vehicle audio is provided, applied to an in-vehicle terminal, including the following steps: Sound signals from multiple component areas of the vehicle are collected, preprocessed, and feature extracted. Anomaly detection is performed using a first model deployed in a first computing unit. These component areas include the engine area, brake area, tire area, and suspension area. The first model detects anomalies in the sound signals, saves the anomaly signals and associated vehicle data, and uploads them to a cloud server. A second model deployed in a second computing unit determines the vehicle fault corresponding to the anomaly signal and generates a warning message. The computing power of the second computing unit is greater than that of the first computing unit. The warning message is output, the repair result of the vehicle fault is obtained, and the repair result is uploaded to the cloud server. Incremental parameters of the model trained by the cloud server based on the anomaly signal, vehicle data, and repair result are obtained. When the vehicle meets preset safety conditions, the incremental parameters are loaded to update the first and second models.

[0006] In conjunction with the first aspect, in determining the vehicle fault corresponding to the abnormal signal through the second model, the method includes: analyzing the abnormal signal and the vehicle data through the second model to obtain the vehicle fault location and fault classification, and determining the vehicle fault based on the fault location and fault classification.

[0007] In conjunction with the first aspect, after the repair result is uploaded to the cloud server, the method further includes: the cloud server using the repair result as a label and associating the abnormal signal with the vehicle data to form an incremental training sample; the cloud server training the incremental parameters of the model based on the incremental training sample and sending the incremental parameters of the model to the vehicle.

[0008] In conjunction with the first aspect, in some implementations of the first direction, the incremental parameters of the model are sent to the vehicle, specifically including: the cloud server analyzes the characteristics of the incremental training sample, determines the applicable scope of the incremental training sample, the applicable scope including vehicle type, configuration and fault mode; and the trained incremental parameters of the model are sent to the vehicle within the applicable scope.

[0009] In conjunction with the first aspect, in some implementations of the first direction, during the process of the cloud server training the incremental parameters of the model based on the incremental training samples, the cloud server adopts an anti-forgetting strategy to maintain the ability to identify known fault categories. The anti-forgetting strategy is achieved by fixing the model weights and constraining the update magnitude of the parameters.

[0010] In conjunction with the first aspect, the method further includes, during the process of generating the early warning information, performing fault prediction on the component area based on historical diagnostic data and vehicle data, and including the generated fault prediction results in the early warning information.

[0011] In conjunction with the first aspect, the preset safety conditions include: the vehicle being parked, charging, or having an idle in-vehicle network.

[0012] It should be noted that, in the absence of conflict, the features in the various embodiments of the first aspect can be combined with each other, and any combination of features in different embodiments is also within the protection scope of this application. That is to say, the various embodiments described above can also be arbitrarily combined according to actual needs.

[0013] Secondly, an in-vehicle sound anomaly diagnostic system is provided for performing the method described in the first aspect or any of the preceding claims. The system includes: an in-vehicle terminal, comprising a sound acquisition module, a signal processing module, a fault identification module, and an interaction module, for detecting sound signals to determine vehicle faults; a maintenance service terminal, comprising a fault display module and a maintenance feedback module, for displaying the vehicle fault, recording and uploading maintenance results; and a cloud server, comprising a data annotation module, an incremental learning training module, and a model distribution management module, for achieving data closure and continuous model evolution.

[0014] Thirdly, a computer is provided, including one or more memories and one or more processors; the memory is coupled to the one or more processors, the memory being used to store computer program code including computer instructions, the one or more processors calling the computer instructions to cause the computer to implement the method as described in the first aspect or any of the embodiments of the first aspect.

[0015] Fourthly, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the method as described in the first aspect or any of the embodiments in the first aspect.

[0016] In the embodiments of this application, the method provided by this application has the following technical advantages: 1. Continuous Model Optimization: After deployment, the diagnostic system provided in this application can automatically utilize real-world maintenance feedback data for continuous and safe incremental learning, constantly absorbing new knowledge and optimizing personalized performance.

[0017] 2. Obtaining high-quality datasets: This application can obtain high-quality data annotations in vehicle repair services, avoiding the problems of data scarcity and high annotation costs in model training.

[0018] 3. Ensure reliable system operation: This application updates the model based on incremental model parameters, with a very small update package size, which can accurately push updates for specific vehicle models or fault modes, and achieve model optimization under the condition of vehicle safety.

[0019] 4. Improve the accuracy of fault diagnosis: The incremental parameters of the model provided in this application allow the model to maintain its original diagnostic capabilities without decay while updating fault characteristics, thus ensuring the reliability and accuracy of the system throughout its entire lifecycle. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an overall architecture block diagram of an in-vehicle sound anomaly diagnostic system provided in an embodiment of this application; Figure 2 This is a flowchart of a method for diagnosing abnormal vehicle audio provided in an embodiment of this application; Figure 3 This is an interaction timing diagram of an in-vehicle terminal provided in an embodiment of this application; Figure 4 This is a module structure diagram of a diagnostic system provided in an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application; Figure 6 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0023] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0024] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. Furthermore, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] This application provides a method, system, and related equipment for diagnosing vehicle sound anomalies, including the following steps: collecting sound signals from multiple component areas of a vehicle; preprocessing and extracting features from the sound signals; and using a first model deployed in a first computing unit to perform anomaly detection. The component areas include the engine area, brake area, tire area, and suspension area. The first model detects anomalies in the sound signals, saves the anomalies and associated vehicle data, and uploads the anomalies and vehicle data to a cloud server. A second model deployed in a second computing unit determines the vehicle fault corresponding to the anomalies and generates a warning message. The computing power of the second computing unit is greater than that of the first computing unit. The warning message is output, the repair results of the vehicle fault are obtained, and the repair results are uploaded to the cloud server. The incremental parameters of the model, trained by the cloud server based on the anomalies, vehicle data, and repair results, are obtained. When the vehicle meets preset safety conditions, the incremental parameters are loaded to update the second model.

[0026] In this way, this application uses an in-vehicle microphone to capture sound signals, extracts the features of the sound signals, and performs anomaly detection through a lightweight first model. After the first model identifies the abnormal signal, a second model is used to accurately classify and predict the fault type, thereby improving the diagnostic accuracy of the system. Vehicle owners or repair personnel can quickly obtain repair suggestions and service recommendations through repair service terminals, such as applications (APPs), thereby improving repair efficiency. The cloud server can obtain abnormal signals, vehicle data, and repair results to update the second model, further improving the accuracy of vehicle fault diagnosis.

[0027] The following four embodiments describe the vehicle audio anomaly diagnosis method, system, and related equipment provided in this application. Embodiment 1 describes the framework of the vehicle audio anomaly diagnosis system, Embodiment 2 describes the method flow of vehicle audio anomaly diagnosis, Embodiment 3 describes the vehicle audio anomaly diagnosis method in a specific vehicle fault scenario, and Embodiment 4 describes the module structure, computer equipment hardware structure, and computer-readable storage medium of the audio anomaly diagnosis system.

[0028] Example 1 Figure 1 This is a block diagram of the overall architecture of the in-vehicle sound anomaly diagnostic system (hereinafter referred to as the system) provided in this application embodiment. Figure 1 As shown, the system consists of an in-vehicle terminal, a cloud server, and a maintenance service terminal working together.

[0029] 1. Vehicle-mounted terminal.

[0030] In this embodiment of the application, the vehicle terminal is used to acquire abnormal signals in the sound signal and determine the vehicle fault. The main units include: a sound signal acquisition unit, a signal preprocessing and feature extraction unit, a first calculation unit, a first model, an encapsulation unit, a communication unit, a second calculation unit, and a second model.

[0031] Sound signal acquisition unit: It consists of microphone arrays arranged in key parts of the vehicle (such as the engine compartment, near the wheel hubs, and the chassis), and is responsible for acquiring raw sound signals with high quality.

[0032] In this embodiment, the microphone array is, for example, a high-precision microelectromechanical system (MEMS) microphone array. The number of microphone arrays may be one or more, and they support beamforming to initially separate the direction of the sound source. For example, to achieve effective sound acquisition from multiple components in a vehicle, the microphone array may employ a spatially distributed layout.

[0033] For example, microphone arrays can be placed in the engine compartment firewall, inside the wheel arches, and near the rearview mirror in the passenger compartment to collect powertrain, chassis suspension, and abnormal noises from the interior, respectively. Microphone arrays placed inside the engine compartment firewall are used to isolate and collect abnormal signals such as high-frequency whistling, knocking, and cylinder knocking from the powertrain, such as the engine and transmission. Microphone arrays placed inside the wheel arches are used to collect abnormal signals such as brake pad wear whistling, shock absorber noises, and bushing aging. Microphone arrays placed in the middle of the passenger compartment roof are used to collect background noise in the vehicle, which can be used as a reference signal for noise suppression.

[0034] The aforementioned microphone array can synchronously acquire sound signals at a fixed sampling rate (such as 44.1kHz or other higher or lower frequencies) and transmit the raw audio stream (i.e., the sound signals generated during vehicle operation) to the processor in the vehicle via a transmission line (such as a dedicated audio bus).

[0035] Signal preprocessing and feature extraction unit: performs real-time processing on the acquired raw audio, including noise reduction, filtering, discrete wavelet transform (DWT) and other methods, and extracts sound features such as Mel frequency cepstral coefficients (MFCC) to provide standardized input for subsequent analysis.

[0036] For example, the sound signals acquired by the microphone array are subjected to gain adjustment, noise reduction, and filtering, and sound features are extracted based on the signal preprocessing and feature extraction modules in the software module. Gain adjustment includes: the processor automatically adjusts the gain according to the signal strength of each microphone to ensure a suitable dynamic range; noise reduction includes: the processor uses an adaptive noise cancellation algorithm, using the in-vehicle background noise signal as a noise reference, and effectively suppresses background noise such as wind noise and road noise in the engine compartment and chassis microphone signals through an adaptive filter; filtering includes: the processor uses bandpass filters (e.g., for the 1-4kHz frequency band of engine knock, and for the 500-1500Hz frequency band of tire squeal) to initially screen the sound signals and identify the main frequency bands of the target fault.

[0037] For example, the processor can perform a short-time Fourier transform on the preprocessed signal to generate a Mel spectrogram; from the Mel spectrogram, it can calculate the MFCC and its first and second-order differences to form an MFCC feature vector. Simultaneously, it can extract acoustic features of the sound signal, such as the spectral centroid, spectral roll-off point, and zero-crossing rate in the time domain. The processor can combine these multiple acoustic features into a high-dimensional fused feature vector, which serves as the input to the subsequent second model.

[0038] The first computing unit runs a lightweight first model, performs real-time calculations on the preprocessed extracted sound features, and continuously detects anomalies. This first computing unit, such as a microcontroller or a low-power embedded processor, has limited computing resources. The first model is used to detect abnormal signals in the sound signal. For example, when the confidence level (e.g., 0.98) of the first model classifying a sound signal as abnormal is higher than a first threshold (e.g., 0.95), then that sound signal is confirmed as abnormal.

[0039] In some implementations, the first model can be implemented using a lightweight network architecture based on depthwise separable convolutions, decomposing standard convolutions into methods such as depthwise convolutions and pointwise convolutions, reducing the number of parameters and computational cost. It takes the sound features of the input sound signal as input and outputs a scalar value indicating whether the sound signal is an abnormal signal, or a two-dimensional vector indicating the probability of the sound signal being normal or abnormal.

[0040] Encapsulation Unit: When the first model detects a high-confidence anomaly, the encapsulation unit can encapsulate a data packet, which may include: an audio signal segment containing the anomaly signal, Controller Area Network (CAN) bus data in the vehicle (such as vehicle speed, RPM, load, etc.), and a Vehicle Identification Number (VIN) and timestamp.

[0041] The second computing unit operates a high-computing-power-demand second model to determine vehicle faults, including the Cockpit Domain Controller (CDC) and a multi-core System-on-Chips (SoC). The second computing unit has greater computing resources than the first unit. The second model analyzes abnormal signals and related vehicle data to obtain fault location and classification, and determines the vehicle fault based on the fault location and classification.

[0042] In some implementations, the second model can be a deep learning model comprising multiple layers of convolutional neural networks (CNNs). This deep learning model includes multiple independent fully connected classification subnetworks, each of which can be specifically optimized and trained for fault sounds of a component subsystem (such as the engine subsystem, braking subsystem, tire and suspension subsystem, etc.). The input to the second model includes abnormal signals, sound features, and vehicle data, and it automatically learns deep abstract features related to faults in the sound signals.

[0043] For example, the second model inputs sound features into sub-networks, and each sub-network outputs a probability vector representing the probability that the input sound signal belongs to a certain type of fault (such as engine knock, misfire) or normal operation of that component. The second model analyzes the output of each sub-network in parallel and identifies which sub-network(s) gave a high-confidence fault judgment by setting a confidence threshold (e.g., 0.8), thereby determining the sound source of the abnormal signal: if the confidence level of the sub-network analyzing engine subsystem faults for knock fault is 0.92, and the confidence level of the sub-network analyzing brake subsystem faults for brake pad wear fault is 0.95, then the system can determine that the abnormal source is the engine subsystem and the brake subsystem, and output the fault type as engine knock and brake pad wear.

[0044] In some implementations, the vehicle terminal can receive vehicle faults identified by the second model and assess the severity of the fault in conjunction with vehicle signals. For example, the vehicle terminal can assess the severity of the vehicle fault by combining the fault type and vehicle data. For instance, if periodic tire noise is detected when the vehicle is traveling at high speed, the severity of the fault can be determined to be high; if slight engine knocking is detected when the vehicle is traveling at low speed, the severity of the fault can be determined to be low.

[0045] Communication unit: Responsible for transmitting data from the vehicle-mounted terminal. For example, it uploads encapsulated data packets to a cloud server via wireless communication technology.

[0046] 2. Cloud server.

[0047] In this embodiment, the cloud server is responsible for receiving relevant data on vehicle malfunctions, determining incremental parameters of the model based on the received data, and sending them to the corresponding second model for updating. The main units include: a data receiving and storage unit, a data alignment and labeling unit, a model training unit, and a training result distribution unit.

[0048] Data receiving and storage unit: Used to receive data packets uploaded from the vehicle terminal, and to store and manage them. Specifically, the data receiving and storage unit is used to store data uploaded by vehicles, and can build a fault sound database covering multiple vehicle models and operating conditions.

[0049] Data Alignment and Labeling Unit: This unit receives vehicle faults (including precise faulty components and fault types) and repair results confirmed at the repair service terminal. It uses a matching algorithm (e.g., based on VIN codes, timestamps, and fault characteristics) to associate the repair results with corresponding audio signal data packets, generating label tags. It also associates abnormal signals within the data packets with vehicle data to form incremental training samples, used to obtain labeled data for model training.

[0050] Model Training Unit: Based on incremental training samples, the model deployed on the cloud server is incrementally trained using an anti-forgetting strategy. The model structure deployed on the cloud server is consistent with the second model deployed on the in-vehicle terminal. The incremental parameters obtained from training the model deployed on the cloud server can be used to optimize the second model deployed on the in-vehicle terminal. Notably, the training on the model on the cloud server does not update the entire model, but rather generates incremental parameters for specific faults or vehicle models.

[0051] Training result distribution unit: Used to manage the acquired model incremental parameters and push them to eligible vehicles based on the applicability of the model incremental parameters (such as for specific vehicle models or fault modes) to achieve accurate model updates.

[0052] 3. Maintenance service terminal.

[0053] In this embodiment, the maintenance service terminal is used to display the detection results of the vehicle-mounted terminal, receive user feedback on vehicle malfunctions, and send it to a cloud server. The maintenance service terminal can exist in the user's mobile electronic device in the form of a software application, and its main units include a fault display unit and a maintenance feedback unit.

[0054] Fault Display Unit: Used to display warning information. This warning information can be generated by vehicle faults determined by the onboard terminal. For example, after detecting abnormal noises in the tires, a suggestion to perform a dynamic balancing check can be generated and prompted to the driver through flashing dashboard icons and voice announcements. Users can also query this warning information.

[0055] In some implementations, the maintenance service terminal can also recommend the nearest maintenance outlet to the user, and can send the appointment request directly to the service system of the outlet, and push early warning information, historical health trend charts and maintenance suggestions, etc.

[0056] In some implementations, the maintenance service terminal can also predict the remaining service life of the component or the time when the next failure may occur based on the historical failure frequency and severity trends of the same component in the vehicle, combined with mileage data, and incorporate maintenance recommendations into the early warning information.

[0057] The maintenance feedback unit is used to obtain the final confirmed cause of the vehicle malfunction and the replaced parts after the actual inspection and repair. Simultaneously, the maintenance feedback unit can upload the confirmed maintenance results to the cloud server. The cloud server uses these results to provide annotation labels for the data alignment and annotation unit, thereby optimizing the model.

[0058] In the embodiments of this application, Figure 1 The illustrated vehicle-mounted audio anomaly diagnostic system uses an in-vehicle terminal, a cloud server, and a maintenance service terminal as an example. In practical applications, this system can have more or fewer devices. For instance, the maintenance service terminal can be integrated into the in-vehicle terminal, with the in-vehicle terminal executing the methods originally performed by the maintenance service terminal. In this case, the vehicle-mounted audio anomaly diagnostic system only requires the in-vehicle terminal and the cloud server. This application does not limit the specific types of devices included in the vehicle-mounted audio anomaly diagnostic system. It is understood that the functional division between the modules illustrated in this application is merely illustrative and does not constitute a functional limitation on the vehicle-mounted audio anomaly diagnostic system. In other embodiments of this application, the vehicle-mounted audio anomaly diagnostic system may also employ different modules or combinations of multiple modules to implement the functions of the system.

[0059] Example 2 Figure 2 This is a flowchart illustrating a method for diagnosing abnormal vehicle audio conditions provided in an embodiment of this application. Figure 2 As shown, the vehicle-mounted sound anomaly diagnosis method is applied to, for example... Figure 1 The in-vehicle audio anomaly diagnostic system shown includes the following steps: S101. The vehicle-mounted terminal performs sound signal acquisition, preprocessing, and sound feature extraction.

[0060] In this embodiment, the vehicle-mounted terminal collects sound signals from multiple component areas as raw data for subsequent analysis. Specifically, the vehicle-mounted terminal can control one or more microphone arrays distributed across different key parts of the vehicle to perform synchronous sampling. Simultaneously, it can acquire and correlate vehicle status parameters in real time via the vehicle bus, including but not limited to: engine speed, vehicle speed, gear position, brake pedal status, and steering angle.

[0061] In this embodiment, audio streams from different sound source regions (such as the engine compartment, left front wheel arch, right front wheel arch, etc.) can be packaged with timestamps and vehicle status parameters to form a raw dataset including time and vehicle data. The method provided in this embodiment can be set to run continuously or automatically start when the vehicle starts or the vehicle speed exceeds a set threshold (such as 20km / h) to ensure coverage of major operating conditions.

[0062] In this embodiment of the application, the sound signal is preprocessed and sound features used to characterize vehicle faults are extracted to ensure the accuracy of vehicle fault diagnosis.

[0063] For example, preprocessing may include framing the continuous audio stream at fixed intervals (e.g., 20 to 40 milliseconds) and processing it using a Hamming window function to reduce spectral leakage. Preprocessing may also employ adaptive filtering algorithms. Using mixed background noise collected by a reference microphone inside the vehicle compartment as a reference signal, interference such as wind noise and road noise is filtered out from target microphone signals from the engine compartment, chassis, etc. Furthermore, based on the fault frequency range of the target's detection components (e.g., engine, brakes, tires, suspension, etc.), parallel digital bandpass filters can be applied to initially separate the signal frequency bands of different components.

[0064] In this embodiment, the extraction of sound features from a sound signal may include: performing a short-time Fourier transform on each frame of the denoised signal to convert it into a time-spectrum graph, thus preserving both time and frequency information; calculating the MFCC of each frame's spectrum and calculating the first and second-order differences to form the features of the spectral envelope. Sound features may also include simultaneously extracting statistical features such as the spectral centroid, spectral bandwidth, spectral roll-off point, time-domain zero-crossing rate, and short-time energy. Concatenating the feature vectors calculated from each frequency band signal can form a high-dimensional fused feature vector. Simultaneously, this vector is standardized to eliminate dimensional differences and accelerate model convergence.

[0065] S102. Anomaly detection is performed using the first model.

[0066] In this embodiment of the application, a first model (such as a lightweight model) is deployed in a first computing unit (such as an in-vehicle edge computing unit) to detect and acquire sound features and perform calculations to determine the confidence score of the corresponding sound signal as an abnormal signal.

[0067] For example, the first calculation unit can preset a confidence threshold (e.g., 0.95). When the sound signal is normal, if the confidence level is lower than this threshold, the system will not trigger subsequent steps. When the first model calculates that the confidence level of the abnormal signal (e.g., 0.98) is higher than the preset confidence threshold, it determines that the corresponding sound signal is an abnormal signal, and the system executes subsequent steps.

[0068] S103. Save abnormal sounds and vehicle data.

[0069] In this embodiment, the vehicle terminal stores the audio signal within the time window of the abnormal signal and acquires vehicle data (such as vehicle speed, RPM, load, temperature, etc.) at the same time through a CAN bus. The vehicle terminal can package the audio signal, vehicle data, identification (such as VIN code), and timestamp to form structured diagnostic data.

[0070] In this embodiment, the vehicle terminal can upload diagnostic data to a cloud server via a communication unit to generate incremental training samples.

[0071] S104. Determine vehicle malfunctions using the second model and generate early warning information.

[0072] In this embodiment, the second computing unit is located in the vehicle terminal, and the second model in the second computing unit performs in-depth analysis of the diagnostic data to determine the vehicle fault corresponding to the abnormal signal.

[0073] In some implementations, the second computing unit may also be located on a cloud server, where a second model performs in-depth analysis of the diagnostic data and returns the vehicle faults corresponding to the abnormal signals to the vehicle terminal.

[0074] In this embodiment of the application, the second model includes a multi-layer CNN deep learning model for identifying and classifying anomalous signals in at least two different vehicle component subsystems.

[0075] For example, abnormal signals and their corresponding sound features are input into a second model, which identifies and determines the location of the sound source in parallel. This includes extracting general features of vehicle faults through shared convolutional layers. These general features can be fed into multiple parallel sub-networks, including an engine sub-network, a braking system sub-network, a tire sub-network, and a suspension sub-network.

[0076] For example, the engine sub-network is used to output the probability of engine fault types such as knocking, cylinder knocking, belt noise, and normal; the braking system sub-network is used to output the probability of braking system fault types such as brake pad wear squealing, caliper reset noise, and normal; the tire sub-network is used to output the probability of tire fault types such as abnormal tire noise, tread groove resonance, and normal; and the suspension sub-network is used to output the probability of suspension fault types such as shock absorber oil leakage noise, bushing aging, and normal.

[0077] In this embodiment, the second model can also have a pre-set confidence threshold. When the vehicle fault probability output by the sub-network exceeds the confidence threshold, it is determined that the component has a corresponding type of anomaly, thereby determining the vehicle's fault location and fault classification. For example, the fault type determined for the braking system is excessive wear of the left front brake pads, with a corresponding confidence level of 0.93; the fault type determined for the engine is normal, with a corresponding confidence level of 0.96, etc. The second model determines structured diagnostic results, which can be warning information including maintenance suggestions.

[0078] In this embodiment, the vehicle terminal can perform fault prediction on component areas based on historical diagnostic data and vehicle data, including conducting a comprehensive risk assessment and generating fault prediction results.

[0079] For example, based on the historical abnormal trends of components, corresponding predictive information can be added, such as predicting that the vehicle's remaining safe mileage is approximately 1500 kilometers. The fault prediction results can generate specific guidance such as: suggesting immediate slowing down to avoid sudden braking; or recommending key checks during maintenance.

[0080] For example, when brake pad wear is detected, if the current vehicle speed is high, the risk level of the vehicle can be determined to be emergency; if the vehicle speed is low, the risk level can be downgraded to high.

[0081] For example, when a slight engine knock is detected, if the engine is under high load, the risk level can be determined as medium; if the vehicle speed is low, the risk level can be downgraded to low.

[0082] S105. Generate fault prediction results.

[0083] In this embodiment of the application, the vehicle terminal can include the fault prediction result in the warning information, send the warning information to the maintenance service terminal, and the maintenance service terminal can output decision information that can be understood by the user and the system.

[0084] In this embodiment of the application, the maintenance service terminal can display warning information to the user based on the user's visual and auditory senses. For example, when the warning information includes a high-risk warning (such as the detection of severe wear on the left front brake pad), the current vehicle speed is high. A warning card can be displayed on the instrument panel or central control screen with a continuously flashing red icon to alert the user to the vehicle's fault information. Alternatively, the vehicle's audio system can be used to broadcast key warning content in synthesized voice to alert the user to the vehicle's fault information, remind the user to drive cautiously, avoid emergency braking, and suggest that the user go to a repair shop for inspection immediately.

[0085] S106. Obtain the repair results.

[0086] In this embodiment, vehicle repair personnel can input or confirm repair results (e.g., the left front wheel bearing was damaged and has been replaced) based on the actual repair situation. For example, after the vehicle completes repairs based on the warning information, the repair technician can confirm the accuracy of the diagnosis through the system or the vehicle owner through a mobile app, including confirming whether the warning information is a correct diagnosis or a false alarm.

[0087] The repair service terminal can receive the repair results of vehicle malfunctions and upload the repair results to the cloud server.

[0088] S107. Incremental training samples are generated by the cloud server and the model is trained.

[0089] In this embodiment, the cloud server can generate incremental training samples from abnormal signals, vehicle data, and repair results, providing a data foundation for the retraining of the second model. For example, the cloud server obtains repair results and diagnostic data, and based on information such as identification identifiers (e.g., VIN codes) and timestamps in the diagnostic data, associates and binds the repair results with the previously received diagnostic data of the corresponding vehicle, generating incremental training samples.

[0090] In this embodiment, the incremental training samples generated by the cloud server can be stored in a database. When the incremental training samples of a specific type of fault accumulate to a certain scale, or when a specified training period is reached, the cloud server can perform model training tasks. For example, the cloud server includes a third model, which has the same structure and model parameters as the second model. The cloud server can use the incremental training samples to periodically perform incremental learning or retraining on the third model to optimize the model parameters.

[0091] In this embodiment, the cloud server can employ an anti-forgetting strategy to train the third model. The anti-forgetting strategy is used for the incremental parameters of the trained model (such as the adapter network). The training objective of the cloud server is to maintain the model's ability to recognize previous vehicle faults while training the model to identify new faults, through algorithmic constraints (such as fixing model weights and constraining the update magnitude of parameters).

[0092] In this embodiment, the cloud server trains the third model to obtain incremental model parameters, which can be used to optimize and update the second model.

[0093] S108. Identify the update target and update the second model.

[0094] In this embodiment, the cloud server can analyze the characteristics of the incremental model parameters (such as specific faults for a specific vehicle model), filter out the target vehicle queue that requires the incremental model parameters from a pre-stored vehicle database, and distribute them to the on-board terminals in the target vehicle queue via over-the-air (OTA) technology. The on-board terminals can automatically complete the model replacement, thereby achieving continuous upgrading of diagnostic capabilities.

[0095] In this embodiment, after receiving an update command, the vehicle-mounted terminal checks whether preset safety conditions (such as the vehicle being parked, turned off, and charging, and a stable network connection) are met. If the safety conditions are met, the vehicle-mounted terminal can perform an update operation, fusing the incremental model parameters into the second model of the vehicle-mounted terminal to enhance the accuracy of vehicle fault identification and detection.

[0096] By cyclically executing steps S101 to S108, this method can not only detect abnormal sound signals, but also accurately locate the fault source and identify the fault mode; provide maintenance suggestions and early warning of vehicle faults by combining historical data; and improve the system's accuracy in identifying and detecting vehicle faults by using the incremental model parameters returned by the cloud server.

[0097] It should be understood that, as mentioned above Figure 2 The steps in the flowcharts are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are performed; they can be executed in other orders. Furthermore, as mentioned above... Figure 2The flowchart may include at least some steps or stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0098] Figure 3 This is a timing diagram of an in-vehicle terminal interaction provided in an embodiment of this application. For example... Figure 3 As shown, the vehicle-mounted terminal can issue a command to collect sound via the vehicle-mounted microphone, or be set to run automatically when the vehicle starts. Specifically, the vehicle-mounted terminal performs a self-check to confirm that the hardware status of each microphone array, communication system, etc., is normal, and loads the pre-trained second model. Simultaneously, it establishes a communication connection with the cloud server and checks for available incremental model parameters to update the second model.

[0099] The vehicle-mounted terminal performs the following steps in sequence: S1. Signal acquisition and transmission.

[0100] After the vehicle terminal is initialized, the microphone arrays located in key parts of the vehicle begin to work, including collecting the operating sounds of the powertrain such as the engine and transmission; the braking, suspension and tire noise of the wheels; and the environmental noise inside the vehicle, for subsequent noise reduction processing.

[0101] The vehicle terminal can acquire vehicle status parameters (such as engine speed, vehicle speed, gear, brake pedal status, etc.) in real time through a transmission line (such as CAN bus), and time-align these parameters with the audio stream to form a data packet containing time and vehicle data, which is then sent to the signal processing module for signal preprocessing.

[0102] S2. Signal preprocessing.

[0103] In some implementations, the signal processing module performs preprocessing such as frame windowing, active noise reduction, and frequency band separation, and then sends the processed audio signal to the feature extraction and classification module.

[0104] For example, frame-by-frame windowing involves shifting the continuous audio stream into frames and applying a Hamming window to each frame to reduce spectral leakage; active noise reduction employs an adaptive filtering algorithm based on Least Mean Squares (LMS), using the signal collected by the in-vehicle reference microphone as a noise reference to filter out background noise from the microphone signals in the engine compartment and chassis; and frequency band separation uses a digital bandpass filter to perform preliminary frequency band separation on the noise-reduced signal based on the fault characteristic frequency bands of different components such as the engine, brakes, tires, and suspension, in order to enhance the target signal.

[0105] In this way, the preprocessed audio signal can be used for subsequent audio feature extraction.

[0106] S3. Extract features and classify them.

[0107] In some implementations, the vehicle terminal obtains sound features from the purified sound signal through a feature extraction and classification module, such as MFCC, time-domain features (such as zero-crossing rate and short-time energy) and frequency-domain features (such as spectral centroid and spectral roll-off point), and combines the above features into a high-dimensional feature vector and performs standardization processing.

[0108] In this embodiment of the application, sound features (such as feature vectors) are input into a loaded second model to determine the source of the sound and any abnormal signals therein.

[0109] For example, the second model extracts high-level abstract features and outputs the failure probability distribution of each component through its sub-networks (for the engine, braking system, tires, and suspension system, respectively).

[0110] For example, the system sets a confidence threshold (e.g., 0.8). If the output probability of a certain sub-network exceeds the threshold, the system determines that the component has a corresponding fault. For example, if the sub-network for the braking system outputs a probability of wear on the left front brake pads of 0.92, and the normal output probabilities of other sub-networks are all higher than 0.80, then the system determines that the fault sound originates from the braking subsystem, specifically from wear on the left front brake pads.

[0111] S4. Fault diagnosis and repair suggestions.

[0112] In this embodiment, the vehicle terminal transmits warning information to the fault diagnosis and warning module, which then assesses the severity level based on the current vehicle operating conditions (such as vehicle speed and brake pedal status). For example, if brake pad wear is detected at high speed, the risk level is rated as high, while at low speed the risk level is rated as medium.

[0113] In some implementations, the fault diagnosis and early warning module can call a historical diagnostic database to analyze the frequency and intensity trends of abnormal signals of faulty components (such as the left front brake pad) over a period of time, and estimate the remaining service life or fault development curve of the component through a fault prediction model.

[0114] For example, the vehicle terminal can predict the expected remaining safe driving range of the brake pad based on historical data.

[0115] S5. Push repair suggestions.

[0116] In this embodiment, the vehicle terminal can generate specific action suggestions based on early warning information and predictive analysis through the maintenance service module.

[0117] For example, based on the vehicle's current location, the in-vehicle terminal obtains relevant information about nearby cooperative repair shops from the cloud server (including distance, user ratings, service prices, appointment availability, etc.), selects one or more of the best shops, and integrates the relevant information into the recommendations.

[0118] In this embodiment, the vehicle-mounted terminal displays warning information in a pop-up window via a vehicle-mounted human-machine interface (such as an instrument panel, central control screen, microphone, etc.), and simultaneously broadcasts key information via voice synthesis. Additionally, network warning information and maintenance suggestions can be pushed to the vehicle owner's linked mobile app, where a detailed report is displayed.

[0119] In this embodiment, after vehicle fault repair is completed, the vehicle owner or repair technician can provide feedback on the diagnostic accuracy via an app or repair shop system. The onboard terminal will upload this feedback and the corresponding diagnostic data to a cloud server for model parameter optimization, thereby continuously improving diagnostic accuracy.

[0120] After this diagnostic process is completed, the vehicle terminal can continue to the next round of data collection and diagnostic cycle according to the settings.

[0121] Example 3 Based on the method flow for diagnosing in-vehicle sound anomalies described in Embodiment 2, the implementation of this application will be illustrated below using a vehicle fault scenario involving brake pad failure. For example, after a period of driving, some of a batch of multiple vehicles exhibit abnormal signals indicating brake pad friction.

[0122] In this embodiment of the application, the vehicle terminal starts working after the vehicle is started. The initial sound signal is acquired by the sound signal acquisition unit (such as the vehicle microphone), and then preprocessed (such as frame windowing, active noise reduction, frequency band separation, etc.) and feature extracted (such as calculating MFCC) by the signal processing module.

[0123] In this embodiment, a first model deployed on a first computing unit (such as a microcontroller unit, MCU) continuously detects sound signals and their corresponding sound features. The first model quickly calculates the confidence level that the sound signal is an abnormal signal based on the sound features.

[0124] In this embodiment, the confidence level calculated by the first model during normal driving time is lower than a preset threshold. When the vehicle exhibits the aforementioned abnormal signal of brake pad friction, the confidence level calculated by the first model exceeds the threshold, and the vehicle terminal can save the sound signal within a time window (e.g., 2 seconds) before and after the detection of the abnormal signal. Simultaneously, vehicle data within this time period is synchronously acquired and saved via the CAN bus, including but not limited to vehicle speed, brake pedal status, wheel speed, and vehicle load. This data can be packaged into a data packet with a timestamp and VIN.

[0125] In this embodiment, the data packet is sent to a second computing unit, which has stronger computing power than the first computing unit, and a cloud server. The second model deployed in the second computing unit is a deep learning model, used for time-frequency analysis and feature extraction of the abnormal signals determined by the first model, and combined with vehicle data analysis to identify vehicle faults, such as suspected abnormal brake pad wear.

[0126] In this embodiment, the vehicle-mounted terminal can generate warning information that includes descriptions of abnormal signals and sound characteristics, vehicle fault classification, associated vehicle data, and confidence levels. This report is uploaded to a cloud server via a wireless network and simultaneously displayed on the maintenance service terminal. For example, the vehicle's display interface may show a message such as "A potential abnormal braking system sound has been detected; it is recommended to check it during the next maintenance."

[0127] In this embodiment, the vehicle undergoes maintenance based on the prompt information, and the maintenance technician can retrieve the warning information stored on the cloud server based on the vehicle's VIN code in the maintenance service terminal.

[0128] After actual inspection, the repair technician confirmed that the cause of the fault was uneven brake pad material leading to localized abnormal wear. The technician then entered the repair results (e.g., the faulty component of the vehicle is the front left brake, and the corresponding vehicle fault type is abnormal wear) into the repair service terminal and uploaded the repair results to the cloud server.

[0129] In this embodiment, the cloud server can generate incremental training samples by matching and binding the vehicle's VIN code, timestamp, and corresponding fault characteristics with the repair results input by the repair technician, and previously uploaded audio signals and vehicle data from the in-vehicle terminal. The label for these incremental training samples is abnormal brake pad wear.

[0130] In this embodiment, the cloud server trains and optimizes the model based on incremental training samples. The cloud server includes a third model, which has the same structure and model parameters as the second model. The cloud server can periodically perform incremental learning or retraining on the third model using incremental training samples to optimize the model parameters.

[0131] In this approach, the cloud server does not need to retrain the entire third model; instead, it trains an adapter inserted into the existing third model. The adapter is a neural network module with a small number of parameters.

[0132] For example, the cloud server uses incremental training samples from different vehicles for training. During training on the cloud server, the update magnitude of the constraint parameters of the third model only updates the parameters of the adapter module. At the same time, a weight solidification algorithm is used for the weights of the third model, which reduces or fixes the model weights by introducing a parameter protection penalty term for important old knowledge (corresponding to other learned faults) into the loss function, thereby preventing the forgetting of old fault features when training new fault features.

[0133] In this embodiment, the adapter parameters obtained by the cloud server after training are the model increment parameters. These model increment parameters are specifically designed to enhance the second model's ability to identify vehicle faults such as abnormal brake pad wear.

[0134] In this embodiment, the cloud server needs to analyze the characteristics of the incremental parameters of the model to determine the applicable vehicle models for updating the second model. The cloud server identifies vehicles that meet the conditions (e.g., using the same brake pads) in the vehicle model database and pushes the incremental parameters of the model in a targeted manner through technologies such as OTA (Over-The-Air) rather than broadcasting them across the entire network, thereby saving network transmission resources.

[0135] In this embodiment, after detecting an available update, the vehicle's onboard terminal needs to check whether preset security conditions are met, such as the vehicle being parked and turned off, connected to a charging station, or in a Wi-Fi environment. If the update fails or verification fails, the system can automatically roll back to a stable version to ensure driving safety is not affected.

[0136] In this embodiment, the second model of the vehicle terminal can acquire the ability to identify the vehicle fault (i.e., abnormal wear of brake pads) after a successful update, without affecting the second model's ability to detect the original vehicle fault.

[0137] Based on the above description of vehicle fault scenarios involving brake pad failure, the in-vehicle sound anomaly diagnosis method provided in this application can iterate and evolve the model based on feedback from repair results. Based on feedback from more cases in the vehicle, the system can continuously optimize the second model, thereby achieving faster and more accurate vehicle fault identification.

[0138] Example 4 Figure 4 This is a schematic diagram of a diagnostic system provided in an embodiment of this application. Figure 4 As shown, the diagnostic system 400 specifically includes the following devices and modules: The vehicle-mounted terminal 410 includes a sound acquisition module 411, a signal processing module 412, a fault identification module 413, and an interaction module 414, used to detect sound signals to determine vehicle faults. Specifically, the sound acquisition module 411 acquires sound signals from multiple component areas during vehicle operation; the signal processing module 412 preprocesses the acquired sound signals and extracts sound features; the fault identification module 413 inputs the sound features into a pre-trained second model to identify and classify abnormal signals in at least two different vehicle component subsystems, and generates warning information containing the faulty component, fault type, and repair suggestions based on the identification results; and the interaction module 414 outputs the warning information on the vehicle and synchronizes relevant data to a cloud server for recording and analysis.

[0139] The maintenance service terminal 420 includes a fault display module 421 and a maintenance feedback module 422, used to display vehicle faults and input and upload maintenance results. Specifically, the fault display module 421 displays warning information obtained by the fault identification module 413; the maintenance feedback module 422 obtains the maintenance results of the vehicle faults and feeds them back to the cloud server 430.

[0140] The cloud server 430 includes a data annotation module 431, an incremental learning and training module 432, and a model distribution and management module 433, which are used to achieve data closure and continuous model evolution. Specifically, the data annotation module 431 generates incremental training samples; the incremental learning and training module 432 receives feedback from users or maintenance personnel on warning information, uses the feedback results as annotation labels, and obtains incremental optimization parameters for optimizing the second model; the model distribution and management module 433 identifies the specific vehicles requiring incremental training parameters, which can be obtained through filtering the vehicle database in the cloud server 430.

[0141] It is understood that the functional division between the modules illustrated in the embodiments of this application is merely illustrative and does not constitute a limitation on the functionality of the diagnostic system 400. In other embodiments of this application, the diagnostic system 400 may also employ different modules than those in the above embodiments, or a combination of multiple modules, to implement the functions of the diagnostic system 400.

[0142] Figure 5 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application. The computer device 500 may include, for example: Figure 1 The in-vehicle sound anomaly diagnostic system shown and such Figure 4 The diagnostic system 400 is shown. (For example...) Figure 5As shown, the computer device 500 includes: a processor 501, a memory 502, a communication module 504, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the aforementioned... Figures 2-3 The execution steps are shown. For example, the computer program 503 described above can be divided into one or more units / modules, which are stored in the memory 502 and executed by the processor 501 to complete this application.

[0143] The aforementioned one or more units / modules may be a series of computer program instruction segments capable of performing a specific function. These instruction segments describe the execution process of the aforementioned computer program 503 within the aforementioned computer device 500. For example, the aforementioned computer program 503 may be used to perform actions such as... Figure 2 The specific functions or mechanisms of the methods shown in steps S101-S108 have been described in the above embodiments and will not be repeated here.

[0144] Those skilled in the art will understand that Figure 5 This is merely an example of computer device 500 and does not constitute a limitation on computer device 500. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device 500 described above may also include input / output devices, network access devices, buses, etc.

[0145] The processor 501 mentioned above can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0146] In some embodiments, the processor 501 may include one or more interfaces. Interfaces may include I2C, I2S, PCM, URAT, MIPI, GPIO, OBD, and / or USB interfaces, etc. It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the computer device 500. In other embodiments of this application, the computer device 500 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0147] In some embodiments, the computer device 500 can connect internal devices and modules through one or more interfaces. The aforementioned memory 502 can be an internal storage unit of the computer device 500, such as a hard disk or RAM. The aforementioned memory 502 can also include both internal storage units and external storage devices. The aforementioned memory 502 is used to store the aforementioned computer program and other programs and data required by the computer device 500. The aforementioned memory 502 can also be used to temporarily store data that has been output or will be output.

[0148] Communication module 504 can provide solutions for wireless communication applications on computer device 500, including Wireless Local Area Network (WLAN), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR). Communication module 504 can be one or more devices integrating at least one communication processing module. The communication module receives electromagnetic waves via an antenna, demodulates and filters the electromagnetic wave signals, and sends the processed signals to processor 501. Communication module 504 can also receive signals to be transmitted from processor 501, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above equipment can be divided into different functional units or modules to complete all or part of the functions described above.

[0150] The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of software functional units.

[0151] In the embodiments of this application, the specific names of each functional unit and module are only for easy distinction and are not intended to limit the scope of protection of this application. It should be understood that each step in the above-described method embodiments provided in this application can be completed by the integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0152] This application also provides a computer program product, which includes: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the vehicle audio anomaly diagnosis method described in the above embodiments.

[0153] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0154] In the embodiments of this application, it can be implemented entirely or partially by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented entirely or partially in the form of a computer program product.

[0155] The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0156] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed by the computer device in any of the foregoing embodiments.

[0157] Figure 6 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. For example... Figure 6 As shown, the computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0158] The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Universal Optical Discs, DVDs), or semiconductor media (e.g., solid-state drives, SSDs), etc.

[0159] Those skilled in the art will understand that implementing all or part of the processes in the foregoing embodiments can be accomplished by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the foregoing method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or RAM, magnetic disks, or optical disks.

[0160] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. A method for diagnosing abnormal vehicle audio, characterized in that, When applied to in-vehicle terminals, the following steps are included: The system collects sound signals from multiple component areas of the vehicle, performs preprocessing and feature extraction on the sound signals, and uses a first model deployed in the first computing unit to perform anomaly detection. The component areas include the engine area, brake area, tire area, and suspension area. The first model detects an abnormal signal in the sound signal, saves the abnormal signal and the vehicle data associated with the abnormal signal, and uploads the abnormal signal and the vehicle data to the cloud server; The vehicle fault corresponding to the abnormal signal is determined by the second model deployed in the second computing unit, and a warning information is generated. The computing power of the second computing unit is greater than that of the first computing unit. Output the warning information, obtain the repair results of the vehicle malfunction, and upload the repair results to the cloud server; The incremental parameters of the model, trained by the cloud server based on the abnormal signal, the vehicle data, and the maintenance results, are obtained. When the vehicle meets the preset safety conditions, the incremental parameters of the model are loaded to update the second model.

2. The method according to claim 1, characterized in that, The step of determining the vehicle fault corresponding to the abnormal signal through a second model deployed in the second computing unit specifically includes: The abnormal signal and vehicle data are analyzed using the second model to obtain the vehicle's fault location and fault classification, and the vehicle fault is determined based on the fault location and fault classification.

3. The method according to claim 1, characterized in that, After uploading the repair results to the cloud server, the method further includes: The cloud server uses the repair results as labels and associates the abnormal signals with the vehicle data to form incremental training samples; The cloud server trains the incremental parameters of the model based on the incremental training samples and then sends the incremental parameters of the model to the vehicle.

4. The method according to claim 3, characterized in that, The step of sending the model incremental parameters to the vehicle specifically includes: The cloud server analyzes the characteristics of the incremental training samples to determine the applicable scope of the incremental training samples, which includes vehicle models, configurations, and fault modes. The trained incremental parameters of the model are distributed to the vehicles within the applicable range.

5. The method according to claim 3, characterized in that, During the process of training the incremental parameters of the model on the cloud server based on the incremental training samples, the cloud server adopts an anti-forgetting strategy to maintain the ability to identify known fault categories. The anti-forgetting strategy is achieved by fixing the model weights and constraining the update magnitude of the parameters.

6. The method according to claim 1, characterized in that, In the process of generating the early warning information, the method further includes: Based on historical diagnostic data and the vehicle data, fault prediction is performed on the component area, and the generated fault prediction results are included in the warning information.

7. The method according to claim 1, characterized in that, The preset safety conditions include: the vehicle is in a parked state, a charging state, or an idle state of the vehicle network.

8. A vehicle-mounted sound anomaly diagnostic system, characterized in that, The system for performing the method as described in any one of claims 1 to 7 includes: The vehicle-mounted terminal includes a sound acquisition module, a signal processing module, a fault identification module, and an interaction module, which are used to detect sound signals to determine vehicle faults; The maintenance service terminal includes a fault display module and a maintenance feedback module, used to display the vehicle faults, record and upload maintenance results; The cloud server includes a data annotation module, an incremental learning and training module, and a model distribution and management module, which are used to achieve data closure and continuous model evolution.

9. A computer device, characterized in that, The device includes one or more memories and one or more processors; the memories are coupled to the one or more processors, the memories are used to store computer program code, the computer program code including computer instructions, and the one or more processors invoke the computer instructions to cause the computer device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the method of any one of claims 1 to 7.

Images