Automobile central control electronic control system test platform and method

Abstract

The invention discloses an automobile central control electronic control system test platform and method, particularly relates to the technical field of automobile central control electronic control, and is used for solving the problem of inaccurate analysis of automobile central control electronic control. According to the method, data redundancy is reduced through acquisition frequency optimization, tagging processing is performed, data source management is performed, after data synchronization, deep feature extraction is performed on different types of data by adopting methods such as polynomial fitting, a convolutional neural network and a Mel-frequency cepstral coefficient, representativeness and integrity of data features are ensured, and the accuracy of data synchronization is improved. And carrying out nonlinear standardization and principal component analysis dimensionality reduction on the extracted feature data, unifying the extracted feature data to a comprehensive feature space, enhancing the consistency and relevance among multi-modal features, carrying out integrated learning optimization fusion, realizing data consistency verification based on an association rule and an anomaly detection model, triggering an alarm according to the anomaly degree, effectively identifying potential faults, and improving the fault diagnosis accuracy. The testing accuracy and stability of the automobile central control electronic control system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile central control electronic control, and more specifically, to an automobile central control electronic control system test platform and method. Background Art

[0002] The automobile central control electronic control system is one of the core electronic modules in the car. It is mainly used to manage and control various functions such as the on-board infotainment system, air-conditioning system, navigation system, window control, seat adjustment, door lock control, etc., to improve the comfort and convenience of the driver and passengers. With the development of automobile intelligence and networking, the functions of the automobile central control system are becoming increasingly rich, and the interaction methods are becoming more diversified, including touch screens, voice control, gesture recognition and other intelligent control methods. Modern central control systems must not only be able to process complex multimedia content, but also work in conjunction with internal and external sensors and actuators in the car, and exchange data with other modules through the in-vehicle local area network (CAN bus, LIN bus) to ensure the overall coordination and safety of the vehicle's operation.

[0003] Deficiencies in existing technologies: During the testing of automobile central control systems, the lack of multi-mode data processing has become a significant problem. The central control system involves multiple data types such as structured data (such as sensor readings, equipment status), unstructured data (such as audio, video), and time series data (such as real-time temperature, speed change curves). The sources and formats of various types of data vary greatly. Existing test systems can usually only process a single data mode separately, making it difficult to perform unified analysis and integration. This defect in unified processing of multi-mode data leads to difficulties in the system in analyzing the correlation between multiple data sources, verifying data consistency, and extracting and fusing information across data modes. It is impossible to fully and accurately restore complex scenarios in actual use and is prone to missing certain potential abnormalities or electronic control system problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a vehicle central control electronic control system test platform and method to solve the problem of inaccurate analysis of the vehicle central control electronic control in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The automobile central control electronic control system test method includes the following steps:

[0007] In the test of the central control electronic control system, data sources from different sources are identified and classified, the data collection frequency of each data source is optimized and outlier detection is performed, and feature recognition and structured management are performed through labeling processing;

[0008] Perform feature extraction on the synchronized data, and use polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method to extract features of structured and unstructured data;

[0009] The extracted feature data are subjected to nonlinear normalization and principal component analysis dimensionality reduction, and unified into the comprehensive feature space to increase the numerical consistency and correlation between multimodal features;

[0010] Use integrated learning methods to optimize multimodal data fusion, verify data consistency based on association rules and anomaly detection models, and trigger corresponding alarm mechanisms according to the degree of anomaly.

[0011] In a preferred embodiment, in the test of the central control electronic control system, data sources from different sources are identified and classified, the data collection frequency of each data source is optimized and outlier detection is performed, and feature recognition and structured management are performed through labeling processing. The specific process is as follows:

[0012] Density-based clustering algorithm clusters feature space vectors according to density similarity, groups data sources and generates labels;

[0013] Define the timestamp sequence of each type of data source, use interpolation to align the time of data sources with different labels, map them into the same time window, and use Lagrange interpolation to complete the time;

[0014] After labeling the data source, different types of data structures and timestamps need to be analyzed;

[0015] In each type of data source, select the initial sampling frequency for testing, monitor the rate of change of the sampled data, determine whether the sampling frequency has oversampling or undersampling, and adjust the collection frequency according to the changes in real-time data.

[0016] In a preferred embodiment, feature extraction is performed on the synchronized data, and polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method are used to extract features of structured and unstructured data. The specific process is as follows:

[0017] Perform feature extraction of structured data, apply polynomial fitting method to sensor readings and vehicle status, optimize through least squares method, and extract trends and features of structured data;

[0018] Use convolutional neural network to extract image features from video data. Set each frame of video image as a matrix, extract image features through multi-layer convolution and pooling operations, and obtain the feature vector of video data.

[0019] For audio data, Mel-frequency cepstral coefficients are used to extract sound features. The audio signal is first framed and windowed, and the time domain signal is converted into a frequency domain signal using fast Fourier transform. The frequency domain signal is mapped to Mel frequency, and the cepstral coefficients are calculated and used to form a feature vector.

[0020] The features extracted from structured data and unstructured data are used as feature data.

[0021] In a preferred embodiment, the extracted feature data is subjected to nonlinear normalization and principal component analysis dimensionality reduction, and unified into a comprehensive feature space to increase the numerical consistency and correlation between multimodal features. The specific process is as follows:

[0022] Define the feature matrix of structured data, mean-center each feature, define the centered data matrix, calculate the covariance matrix of the centered feature matrix, perform eigenvalue decomposition on the covariance matrix, and obtain the eigenvalues ​​and corresponding eigenvectors;

[0023] Use convolutional neural network to extract video frame image features, obtain local features in the image, use convolution kernel to perform convolution operation on the image, obtain convolution output feature map, use ReLU function to perform nonlinear transformation on the convolved feature map, obtain activated feature map, and perform maximum pooling operation on the activated feature map to reduce dimension;

[0024] Divide the audio signal into frames, perform fast Fourier transform on each frame, calculate the power spectrum of each frame, map the power spectrum to the Mel frequency through the Mel filter bank, extract the Mel frequency features of each audio frame, and apply average pooling to the Mel frequency features to obtain the low-dimensional feature vector of the audio;

[0025] Concatenate the structured, unstructured and time series feature extraction results into an initial multimodal feature vector;

[0026] The multimodal autoencoder performs dimensionality reduction and feature mapping;

[0027] To further optimize the feature representation, the comprehensive features generated by the autoencoder are passed into the deep belief network, and in each layer of the restricted Boltzmann machine, the joint probability distribution of the visible layer and the hidden layer is defined;

[0028] The restricted Boltzmann machine of each layer is optimized through greedy layer-by-layer training method, and the comprehensive features are converted into high-level shared feature representation to determine the deep correlation of the data.

[0029] In a preferred embodiment, an integrated learning method is used to optimize multimodal data fusion, and data consistency is verified based on association rules and anomaly detection models, and corresponding alarm mechanisms are triggered according to the degree of anomaly. The specific process is as follows:

[0030] After obtaining the shared feature representation, random forest is used to build an integrated learning model;

[0031] The shared feature representation is input into a decision tree. Each decision tree is trained on a subset of features from a different modality and calculates the importance of each feature in the decision.

[0032] The outputs of the decision trees are fused by voting or weighted averaging to generate the final fusion decision output;

[0033] Integrate structured data and unstructured data in a unified feature space for multimodal data fusion.

[0034] In a preferred implementation, the data consistency is verified based on the association rules and the anomaly detection model, and the corresponding alarm mechanism is triggered according to the anomaly level. The specific steps are as follows:

[0035] Preprocess the fused multimodal data and define the typical response mode of each data source under events, including sudden braking, turning, and acceleration, and discretize the multimodal data into state tags;

[0036] The Apriori algorithm is used to construct event sets and association rules of multimodal data between different data sources;

[0037] The event set includes the state combination of all modal data within the time window, and the association rules represent the conditional probability relationship between events;

[0038] Match the current event pattern with the association rules. If the confidence of the rules matched by the event meets the conditions in the association rules, it means that the data sources are synchronized. If the conditions are not met, calculate the abnormality of the event and mark it as a potential abnormality.

[0039] The anomaly values ​​are divided into three levels of anomaly types, including low-level anomalies, medium-level anomalies and high-level anomalies.

[0040] The automobile central control electronic control system test platform is used to implement the above automobile central control electronic control system test method, including:

[0041] The label classification module is used to identify and classify data sources from different sources in the central control electronic control system test, optimize the data collection frequency and detect outliers for each data source, and perform feature recognition and structured management through label processing;

[0042] The feature extraction module is used to extract features from the synchronized data, using polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method to extract features from structured and unstructured data;

[0043] Dimensionality reduction association module, which is used to perform nonlinear normalization and principal component analysis on the extracted feature data and unify them into the comprehensive feature space to increase the numerical consistency and correlation between multimodal features;

[0044] The verification alarm module is used to optimize multimodal data fusion using integrated learning methods, verify data consistency based on association rules and anomaly detection models, and trigger corresponding alarm mechanisms according to the degree of anomaly.

[0045] Technical effects and advantages of the present invention:

[0046] The present invention reduces data redundancy by optimizing the data collection frequency of each data source, improves data quality through outlier detection, realizes accurate identification and structured management of various data features through labeling processing, effectively unifies the feature expression methods of multi-source data, and after data synchronization is completed, uses polynomial fitting, convolutional neural network and Mel-frequency cepstral coefficients and other methods for different types of data to perform in-depth feature extraction of structured data (such as sensor data) and unstructured data (such as video and audio), ensuring the representativeness and integrity of data features, and uses nonlinear normalization and principal component analysis to reduce the dimension of the extracted feature data, and uniformly maps them to the comprehensive feature space, thereby enhancing the numerical consistency and correlation between multimodal features and improving the reliability of data fusion.

[0047] In addition, the ensemble learning method is used to optimize the fusion of multimodal data, and the data consistency is verified based on association rules and anomaly detection models, which enables accurate classification of system anomalies. The corresponding alarm mechanism is triggered according to the degree of anomaly, which can effectively identify potential faults, improve the accuracy and response speed of system testing, and thus enhance the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The present invention is a flow chart of a method for testing a central electronic control system of an automobile.

[0049] Figure 2 It is a structural schematic diagram of the automobile central control electronic control system test platform of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Example 1: Figure 1As shown, the automobile central control electronic control system test method includes the following steps:

[0052] In the test of the central control electronic control system, data sources from different sources are identified and classified, the data collection frequency of each data source is optimized and outlier detection is performed, and feature recognition and structured management are performed through labeling processing;

[0053] Perform feature extraction on the synchronized data, and use polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method to extract features of structured and unstructured data;

[0054] The extracted feature data are subjected to nonlinear normalization and principal component analysis dimensionality reduction, and unified into the comprehensive feature space to increase the numerical consistency and correlation between multimodal features;

[0055] Use integrated learning methods to optimize multimodal data fusion, verify data consistency based on association rules and anomaly detection models, and trigger corresponding alarm mechanisms according to the degree of anomaly.

[0056] There are many types of data in the automobile central control system test. Identify and classify data from different sources, and unify various types of data through standardization and preprocessing methods. The specific steps are as follows:

[0057] In the initial stage, it is necessary to identify the various data sources in the car's central control electronic system and add a unique label to each type of data source to facilitate data classification in the subsequent processing;

[0058] Use feature space vector X to represent different features in multi-source data: ,in, Represents the nth data feature in various data sources, such as sensor readings, video frame features, audio signals, etc. Through the density-based clustering algorithm, X is clustered according to density similarity, the data sources are divided into several groups and labels are generated : ,in, represents the j-th data source label, Indicates the maximum radius of the data feature distance, is the minimum number of data points in each cluster. This step groups data from different sources and assigns labels , so as to track the characteristics of various types of data in subsequent processing;

[0059] For each type of data source, determine the required sampling frequency range based on its characteristics and importance in the central control system. For example, high-speed changing sensor data (such as acceleration and rotation speed) requires a higher frequency, while the video frame rate and audio sampling rate are adjusted according to the analysis requirements.

[0060] In each type of data source, select the initial sampling frequency for testing to determine whether the frequency can meet the data integrity and accuracy requirements. By monitoring the change rate of the sampled data in real time, determine whether the frequency is over-sampling or under-sampling. For example, if the sensor data fluctuates gently, the sampling frequency can be appropriately reduced; if the change is drastic, the frequency needs to be increased.

[0061] Introduce a dynamic adjustment mechanism to adjust the acquisition frequency according to the changes in real-time data; for example, use an adaptive frequency control algorithm (such as PID control or fuzzy control) to automatically increase the sampling frequency when a dramatic data change is detected, and conversely reduce the sampling frequency when the data change tends to be stable, so as to save storage and computing resources;

[0062] Perform data structure analysis and timestamp synchronization. After labeling the data source, analyze different types of data structures and timestamps to ensure consistency in the time dimension and define the timestamp sequence for each type of data source. , for different labels The data sources under need to be time-aligned by interpolation so that all data sources are mapped to the same time window. Lagrange interpolation is used for time completion and the interpolation function is defined as: , where x represents the interpolation node, It is the characteristic value of the data source. The missing timestamp data is supplemented by interpolation to ensure that different data sources can be aligned in the time dimension to ensure the synchronization of subsequent data processing.

[0063] After data synchronization is completed, key information is extracted from the feature vector of each data source to ensure that the data is concise and representative. Structured data (such as sensor readings, vehicle status parameters, etc.) can be nonlinearly transformed through the feature extraction method of polynomial fitting. Unstructured data such as audio, video, and images need to be extracted through convolutional neural networks (CNN) to extract deep features and standardize the features so that different features can be analyzed under the same dimension. The specific steps are as follows:

[0064] For structured data feature extraction, for structured data such as sensor readings and vehicle status, a polynomial fitting method is applied to identify its trends and important features. Suppose the structured data feature vector is , extract features through the polynomial fitting model P(x), and fit the polynomial using the following formula: ,in, is the fitting parameter, which represents the highest order of the polynomial; it is optimized by the least squares method value, so that the polynomial P(x) can best express the trend and main characteristics of structured data;

[0065] Video data feature extraction: For video data, a convolutional neural network (CNN) is used to extract image features. Each frame of video image is assumed to be a matrix. CNN extracts image features F through multi-layer convolution and pooling operations to obtain a feature vector after dimensionality reduction.

[0066] Audio data feature extraction: For audio data, the Mel-frequency cepstral coefficient (MFCC) is used to extract sound features. The audio signal is first framed and windowed, and then the fast Fourier transform (FFT) is used to convert the time domain signal into a frequency domain signal. Finally, the frequency domain signal is mapped to the Mel frequency, and the cepstral coefficient is calculated to form a feature vector.

[0067] The extracted data features are compared under the same dimension, and nonlinear normalization is applied to process the extracted features. The data vector obtained from feature extraction is , n is the total number of vectors. The goal of standardization is to compress the eigenvalues ​​of XL to a similar range while retaining the numerical distribution characteristics. The data is standardized using a nonlinear compression function: ;

[0068] Logarithmic function , This part is used for nonlinear compression, which adjusts the extreme values ​​of the data to a reasonable range to avoid the influence of outliers on subsequent analysis. The logarithmic transformation introduces nonlinearity, which makes the compression amplitude of large values ​​larger and the compression amplitude of small values ​​smaller, so as to maintain consistency between data of different dimensions;

[0069] is the nonlinear control coefficient, which determines the compression strength of the data. A larger value will increase the difference between the values, while a smaller value will The value will make the data compression smoother. In the automotive central control electronic control system, it is often determined according to the actual distribution of the data. value to ensure moderate data compression;

[0070] Maximum Normalization It is to ensure that the range of data features is between [0, 1] through maximum normalization, so that different features can be compared under the same dimension. Maximum normalization standardizes the maximum value of the data to 1, making the feature values ​​from different sources comparable;

[0071] After the multimodal data is standardized, it needs to be further reduced in dimension to eliminate redundant information and retain the main components of the data. For structured data, principal component analysis is used to project the feature data into a low-dimensional space through linear transformation, retain the main components of the data and remove redundant information.

[0072] Similarly, define the feature matrix of structured data , where m is the number of samples and n is the number of features; first, each feature is mean-centered to define the feature matrix after centering ,in, is the feature mean vector;

[0073] Calculate the covariance matrix of the centered feature matrix: ;

[0074] Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalue and the corresponding eigenvector ;

[0075] Select the eigenvectors corresponding to the largest k eigenvalues ​​to form the projection matrix , project the data into k-dimensional space: ,in, is the data matrix after dimensionality reduction;

[0076] Use convolutional neural network to extract video frame image features, and obtain local features in the image through multi-layer convolution, activation and pooling operations. Suppose the video frame image matrix is , where h is the height, w is the width, and c is the number of channels;

[0077] Using convolution kernel Perform a convolution operation on the image to obtain the convolution output feature map: ,in, and is the height and width of the convolution kernel, i and j are the positions in the image;

[0078] The convolutional feature map is transformed nonlinearly through an activation function (such as ReLU) to obtain the activated feature map A;

[0079] Pool the activated feature map (such as maximum pooling or average pooling) to further reduce the dimension and obtain the pooled feature P, where the pooling window size is p×p (usually 2×2 or 3×3);

[0080] Audio data processing requires converting time domain signals into frequency domain features. Mel-frequency cepstral coefficients (MFCC) are often used to extract sound features in audio. The specific steps are as follows:

[0081] Divide the audio signal A(t) into frames and add a window (such as a Hamming window) to each frame. Suppose each frame is ;

[0082] Perform fast Fourier transform (FFT) on each frame signal to obtain frequency domain representation : ;

[0083] Compute the power spectrum of each frame: ;

[0084] The power spectrum is mapped to the Mel frequency through the Mel filter bank, and the filter is assumed to be , then the Mel frequency energy is: ;

[0085] Mel frequency energy Perform discrete cosine transform to obtain MFCC feature coefficients: , where M is the number of filters and k is the order of the cepstral coefficients;

[0086] Extract the Mel frequency features of each audio frame, and apply average pooling to the Mel frequency features to obtain a low-dimensional feature vector of the audio;

[0087] Feature extraction of time series data can be achieved through frequency domain analysis or wavelet transform to identify periodicity, trend and transient characteristics in the signal;

[0088] Perform frequency domain analysis and apply fast Fourier transform (FFT) to the time series signal S(t) to obtain the frequency domain feature S(f). The frequency domain feature contains the frequency component of the signal and can be used to identify periodic features.

[0089] In the test of automobile central control electronic control system, multimodal data fusion aims to unify the features of structured, unstructured, and time series data into a comprehensive feature space to capture the correlation between different data types. This process relies on the feature mapping and decision fusion algorithm of the multimodal deep learning model for in-depth analysis. The specific steps are as follows:

[0090] First, the feature extraction results from structured, unstructured and time series are spliced ​​into the initial multimodal feature vector. The feature vectors of each modal data are: structured data feature vector , video data feature vector , audio data feature vector , time series data feature vector ;

[0091] Concatenate the features of each mode to form the initial multimodal feature vector :in, is the dimension of the concatenated feature vector;

[0092] The initial multimodal feature vector F has a high dimension, and the feature distribution of different modes is different. In order to unify the feature representation, a multimodal autoencoder is used to reduce the dimension and map the features. The multimodal autoencoder includes an encoder and a decoder, which aims to map the features to a low-dimensional comprehensive feature space. The encoder function is defined as and the decoder function Among them, the comprehensive feature representation Generated by the encoder: ,in, and are the weight matrix and bias term of the encoder respectively, Represents a nonlinear activation function (such as ReLU), which is encoded and passed through the decoder Reconstruct the original eigenvectors: , by minimizing the reconstruction loss Adjust the encoder weight matrix and the decoder weight matrix The model learns the best representation of the low-dimensional comprehensive feature z;

[0093] To further optimize the feature representation, the comprehensive features generated by the autoencoder are passed to the deep belief network (DBN) to further extract representative high-level features. The DBN is composed of multiple stacked restricted Boltzmann machines (RBM) layers to capture nonlinear associations in multimodal data. In each layer of RBM, the joint probability distribution of the visible layer and the hidden layer is defined: , where v and h are the visible layer and the hidden layer respectively, and is the bias term, is the connection weight, Z is the normalization factor;

[0094] By optimizing each layer of RBM through greedy layer-by-layer training, the comprehensive feature z is converted into a high-level shared feature representation as , thereby capturing the deep correlation of the data;

[0095] In obtaining the shared feature representation Finally, an integrated learning method (such as AdaBoost or random forest) is used to further optimize the decision fusion of multimodal features. The specific steps are as follows:

[0096] Integrated learning model construction Input multiple base learners (e.g., in random forest, the base learners are decision trees), each of which is trained on a feature subset of a different modality;

[0097] Perform feature importance calculations. For the random forest model, calculate the importance of each feature in the decision: ,in, is the feature in the tth decision tree The information gain caused by, T is the number of decision trees in the forest;

[0098] The final decision is generated by fusing the outputs of each decision tree by voting or weighted averaging to generate the final fusion decision output: ,in, represents the output of the kth decision tree, is the weight of the decision tree;

[0099] Multimodal data fusion integrates various types of information such as structured data (such as vehicle status) and unstructured data (such as video and audio) into a unified feature space. In this way, the multi-dimensional performance of the car's central control can be analyzed from different angles and levels at the same time, avoiding information bias or omissions that may be caused by relying solely on a single data source. The fused feature space can comprehensively reflect the inherent correlation between various types of data, making the test more comprehensive and in-depth. Through multimodal data fusion, the test of the car's central control system not only obtains a more comprehensive analysis perspective, but also improves the accuracy and sensitivity of system anomaly detection, optimizes user experience evaluation, and brings significant improvements in computing efficiency and model generalization capabilities.

[0100] In the test of automobile central control electronic control system, data consistency verification and anomaly detection are steps to ensure the logical coordination and consistency of multimodal data, so as to systematically verify the consistency of each data source in a specific event and detect potential anomalies. The specific processing steps of anomaly detection based on association rules are as follows:

[0101] Preprocess the fused multimodal data to identify the consistency pattern of the data, define the typical response pattern of each data source under key events (such as sudden braking, turning, acceleration, etc.), and discretize the multimodal data (such as sensor data, audio signals, video frames) into state tags, such as "sudden braking", "acceleration", "audio mutation", etc. The discretized data pattern is: ,in, represents the i-th discretized event pattern, such as a sudden change in sensor readings, a movement change detected in a video frame, etc.

[0102] Association rule mining algorithms (such as the Apriori algorithm) are used to construct logical association rules between different data sources to discover data synchronization relationships in specific events. Define the event set E and association rules R of multimodal data:

[0103] Event Set: ,event Includes the state combination of all modal data within a specific time window, such as the state of sensors, audio, and video during emergency braking;

[0104] Association rules: ,rule Represents the conditional probability relationship between events, such as "if you brake suddenly, the audio signal increases and the video shows that the vehicle shakes";

[0105] The Apriori algorithm generates association rules, and event set combinations that meet the conditions will be identified as normal event synchronization patterns, otherwise they will be marked as potential anomalies;

[0106] Support(e) represents the frequency of an event in all event sets and is used to measure the prevalence of an event.

[0107] Confidence(r) indicates the probability of a rule occurring in all events and is used to measure the reliability of synchronization events from different data sources. By specifying the support and confidence thresholds, relevant rules can be screened out as the basis for consistency verification.

[0108] When a critical event occurs (such as sudden braking), the multimodal data collected in real time is input into the association rule model, the synchronization performance of different data sources is checked, and the current event pattern is matched with the association rules. If the conditions in the association rule are met, the data sources are considered to be synchronized. If the conditions are not met, the abnormality degree A(e) of the event is calculated and marked as a potential abnormality: ,in, Representing event mode The confidence of the matched rule. If the abnormality A(e) is higher than the preset threshold, it is considered a serious abnormality, otherwise it is considered a slight abnormality.

[0109] In anomaly detection, the anomaly value A(e) is used to quantify the synchronization deviation of multimodal data under specific events. According to specific requirements, the anomaly value can be divided into three levels of anomaly types, and each type corresponds to different alarm measures:

[0110] Low-level abnormality (mild abnormality), abnormality degree A(e)<0.3, intermediate abnormality (moderate abnormality), 0.3≤A(e)<0.6, high-level abnormality (serious abnormality), abnormality degree A(e)≥0.6;

[0111] The system sets different alarm mechanisms for different levels of abnormalities in order to provide appropriate response measures:

[0112] Low-level anomaly alarm: Low-level anomalies represent minor inconsistencies and usually do not affect the overall performance of the system. Therefore, there is no need for immediate alarm. The handling measure is that the system records the anomaly in the data log and marks it as a "low-level anomaly" for subsequent analysis. At this level, no real-time alarm is triggered to reduce unnecessary interference;

[0113] For example, when the vehicle turns slightly, the steering wheel angle sensor reading fluctuates slightly, but the video data and audio data fail to synchronously detect the turning state. This slight inconsistency will be marked as a low-level abnormal record.

[0114] Intermediate anomalies point to significant differences in the system, indicating that the multimodal data is not completely consistent but has not yet seriously deviated from the normal state. When an intermediate anomaly is detected, the system will generate a prompt alarm to alert the tester to the event. The system continuously monitors the subsequent performance of the abnormal event in the background. If the intermediate anomaly continues to occur or is frequently related to other abnormal events, it will be upgraded to a high-level abnormal alarm; for example, in an emergency braking event, the sensor data quickly indicates the braking state (such as a sharp increase in brake pressure), but the shaking in the video image is small, and the emergency stop state of the vehicle is not fully detected synchronously. In this case, the degree of abnormality may be in the range of 0.3≤A(e)<0.6, and the system generates a prompt alarm and records the abnormality;

[0115] Advanced anomalies usually point to serious system inconsistencies or potential failures. Multimodal data show significant inconsistency in this event, which may affect system performance or safety. When the anomaly degree A(e) ≥ 0.6, the system immediately triggers an advanced alarm and suspends the current test process. The advanced alarm will prompt the tester to quickly check the system status and start the emergency troubleshooting process. The system generates a complete anomaly report, including the status of each modal data at the time of the anomaly, the anomaly value, and the matching status with the relevant rules;

[0116] For example, during driving, the system detects emergency acceleration, and the engine data and sensors show a significant acceleration state, but the video fails to identify any visual changes of the vehicle moving forward, and there is no change such as an increase in the engine sound in the audio data. This inconsistency may cause A(e) ≥ 0.6, and the system will determine it as a high-level abnormality, immediately trigger an alarm and suspend the test;

[0117] Assuming that in the event of an emergency braking of a vehicle, the system's multimodal data includes the following information: sensor data shows that the brake pressure rises rapidly, indicating an emergency braking state; video data shows that the vehicle is tilting forward, which is consistent with the visual effect of braking; audio data shows that the tire friction sound is detected, which is consistent with the sound characteristics of an emergency braking;

[0118] Case 1 (low-level anomaly): The tire friction sound delay of the audio data lags slightly behind the sensor and video data, resulting in A(e)=0.2, which meets the low-level anomaly threshold. This anomaly is recorded in the log, but no alarm is triggered.

[0119] Case 2 (Intermediate Abnormality): The sensor and video data synchronously indicate the emergency braking state, but the tire friction sound of the audio data is not detected or the volume is insufficient, and the abnormality degree A(e)=0.4. This inconsistency will trigger an intermediate abnormality alarm, reminding the tester to pay attention and conduct subsequent observations;

[0120] Case 3 (Advanced Abnormal): Both the sensor and video data detected an emergency brake, but the audio data did not show any friction sound, and the video data did not capture any obvious tilt angle. This multimodal data performance was seriously inconsistent, with an abnormality degree A(e)=0.7, which met the advanced abnormal alarm conditions. The system will trigger an advanced alarm, suspend the test process, and record the event details to facilitate troubleshooting;

[0121] Each time an exception occurs, the system generates an exception report, which includes the following: event type (such as "sudden braking", "turning", "acceleration", etc.); exception level (low, medium or high); anomaly value (consistency deviation value of the event in different data sources); data source status (consistency deviation value of the event in different data sources); association rule matching status (detection event meets the established association rules);

[0122] These reports are saved in the exception database to facilitate subsequent analysis and problem tracing. Through the hierarchical alarm mechanism, the consistency of multimodal data can be reflected in a timely and accurate manner, and targeted data support can be provided to testers.

[0123] It should be noted that the thresholds involved in the embodiments can be determined according to specific scenarios and requirements.

[0124] The present invention reduces data redundancy by optimizing the data collection frequency of each data source, improves data quality through outlier detection, realizes accurate identification and structured management of various data features through labeling processing, effectively unifies the feature expression methods of multi-source data, and after data synchronization is completed, uses polynomial fitting, convolutional neural network and Mel-frequency cepstral coefficients and other methods for different types of data to perform in-depth feature extraction of structured data (such as sensor data) and unstructured data (such as video and audio), ensuring the representativeness and integrity of data features, and uses nonlinear normalization and principal component analysis to reduce the dimension of the extracted feature data, and uniformly maps them to the comprehensive feature space, thereby enhancing the numerical consistency and correlation between multimodal features and improving the reliability of data fusion.

[0125] In addition, the ensemble learning method is used to optimize the fusion of multimodal data, and the data consistency is verified based on association rules and anomaly detection models, which enables accurate classification of system anomalies. The corresponding alarm mechanism is triggered according to the degree of anomaly, which can effectively identify potential faults, improve the accuracy and response speed of system testing, and thus enhance the stability and reliability of the system.

[0126] Example 2: Automobile central electronic control system test platform, such as Figure 2 As shown, specifically including:

[0127] The label classification module is used to identify and classify data sources from different sources in the central control electronic control system test, optimize the data collection frequency and detect outliers for each data source, and perform feature recognition and structured management through label processing;

[0128] The feature extraction module is used to extract features from the synchronized data, using polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method to extract features from structured and unstructured data;

[0129] Dimensionality reduction association module, which is used to perform nonlinear normalization and principal component analysis on the extracted feature data and unify them into the comprehensive feature space to increase the numerical consistency and correlation between multimodal features;

[0130] The verification alarm module is used to optimize multimodal data fusion using integrated learning methods, verify data consistency based on association rules and anomaly detection models, and trigger corresponding alarm mechanisms according to the degree of anomaly.

[0131] The above formulas are all dimensionless and calculated numerically. Specific dimension removal can be achieved by various means such as standardization, which will not be elaborated here. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0132] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, an ATA hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state ATA hard disk.

[0133] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0134] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0135] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for testing a central electronic control system of an automobile, characterized in that: The steps include: In the test of the central control electronic control system, data sources from different sources are identified and classified, the data collection frequency of each data source is optimized and outlier detection is performed, and feature recognition and structured management are performed through labeling processing; Perform feature extraction on the synchronized data, and use polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method to extract features of structured and unstructured data; The extracted feature data are subjected to nonlinear normalization and principal component analysis dimensionality reduction, and unified into the comprehensive feature space to increase the numerical consistency and correlation between multimodal features; Use integrated learning methods to optimize multimodal data fusion, verify data consistency based on association rules and anomaly detection models, and trigger corresponding alarm mechanisms according to the degree of anomaly.

2. The automobile central control electronic control system testing method according to claim 1 is characterized in that: In the test of the central control electronic control system, data sources from different sources are identified and classified, the data collection frequency of each data source is optimized and outliers are detected, and feature recognition and structured management are performed through labeling. The specific process is as follows: Density-based clustering algorithm clusters feature space vectors according to density similarity, groups data sources and generates labels; Define the timestamp sequence of each type of data source, use interpolation to align the time of data sources with different labels, map them into the same time window, and use Lagrange interpolation to complete the time; After labeling the data source, different types of data structures and timestamps need to be analyzed; In each type of data source, select the initial sampling frequency for testing, monitor the rate of change of the sampled data, determine whether the sampling frequency has oversampling or undersampling, and adjust the collection frequency according to the changes in real-time data.

3. The automobile central control electronic control system testing method according to claim 2 is characterized in that: The synchronized data is subjected to feature extraction, and the features of structured and unstructured data are extracted using polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method. The specific process is as follows: Perform feature extraction of structured data, apply polynomial fitting method to sensor readings and vehicle status, optimize through least squares method, and extract trends and features of structured data; Use convolutional neural network to extract image features from video data. Set each frame of video image as a matrix, extract image features through multi-layer convolution and pooling operations, and obtain the feature vector of video data. For audio data, Mel-frequency cepstral coefficients are used to extract sound features. The audio signal is first framed and windowed, and the time domain signal is converted into a frequency domain signal using fast Fourier transform. The frequency domain signal is mapped to Mel frequency, and the cepstral coefficients are calculated and used to form a feature vector. The features extracted from structured data and unstructured data are used as feature data.

4. The automobile central control electronic control system testing method according to claim 3 is characterized in that: The extracted feature data are subjected to nonlinear normalization and principal component analysis dimensionality reduction, and unified into the comprehensive feature space to increase the numerical consistency and correlation between multimodal features. The specific process is as follows: Define the feature matrix of structured data, mean-center each feature, define the centered data matrix, calculate the covariance matrix of the centered feature matrix, perform eigenvalue decomposition on the covariance matrix, and obtain the eigenvalues ​​and corresponding eigenvectors; Use convolutional neural network to extract video frame image features, obtain local features in the image, use convolution kernel to perform convolution operation on the image, obtain convolution output feature map, use ReLU function to perform nonlinear transformation on the convolved feature map, obtain activated feature map, and perform maximum pooling operation on the activated feature map to reduce dimension; Divide the audio signal into frames, perform fast Fourier transform on each frame, calculate the power spectrum of each frame, map the power spectrum to the Mel frequency through the Mel filter bank, extract the Mel frequency features of each audio frame, and apply average pooling to the Mel frequency features to obtain the low-dimensional feature vector of the audio; Concatenate the structured, unstructured and time series feature extraction results into an initial multimodal feature vector; The multimodal autoencoder performs dimensionality reduction and feature mapping; The comprehensive features generated by the autoencoder are passed into the deep belief network, and in each layer of the restricted Boltzmann machine, the joint probability distribution of the visible layer and the hidden layer is defined; The restricted Boltzmann machine of each layer is optimized through greedy layer-by-layer training method, and the comprehensive features are converted into high-level shared feature representation to determine the deep correlation of the data.

5. The automobile central control electronic control system testing method according to claim 4 is characterized in that: Use ensemble learning methods to optimize multimodal data fusion, verify data consistency based on association rules and anomaly detection models, and trigger corresponding alarm mechanisms according to the degree of anomaly. The specific process is as follows: After obtaining the shared feature representation, random forest is used to build an integrated learning model; The shared feature representation is input into a decision tree. Each decision tree is trained on a subset of features from a different modality and calculates the importance of each feature in the decision. The outputs of the decision trees are fused by voting or weighted averaging to generate the final fusion decision output; Integrate structured data and unstructured data in a unified feature space for multimodal data fusion.

6. The automobile central control electronic control system testing method according to claim 5 is characterized in that: The data consistency is verified based on the association rules and anomaly detection model, and the corresponding alarm mechanism is triggered according to the anomaly level. The specific steps are as follows: Preprocess the fused multimodal data and define the typical response mode of each data source under events, including sudden braking, turning, and acceleration, and discretize the multimodal data into state tags; Use the Apriori algorithm to construct event sets and association rules for multimodal data between different data sources; The event set includes the state combination of all modal data within the time window, and the association rules represent the conditional probability relationship between events; Match the current event pattern with the association rules. If the confidence of the rules matched by the event meets the conditions in the association rules, it means that the data sources are synchronized. If the conditions are not met, calculate the abnormality of the event and mark it as a potential abnormality. The anomaly values ​​are divided into three levels of anomaly types, including low-level anomalies, medium-level anomalies and high-level anomalies.

7. A vehicle central control electronic control system test platform, used to implement the vehicle central control electronic control system test method according to any one of claims 1 to 6, characterized in that: include: The label classification module is used to identify and classify data sources from different sources in the central control electronic control system test, optimize the data collection frequency and detect outliers for each data source, and perform feature recognition and structured management through label processing; The feature extraction module is used to extract features from the synchronized data, using polynomial fitting, convolutional neural network and Mel frequency cepstral coefficient method to extract features from structured and unstructured data; Dimensionality reduction association module, which is used to perform nonlinear normalization and principal component analysis on the extracted feature data and unify them into the comprehensive feature space to increase the numerical consistency and correlation between multimodal features; The verification alarm module is used to optimize multimodal data fusion using integrated learning methods, verify data consistency based on association rules and anomaly detection models, and trigger corresponding alarm mechanisms according to the degree of anomaly.

Images