Commercial vehicle fault diagnosis method and device based on spatial-temporal feature fusion

Abstract

The invention provides a commercial vehicle fault diagnosis method and device based on spatial-temporal feature fusion, and the method comprises the steps: collecting sensor data of a plurality of part sensors in the operation process of a commercial vehicle in real time, dividing the sensor data according to a preset time interval, and obtaining a plurality of time slices; constructing a spatial incidence matrix among the sensor data in each time slice; performing time dimension feature extraction on each piece of sensor data in each time slice, so as to obtain a time dimension feature vector of each piece of sensor data based on change features of the data in the time slices; fusing the space incidence matrix of each time slice with the corresponding time dimension feature vector to obtain a space-time feature vector; and performing comparative analysis on the spatial-temporal feature vector and a reference spatial-temporal feature vector in a pre-established reference library, and determining a fault type and a fault position of the commercial vehicle according to a comparative analysis result. According to the invention, the accuracy and reliability of fault diagnosis are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a commercial vehicle fault diagnosis method and device based on spatiotemporal feature fusion. Background Art

[0002] Currently, commercial vehicle fault diagnosis methods often focus on a single-dimensional analysis of vehicle operating data, such as relying solely on real-time sensor data for fault diagnosis. This diagnostic approach has a significant drawback: it ignores the temporal variations in vehicle operating data and the spatial correlations between different sensor data. This makes it difficult to accurately identify faults under complex operating conditions, and can easily lead to misdiagnosis or omissions. In particular, when a vehicle fails due to the combined effects of multiple factors, traditional diagnostic methods are unable to fully capture fault characteristic information due to the ineffective integration of the temporal and spatial characteristics of the data. This reduces the accuracy and reliability of fault diagnosis and makes it difficult to meet the requirements for efficient and safe operation of modern commercial vehicles. Summary of the Invention

[0003] The present invention provides a commercial vehicle fault diagnosis method and device based on spatiotemporal feature fusion, which are used to improve the accuracy and reliability of fault diagnosis and better meet the needs of efficient and safe operation of modern commercial vehicles.

[0004] In a first aspect, the present invention provides a commercial vehicle fault diagnosis method based on spatiotemporal feature fusion, comprising:

[0005] Collect sensor data from multiple sensors in the operation of commercial vehicles in real time, and divide the sensor data into multiple time segments according to preset time intervals;

[0006] Constructing a spatial correlation matrix between the sensor data in each time segment; the spatial correlation matrix is used to characterize the mutual influence relationship between different sensor data;

[0007] Extracting time dimension features from each sensor data in each time segment to obtain a time dimension feature vector for each sensor data based on the change characteristics of the data in the time segment;

[0008] The spatial correlation matrix of each time segment is fused with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector;

[0009] The spatiotemporal feature vector is compared and analyzed with a reference spatiotemporal feature vector in a pre-established reference library, and the fault type and fault location of the commercial vehicle are determined according to the comparison and analysis results.

[0010] In a second aspect, the present invention further provides a commercial vehicle fault diagnosis device based on spatiotemporal feature fusion, which is applied to the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion as described in the first aspect; the commercial vehicle fault diagnosis device based on spatiotemporal feature fusion includes:

[0011] A data processing module is used to collect sensor data from multiple sensors in the operation of a commercial vehicle in real time and divide the sensor data into multiple time segments according to preset time intervals;

[0012] A correlation matrix construction module is used to construct a spatial correlation matrix between each sensor data in each time segment; the spatial correlation matrix is used to characterize the mutual influence relationship between different sensor data;

[0013] A time feature extraction module is used to extract time dimension features of each sensor data in each time segment, so as to obtain a time dimension feature vector of each sensor data based on the change characteristics of the data in the time segment;

[0014] The feature fusion module is used to fuse the spatial correlation matrix of each time segment with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector;

[0015] The fault diagnosis module is used to compare and analyze the spatiotemporal feature vector with a reference spatiotemporal feature vector in a pre-established reference library, and determine the fault type and fault location of the commercial vehicle based on the comparison and analysis results.

[0016] In a third aspect, the present invention also provides an electronic device comprising: a memory for storing a computer software program; and a processor for reading and executing the computer software program, thereby implementing any of the commercial vehicle fault diagnosis methods based on spatiotemporal feature fusion as described above.

[0017] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, wherein the storage medium stores a computer software program, and when the computer software program is executed by a processor, it implements any of the commercial vehicle fault diagnosis methods based on spatiotemporal feature fusion as described above.

[0018] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the commercial vehicle fault diagnosis methods based on spatiotemporal feature fusion as described above.

[0019] The commercial vehicle fault diagnosis method based on spatiotemporal feature fusion provided by the embodiment of the present invention fully considers the spatial correlation relationship between different sensor data by constructing a spatial correlation matrix, can capture the potential connection between the data, and effectively mines the changing law of the data in the time series by extracting the time dimension feature vector. The spatiotemporal feature vector obtained by fusing the spatial correlation matrix with the time dimension feature vector comprehensively and accurately reflects the vehicle operation status. Finally, through comparative analysis with the benchmark library, it can identify faults based on the comprehensive judgment of spatiotemporal features, thereby improving the accuracy and reliability of fault diagnosis, effectively avoiding the problems of misjudgment and missed judgment caused by ignoring the spatiotemporal features of the data, and can better meet the needs of efficient and safe operation of modern commercial vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 1 is a flow chart of a commercial vehicle fault diagnosis method based on spatiotemporal feature fusion provided by an embodiment of the present invention;

[0021] Figure 2 1 is a schematic structural diagram of a commercial vehicle fault diagnosis device based on spatiotemporal feature fusion provided by an embodiment of the present invention;

[0022] Figure 3 An embodiment diagram of an electronic device provided by an embodiment of the present invention;

[0023] Figure 4 An embodiment diagram of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0026] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0027] See Figure 1 , Figure 1 : This is a flow chart of a commercial vehicle fault diagnosis method based on spatiotemporal feature fusion provided by the present invention. In an embodiment of the present invention, the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion is executed by a fault diagnosis device. Therefore, the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion includes:

[0028] Step 10: collect sensor data from sensors at multiple locations during the operation of the commercial vehicle in real time, and divide the sensor data into multiple time segments according to preset time intervals.

[0029] Optionally, the fault diagnosis device connects to sensors in multiple locations on a commercial vehicle (such as the engine, transmission, and braking system) to collect sensor data in real time. These sensors include temperature sensors, pressure sensors, and vibration sensors, each used to monitor parameters such as temperature, pressure, and vibration at the corresponding location. Furthermore, the fault diagnosis device divides the collected data into multiple time segments according to preset time intervals (e.g., 10 seconds), each of which contains all data collected by each sensor during that time period.

[0030] In one embodiment, an engine temperature sensor, a transmission pressure sensor, and a brake system vibration sensor are installed on a commercial vehicle. The fault diagnosis device collects data from these three sensors in real time, with a preset time interval of 10 seconds. At a certain moment, the data collected by the engine temperature sensor is 80°C, 82°C, 83°C, 84°C, 85°C, 86°C, 87°C, 88°C, 89°C, and 90°C in sequence within 0-10 seconds; the data collected by the transmission pressure sensor is 10MPa, 11MPa, 12MPa, 13MPa, 14MPa, 15MPa, 16MPa, 17MPa, 18MPa, and 19MPa in sequence; the data collected by the brake system vibration sensor is 5m / s in sequence. 2, 6m / s 2 , 7m / s 2 , 8m / s 2 , 9m / s 2 、10m / s 2 , 11m / s 2 , 12m / s 2 、13m / s 2 、14m / s 2 In this way, 0-10 seconds constitutes a time segment, and the fault diagnosis device continues to collect and divide subsequent time segments in this way.

[0031] Step 20: Construct a spatial correlation matrix between the sensor data in each time segment. The spatial correlation matrix is used to represent the mutual influence relationship between different sensor data.

[0032] Furthermore, within each time segment, the fault diagnosis device constructs a spatial correlation matrix based on the mutual influence relationships between different sensor data. Specifically, the fault diagnosis device analyzes the correlation, causal relationship, and other relationships between the various sensor data. For example, if an increase in engine temperature may cause a change in transmission pressure, then there is a certain mutual influence relationship between the two sensor data. By calculating indicators such as the correlation coefficient between the various sensor data, the values of the elements in the matrix are determined, and a spatial correlation matrix is constructed. The spatial correlation matrix is used to characterize the degree of mutual influence between the different sensor data, as described in detail in steps 201 to 205.

[0033] Step 30 : extracting time dimension features from each sensor data in each time segment, so as to obtain a time dimension feature vector of each sensor data based on the change characteristics of the data in the time segment.

[0034] Furthermore, for each sensor data in each time segment, the fault diagnosis device extracts time dimension features based on its change characteristics in the time segment, and by analyzing the data's change trend (increase, decrease, stability), change amplitude, fluctuation frequency and other characteristics, using algorithms such as sliding windows and Fourier transform, these change characteristics are converted into time dimension feature vectors to obtain the time dimension feature vectors of each sensor data in each time segment, as specifically described in steps 301 to 304.

[0035] Step 40: Fusing the spatial correlation matrix of each time segment with the corresponding time dimension feature vector to obtain a spatiotemporal feature vector.

[0036] Furthermore, the fault diagnosis device fuses the spatial correlation matrix of each time segment with the corresponding time dimension feature vector. For example, the spatial correlation information and the time dimension feature information can be integrated by performing specific mathematical operations on the matrix and the vector, such as matrix multiplication, vector concatenation, etc., to obtain a spatio-temporal feature vector that can reflect the spatio-temporal characteristics of the data simultaneously. Specifically, refer to steps 401 to 404.

[0037] Step 50: Compare and analyze the spatio-temporal feature vector with the reference spatio-temporal feature vectors in the pre-established reference library, and determine the fault type and fault location of the commercial vehicle according to the comparison and analysis results.

[0038] Furthermore, the fault diagnosis device compares and analyzes the spatio-temporal feature vector with the reference spatio-temporal feature vectors in the pre-established reference library. Among them, the reference library stores the reference spatio-temporal feature vectors corresponding to various normal operating states, different fault types and positions.

[0039] Therefore, the fault diagnosis device calculates the distance (such as Euclidean distance, cosine similarity, etc.) between the spatio-temporal feature vector and the reference spatio-temporal feature vector, and judges the current operating state of the commercial vehicle according to the set threshold and the distance calculation result, and determines the fault type and fault location.

[0040] In an embodiment, the reference library has a reference spatio-temporal feature vector A in the normal operating state, a reference spatio-temporal feature vector B of engine fault type 1, and a reference spatio-temporal feature vector C of transmission fault type 2. The fault diagnosis device calculates the Euclidean distances between the obtained spatio-temporal feature vector and A, B, and C, which are d1, d2, and d3 respectively. If d2 < d! and d2 < d3, and d2 is less than the set fault judgment threshold, then it can be determined that the commercial vehicle has engine fault type 1; if d3 < d1 and d3 < d2, and d3 is less than the set fault judgment threshold, then it can be determined that the commercial vehicle has transmission fault type 2. In this way, the fault type and fault location of the commercial vehicle are determined according to the comparison and analysis results.

[0041] In the embodiment of the present invention, by constructing a spatial correlation matrix, the spatial correlation relationship between different sensor data is fully considered, and the potential connection between data can be captured. By extracting the time dimension feature vector, the change law of the data in the time series is effectively mined. The spatio-temporal feature vector obtained by fusing the spatial correlation matrix and the time dimension feature vector comprehensively and accurately reflects the vehicle operating state. Finally, through comparison and analysis with the reference library, faults can be identified based on the comprehensive judgment of spatio-temporal features, improving the accuracy and reliability of fault diagnosis, effectively avoiding misjudgment and missed judgment problems caused by ignoring the spatio-temporal features of data, and better meeting the requirements of the efficient and safe operation of modern commercial vehicles.

[0042] In one embodiment, steps 201 to 205 are described as follows:

[0043] Step 201 : for sensor data in each time segment, the time series data of each sensor is processed based on a preset local window size to obtain a local extreme value characteristic value of each sensor data.

[0044] Optionally, for the sensor data in each time segment, the fault diagnosis device performs sliding window processing on the time series data of each sensor according to a preset local window size. In each local window, the maximum and minimum values of the data are found, and the local extreme value eigenvalue of each sensor data is determined based on the ratio of the extreme value difference of the maximum and minimum values in the local window to the extreme value sum. Therefore, the local extreme value eigenvalue reflects the degree of fluctuation of the data in the local window. The local extreme value eigenvalue E i The specific formula is:

[0045]

[0046] Among them, x ij represents the data of the i-th sensor at time j, t represents the starting time of the window, and w represents the local window size.

[0047] In one embodiment, using engine temperature sensor data from a time segment of 0-10 seconds at 80°C, 82°C, 83°C, 84°C, 85°C, 86°C, 87°C, 88°C, 89°C, and 90°C as an example, the local window size w = 3. The first local window is [80°C, 82°C, 83°C], and the local extreme eigenvalue E1 is calculated according to the formula: E1 = (83-80) / (83+80)≈0.018. Similarly, the local extreme eigenvalues corresponding to subsequent local windows can be calculated, ultimately obtaining a set of local extreme eigenvalues for the engine temperature sensor data within that time segment.

[0048] Step 202 : Calculate the data difference between each sensor data at adjacent times, and determine the fluctuation trend index of each sensor data within the time segment.

[0049] Furthermore, the fault diagnosis device calculates the data difference of each sensor data at adjacent times, and determines the fluctuation trend index by analyzing the change pattern of these data differences. In this embodiment of the present invention, discrete wavelet transform is used to process the data difference sequence to extract high-frequency and low-frequency components, and the energy proportion of the high-frequency component is used as the fluctuation trend index T i , the calculation formula is as follows:

[0050]

[0051] Among them, x i,kis the difference sequence of the i-th sensor data at adjacent times, DWT high (x i,k ) is the high frequency component after discrete wavelet transform, DWT(x i,k ) is the result of the entire discrete wavelet transform, and n is the length of the difference sequence. The higher the proportion of high-frequency component energy, the more severe the data fluctuation.

[0052] For the engine temperature sensor data difference sequence [2, 1, 1, 1, 1, 1, 1, 1, 1] at adjacent times, after discrete wavelet transform, the high-frequency component energy ratio is calculated to obtain the fluctuation trend indicator T1. If the calculated result T1 = 0.3, it indicates that the engine temperature sensor data has a stable fluctuation trend within this time segment.

[0053] Step 203 : determining the initial correlation strength between the sensor data based on the local extreme value characteristic values and the fluctuation trend index between the sensor data.

[0054] Furthermore, the fault diagnosis device calculates the initial correlation strength using mutual information based on the local extreme value characteristic value and fluctuation trend index between each sensor data. Mutual information is used to measure the degree of dependence between two random variables. The initial correlation strength S between two sensor data i and j is ij The calculation formula is:

[0055] S ij =MI(E i ,E j )*MI(T i ,T j ). Among them, MI(E i ,E j ) is the mutual information between the local extreme eigenvalues of sensor data i and j, MI(T i ,T j ) is the mutual information between the fluctuation trend indicators of sensor data i and j. The larger the mutual information value, the stronger the correlation between the two sensor data.

[0056] In one embodiment, the local extreme value characteristic value sequence E1 and the fluctuation trend index T1 of the engine temperature sensor, and the local extreme value characteristic value sequence E2 and the fluctuation trend index T2 of the transmission pressure sensor are known. By calculating the mutual information, MI(E1, E2) = 0.6, MI(T1, T2) = 0.7, then the initial correlation strength S between the engine temperature sensor and the transmission pressure sensor is 12 For S 12 =0.6*0.7=0.42. Similarly, the initial correlation strength between other sensor data can be calculated.

[0057] Step 204 : determining the influence direction between the sensor data based on the data change time delay and the correlation coefficient between the sensor data.

[0058] Furthermore, the fault diagnosis device determines the direction of influence based on the data change time delay and correlation coefficient between each sensor data. In this embodiment of the present invention, the Granger causality test is used to determine whether sensor data i has a causal influence on sensor data j. By calculating the predictive ability of sensor data i's historical data on sensor data j's current data, if, under a certain confidence level, sensor data i's historical data can significantly improve the prediction accuracy of sensor data j's current data, then it is considered that sensor data i has an influence on sensor data j, and the influence direction is i→j. In the specific calculation, a vector autoregression model is constructed:

[0059] Among them, Y t =[x it ,x jt ] T is the vector containing sensor data i and j, p is the lag order, A k is the coefficient matrix, ∈ t is the error term. The direction of influence is determined by testing whether the elements in the coefficient matrix are significantly different from zero.

[0060] In one embodiment, a VAR model is constructed for engine temperature sensor data and transmission pressure sensor data, for example, with a lag order of p = 2. A Granger causality test shows that considering the previous two periods of engine temperature sensor data significantly improves the prediction accuracy of the current transmission pressure sensor data. The Granger causality test rejects the hypothesis at a 5% significance level, thus determining the direction of influence to be engine temperature sensor → transmission pressure sensor.

[0061] Step 205 : constructing a spatial correlation matrix between the sensor data in each time segment based on the initial correlation strength and influence direction between the sensor data.

[0062] Furthermore, the fault diagnosis apparatus constructs a spatial correlation matrix between the sensor data in each time segment according to the initial correlation strength and influence direction between the sensor data, as specifically described in steps 2051 to 2054 .

[0063] The embodiments of the present invention can comprehensively consider the local fluctuation characteristics of sensor data, the time series change trend and the causal relationship between data, and construct a spatial correlation matrix that accurately reflects the relationship between each sensor data. The spatial correlation matrix not only quantifies the correlation strength between sensor data, but also clarifies the direction of influence, which helps to more accurately locate the fault source and determine the fault type, thereby improving the accuracy of fault diagnosis.

[0064] In one embodiment, steps 2051 to 2054 are described as follows:

[0065] Step 2051 : Correcting the initial correlation strength based on the influence direction between each sensor data to obtain the corrected correlation strength between each sensor data.

[0066] Optionally, the fault diagnosis device corrects the initial correlation strength according to the influence direction between the sensor data. Considering that the influence direction has unidirectionality and different degrees of causal relationship, the embodiment of the present invention adopts an asymmetric correction function. If sensor data i has an influence direction i→j on sensor data j, then the corrected correlation strength is The calculation formula is:

[0067] Among them, α is a custom adjustment parameter (0<α<1), which is used to control the correction amplitude, sign(S ij ) is a sign function, when S ij >0, sign(S ij )=1; when S ij = 0, sign(S ij )=0; when S ij When <0, sign(S ij )=-1. If there is no influence direction, that is, S ij =0, then the corrected correlation strength

[0068] In one embodiment, the initial correlation strength S between the engine temperature sensor and the transmission pressure sensor is known. 12 =0.42, and the impact direction is engine temperature sensor → transmission pressure sensor, for example, the adjustment parameter α = 0.3. Calculate the corrected correlation strength according to the formula

[0069]

[0070] Initial correlation strength S between the engine temperature sensor and the brake system vibration sensor 13 =0.1, no obvious influence direction, then Initial correlation strength S between the transmission pressure sensor and the brake system vibration sensor 23 =0.2, no obvious influence direction, then

[0071] Step 2052 : In each time segment, based on the actual spatial position of each sensor on the commercial vehicle, determine a spatial position mapping factor between each sensor data corresponding to each sensor.

[0072] Furthermore, within each time segment, the fault diagnosis device determines the spatial position mapping factor between the sensor data corresponding to each sensor data based on the actual spatial position of each sensor on the commercial vehicle. The closer the distance between sensors, the greater the influence of spatial position on their data association. Therefore, the embodiment of the present invention adopts a calculation method based on the distance attenuation principle. For example, the spatial distance between sensors i and j on the commercial vehicle is d ij , the spatial position mapping factor F between the sensor data corresponding to sensors i and j ij The calculation formula is: Where β is the distance attenuation coefficient (β>0), which is used to adjust the influence of distance on the mapping factor. ij It can be calculated through 3D modeling of commercial vehicles and sensor installation coordinates.

[0073] In one embodiment, the engine temperature sensor is installed at coordinates (x1, y1, z1), the transmission pressure sensor is installed at coordinates (x2, y2, z2), and the brake system vibration sensor is installed at coordinates (x3, y3, z3). After calculation, the spatial distance d between the engine temperature sensor and the transmission pressure sensor is 12 = 2 meters, the spatial distance d between the engine temperature sensor and the brake system vibration sensor 13 = 5 meters, the spatial distance d between the transmission pressure sensor and the brake system vibration sensor 23 = 4 meters, for example, the distance reduction coefficient β = 0.1. Then the spatial position mapping factor F of the engine temperature sensor and the transmission pressure sensor is 12 for

[0074]

[0075] Similarly,

[0076]

[0077] Step 2053 : determining the final correlation strength between the sensor data based on the spatial position mapping factor and the corrected correlation strength between the sensor data.

[0078] Furthermore, the fault diagnosis device combines the spatial position mapping factor and the corrected correlation strength between each sensor data to calculate the final correlation strength. In this embodiment of the present invention, the product method is adopted. Therefore, the final correlation strength C between sensor data i and data j is ij The calculation formula is:

[0079]

[0080] Continuing with the above embodiment, the corrected correlation strength between the engine temperature sensor and the transmission pressure sensor is Spatial position mapping factor F 12 ≈0.714, then the final correlation strength C 12 for:

[0081] C 12 =0.546*0.714≈0.39.

[0082] Similarly,

[0083] Step 2054 : constructing a spatial correlation matrix between the sensor data in each time segment based on the final correlation strength between the sensor data.

[0084] Furthermore, the fault diagnosis device constructs a spatial correlation matrix based on the final correlation strength between each sensor data. In the spatial correlation matrix, the main diagonal elements are 1, indicating that the sensor itself is completely correlated with itself; for the non-diagonal elements, if the final correlation strength between sensor data i and j is C ij , then the matrix element M ij =C ij ,M ji =C ji .

[0085] The spatial correlation matrix constructed by continuing the above embodiment is as follows:

[0086]

[0087] The spatial correlation matrix constructed in the embodiment of the present invention integrates the causal relationship and spatial position relationship between sensor data. The modified initial correlation strength highlights the directional difference of data influence. The spatial position mapping factor takes into account the influence of the physical position of the sensor on data correlation. The final correlation strength combines the two, so that the moment space correlation matrix more accurately reflects the actual correlation of sensor data in the spatial dimension, which helps to accurately determine the fault type and locate the fault location, and improves the accuracy and reliability of fault diagnosis.

[0088] In one embodiment, steps 301 to 304 are described as follows:

[0089] Step 301 : For each sensor data in each time segment, the data sequence is discretized to convert the continuous time data sequence into a discrete point set.

[0090] Optionally, for each sensor data sequence within each time slice, the fault diagnosis device discretizes the data sequence. In the embodiment of the present invention, a specific sampling interval Δt is set to convert the continuous time data sequence into a discrete point set. For example, the original data sequence is x(t), and the discretized data point set is {x(t i )}, where t i =i·Δt, where i=0, 1, 2, ..., n, where n is the number of discrete points. In one embodiment, the engine temperature sensor data within a time segment of 0-10 seconds is [80°C, 82°C, 83°C, 84°C, 85°C, 86°C, 87°C, 88°C, 89°C, 90°C]. For example, if the sampling interval Δt is 1 second, the discretized data point set is {80, 82, 83, 84, 85, 86, 87, 88, 89, 90}, where each data point corresponds to a discrete moment.

[0091] Step 302 : constructing a local slope matrix based on the local slopes calculated for each group of three adjacent data points in the discretized data points.

[0092] Furthermore, based on the discretized data points, the fault diagnosis device calculates the local slope for each group of three adjacent data points. i-1 ,x i ,x i+1 , local slope k i The calculation formula is:

[0093]

[0094] Furthermore, the fault diagnosis device sequentially arranges all calculated local slopes to construct a local slope matrix K, where each row in the local slope matrix corresponds to the local slopes calculated for a data point and its adjacent points. In one embodiment, local slopes are calculated for the discretized engine temperature sensor data point set {80, 82, 83, 84, 85, 86, 87, 88, 89, 90}. Taking the first local slope calculation as an example, with x0 = 80, x1 = 82, x2 = 83, and Δt = 1 second, k1 = (83 - 80) / (2 * 1) = 1.5.

[0095] Calculate other local slopes in sequence and get the local slope sequence as [1.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5]. The constructed local slope matrix K is:

[0096]

[0097] Step 303: determine local extreme points based on the second-order difference, and construct a trend line based on the local extreme points in combination with piecewise cubic Hermite interpolation.

[0098] Furthermore, the fault diagnosis device determines the local extreme value points in the discrete data points by second-order difference. For example, the discrete data point sequence is {x i}, second-order difference Δ 2 x i The calculation formula is: 2 x i =x i+1 -2x i +x i-1 When Δ 2 x i When it changes from positive to negative, the corresponding data point is a local maximum point; when Δ 2 x i When it changes from negative to positive, the corresponding data point is the local minimum point. After determining the local extreme point, based on the piecewise cubic Hermite interpolation method, the slope information of the local extreme point and the data point is combined to construct the trend line of the data. The piecewise cubic Hermite interpolation method requires that in each subinterval [x i ,x i+1 ] construct a cubic polynomial H(x) that satisfies H(x i )=x i ,H(x i+1 )=x i+1 ,H'(x i )=k i ,H'(x i+1 )=k i+1 , where k i and k i+1 is the local slope.

[0099] For the discrete data point set {80,82,83,84,85,86,87,88,89,90} of the engine temperature sensor, calculate the second-order difference: Δ 2 x1=83-2*82+80=-1,Δ 2 x2=84-2*83+82=0.

[0100] Our analysis revealed no cases where the second-order difference changed from positive to negative or vice versa, indicating that the data series lacks obvious local extreme points (this may exist in practice, but is merely an example). For example, if local extreme points do exist, a trend line can be constructed using the slope information in the local slope matrix K using piecewise cubic Hermite interpolation. This trend line can better reflect the overall trend of the data.

[0101] Step 304 : Analyze the change trend and fluctuation range of the data in the time segment based on the local slope matrix and trend line to obtain the time dimension feature vector of each sensor data in each time segment.

[0102] Furthermore, the fault diagnosis device analyzes the changing trend and fluctuation range of the data within the time segment based on the local slope matrix and trend line, and obtains the time dimension feature vector of each sensor data in each time segment, as described in steps 3041 to 3044.

[0103] The embodiments of the present invention can deeply extract time dimension features from commercial vehicle sensor data. The local slope matrix quantifies the local trend of data changes, and the trend line construction clearly presents the overall change trend of the data. The time dimension feature vector finally integrated fully reflects the change trend and fluctuation range of the data within the time segment, so that the time dimension feature vector provides rich and accurate time dimension information for subsequent spatiotemporal feature fusion and fault diagnosis, which helps to more accurately identify abnormal changes in the operation process of commercial vehicles and improve the accuracy and reliability of fault diagnosis.

[0104] In one embodiment, steps 3041 to 3044 are described as follows:

[0105] Step 3041 : For each sensor data in each time segment, the degree of difference between adjacent local slopes is calculated based on the local slope matrix to determine the trend complexity.

[0106] Optionally, for each sensor data in each time segment, the fault diagnosis device calculates the difference between adjacent local slopes according to the local slope matrix to determine the trend complexity. In this embodiment of the present invention, discrete cosine transform is used to perform frequency domain analysis on the local slope sequence, converting the local slope sequence from the time domain to the frequency domain, and measuring the trend complexity by calculating the energy proportion of the high-frequency component in the frequency domain. For example, the local slope sequence is {k i}, its discrete cosine transform is DCT({k i}), trend complexity C trend The calculation formula is:

[0107]

[0108] Where m is the starting index of the high-frequency component, and n is the length of the frequency domain sequence. The higher the proportion of high-frequency component energy, the more complex the data trend changes.

[0109] In one embodiment, the local slope matrix for the engine temperature sensor data is

[0110] Extract the local slope sequence as

[0111] [1.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5]. Perform discrete cosine transform on the sequence. For example, the high-frequency component starting index m = 3, the frequency domain sequence length n = 9, and the trend complexity C is calculated.trend :

[0112]

[0113] Step 3042: Determine the trend angle based on the angle between the line connecting the starting point and the ending point of the trend line in the time segment and the horizontal axis.

[0114] Furthermore, the fault diagnosis device determines the trend angle based on the angle between the line connecting the starting point and the ending point of the trend line in the time segment and the horizontal axis, where the trend angle reflects the overall direction of change of the data in the time segment. For example, if the starting point of the trend line is (t1, x1) and the ending point is (t2, x2), the calculation formula for the trend angle θ is:

[0115] Where arctan is the inverse tangent function, and the result is in radians.

[0116] In one embodiment, the starting point of the engine temperature sensor data trend line is (0, 80) and the ending point is (10, 90), so the trend angle θ is

[0117]

[0118] Step 3043: determine the fluctuation amplitude based on the maximum value of the absolute value of the difference between all data points in the data sequence and the points corresponding to the trend line, and determine the fluctuation frequency based on the number of times the data sequence crosses the trend line.

[0119] Furthermore, the fault diagnosis device determines the fluctuation amplitude based on the maximum value of the absolute value of the difference between all data points in the data sequence and the corresponding points of the trend line. For example, the data sequence is {x i}, the trend line corresponding point sequence is {y i}, the calculation formula of the fluctuation amplitude A is:

[0120] The fluctuation frequency f is determined by counting the number of times the data sequence crosses the trend line. This is the number of times two adjacent points in the data sequence lie on opposite sides of the trend line. In one embodiment, for the engine temperature sensor data sequence {80, 82, 83, 84, 85, 86, 87, 88, 89, 90}, the corresponding trend line point sequence is {80, 81, 82, 83, 84, 85, 86, 87, 88, 89}. The absolute value sequence of the difference between the two is {0, 1, 1, 1, 1, 1, 1, 1, 1, 1}, resulting in a fluctuation amplitude A = 1. Statistically, the data sequence crosses the trend line 0 times, so the fluctuation frequency f = 0.

[0121] In step 3044, the trend complexity, trend angle, fluctuation amplitude, and fluctuation frequency are integrated to obtain a time dimension feature vector of each sensor data in each time segment.

[0122] Furthermore, the fault diagnosis device fuses the trend complexity, trend angle, fluctuation amplitude and fluctuation frequency, and directly arranges these features in order to obtain the time dimension feature vector V of each sensor data in each time segment: V = [C trend ,θ,A,f].

[0123] Continuing with the above embodiment, the trend complexity C of the engine temperature sensor data calculated above is irend ≈0.1, trend angle θ≈45° (radian values can be retained in actual calculations), fluctuation amplitude A=1, and fluctuation frequency f=0 are fused to obtain the time dimension feature vector:

[0124]

[0125] The time dimension feature vector generated by the embodiment of the present invention comprehensively and deeply characterizes the characteristics of sensor data in the time dimension. The trend complexity quantifies the complexity of the data trend from the frequency domain perspective. The trend perspective clarifies the direction of data change, and the fluctuation amplitude and fluctuation frequency accurately describe the fluctuation characteristics of the data. The fused time dimension feature vector can effectively capture the dynamic change pattern of the data within the time segment, which helps to more accurately identify the differences in data characteristics under normal operating conditions and fault conditions, and improve the accuracy and reliability of fault diagnosis.

[0126] In one embodiment, steps 401 to 404 are described as follows:

[0127] Step 401 : For each time segment, dimension expansion is performed on the time dimension feature vector according to the number of rows and columns of the spatial correlation matrix to obtain a dimension expanded feature vector.

[0128] Optionally, for each time segment, the fault diagnosis device performs dimension expansion on the time dimension feature vector according to the number of rows and columns of the spatial correlation matrix. For example, the spatial correlation matrix is M, the number of rows and columns is n*n, and the time dimension feature vector is v=[v1,v2,…,v m In order to achieve dimensional expansion, the elements in the time dimension feature vector are filled into a new vector through loop and copy operations, so that its length is the same as the number of elements n in the spatial correlation matrix. 2 The specific operation is to reuse the elements in the time dimension feature vector in sequence until the length of the new vector reaches n 2 .

[0129] In one embodiment, the spatial correlation matrix M of the time segment is a 3*3 matrix, that is, n=3, n 2 = 9. Time dimension feature vector When expanding the dimension, first take the elements in v in order. Since 9 / 4=2...1, first copy v twice completely to get Then take the first element of v 0.1 and add it to the back to get the dimension-expanded feature vector

[0130]

[0131] In step 402, the dimension-expanded feature vector is converted into a three-dimensional tensor, and a feature embedding operation is performed on the spatial correlation matrix based on the embedding coefficient matrix combined with the three-dimensional tensor to obtain a feature-embedded correlation matrix.

[0132] Furthermore, the fault diagnosis device converts the dimension-expanded feature vector into a three-dimensional tensor, for example, the dimension-expanded feature vector v ext The length is n 2 , reshape it into a three-dimensional tensor T of n*n*1. Then, introduce an embedding coefficient matrix E, whose size is also n*n. By multiplying the elements one by one, combine the three-dimensional tensor T with the embedding coefficient matrix E and act on the spatial correlation matrix M to realize the feature embedding operation and obtain the feature embedded correlation matrix M embede The calculation formula is M cmbcd (i,j)=M(i,j)*E(i,j)*T(i,j,1), where i=1,…,n,j=1,…,n.

[0133] Continue to use the above 3*3 spatial correlation matrix

[0134] Dimensionality-expanded feature vector Convert to a three-dimensional tensor (This is prioritized.) For example, the embedding coefficient matrix M embed (1,1) as an example, M emiked (1,1)=M(1,1)×E(1,1)×T(1,1,1)=1×0.2×0.1=0.02.

[0135] Similarly, calculate other elements to obtain the feature embedding correlation matrix:

[0136] (here Take the approximate value 0.785 for calculation).

[0137] Step 403 , starting from the central element of the feature-embedded correlation matrix, perform a spiral transformation on the elements in the feature-embedded correlation matrix according to a preset spiral order to obtain a spiral-transformed correlation matrix.

[0138] Furthermore, the fault diagnosis device is based on the feature embedding correlation matrix M embed Starting from the central element of , the elements in the matrix are rearranged according to the preset spiral order (for example, clockwise spiral) to obtain the spiral transformation correlation matrix M spirale The specific operation is to first determine the center position of the matrix, then start from the center and take out the matrix elements in sequence along a spiral path to form a new vector, and then rearrange this vector into a matrix form.

[0139] Continuing with the above example, for the 3*3 feature embedding correlation matrix: The central element is M embed (2,2)=0.0491. In clockwise spiral order, take out the elements 0.0491, 0.0468, 0, 0, 0.2, 0, 0.0117, 0.02, and rearrange them into a 3*3 spiral transformation correlation matrix

[0140] Step 404 : Perform multi-dimensional fusion based on the spiral transformation correlation matrix to obtain the spatiotemporal feature vector in each time segment.

[0141] Furthermore, the fault diagnosis device performs multi-dimensional fusion based on the correlation matrix after spiral transformation to obtain the spatiotemporal feature vector in each time segment, as specifically described in steps 4041 to 4043.

[0142] The embodiments of the present invention realize the deep fusion of the spatial correlation matrix and the time dimension feature vector. The dimension expansion enables the time dimension feature to be expanded in the dimension of the spatial correlation matrix. The feature embedding operation integrates the time dimension feature information into the spatial correlation matrix. The spiral transformation rearranges the matrix elements after feature embedding, highlights the specific sequential relationship between the matrix elements, and mines the potential feature patterns, so that the spatiotemporal feature vector integrates the characteristics of the sensor data in the spatial and temporal dimensions, and can more comprehensively and accurately reflect the status information of the commercial vehicle during operation, thereby improving the accuracy and reliability of fault type and fault location judgment, and improving the accuracy and reliability of fault diagnosis.

[0143] In one embodiment, steps 4041 to 4043 are described as follows:

[0144] Step 4041: For each time segment, perform a multi-dimensional convolution operation on each matrix element in the correlation matrix after helical transformation based on a multi-dimensional convolution kernel to obtain a correlation matrix after convolution fusion.

[0145] Optionally, for the correlation matrix after helical transformation of each time segment, the fault diagnosis device uses a multi-dimensional convolution kernel to perform a multi-dimensional convolution operation on each element in the matrix. For example, the correlation matrix after helical transformation is M spiral , with a dimension of n*n, and the multi-dimensional convolution kernel is K, with a dimension of m*m (m < n). For each element M spiral in the matrix M spirol (i,j), select a sub-matrix S ij of the same size as the convolution kernel centered on this element, and calculate the element value at the corresponding position in the correlation matrix C after convolution fusion through the convolution formula . Traverse the entire M spiral matrix to obtain the complete correlation matrix C after convolution fusion, which can extract the local features and spatial relationships of matrix elements and enhance the expression ability of features.

[0146] In one embodiment, the correlation matrix M spiral after helical transformation is a 3*3 matrix:

[0147]

[0148] Select a 2*2 multi-dimensional convolution kernel K:

[0149]

[0150] Taking the calculation of C(1,1) as an example, take M spiral (1,1) = 0.2 as the center to select the sub-matrix S 11 :

[0151]

[0152] Calculate according to the convolution formula:

[0153] C(1,1) = 0.0468 * 0.5 + 0 * 0.3 + 0.02 * 0.2 + 0 * 0.1 = 0.0274.

[0154] Similarly, calculate other elements to obtain the correlation matrix C after convolution fusion:

[0155]

[0156] The edge elements are not calculated temporarily due to the limitation of convolution kernel selection. In actual operation, padding can be used for processing.

[0157] Step 4042: Map each first matrix element in the association matrix after convolution fusion to the second matrix element under the corresponding two-dimensional coordinates according to the preset mapping relationship and the three-dimensional coordinates of the first matrix element, and construct the association matrix after topological reconstruction based on each second matrix element.

[0158] Furthermore, the fault diagnosis device maps each first matrix element to the corresponding second matrix element under the two-dimensional coordinate according to the preset mapping relationship and in combination with the three-dimensional coordinates (for example, (i, j, 0), where the third dimension is fixed to 0 to simplify the position in the matrix) of the first matrix element (the element in the association matrix after convolution fusion). The preset mapping relationship can be a custom function or rule, for example, the three-dimensional coordinates are converted into two-dimensional coordinates through the coordinate transformation formula x'=f1(i, j), y'=f2(i, j), where f1 and f2 are specific functions. The topologically reconstructed association matrix R is constructed based on the mapped second matrix elements, so that the topological structure of the matrix elements changes to mine different feature relationships.

[0159] In one embodiment, for example, the preset mapping relationship is x'=i+j, y'=ij. For the element C(1,1)=0.0274 in the correlation matrix C after convolution fusion, its three-dimensional coordinates are (1,1,0), and the two-dimensional coordinates are calculated by the mapping formula: x'=1+1=2, y'=1-1=0, that is, mapped to the position with coordinates (2,0) in the correlation matrix R after topological reconstruction. All elements in C are mapped in this way to construct the correlation matrix R after topological reconstruction. For example, after mapping, the following is obtained:

[0160]

[0161] Step 4043, perform feature cross-fusion based on each second matrix element in the correlation matrix after topological reconstruction to obtain a cross-fused correlation matrix, and perform aggregation operation on each matrix element in the cross-fused correlation matrix to generate a spatiotemporal feature vector in each time segment.

[0162] Furthermore, the fault diagnosis device performs feature cross-fusion on each second matrix element in the topology reconstructed correlation matrix R. In this embodiment, nonlinear operations between matrix elements, such as power operations and exponential operations of elements, are used to construct the cross-fusion correlation matrix F. For example, for the element R(i,j) in R, the formula F(i,j)=R(i,j) 2 +exp(R(i,j)) calculates the element value of the corresponding position in the cross-fusion correlation matrix F. Then, all elements in the cross-fusion correlation matrix F are aggregated. Aggregation methods such as averaging, maximum value, and minimum value can be used to aggregate all elements into a vector to generate the spatiotemporal feature vector v in each time segment. st .

[0163] Continuing with the above embodiment, for the correlation matrix R after topology reconstruction:

[0164]

[0165] Use the formula F(i,j)=R(i,j) 2 +exp(R(i,j)) calculates the correlation matrix F after cross fusion:

[0166]

[0167] Use the mean aggregation method to calculate the aggregate value:

[0168]

[0169] Get the spatiotemporal feature vector v st =[1.0273]. In practical applications, if there are more matrix elements, the aggregation results in a vector containing multiple values.

[0170] The embodiment of the present invention realizes the deep processing of the correlation matrix after spiral transformation, and the generated spatiotemporal feature vector has stronger feature expression ability and discrimination. The multidimensional convolution operation extracts the local features and spatial relationships of the matrix elements, enhancing the detailed features of the data; the topological reconstruction changes the topological structure of the matrix elements and mines different feature association patterns; the feature cross-fusion further enriches the feature information through nonlinear operations, and the aggregation operation integrates these complex features into a concise and representative spatiotemporal feature vector, so that the spatiotemporal feature vector can more comprehensively and accurately integrate the features of the commercial vehicle sensor data in the spatial and temporal dimensions, which helps to more accurately judge the fault type and fault location of the commercial vehicle, and improve the accuracy and reliability of fault diagnosis.

[0171] Furthermore, the commercial vehicle fault diagnosis device based on spatiotemporal feature fusion provided by the present invention is described below. The commercial vehicle fault diagnosis device based on spatiotemporal feature fusion described below and the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion described above can be referenced to each other.

[0172] Optional, see Figure 2 , Figure 2 It is a structural diagram of a commercial vehicle fault diagnosis device based on spatiotemporal feature fusion provided by the present invention. The commercial vehicle fault diagnosis device based on spatiotemporal feature fusion includes.

[0173] The data processing module 210 is used to collect sensor data from sensors at multiple locations during the operation of the commercial vehicle in real time and divide the sensor data into multiple time segments according to preset time intervals;

[0174] The correlation matrix construction module 220 is used to construct a spatial correlation matrix between each sensor data in each time segment; the spatial correlation matrix is used to represent the mutual influence relationship between different sensor data;

[0175] The time feature extraction module 230 is used to extract the time dimension feature of each sensor data in each time segment, so as to obtain the time dimension feature vector of each sensor data based on the change feature of the data in the time segment;

[0176] A feature fusion module 240 is used to fuse the spatial correlation matrix of each time segment with the corresponding time dimension feature vector to obtain a spatiotemporal feature vector;

[0177] The fault diagnosis module 250 is used to compare and analyze the spatiotemporal feature vector with the reference spatiotemporal feature vectors in a pre-established reference library, and determine the fault type and fault location of the commercial vehicle based on the comparison and analysis results.

[0178] The embodiment of the present invention fully considers the spatial correlation relationship between different sensor data by constructing a spatial correlation matrix, can capture the potential connection between the data, and effectively mines the changing pattern of the data in the time series by extracting the time dimension feature vector. The spatiotemporal feature vector obtained by fusing the spatial correlation matrix with the time dimension feature vector comprehensively and accurately reflects the vehicle operation status. Finally, through comparative analysis with the benchmark library, it can identify faults based on the comprehensive judgment of spatiotemporal features, thereby improving the accuracy and reliability of fault diagnosis, effectively avoiding the problems of misjudgment and missed judgment caused by ignoring the spatiotemporal features of the data, and can better meet the needs of efficient and safe operation of modern commercial vehicles.

[0179] See also Figure 3 , Figure 3 This is a diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented:

[0180] Collect sensor data from multiple sensors in the operation of commercial vehicles in real time, and divide the sensor data into multiple time segments according to preset time intervals;

[0181] Construct a spatial correlation matrix between the sensor data in each time segment; the spatial correlation matrix is used to characterize the mutual influence relationship between different sensor data;

[0182] Extracting time dimension features from each sensor data in each time segment to obtain a time dimension feature vector for each sensor data based on the change characteristics of the data in the time segment;

[0183] The spatial correlation matrix of each time segment is fused with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector;

[0184] The spatiotemporal feature vector is compared and analyzed with the benchmark spatiotemporal feature vectors in the pre-established benchmark library, and the fault type and fault location of the commercial vehicle are determined based on the comparative analysis results.

[0185] See also Figure 4 , Figure 4 Detailed description of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. Figure 4 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, the following steps are implemented:

[0186] Collect sensor data from multiple sensors in the operation of commercial vehicles in real time, and divide the sensor data into multiple time segments according to preset time intervals;

[0187] Construct a spatial correlation matrix between the sensor data in each time segment; the spatial correlation matrix is used to characterize the mutual influence relationship between different sensor data;

[0188] Extracting time dimension features from each sensor data in each time segment to obtain a time dimension feature vector for each sensor data based on the change characteristics of the data in the time segment;

[0189] The spatial correlation matrix of each time segment is fused with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector;

[0190] The spatiotemporal feature vector is compared and analyzed with the benchmark spatiotemporal feature vectors in the pre-established benchmark library, and the fault type and fault location of the commercial vehicle are determined based on the comparative analysis results.

[0191] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion provided by the above methods, which includes:

[0192] Collect sensor data from multiple sensors in the operation of commercial vehicles in real time, and divide the sensor data into multiple time segments according to preset time intervals;

[0193] Construct a spatial correlation matrix between the sensor data in each time segment; the spatial correlation matrix is used to characterize the mutual influence relationship between different sensor data;

[0194] Extracting time dimension features from each sensor data in each time segment to obtain a time dimension feature vector for each sensor data based on the change characteristics of the data in the time segment;

[0195] The spatial correlation matrix of each time segment is fused with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector;

[0196] The spatiotemporal feature vector is compared and analyzed with the benchmark spatiotemporal feature vectors in the pre-established benchmark library, and the fault type and fault location of the commercial vehicle are determined based on the comparative analysis results.

[0197] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0198] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A commercial vehicle fault diagnosis method based on spatiotemporal feature fusion, characterized in that: include: Collect sensor data from multiple sensors in the operation of commercial vehicles in real time, and divide the sensor data into multiple time segments according to preset time intervals; Constructing a spatial correlation matrix between the sensor data in each time segment; the spatial correlation matrix is used to characterize the mutual influence relationship between different sensor data; Extracting time dimension features from each sensor data in each time segment to obtain a time dimension feature vector for each sensor data based on the change characteristics of the data in the time segment; The spatial correlation matrix of each time segment is fused with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector; The spatiotemporal feature vector is compared and analyzed with a reference spatiotemporal feature vector in a pre-established reference library, and the fault type and fault location of the commercial vehicle are determined according to the comparison and analysis results.

2. The commercial vehicle fault diagnosis method based on spatiotemporal feature fusion according to claim 1 is characterized in that: The spatial correlation matrix of each time segment is fused with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector, including: For each time segment, dimensionally expand the time dimension feature vector according to the number of rows and columns of the spatial correlation matrix to obtain a dimensionally expanded feature vector; Converting the dimensionally expanded feature vector into a three-dimensional tensor, and performing a feature embedding operation on the spatial correlation matrix based on an embedding coefficient matrix combined with the three-dimensional tensor to obtain a feature-embedded correlation matrix; Starting from the central element of the feature-embedded correlation matrix, the elements in the feature-embedded correlation matrix are spirally transformed according to a preset spiral order to obtain a spirally transformed correlation matrix; Multi-dimensional fusion is performed based on the spiral transformed correlation matrix to obtain the spatiotemporal feature vector in each time segment.

3. The commercial vehicle fault diagnosis method based on spatiotemporal feature fusion according to claim 2 is characterized in that: The multi-dimensional fusion based on the spiral transformation correlation matrix to obtain the spatiotemporal feature vector in each time segment includes: For each time segment, a multidimensional convolution operation is performed on each matrix element in the spiral transformed correlation matrix based on a multidimensional convolution kernel to obtain a convolution fusion correlation matrix; According to a preset mapping relationship and in combination with the three-dimensional coordinates of the first matrix elements, each first matrix element in the convolution fusion association matrix is mapped to a second matrix element under the corresponding two-dimensional coordinates, and a topologically reconstructed association matrix is constructed based on each second matrix element; Based on each second matrix element in the topology reconstructed association matrix, feature cross-fusion is performed to obtain a cross-fused association matrix, and each matrix element in the cross-fused association matrix is aggregated to generate a spatiotemporal feature vector in each time segment.

4. The commercial vehicle fault diagnosis method based on spatiotemporal feature fusion according to claim 1, characterized in that: The step of constructing a spatial correlation matrix between sensor data in each time segment includes: For the sensor data in each time segment, the time series data of each sensor is processed based on the preset local window size to obtain the local extreme value feature value of each sensor data; Calculate the data difference of each sensor data at adjacent times and determine the fluctuation trend index of each sensor data within the time segment; Determine the initial correlation strength between the sensor data based on the local extreme value characteristic value and fluctuation trend index between the sensor data; Determine the influence direction between the sensor data based on the data change time delay and correlation coefficient between the sensor data; Based on the initial correlation strength and influence direction between the sensor data, a spatial correlation matrix between the sensor data in each time segment is constructed.

5. The commercial vehicle fault diagnosis method based on spatiotemporal feature fusion according to claim 4 is characterized in that: The method of constructing a spatial correlation matrix between the sensor data in each time segment based on the initial correlation strength and influence direction between the sensor data includes: Correcting the initial correlation strength based on the influence direction between the sensor data to obtain the corrected correlation strength between the sensor data; In each time segment, based on the actual spatial position of each sensor on the commercial vehicle, determining a spatial position mapping factor between each sensor data corresponding to each sensor; Determining a final correlation strength between each sensor data based on the spatial position mapping factor and the corrected correlation strength between each sensor data; The spatial correlation matrix between the sensor data in each time segment is constructed based on the final correlation strength between the sensor data.

6. The commercial vehicle fault diagnosis method based on spatiotemporal feature fusion according to claim 1 is characterized in that: The time dimension feature extraction of each sensor data in each time segment to obtain a time dimension feature vector of each sensor data based on the change characteristics of the data in the time segment includes: For each sensor data in each time segment, the data sequence is discretized to convert the continuous time data sequence into a discrete point set; Based on the local slopes calculated for each group of three adjacent data points in the discretized data points, a local slope matrix is constructed; Determine local extreme points based on second-order differences, and construct trend lines based on local extreme points combined with piecewise cubic Hermite interpolation; Based on the local slope matrix and trend line, the changing trend and fluctuation range of the data in the time segment are analyzed to obtain the time dimension feature vector of each sensor data in each time segment.

7. The commercial vehicle fault diagnosis method based on spatiotemporal feature fusion according to claim 6 is characterized in that: The change trend and fluctuation range of the data in the time segment are analyzed based on the local slope matrix and trend line to obtain the time dimension feature vector of each sensor data in each time segment, including: For each sensor data in each time segment, the difference between adjacent local slopes is calculated based on the local slope matrix to determine the trend complexity; Determining a trend angle based on an angle between a line connecting a starting point and an end point of the trend line within the time segment and a horizontal axis; determining a fluctuation amplitude based on a maximum value of absolute values of differences between all data points in the data sequence and points corresponding to the trend line, and determining a fluctuation frequency based on the number of times the data sequence crosses the trend line; The trend complexity, the trend angle, the fluctuation amplitude and the fluctuation frequency are integrated to obtain a time dimension feature vector of each sensor data in each time segment.

8. A commercial vehicle fault diagnosis device based on spatiotemporal feature fusion, characterized in that: Applicable to the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion as claimed in any one of claims 1 to 7; The commercial vehicle fault diagnosis device based on spatiotemporal feature fusion includes: A data processing module is used to collect sensor data from multiple sensors in the operation of a commercial vehicle in real time and divide the sensor data into multiple time segments according to preset time intervals; A correlation matrix construction module is used to construct a spatial correlation matrix between each sensor data in each time segment; the spatial correlation matrix is used to characterize the mutual influence relationship between different sensor data; A time feature extraction module is used to extract time dimension features of each sensor data in each time segment, so as to obtain a time dimension feature vector of each sensor data based on the change characteristics of the data in the time segment; The feature fusion module is used to fuse the spatial correlation matrix of each time segment with the corresponding time dimension feature vector to obtain the spatiotemporal feature vector; The fault diagnosis module is used to compare and analyze the spatiotemporal feature vector with a reference spatiotemporal feature vector in a pre-established reference library, and determine the fault type and fault location of the commercial vehicle based on the comparison and analysis results.

9. An electronic device comprising: Memory for storing computer software programs; A processor for reading and executing the computer software program, characterized in that when the processor executes the computer software program, it implements the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing a computer software program, wherein: When the computer software program is executed by a processor, the commercial vehicle fault diagnosis method based on spatiotemporal feature fusion as claimed in any one of claims 1 to 7 is implemented.

Images