Vehicle-mounted CAN bus data decoding method based on deep learning

Abstract

The invention discloses a vehicle-mounted CAN bus data decoding method based on deep learning, and the method comprises the steps: firstly collecting original CAN messages, then carrying out the de-noising processing of the data, and grouping the data into time sequence subsets according to the ID of the messages; then, multi-level features are extracted from a bit level, a byte level and a message level by using a deep learning model, and specifically, bit flipping rate and local space feature analysis, cross-byte dependency analysis and capture of context and periodic modes in a time sequence are involved; and then integrating the multi-level features through a deep learning model to automatically identify the initial boundary of the signal, and inferring the most significant bit and the byte sequence of the signal by using a mixed byte sequence scanning algorithm, thereby generating detailed signal domain information. And finally, outputting a signal decoding result. According to the method, the problems of complex manual analysis, low precision and insufficient adaptability in a traditional method are solved through conjoint analysis of multi-level features and accurate inference of the signal boundary and the signal domain.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle network security, and particularly relates to a method for decoding in-vehicle CAN bus data based on a deep learning algorithm. Background Art

[0002] The Controller Area Network (CAN) bus in vehicles is a key protocol for communication between electronic control units (ECUs) in modern vehicles and is widely used in fields such as vehicle diagnosis, fault troubleshooting, security vulnerability analysis, and personalized modification. The CAN bus adopts a broadcast communication mode and identifies and parses message data through a unique message ID. However, due to the lack of a unified CAN message format among different manufacturers and vehicle models, it is very challenging to parse CAN data. This proprietariness makes it difficult for third-party developers to effectively decode CAN data without detailed technical documentation. Currently, the main method for parsing CAN data is to infer the meaning of CAN messages through reverse engineering. Traditional manual reverse engineering usually requires experts to capture and analyze the CAN bus communication flow and verify the function of specific messages or their correlation with known vehicle behaviors through experiments. Although this method is effective, it is labor-intensive, time-consuming, and requires professional knowledge, making it difficult to apply on a large scale.

[0003] With the rapid development of artificial intelligence technology, researchers have begun to explore automated CAN bus reverse engineering methods. Some studies have introduced machine learning techniques to identify the boundaries of CAN signals through low-level features such as bit flip rates. However, these methods have several limitations:

[0004] (1) Single feature: Most existing methods mainly focus on bit-level features (such as bit flip rates) while ignoring the spatio-temporal relationships at the byte level and message level, resulting in insufficient ability to parse complex signals.

[0005] (2) Byte order problem: Many methods assume that all signals use a unified byte order (big endian or little endian), but in actual CAN data, the situation of mixed byte orders is very common. This assumption causes existing methods to perform poorly when processing actual vehicle data.

[0006] (3) Limited generality: Some studies rely on manually defined thresholds or assume specific data structures, and these methods are difficult to generalize across different vehicle models or environments.

[0007] (4) Limited accuracy: Due to problems such as fuzzy signal boundaries, signal noise, and incomplete data, existing methods still have obvious deficiencies in the accuracy of signal boundary and field identification.

[0008] To address the above problems, researchers have proposed some more advanced algorithms in recent years. For example, Markovitz et al. proposed a CAN signal classification method based on data domain segmentation, which uses clustering techniques to perform a rough classification of CAN data. However, these methods can only provide preliminary signal boundary recognition results and still require further manual analysis. Nolan et al. improved the signal marking process through a bit flip counting algorithm, attempting to automatically identify signal boundaries. However, due to its reliance on bit-level features, it is difficult to comprehensively reflect the relationships between signals at the byte and message levels. In view of the above technical limitations, the present invention proposes a brand-new end-to-end deep learning algorithm framework, aiming to break through the limitations of existing methods in signal boundary recognition and field parsing, improve decoding accuracy and reduce the complexity of reverse engineering by introducing deep learning techniques, and provide a new solution for the field of vehicle data parsing. Summary of the Invention

[0009] In view of the above problems, the present invention proposes a method for decoding vehicle CAN bus data based on deep learning, which uses a deep learning model to perform multi-level feature extraction and intelligent decoding on vehicle CAN bus data. By jointly analyzing bit-level, byte-level, and message-level features, as well as accurately inferring signal boundaries and signal domains, the problems of complex manual parsing, low accuracy, and insufficient adaptability in traditional methods are solved.

[0010] Technical Solution:

[0011] A method for decoding vehicle CAN bus data based on deep learning according to the present invention includes the following steps:

[0012] Step 1, collect the original message data of the vehicle CAN bus, including data frames and message IDs;

[0013] Step 2, perform denoising and partitioning preprocessing on the original message data, remove redundant information and invalid data, and group them into time series subsets according to message IDs;

[0014] Step 3, based on the deep learning model, perform multi-level feature extraction including bit-level features, byte-level features, and message-level features. Among them,

[0015] The bit-level feature extraction is: analyze the bit flip rate and local spatial characteristics; the byte-level feature extraction is: parse the cross-byte dependency relationship; the message-level feature extraction is: capture the context and periodic patterns in the time series;

[0016] Step 4, use the deep learning model to fuse the multi-level features extracted in Step 3 to automatically identify the start boundary of the signal; Step 5, combine the identified boundary and the mixed byte order scanning algorithm to infer the most significant bit and byte order of the signal, and generate signal domain information;

[0017] Step 6, output the complete signal decoding result, including signal boundaries, byte order, and the decoding formula for the mapped physical quantity.

[0018] Preferably, in step 1, the original message data including data frames and message IDs is collected through the CAN bus.

[0019] Preferably, in step 2, the original message data is denoised, then grouped according to message IDs, and further divided into time series subsets, each subset containing a fixed number of messages.

[0020] Preferably, the denoising method includes one or more of static bit filtering, low variance filtering, and outlier detection. The static bit filtering is to remove the fields with unchanged bit values in the data. The low variance filtering is to delete the signals with variances lower than the set threshold. The outlier detection is to detect and remove abnormal data using statistical methods.

[0021] Preferably, in step 3, the bit-level feature extraction is as follows: convert the data field of the message into a binary matrix, and use convolutional operations to extract local spatial features. The byte-level feature extraction is as follows: capture the signal patterns spanning multiple bytes by analyzing the dependencies across bytes. The message-level feature extraction is as follows: analyze the message data in the time series to extract cross-message context information and periodic features.

[0022] Preferably, in the bit-level feature extraction, first calculate the flipping frequency of each bit to detect the region where the signal boundary is located, and then extract local spatial patterns through a convolutional kernel to identify the detailed features of the signal start position.

[0023] Preferably, in the byte-level feature extraction, first use a convolutional kernel to capture the dependencies between bytes, detect the signal boundaries across bytes, and then identify the situation where big-endian and little-endian coexist to solve the inconsistency of the signal storage order.

[0024] Preferably, in the message-level feature extraction, first analyze the periodicity and context information of the signal, and then identify the correlation of the signal in multiple messages.

[0025] Preferably, in step 4, use a deep convolutional neural network model, combine bit-level, byte-level, and message-level features to generate multi-channel inputs, extract high-dimensional features through convolutional networks and residual connections to generate high-dimensional feature expressions, and automatically identify the start position of the signal (i.e., the least significant bit, LSB) through a multi-label classifier to achieve precise positioning of the signal boundary.

[0026] Preferably, in step 5, the hybrid endian scanning algorithm is used to infer the byte storage order of the signal, distinguishing between big endian and little endian; by analyzing the bit flip rate and boundary characteristics, the most significant bit of the signal is inferred to generate complete signal domain information, including the start position, length, byte order, etc. of the signal.

[0027] Preferably, the scanning directions of big endian and little endian are different. When the signal is stored in big endian byte order, the arrow advances to the right, scanning from the high bit to the low bit gradually until the flip rate is 0 or other signal boundaries are encountered; for little endian byte order, the arrow advances to the left, and the signal starts from the low bit and extends in the direction of the high bit.

[0028] Preferably, in step 6, the decoding formula is: V phy =(V raw ×Factor)+Offset, where: V raw is the original signal value; Factor is the scaling factor; Offset is the offset.

[0029] Beneficial effects:

[0030] (1) The vehicle-mounted CAN bus data decoding method based on deep learning disclosed in this application realizes multi-level feature extraction from three levels of bits, bytes and messages, comprehensively captures local and global information of CAN data, especially significantly enhances the ability to process signals with mixed byte order, and can adapt to data complexity in different vehicle models and real driving scenarios;

[0031] (2) The vehicle-mounted CAN bus data decoding method based on deep learning disclosed in this application combines the deep learning model to extract features by bit flip rate, cross-byte dependence and timing pattern; adopts a hybrid byte order dynamic inference algorithm to adapt to data formats of different vehicle models; improves the accuracy by 10-30% compared with traditional methods in automatic signal boundary inference;

[0032] (3) The vehicle-mounted CAN bus data decoding method based on deep learning disclosed in this application designs a dedicated convolutional neural network (CNN) structure, and improves the generalization ability and noise resistance of the model through residual connection, batch normalization and Dropout techniques;

[0033] (4) The vehicle-mounted CAN bus data decoding method based on deep learning disclosed in this application combines a signal boundary and field recognition module, reduces manual dependence, adopts an automated deep learning method, does not require manual participation in parsing CAN data, reduces the dependence on expert experience, and saves labor costs. Description of the Drawings

[0034] Figure 1 It is a schematic flowchart of an embodiment of the present invention;

[0035] Figure 2 Structure diagram of CAN message format for an embodiment of the present invention;

[0036] Figure 3 Example diagram of signal distribution for an embodiment of the present invention;

[0037] Figure 4 Schematic diagram for identifying signal boundaries for an embodiment of the present invention;

[0038] Figure 5 Structure diagram of the AutoDBC deep learning model for an embodiment of the present invention. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0040] This application discloses a vehicle-mounted CAN bus data decoding method based on deep learning, which uses a deep learning model to perform multi-level feature extraction and intelligent decoding on vehicle-mounted CAN bus data. Through the joint analysis of bit-level, byte-level, and message-level features, as well as the accurate inference of signal boundaries and signal domains, it solves the problems of complex manual parsing, low accuracy, and insufficient adaptability in traditional methods. Through a deep convolutional neural network model, multi-dimensional features are fused to improve the accuracy of signal boundary recognition, and further achieve efficient decoding of mixed byte-order signals. The present invention does not need to rely on manufacturer-specific DBC files, has good universality and generality, and is applicable to various vehicle models and complex driving scenarios.

[0041] As Figure 1 shown, it includes the following steps:

[0042] Step 1, collect the original message data of the vehicle-mounted CAN bus, including data frames and message IDs.

[0043] The structure composition of a standard CAN (Controller Area Network) frame is as Figure 2 shown, which describes the key fields of the CAN data frame used in the vehicle communication process.

[0044] SOF (Start of Frame): Start frame flag, used for frame synchronization, indicating the start of a new CAN frame.

[0045] AID (Arbitration ID): The arbitration ID is the unique identifier of the data frame and determines the priority of the frame on the bus. In a standard frame, this field is 11 bits, and in an extended frame, it is 29 bits.

[0046] RTR (Remote Transmission Request): The remote transmission request bit is used to distinguish between data frames and remote frames.

[0047] Control field: It contains the IDE bit (indicating the extended frame format), reserved bits, and the data length code (DLC), where the latter indicates the number of bytes contained in the data field.

[0048] Data field: The data field contains the actual CAN signals, up to 8 bytes (64 bits). These signals usually represent vehicle sensor data or control instructions.

[0049] CRC (Cyclic Redundancy Check): The cyclic redundancy check field is used to detect errors in data transmission.

[0050] ACK (Acknowledge): The acknowledge field is used by the receiving node to confirm to the sending node that the message has been received.

[0051] EOF (End of Frame): The end of frame flag indicates the end of a CAN frame.

[0052] In the figure, the logical areas of each field are distinguished by color, highlighting the importance of the data field. The data field is the part of the CAN message that actually carries signal information, and the boundaries and field definitions of the signals are the core issues in signal parsing.

[0053] In this embodiment, Message ID: 0x1A1 in the CAN acquisition data is selected from the experimental dataset

[0054] (Accelerator Pedal), and this message contains 8 bytes (64 bits) of raw data as follows:

[0055] Byte[0]: 0x32 Byte[1]: 0x45 Byte[2]: 0x91 Byte[3]: 0xF2Byte[4]: 0x67 Byte[5]: 0xB3 Byte[6]: 0x29 Byte[7]: 0xD5.

[0056] This data comes from the ECU (Electronic Control Unit) of the vehicle and contains signals of multiple physical quantities, such as the accelerator pedal position. Since the DBC file is unknown, it is necessary to automatically parse the signal boundaries and byte order through AutoDBC.

[0057] Step 2: Perform denoising and partitioning preprocessing on the original message data, remove redundant information and invalid data, and group them into time series subsets according to the message ID.

[0058] Specifically, the denoising process adopted includes methods such as static bit filtering, low-variance filtering, and outlier detection to remove invalid data and redundant information and ensure the quality and validity of the data. Among them, static bit filtering is to remove the fields with unchanged bit values (bit flipping rate is 0) in the data to avoid the interference of invalid information on model training; low-variance filtering is to delete the signals with variances lower than the set threshold and remove the noise signals with too little change; outlier detection is to use statistical methods (such as Z-score or IQR method) to detect and remove the abnormal data caused by sensor failures or communication errors.

[0059] Group the denoised data according to the message ID and further divide it into time series subsets. Each subset contains a fixed number of messages (such as 1000), and retain the time-dependent characteristics of the data to improve the generalization ability of the model.

[0060] In this embodiment, in order to identify the signal boundary, we extract the time series (ID Trace) from the multiple occurrences of Message ID: 0x1A1 in the entire data stream to form the following matrix:

[0061] T0: 0x32 45 91F2 67B3 29D5

[0062] T1: 0x34 44 90F3 68B2 28D4

[0063] T2: 0x33 46 91F1 67B4 2A D6 ...

[0065] This time series is used to analyze the bit flipping rate to find the least significant bit (LSB) and the most significant bit (MSB) of the signal, and provide a feature matrix for the subsequent neural network input.

[0066] Step 3: Based on the deep learning model, perform multi-level feature extraction on the same original CAN signal, including bit-level features, byte-level features, and message-level features. Among them,

[0067] (1) Bit-level feature extraction: Convert the data field of each message into an 8×8 binary matrix, and use convolution operations to extract local spatial features, such as the bit-flipping rate, to analyze the fine-grained information of the signal boundary, which can be used to detect the least significant bit (LSB) of the signal. The bit-flipping rate is calculated as follows: calculate the flipping frequency of each bit to detect the region where the signal boundary is located; analyzing the fine-grained information of the signal boundary means extracting local spatial patterns through a 3×3 convolution kernel to identify the detailed features at the starting position of the signal.

[0068] The identification of possible signal boundaries (bit-flipping rate) for each bit is expressed as:

[0069]

[0070] Bits with a low flipping rate may be LSBs, and bits with a high flipping rate may be MSBs. The flipping rate is used to identify possible signal boundaries, discover those relatively stable bits (usually LSBs), and filter out noise bits to improve the recognition accuracy of signal boundaries.

[0071] For example:

[0072]

[0073] If the flipping rate of a certain bit is much lower than the surrounding bits, it may be a signal boundary (LSB).

[0074] (2) Byte-level feature extraction: By analyzing the dependencies across bytes, use an 8×8 convolution kernel to capture the dependencies between bytes, detect signal boundaries across bytes, and capture signal patterns spanning multiple bytes; solve the problem of mixed byte order, identify the situation where big-endian and little-endian coexist, and solve the inconsistency of signal storage order.

[0075] Feature mapping across bytes helps identify big-endian (Big-Endian) and little-endian (Little-Endian) storage formats. Extract dependencies across bytes through an 8×8 convolution kernel to detect how the signal is stored in bytes. If the high-order part of the signal is at the low-byte position, it is in little-endian format; if the high-order part of the signal is at the high-byte position, it is in big-endian format. Feature mapping across bytes is used to solve the problem of mixed byte order, ensure the correctness of signal parsing, and prevent decoding errors caused by different storage orders.

[0076] For example:

[0077] Storage method Byte order (stored by address)

[0078] Big-Endian MSB →... → LSB

[0079] Little-Endian, LSB →... → MSB.

[0080] (3) Message-level feature extraction: Analyze the message data in the time series, analyze the periodicity and context information of the signal, extract cross-message context information and periodic features, identify the correlation of the signal in multiple messages, optimize the signal boundary positioning accuracy, and enhance the model's understanding ability of complex signals.

[0081] Message-level feature extraction observes the periodicity and change patterns of the signal through a time window, combines time series data through convolution + pooling operations, and extracts the context features of the signal. Message-level periodic pattern detection can identify periodic signals in the CAN bus, such as speed, rotational speed, fuel consumption, etc., which can help identify whether there are signal boundaries at fixed positions, ensure signal boundary consistency, and improve the reliability of boundary inference.

[0082] For example:

[0083]

[0084] If the signal presents a fixed periodic pattern on the time axis, it indicates that its boundary is stable and can be used to enhance the model's recognition ability.

[0085] The results of multi-dimensional extraction are feature vectors at different levels, which jointly determine the signal boundary and storage format.

[0086] In this embodiment, the deep learning model of the AutoDBC framework performs three-layer feature extraction: bit-level, byte-level, and message-level.

[0087] Bit-level input: 64×N-dimensional matrix, representing the flipping situation of each bit.

[0088] Byte-level input: 8×N-dimensional matrix, representing the change of each byte in the time dimension.

[0089] Message-level input: Contains multiple data frames, providing the ability to learn timing patterns.

[0090] AutoDBC adopts a CNN structure:

[0091] Bit-level convolution (3×3 convolution kernel): Extract local bit flip rate features.

[0092] Byte-level convolution (8×8 convolution kernel): Extract cross-byte signal dependencies.

[0093] Message-level convolution (3×3 convolution kernel): Capturing time series patterns.

[0094] Extraction results:

[0095] Bit-level features:

[0096] Byte[4] bits [3]-bit[7] have a relatively high flipping rate and may be the MSB.

[0097] Byte[6] bits [0]-bit[2] have a flipping rate close to 0 and may be the LSB.

[0098] Byte-level features:

[0099] The signals between Byte[2] and Byte[4] have strong correlations and may belong to the same physical signal.

[0100] Byte[5]-Byte[7] are stable in all time steps and may be status signals.

[0101] Message-level features:

[0102] This ID is sent at a frequency of 10 ms, indicating that this signal belongs to a real-time dynamic control signal.

[0103] Step 4: Use a deep learning model to fuse the multi-level features extracted in Step 3 to accurately identify the signal boundaries in the data field, thereby solving the challenge of being unable to parse signals due to the manufacturer's proprietary format.

[0104] As Figure 5 shown, in this embodiment, the method for the deep learning model to fuse features includes feature concatenation, residual connection, and pooling operations, where

[0105] Feature concatenation: Map features at different levels to a high-dimensional space and perform feature concatenation in the channel dimension. Specifically, the bit-level, byte-level, and message-level features are respectively pre-processed through their respective network layers, and then concatenated in a higher-dimensional space, enabling the model to capture multi-level information in the data and thus improving the ability to understand complex patterns.

[0106] Residual connection: Preserve the original information in the convolutional layer to reduce information loss and improve recognition accuracy. By introducing direct input-output connections in the convolutional layer, the residual connection can effectively reduce information loss and ensure that the key features of the original data are retained. As Figure 5In the architecture, residual connections are connected between each layer to ensure the continuity and integrity of information.

[0107] Pooling: Normalize the features at different levels to extract global information. Figure 5 In the architecture, by downsampling the feature map, the pooling operation can extract global information and help the model better understand the overall structure of the data. Figure 5 In the architecture shown, the custom pooling layer is combined with the reshaping operation to further optimize the representation of the features, allowing the model to capture key information over a wider range.

[0108] If only bit-level features are used, global context information will be lost, resulting in reduced boundary recognition accuracy. If only byte-level features are used, mixed byte order signals may not be processed correctly. Message-level features can provide timing information, but without combining the first two, it is difficult to determine the specific boundary location. Therefore, it is particularly important to use a deep learning model to integrate the above multi-level features. Figure 5 The architecture shown can achieve effective integration of bit-level, byte-level, and message-level features. After a series of processing such as feature extraction, residual connection, and pooling operation, a more accurate and comprehensive output result is finally obtained through the fully connected layer and activation function.

[0109] An example of signal distribution is Figure 3 As shown, it reflects the signal's start bit, length, and distribution in the data frame, which is used to explain the method of identifying signal boundaries in CAN data frames. In this figure, each small square represents a bit of the CAN message data frame, and the color represents the start bit of a specific signal, the signal length, and the bit range it occupies. Among them, the blue area represents the start bit and background information, which is the first part to be excluded in bit-level analysis to distinguish the actual signal. The green, red, yellow, and orange areas respectively represent the bit distribution range of different signals. The specific starting positions and lengths of these signals in the data frame are different, reflecting the complexity of signal distribution in mixed byte order (big endian and little endian). The gray area represents the rest of the message frame, which can be regarded as the background of unassigned signals.

[0110] In this embodiment, USER_BRAKE is used to indicate the user brake status, bit pattern: 0 1 1 0 0 1 1 0.

[0111] ESP_DISABLED is used to indicate whether the Electronic Stability Program (ESP) is disabled. Bit pattern: 0 0 0 0 0 0 00. Specific bit: The 4th bit is orange, indicating that this bit is used to indicate whether the ESP is disabled. BRAKE_HOLD_ACTIVE: and BRAKE_HOLD_ENABLED: are used to indicate the status of the brake hold function. Bit pattern: 0 0 0 0 0 0 0 0. Specific bits: The 2nd and 3rd bits are green and red respectively, used to indicate whether the brake hold function is active or available. COUNTER is used for synchronization or other control purposes. Bit pattern: 0 0 0 0 0 0 0 0. Specific bit: The 4th bit is yellow, indicating that this bit is used for the counter. CHECKSUM is used for error detection in data transmission. Bit pattern: 0 0 0 0 1 1 0 1. Specific bit: The entire 8 bits are used to calculate the checksum.

[0112] The signal distribution identified by colors in the figure shows how bit-level, byte-level, and message-level analyses are combined in multi-level feature extraction for signal boundary recognition. Especially when dealing with mixed-endian data, correctly identifying signal boundaries is crucial for parsing the meaning of each signal in CAN data.

[0113] At the same time, this example illustrates the complexity of signal boundary recognition and the key steps adopted by the AutoDBC framework during the parsing process, namely automatically inferring the start bit, length, and field range of signals through multi-dimensional feature extraction and deep learning models. The core strategy of this method lies in multi-level feature extraction, which enables the system to effectively cope with the challenges of mixed-endian and uneven signal distribution.

[0114] Multi-dimensional feature extraction is the core mechanism for the AutoDBC framework to achieve signal boundary recognition. The key to understanding this mechanism lies in recognizing how the information provided by different levels of features (including bit-level, byte-level, and message-level) complements each other to accurately infer the start bit, length, and field range of signals. Specifically: Bit-level features: Analyze the bit flip rate and local spatial characteristics to help identify static signals; Byte-level features: Parse cross-byte dependencies to ensure accurate data alignment; Message-level features: Capture context and periodic patterns in time series to ensure the timing consistency of signals.

[0115] If multi-dimensional feature extraction is not performed, the following problems may occur in subsequent steps:

[0116] 1. Relying solely on the bit flip rate may lead to misjudgment because the flip rate of some static signals is low and difficult to identify.

[0117] 2. The lack of byte-level features may prevent the correct parsing of cross-byte signals, resulting in data alignment errors.

[0118] 3. The absence of message-level features affects the timing consistency of signals, making it difficult to capture the signal change patterns.

[0119] Therefore, multi-dimensional feature extraction not only improves the recognition accuracy of signal boundaries but also effectively overcomes the challenges brought by mixed byte order and uneven signal distribution.

[0120] Details of the signal boundary recognition process are as Figure 4 shown, which mainly illustrates how to determine the signal field from the least significant bit (LSB) to the most significant bit (MSB) in a mixed byte order environment.

[0121] Green marker (LSB): The green cells in the figure represent the least significant bit (Least Significant Bit) of the signal, which is the starting position of the signal range. The determination of the LSB is a key step in signal boundary recognition.

[0122] Orange marker (MSB): The orange cells represent the most significant bit (Most Significant Bit) of the signal, which is the end position of the signal range. By gradually deriving from the LSB, the position of the MSB can be accurately identified.

[0123] Grey path (bit tracing): The grey area and the blue arrow in the figure represent the bit flip path. By calculating the bit flip rate, trace from the LSB along the byte order (big endian or little endian) of the signal to the MSB in sequence.

[0124] Boundary recognition logic: Bit flip rate: Calculate the number of changes of each bit from 0 to 1 or from 1 to 0. Select the bit with a lower bit flip rate as the least significant bit LSB of the signal, and select the bit with a higher bit flip rate as the most significant bit MSB of the signal.

[0125] The calculation method of the flip rate is as follows:

[0126]

[0127] In the formula, R b The value range is [0, 1]. 0 means that this bit remains unchanged throughout the dataset, and 1 means that this bit flips in each sampling period.

[0128] Bits with a lower flip rate (LSB): Usually R b <0.2, that is, in most cases this bit remains stable.

[0129] Bits with a higher flip rate (MSB): Usually R b >0.6, indicating that this bit changes frequently.

[0130] When the signal is stored in big-endian byte order, the arrow advances to the right, scanning from the high-order bits to the low-order bits step by step until the flip rate is 0 or other signal boundaries are encountered. For little-endian byte order, the arrow advances to the left, and the signal starts from the low-order bits and extends in the direction of the high-order bits. Among them, in big-endian, the most significant byte (MSB) is stored at the lowest address, and the data is arranged from high-order to low-order; in little-endian, the least significant byte (LSB) is stored at the lowest address, and the data is arranged from low-order to high-order. The recognition method is as follows: by analyzing the trend of the bit flip rate, if the flip rate decreases from high-order to low-order, it is inferred as big-endian; if the flip rate decreases from low-order to high-order, it is inferred as little-endian; in the case of mixed storage, AutoDBC adopts a dynamic byte order detection algorithm to infer the byte order of each signal one by one and select the correct scanning direction.

[0131] Signal distribution complexity: The diversity of signal distribution in the figure (such as the mixture of big-endian and little-endian signals) shows the complexity of parsing mixed byte-order signals. AutoDBC can accurately infer signal boundaries by combining the feature extraction ability of the deep learning model.

[0132] In this embodiment, AutoDBC adopts a multi-layer feature fusion strategy: including channel concatenation (Concatenation):

[0133] Bit-level, byte-level, and message-level features are concatenated in the channel dimension; attention mechanism (Attention Mechanism): enhancing the feature weights at the signal boundaries; residual connection (Residual Connection): preventing the disappearance of feature gradients and improving the signal parsing accuracy.

[0134] Fusion result:

[0135] Byte[2]-Byte[4] may belong to a 16-bit signal.

[0136] Byte[6]-Byte[7] may be status flags.

[0137] Step 5, combining the recognized boundaries and the mixed byte-order scanning algorithm, infer the most significant bit and byte order of the signal, and generate signal domain information.

[0138] Specifically, the hybrid endian scanning algorithm is used to infer the byte storage order of the signal, distinguishing between big endian and little endian; by analyzing the bit flip rate and boundary characteristics, the most significant bit (MSB) of the signal is inferred to generate complete signal domain information, including the start position, length, endianness, etc. of the signal. Among them, the method for inferring the MSB is as follows: First, locate the least significant bit (LSB), then starting from the LSB, scan along the direction of decreasing bit flip rate. When the flip rate drops to 0 or the next signal boundary is encountered, identify the MSB; finally, optimize the MSB inference in combination with the deep learning model of AutoDBC to avoid errors.

[0139] In this embodiment,

[0140] Flip rate calculation:

[0141] The flip rate of Byte[4] bit[3]-bit[7] is relatively high, inferred as the MSB;

[0142] The flip rate of Byte[6] bit[0]-bit[2] is relatively low, inferred as the LSB.

[0143] Boundary inference:

[0144] If the flip rate increases with the bit position, it is inferred as little endian.

[0145] If the flip rate decreases with the bit position, it is inferred as big endian.

[0146] Final inference result:

[0147] Signal 1: Byte[2] bitte[4] bit[2] (big endian format).

[0148] Signal 2: Byte[6] bit[0] → Byte[7] bit[7] (little endian format).

[0149] Signal domain information generation, the parsed signal information is as follows:

[0150]

[0151] Step 6, output the complete signal decoding result, including the signal boundary, endianness, and the decoding formula of the mapped physical quantity.

[0152] The calculation formula of the signal physical quantity is expressed as:

[0153] V phy = (V raw × Factor) + Offset

[0154] Where: V rawis the original signal value; Factor is the scaling factor; Offset is the offset;

[0155] This formula is used to convert CAN data into physical units such as speed (km / h), rotational speed (rpm), etc.

[0156] In this embodiment, the final decoding result of AutoDBC is: Message ID: 0x1A1, accelerator pedal = 78%.

[0157] Figure 4 It clearly illustrates the dynamic process of signal boundary recognition and demonstrates the automated parsing method from the LSB to the MSB signal fields. This process is of great significance for correctly decoding and deriving signals in complex CAN data and is also one of the core functions of the AutoDBC framework.

[0158] The deep learning model structure for signal boundary recognition in the AutoDBC framework is as Figure 5 shown. AutoDBC adopts a CNN structure with multi-level feature extraction, combined with residual connections and pooling techniques, to improve the accuracy of signal boundary recognition, highlighting how the model extracts information from different feature levels and completes the signal boundary recognition task.

[0159] (1) Input Layer:

[0160] Bit-level input (left): The data frame is represented as an 8×8 binary matrix, with each cell corresponding to a bit. This type of input captures the local interaction between bits.

[0161] Byte-level input (middle): The data frame is decomposed into multiple bytes to further extract the global relationship across bytes and input the byte-level data.

[0162] Message-level input (right): The data frame sequence is processed into a time series matrix to capture the temporal dependence between different messages.

[0163] (2) Feature Extraction Layer:

[0164] Three convolutional modules respectively process the bit-level, byte-level, and message-level inputs, and use convolutional kernels of different sizes (such as 3×3 and 8×8) for bit-level feature extraction, byte-level feature extraction, and message-level feature extraction layers respectively.

[0165] Among them, the bit-level features use a 3×3 convolutional kernel, which is mainly used for local bit interaction to extract microscopic features such as bit flip rate. Reason: Signal changes at the bit level usually have local correlations, especially near signal boundaries. Basis: The 3×3 convolutional kernel can slide in an 8×8 bit matrix to extract the relationship between each bit and its adjacent bits; in the field of image processing, 3×3 is also commonly used for edge detection and is suitable for signal boundary recognition tasks.

[0166] The byte-level features use an 8×8 convolutional kernel, covering the entire byte, to extract the dependencies between bytes, especially suitable for dealing with the situation of cross-byte signals. Reason: Signals are often stored across bytes, and it is necessary to analyze the dependencies between bytes, especially in a mixed byte-order environment. Basis: The 8×8 convolutional kernel can directly cover a complete byte to ensure that complete features can still be extracted when signals are stored across bytes; since CAN data frames are at most 8 bytes, the 8×8 convolutional kernel can capture the local structure of the entire data frame and improve the recognition accuracy.

[0167] The message-level features use a 3×3 convolutional kernel to analyze the time series patterns at the message level. Reason: CAN data is time series data, and there may be periodicity or patterns between adjacent messages. Basis: The 3×3 convolutional kernel can extract local features in the time dimension, which helps to detect pattern changes between adjacent data frames; this size can reduce the computational complexity while retaining enough information for signal pattern recognition.

[0168] After each layer of convolution, a residual connection is added, which retains the original feature information, accelerates the training process and reduces the problem of gradient vanishing, enhancing the learning ability of the model.

[0169] (3) Feature fusion:

[0170] The feature maps from the bit-level, byte-level, and message-level are fused into a multi-channel input, and a complete data representation is formed through feature stacking. The fused features are sent to the pooling layer to extract global information and compress the feature dimension.

[0171] (4) Regularization layer:

[0172] To prevent overfitting, the model adds a Batch Normalization layer to standardize the features for full connection layer processing, while improving the training efficiency and stability of the model.

[0173] (5) Output layer:

[0174] The final features are converted into a vector of length 64 through a fully connected layer (Perceptron). The Sigmoid activation function is used for multi-label classification to mark the signal boundary information for each bit.

[0175] Figure 5 It shows the hierarchical feature extraction process of this deep learning model from bit to message level. Combining residual connections and regularization strategies ensures that the model can accurately identify complex patterns in CAN data. This multi-level feature extraction and fusion method enables AutoDBC to handle the signal boundary recognition challenges in different vehicle and data environments.

[0176] The following experimental verification results are given in this embodiment:

[0177] 1. Performance comparison of signal boundary recognition

[0178] Precision, Recall, F1-score, and Accuracy are used in the experiment to evaluate the performance of AutoDBC and other methods, and a comparison is made with READ, LibreCAN (Phase0), and CAN-D (Step1).

[0179]

[0180] Experimental conclusion:

[0181] AutoDBC is superior to the SOTA method in all evaluation metrics. Especially, there are significant improvements in Precision, Recall, and F1-score, indicating that it is more accurate in signal boundary recognition.

[0182] 2. Performance comparison of signal domain recognition

[0183] The experiment also evaluates the ability of AutoDBC in signal domain recognition (Step 5) and makes a comparison with READ, LibreCAN (Phase0), and CAN-D (Step1&Step2). The results are as follows.

[0184]

[0185]

[0186] Experimental conclusion:

[0187] AutoDBC has the highest accuracy (CE / TE = 61.45%) and recall rate (CE / TDBC = 67.55%) in signal domain recognition, and can identify the signal domain more accurately than the CAN-D method. This proves that AutoDBC improves the accuracy of signal boundary recognition and enhances the signal domain inference ability by fusing multi-level feature extraction.

[0188] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle-mounted CAN bus data decoding method based on deep learning, characterized in that, Including the following steps: Step 1, collect the original message data of the in-vehicle CAN bus, including data frames and message IDs; Step 2, perform denoising and partition preprocessing on the original message data, remove redundant information and invalid data, and group them into time series subsets according to message IDs; Step 3, based on the deep learning model, perform multi-level feature extraction including bit-level features, byte-level features, and message-level features. Among them, The bit-level feature extraction is: analyze the bit flip rate and local spatial characteristics; the byte-level feature extraction is: parse the cross-byte dependency relationship; the message-level feature extraction is: capture the context and periodic patterns in the time series; Step 4, use the deep learning model to fuse the multi-level features extracted in Step 3, and automatically identify the start boundary of the signal; Step 5, combine the identified boundary and the mixed byte order scanning algorithm to infer the most significant bit and byte order of the signal, and generate signal domain information; Step 6, output the complete signal decoding result, including the signal boundary, byte order, and decoding formula for mapping physical quantities.

2. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 1, wherein In Step 2, the original message data is denoised, then grouped according to message IDs, and further divided into time series subsets, each subset containing a fixed number of messages; the denoising method includes one or more of static bit filtering, low variance filtering, and outlier detection. The static bit filtering is to remove the fields with unchanged bit values in the data, the low variance filtering is to delete the signals with variances lower than the set threshold, and the outlier detection is to detect and remove abnormal data using statistical methods.

3. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 1, characterized in that In Step 3, the bit-level feature extraction is: convert the data field of the message into a binary matrix, and use convolution operations to extract local spatial features; the byte-level feature extraction is: capture the signal patterns spanning multiple bytes by analyzing the cross-byte dependency relationship; the message-level feature extraction is: analyze the message data in the time series, and extract cross-message context information and periodic features.

4. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 3, characterized in that In the bit-level feature extraction, first calculate the flip frequency of each bit, detect the region where the signal boundary is located, and then extract the local spatial pattern through the convolution kernel to identify the detailed features of the signal start position.

5. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 3, characterized in that, In the byte-level feature extraction, first use the convolution kernel to capture the dependency between bytes, detect the signal boundary across bytes, and then identify the situation where big-endian and little-endian coexist to solve the inconsistency of the signal storage order.

6. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 3, characterized in that, In the message-level feature extraction, first analyze the periodicity and context information of the signal, and then identify the correlation of the signal in multiple messages.

7. The vehicle-mounted CAN bus data decoding method based on deep learning according to any one of claims 1-6, characterized in that In Step 4, use the deep convolutional neural network model, combine bit-level, byte-level, and message-level features to generate multi-channel inputs, extract high-dimensional features through convolutional networks and residual connections, generate high-dimensional feature representations, and automatically identify the start position of the signal, that is, the least significant bit LSB, through a multi-label classifier to achieve precise positioning of the signal boundary.

8. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 7, characterized in that, In step 5, the hybrid endian scanning algorithm is used to infer the byte storage order of the signal, distinguishing between big-endian and little-endian; by analyzing the bit flip rate and boundary characteristics, the most significant bit of the signal is inferred to generate complete signal domain information, including the start position, length, byte order, etc. of the signal.

9. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 8, characterized in that The scanning directions of big-endian and little-endian are different. When the signal is stored in big-endian byte order, the arrow advances to the right, scanning from the high-order bit to the low-order bit step by step until the flip rate is 0 or other signal boundaries are encountered; for little-endian byte order, the arrow advances to the left, and the signal starts from the low-order bit and extends in the high-order bit direction.

10. The vehicle-mounted CAN bus data decoding method based on deep learning according to claim 1, characterized in that In step 6, the decoding formula is: V phy =(V raw ×Factor)+Offset, where V raw is the original signal value; Factor is the scaling factor; Offset is the offset.

Images