A DSSAD data parsing method, medium, and computer equipment

Abstract

This invention discloses a DSSAD data parsing method, medium, and computer equipment. The method includes: raw data acquisition and preprocessing, frame boundary location and segmentation, candidate field boundary identification, establishing a mapping relationship between physical quantities and binary values, encoding rule inversion and verification, generating parsing templates and batch parsing, and additional processing for different architectures. Most DSSAD data parsing methods rely on communication protocol documents or dedicated decoding software provided by automakers, but this invention provides a universal reverse parsing method that does not depend on any prior information and is entirely based on the characteristics of the data itself.

Description

Technical Field

[0001] This invention relates to the field of intelligent connected vehicle electronic data parsing technology, and more specifically to a DSSAD data parsing method, medium, and computer equipment. Background Technology

[0002] The Automated Driving Data Recording System (DSSAD) is a legally mandated data recording device for intelligent connected vehicles, and the data it records is core objective evidence for determining liability in traffic accidents. National standards clearly define the names, physical definitions, and units of the data elements that DSSAD should record, but they do not uniformly specify the specific binary encoding of these data elements in the DSSAD storage medium. This is because different automakers, and even different models from the same automaker, have technical autonomy in implementing DSSAD functions, freely choosing encoding parameters such as data type, byte length, endianness, dimensional transformation coefficients, and invalid value representation methods. This situation of "unified semantics at the application layer, free encoding at the presentation layer" means that after law enforcement agencies or third-party appraisal institutions legally read the raw DSSAD data, they can only obtain a string of hexadecimal numbers, unable to convert them into physically meaningful values. Furthermore, data from different brands of vehicles is completely incompatible for interpretation, and even different models from the same brand may have different data formats. The root cause of these problems is that data encoding rules are considered corporate trade secrets and are usually not disclosed, making them difficult for external institutions to obtain. Summary of the Invention

[0003] The present invention provides a DSSAD data parsing method, medium, and computer equipment that does not rely on any prior information and is entirely based on the characteristics of the data itself, which can at least solve one of the above-mentioned technical problems.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A DSSAD data parsing method includes the following steps: S1. The data extraction device reads the raw binary data from the target vehicle's DSSAD without loss and organizes it into a unified format for subsequent analysis. S2. Determine the starting position and length of each event record frame inside DSSAD, and divide the continuous byte stream formed by binary data into independent record frames; S3. Within each frame, the continuous byte sequence is initially divided into several independent candidate data fields to prepare for subsequent mapping. S4. Obtain a batch of paired samples of "known real physical quantities" and "corresponding binary original values ​​in DSSAD", and establish a mapping relationship in two ways, including path A - laboratory bench injection method and path B - real vehicle bus monitoring method. S5. Using the mapping relationship, deduce the encoding rules for each candidate data field; S6. Solidify the verified encoding rules into parsing templates for batch processing of data for vehicles of the same model; S7 provides additional processing for different architecture scenarios.

[0005] Furthermore, S1 further includes: S1.1. Establish a connection with DSSAD through the target vehicle's OBD interface using the Ethernet diagnostic communication protocol and a standard port; S1.2 The data extraction device sends a general diagnostic command to the DSSAD to read all stored data. The DSSAD returns the data as a continuous byte stream according to its internal storage order. After receiving the data, the data extraction device saves the byte stream as a raw binary file raw_data.bin and records the total byte length L of the file. total ; S1.3 Obtain the following metadata from the DSSAD through the diagnostic service: DSSAD type, storage medium identifier, firmware version number, and vehicle identification number, for subsequent parsing template indexing.

[0006] Furthermore, S2 further includes: S2.1. Based on the "Get Event Count" or "Get Stored Record Count" request in the DSSAD standard diagnostic service, obtain the total number of records N stored internally in DSSAD. If the standard diagnostic service does not support the request, then estimate the total number of records N using fixed-length frame detection or variable-length frame detection. Fixed-length frame detection: Calculate L total / N, if the result is an integer L frame If it is determined to be a fixed-length frame structure, it will directly start from the beginning of the file and divide into L-shaped frames. frame Byte segmentation yields N frames; If L total If the length is not divisible by N, or if N is unknown, variable-length frame detection is used: scan the file, find the position of the length field with a high probability, assuming that the first 1 or 2 bytes of the frame header are the length field, take the value len of that byte, then the current frame length is len + header. len Starting from the beginning of the file, read the first frame sequentially, jump to the second frame based on the length field of the first frame header, and so on, until the end of the file; S2.2, Check whether the header of each frame has reasonable feature values, or check whether the remaining length after segmentation is 0, where: For integrated memory, read-only partitioning does not modify the original data in any way; For split-type storage, the binary files of each storage unit are read separately, split into their own segments, and aligned by timestamps in subsequent steps.

[0007] Furthermore, S3 further includes: S3.1. Take all frames and calculate the numerical entropy or variance at each byte position. Positions with high entropy values ​​belong to data fields that change frequently, while positions with low entropy values ​​belong to fixed values. Fixed value fields are temporarily excluded. S3.2 Starting from the first byte of the frame, try different window widths using the sliding window method and observe the correlation between the values ​​of adjacent windows: if the value in a window shows an obvious linear or monotonic relationship with the change of the frame number, and is highly independent of the value changes of its adjacent windows, then it is determined to be an independent field. S3.3, DSSAD: Each record must contain a trigger timestamp. Scan all frames and look for a 4-byte or 8-byte field. If the increment of the corresponding value between frames is basically constant and has an approximately linear relationship with the frame sequence number, then mark the field as a candidate timestamp.

[0008] Furthermore, in S4, the path A-laboratory bench injection method is suitable for scenarios where input signals can be applied, and the implementation steps include: S4.A.1. For the physical quantity to be analyzed, design a set of test sequences covering the entire range. Each time a value is injected, maintain the state for at least 1 second to ensure that DSSAD can record the state at that moment. At the same time, the bench control software records each injected value and the corresponding precise timestamp. S4.A.2 During the injection process, ensure that the DSSAD is in recording mode, and the DSSAD automatically records the state corresponding to the injection signal; S4.A.3. Read the original DSSAD data within the injection time window through the diagnostic interface, and segment the frame sequence according to S2. Since the injection time is known, locate the frame at the injection time through the candidate timestamp field. S4.A.4. For each frame located, extract the binary values ​​of each candidate field and pair them with the corresponding injected physical quantities to obtain several mapping pairs (P). i V ij ), where P i V is the physical quantity injected. ij is the binary value of candidate field j in the frame.

[0009] Furthermore, in S4, the path B-vehicle bus monitoring method is suitable for scenarios where signals cannot be injected, and the implementation steps include: S4.B.1 Connect the high-bandwidth bus monitoring tool to the Ethernet diagnostic port of the target vehicle or directly connect it to the vehicle's Ethernet bus. The monitoring tool should support precise time synchronization to ensure the accuracy of the timestamps of the collected data. S4.B.2. Based on the physical quantities to be analyzed, design a sequence of driving operations that includes significant changes; S4.B.3. Start the monitoring tool to begin recording all messages on the bus. At the same time, read the raw binary data of DSSAD through another diagnostic interface and record the extraction time window. S4.B.4 During driving, a characteristic event is artificially created that can be recorded simultaneously in both the monitoring data stream and the DSSAD data frame. Based on the timestamp deviation of the characteristic event in the two data streams, the two timelines are aligned. S4.B.5. In the monitored data stream, the actual physical quantity at each moment is calculated using the publicly available sensor protocols or physical relationships. S4.B.6. In the DSSAD data frame, based on the aligned time axis, find the frame that is close to the time of the monitored physical quantity, extract the binary values ​​of each candidate field in the frame, and form a mapping pair with the physical quantity.

[0010] Furthermore, S5 further includes: S5.1. For the binary value sequence in the mapping relationship between physical quantity P and binary value V, analyze its statistical characteristics: If the binary value sequence takes only a few discrete states and has a non-linear relationship with physical quantities, but rather a state-to-state relationship, then it is determined to be an enumeration type. If the sequence of binary values ​​is continuous and the mapping relationship shows that the physical quantity ≈ constant × binary value, then it is determined to be an integer. If the binary value is 4 or 8 bytes long, and after attempting to interpret it according to IEEE 754 floating-point, the error with the physical quantity is within a reasonable range, then it is determined to be a floating-point type. S5.2 For 2 / 4 / 8 byte fields, interpret the value V in big-endian and little-endian order respectively. big and V little Calculate V respectively big The linear correlation coefficient with physical quantity P, and V little The byte order is determined by the linear correlation coefficient with the physical quantity P, with the byte order being the one whose correlation coefficient is closer to 1. S5.3 For the linear model P = k * V + b, use the least squares method to perform linear regression on all mapping pairs to obtain the optimal linear correlation coefficients k and b. At the same time, calculate the fitting error. If the error is within the allowable range, the linear model is confirmed to be effective. S5.4 Check if there are any binary values ​​in the mapping relationship whose corresponding physical quantities are obviously abnormal or do not conform to physical laws, and mark such values ​​as "invalid values" or "fault flags"; S5.5. Verify the correctness of the analysis using vehicle kinematics relationships. If the physical quantities obtained after independent analysis of multiple fields can corroborate each other, the confidence level of the analysis is increased.

[0011] Furthermore, the additional processing provided by S7 for different architecture scenarios includes: Separate storage: Before the splitting in S2, the original data of each storage is read. After the splitting, the frames of different storages are merged into a unified frame sequence in ascending order of timestamp using the timestamp field identified in S4. During the merging, the timestamp interval is checked. If the time difference between adjacent frames of two storages is less than 100ms, they are inserted in chronological order. If the time difference exceeds 100ms, there is a possibility of data loss. Event-triggered system: The frame content of the event-triggered system changes depending on the event type. When identifying the candidate field in S3, the frames should first be classified according to a certain flag bit, and then the field boundary identification should be performed for each type of frame. Data compression: A few DSSADs may perform simple data compression. If the mapping relationship in S4 cannot be fitted, compression is suspected. The compression mode is detected by using the positions of a large number of repeated values ​​in the injected sequence. Decompression is performed before subsequent steps.

[0012] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the DSSAD data parsing method described above.

[0013] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the DSSAD data parsing method described above.

[0014] The beneficial effects of this invention are reflected in: This invention enables pluggable pre-consensus, selectable block generation methods, and separates the responsibilities of packaging blocks from those of confirming blocks. It can transform the probabilistic consensus of pre-consensus into the deterministic consensus of post-consensus, facilitating the integration of the original chain consensus as pre-consensus during cross-chain operations. The adaptation layer performs adaptation, and the target chain uses post-consensus for confirmation. Attached Figure Description

[0015] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0016] Figure 1 This is a schematic diagram of the overall process of DSSAD data parsing according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that the meaning of "and / or" throughout the text includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or a solution that simultaneously satisfies A and B. Furthermore, "multiple" refers to two or more. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0019] See Figure 1 This invention provides a DSSAD data parsing method, comprising the following steps: S1. Raw Data Acquisition and Preprocessing The data extraction device reads the raw binary data from the target vehicle's DSSAD without loss and organizes it into a unified format for subsequent analysis.

[0020] S1.1 Establish diagnostic communication link A connection is established with DSSAD via the target vehicle's OBD interface using the Ethernet Diagnostic Communication Protocol (DoIP) and standard port 13400.

[0021] Since DSSAD may return a large amount of data at once (up to hundreds of megabytes in continuous recording mode), this embodiment adopts a segmented flow control reading method: each time a read request is sent, DSSAD returns several frames of data. The data extraction device adjusts the flow control parameters (such as the number of frames per batch and the interval time) to stably obtain complete data, avoiding link congestion or frame loss due to excessive data volume.

[0022] S1.2 Extract the complete binary data block The data extraction device sends a general diagnostic command to DSSAD to "read all stored data" (without relying on any private parameters). DSSAD returns the data as a continuous byte stream according to its internal storage order. After receiving the data, the data extraction device saves the byte stream as a raw binary file raw_data.bin and records the total byte length L_total of the file.

[0023] S1.3 Record auxiliary information The following metadata is obtained from the DSSAD through the diagnostic service: DSSAD type (Type I or Type II system, i.e., event-triggered system or continuous recording system), storage media identifier (integrated or separate), firmware version number, and vehicle identification number (VIN code). The above metadata information is used for subsequent parsing template indexing.

[0024] S2, Frame Boundary Location and Segmentation Determine the start position and length of each event record frame within DSSAD, and divide the continuous byte stream into independent record frames.

[0025] S2.1 Count of records The total number of records N stored internally in DSSAD is obtained by requesting either "Get Event Count" or "Get Stored Records" in the DSSAD standard diagnostic service. If the standard diagnostic service does not support the request, the following method is used for estimation: (1) Fixed-length frame detection: Calculate L_total / N. If the result is an integer L_frame, it is determined to be a fixed-length frame structure. In this case, directly divide the file from the beginning position into N frames, each L_frame byte.

[0026] (2) Variable-length frame detection: If L_total is not divisible by N, or if N cannot be known, then a variable-length frame detection method is used: Scan the file to find possible locations of the length field. Assuming the first 1 or 2 bytes of the frame header are the length field, take the value len of that byte; then the current frame length is len + header_len. Starting from the beginning of the file, read the first frame sequentially, jump to the second frame based on the length field in the first frame's header, and so on, until the end of the file.

[0027] S2.2 Verify the rationality of the segmentation Check if the header of each frame has a reasonable feature value (such as 0xAA, 0x55, or other common synchronization bytes), or check if the remaining length after segmentation is 0.

[0028] (1) For integrated memory: read-only partitioning, without modifying the original data.

[0029] (2) For split memory: read the binary file of each memory separately, split it according to the above method, and align it by timestamp in subsequent steps.

[0030] S3, Candidate Field Boundary Recognition Within each frame, the continuous byte sequence is initially divided into several independent candidate data fields to prepare for subsequent mapping.

[0031] S3.1, Statistical Byte Variation Take all frames and calculate the numerical entropy or variance at each byte position. Positions with high entropy values ​​belong to frequently changing data fields; positions with low entropy values ​​(close to 0) belong to fixed values ​​(such as frame header identifiers, reserved bits, etc.), and fixed value fields can be temporarily excluded.

[0032] S3.2 Adaptive Boundary Detection Starting from the first byte of the frame, the sliding window method is used to try different window widths (1, 2, 4, 8 bytes) to observe the correlation between the values ​​of adjacent windows: if the value in a window shows a clear linear or monotonic relationship with the change of the frame number, and the value changes of its adjacent windows are highly independent, then it is determined to be an independent field.

[0033] Specifically, data transmitted via vehicular Ethernet often uses network byte order (big-endian), and fields are typically aligned to 4 bytes. In this embodiment, we prioritize using 4-byte alignment for boundary segmentation.

[0034] S3.3, Timestamp Field Location Each DSSAD record must contain a trigger timestamp. The timestamp field typically has the following characteristics: the difference in this field is stable between consecutive frames (e.g., recorded at fixed time intervals); or it is a number of seconds or milliseconds in Coordinated Universal Time (UTC), with the value monotonically increasing within a certain time interval. In this embodiment, by scanning all frames, a 4-byte or 8-byte field is searched for whose value increases at a substantially constant step size between frames and is approximately linearly related to the frame sequence number; this field is then marked as a candidate timestamp.

[0035] S4. Establish the mapping relationship between physical quantities and binary values. A batch of paired samples of "known real physical quantities" and "corresponding binary original values ​​in DSSAD" were obtained, and a mapping relationship was established through two implementation paths: Path A - laboratory bench injection method, and Path B - real vehicle bus monitoring method.

[0036] The Path A-Laboratory Bench Injection Method is suitable for scenarios where input signals can be applied. Its applicability lies in the ability to connect the target vehicle or relevant control unit to a hardware-in-the-loop (HiL) bench and to accurately transmit excitation signals. The Path A-Laboratory Bench Injection Method implementation includes: S4.A.1, Inject test signal For the physical quantity to be analyzed (e.g., "the steering angle requested by ADS"), a set of test sequences covering the entire range is designed. For example, from the minimum to the maximum value, incrementing and then decreasing in fixed steps. Each time a value is injected, the state is maintained for at least 1 second to ensure that DSSAD can record the state at that moment. At the same time, the bench control software records each injected value and its corresponding precise timestamp (accuracy 1ms). S4.A.2, Trigger DSSAD recording During the injection process, ensure that DSSAD is in recording mode (for Type I systems, i.e., event-triggered systems, it is necessary to simulate the triggering event; for Type II systems, i.e., continuous recording systems, it is sufficient to keep the autopilot system active). DSSAD will automatically record the state corresponding to the injection signal. S4.A.3, Read and locate the corresponding frame The original DSSAD data within the injection time window is read through the diagnostic interface, and the frame sequence is segmented according to S2. Since the injection time is known, the frame at the injection time can be roughly located through the candidate timestamp field (not yet parsed but can be compared relatively). For greater accuracy, an "auxiliary known quantity" can be used for time alignment: for example, a clear level change signal is sent at the beginning and end of the injection sequence, and this signal will also be recorded in DSSAD (such as light status), thereby aligning the time axis. S4.A.4 Extracting Binary Values For each frame located, the binary values ​​of each candidate field are extracted and paired with the corresponding injected physical quantity to obtain several mapping pairs (P_i, V_ij), where P_i is the injected physical quantity and V_ij is the binary value of candidate field j in the frame.

[0037] Path B - Vehicle Bus Monitoring Method is suitable for scenarios where signals cannot be injected. Its applicable conditions are: data can only be acquired under actual vehicle operating conditions, and precise input control is not possible. The implementation method of Path B - Vehicle Bus Monitoring Method includes: S4.B.1 Deploy bus monitoring device Connect a high-bandwidth bus monitoring tool (such as the Vector VN56xx series, which supports 100BASE-T1 / 1000BASE-T1 automotive Ethernet) to the vehicle's Ethernet diagnostic port or directly to the automotive Ethernet bus. The monitoring tool should support precise time synchronization (PTP / gPTP) to ensure that the timestamp accuracy of the collected data is better than 1ms. S4.B.2, Design typical driving conditions Based on the physical quantities to be analyzed, design a sequence of driving operations that includes significant changes. For example, by analyzing the "vehicle speed" field, a complete process of accelerating from a standstill to 100 km / h and then decelerating back to a standstill can be designed; by analyzing the "target distance", a following scenario where the vehicle in front moves from far to near can be designed. S4.B.3, Synchronize recording bus data and DSSAD data The monitoring tool is started to record all messages on the bus (including sensor signals, actuator status, ADS internal status, etc.). Although these messages are also encoded, they usually come from the standard protocols of the sensor manufacturer, or they can establish a relationship with real physical quantities through physical laws (such as the relationship between wheel speed pulse and vehicle speed). At the same time, the raw binary data of DSSAD is read through another diagnostic interface, and the extraction time window is recorded. S4.B.4, Timeline Alignment The monitoring data stream has its own timestamp (based on the clock of the monitoring device), and the DSSAD data also has its internal timestamp. To align the two, a characteristic event that can be recorded simultaneously in both links can be artificially created during driving. For example, when starting the monitoring, the vehicle's emergency hazard lights switch is pressed. This operation will be recorded by both the bus monitoring (light status change) and the DSSAD (light signal). Based on the timestamp deviation of this event in the two data streams, the two timelines are aligned. S4.B.5 Extracting reference values ​​for physical quantities By listening to the data stream and using publicly available sensor protocols or physical relationships, the real physical quantities at each moment can be calculated. For example, vehicle speed can be calculated from wheel speed sensor messages, acceleration can be calculated from IMU messages, and the distance to a target can be obtained from a radar target list. S4.B.6 Matching binary values In the DSSAD data frame, based on the aligned time axis, find the frame that is closest to the time of the monitored physical quantity (time difference less than 50ms), extract the binary values ​​of each candidate field in the frame, and form a mapping pair with the physical quantity.

[0038] S5. Encoding Rule Inversion and Verification By using the mapping relationship, the encoding rules (data type, byte order, coefficient, offset, and invalid value definition) of each candidate data field can be deduced.

[0039] S5.1 Initial Data Type Judgment For the binary value sequence in the mapping relationship, analyze its statistical characteristics: (1) If the value takes only a few discrete states and is not linearly related to the physical quantity, but rather a state correspondence (e.g., 0 → closed, 1 → open), then it is determined to be an enumeration type, and the mapping relationship is directly enumerated. (2) If the value is continuous and the mapping relationship shows that the physical quantity ≈ constant × binary value, then it is determined to be an integer; (3) If the length of the binary value is 4 bytes or 8 bytes, and after attempting to interpret it according to IEEE 754 floating-point numbers, the error with the physical quantity is within a reasonable range, then it is determined to be a floating-point type; S5.2 Byte Order Determination (Multi-byte Fields) For 2 / 4 / 8 byte fields, the values ​​V_big and V_little are interpreted in big-endian and little-endian order respectively. The linear correlation coefficient between V_big and physical quantity P and the linear correlation coefficient between V_little and physical quantity P are calculated respectively. The byte order is taken as the one with the correlation coefficient closer to 1. S5.3 Coefficients and Offset Fitting For the linear model P = k * V + b, the least squares method is used to perform linear regression on all mapping pairs to obtain the best linear correlation coefficients k and b. At the same time, the fitting error (root mean square error) is calculated. If the error is within the allowable range (e.g., vehicle speed ±1%), then the linear model is confirmed to be effective. S5.4 Special Value Identification Check if there are any binary values ​​in the mapping relationship that correspond to physical quantities that are obviously abnormal (such as a vehicle speed of 999 km / h) or do not conform to physical laws (such as a large acceleration when the vehicle is stationary). Mark these values ​​as "invalid values" or "fault signs". Invalid values ​​are usually represented by all 1s (such as 0xFFFF) or extreme values. S5.5, Multi-field Joint Validation The correctness of the analysis can be verified by using vehicle kinematic relationships. For example, the vehicle speed and wheel speed should satisfy the transmission ratio relationship, and the rate of change of heading angle should be consistent with the yaw rate. If the physical quantities obtained after independent analysis of multiple fields can corroborate each other, the confidence of the analysis is increased.

[0040] S6. Generate parsing templates and perform batch parsing The verified encoding rules are solidified into parsing templates for batch processing of data from vehicles of the same model.

[0041] S6.1 Template Structure The encoding rules obtained in S5 are saved in JSON format, with the following structure: { "vehicle_model": "XXXX", "dssad_firmware": "1.0.2", "system_type": "II", "memory_layout": "integrated", "fields": [ {"name": "timestamp", "offset": 0, "length": 4, "type": "uint32", "endian": "little", "scale": 0.001, "offset": 0}, {"name": "vehicle_speed", "offset": 4, "length": 2, "type": "uint16","endian": "little", "scale": 0.01, "offset": 0, "invalid":

[65535] }, ... ] } S6.2 Batch Parsing Program Develop a general-purpose parsing software. The software takes the original binary file and the corresponding parsing template as input, reads, converts and outputs the data frame by frame according to the template rules, and outputs it in CSV or XLSX format. At the same time, the software supports differentiated processing of Type I and Type II data (Type I system needs to distinguish between time period events and timestamp events, while Type II system outputs data continuously).

[0042] S6.3 Template Library Management Establish a parsing template library. Each time a car model is successfully parsed, the template is added to the library. When the same car model is encountered in the future, the template can be directly called without repeating the S1-S5 steps mentioned above.

[0043] S7 provides additional processing for different architecture scenarios.

[0044] Case 1: Separate memory Before the S2 segmentation, the original data of each memory needs to be read separately. After each segmentation, the frames from different memories are merged into a unified frame sequence by using the timestamp field identified in the S4 (which can be parsed or unparsed, as long as it is relative time). During the merging, the timestamp interval is checked: if the time difference between adjacent frames of two memories is less than 100ms, they are inserted in chronological order; if it exceeds 100ms, there may be missing data. Scenario 2: Type I System The frame content of the Type I system varies depending on the event type (time period events contain multiple frames, while timestamp events contain only a single frame). When identifying the S3 candidate field, the frames should first be classified according to a certain flag bit (event type field), and then the field boundary identification should be performed for each type of frame. The parsing template should include event type discrimination rules. Scenario 3: Data Compression A few DSSADs may perform simple compression on the data (such as run-length encoding). If the mapping relationship in S4 cannot be fitted (i.e. the binary value changes too drastically and irregularly), compression is suspected. In this case, the compression mode can be detected by using the positions of a large number of repeated values ​​in the injected sequence. Decompression can be performed first before continuing with the subsequent steps.

[0045] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the DSSAD data parsing method described above.

[0046] This invention also provides a computer device, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the DSSAD data parsing method described above.

[0047] This invention also provides a computer program product containing instructions that, when run on a computer, causes the computer to perform the steps of the DSSAD data parsing method described above.

[0048] It is understood that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above DSSAD data parsing method.

[0049] It should be noted that those skilled in the art will understand that all or part of the steps implemented in the embodiments of the present invention can be implemented entirely or partially by software, hardware, firmware, or any combination thereof. When implemented in hardware, it can be implemented entirely or partially by purchasing standard parts or modifications. When implemented in software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0050] In summary, this invention addresses the problem that most existing DSSAD data parsing methods rely on communication protocol documents or dedicated decoding software provided by automakers, leading to data incomprehensibility due to inconsistent proprietary encoding protocols. It provides a reverse parsing method for binary data stored in the Automated Driving Data Recording System (DSSAD), which is entirely based on the characteristics of the data itself and does not rely on any prior information. Specifically, this method is mainly used to reverse-parse the raw binary data stored in the Automated Driving Data Recording System (DSSAD). It does not depend on any automaker's proprietary protocols. By establishing a mapping relationship between known physical quantities and binary values, it progressively reverse-engineers the encoding rules of each data field (data type, byte order, coefficients, offsets, and invalid value definitions), ultimately generating a reusable parsing template to achieve automated batch parsing of DSSAD data for the current vehicle model.

[0051] It should be understood that the examples and embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Those skilled in the art can make various modifications or changes based on them. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A DSSAD data parsing method, characterized in that, Includes the following steps: S1. The data extraction device reads the raw binary data from the target vehicle's DSSAD without loss and organizes it into a unified format for subsequent analysis. S2. Determine the starting position and length of each event record frame inside DSSAD, and divide the continuous byte stream formed by binary data into independent record frames; S3. Within each frame, the continuous byte sequence is initially divided into several independent candidate data fields to prepare for subsequent mapping. S4. Obtain a batch of paired samples of "known real physical quantities" and "corresponding binary original values ​​in DSSAD", and establish a mapping relationship in two ways, including path A - laboratory bench injection method and path B - real vehicle bus monitoring method. S5. Using the mapping relationship, deduce the encoding rules for each candidate data field; S6. Solidify the verified encoding rules into parsing templates for batch processing of data for vehicles of the same model; S7 provides additional processing for different architecture scenarios.

2. The DSSAD data parsing method as described in claim 1, characterized in that, S1 further includes: S1.

1. Establish a connection with DSSAD through the target vehicle's OBD interface using the Ethernet diagnostic communication protocol and a standard port; S1.2 The data extraction device sends a general diagnostic command to the DSSAD to read all stored data. The DSSAD returns the data as a continuous byte stream according to its internal storage order. After receiving the data, the data extraction device saves the byte stream as a raw binary file raw_data.bin and records the total byte length L of the file. total ; S1.3 Obtain the following metadata from the DSSAD through the diagnostic service: DSSAD type, storage medium identifier, firmware version number, and vehicle identification number, for subsequent parsing template indexing.

3. The DSSAD data parsing method as described in claim 1, characterized in that, S2 further includes: S2.

1. Based on the "Get Event Count" or "Get Stored Record Count" request in the DSSAD standard diagnostic service, obtain the total number of records N stored internally in DSSAD. If the standard diagnostic service does not support the request, then estimate the total number of records N using fixed-length frame detection or variable-length frame detection. Fixed-length frame detection: Calculate L total / N, if the result is an integer L frame If it is determined to be a fixed-length frame structure, it will directly start from the beginning of the file and divide into L-shaped frames. frame Byte segmentation yields N frames; If L total If the length is not divisible by N, or if N is unknown, then variable-length frame detection is used: scan the file, find the position of the length field with a high probability, assuming that the first 1 or 2 bytes of the frame header are the length field, take the value len of that byte, then the current frame length is len + header. len Starting from the beginning of the file, read the first frame sequentially, jump to the second frame based on the length field of the first frame header, and so on, until the end of the file; S2.2, Check whether the header of each frame has reasonable feature values, or check whether the remaining length after segmentation is 0, where: For integrated memory, read-only partitioning does not modify the original data in any way; For split-type storage, the binary files of each storage unit are read separately, split into their own segments, and aligned by timestamps in subsequent steps.

4. The DSSAD data parsing method as described in claim 1, characterized in that, S3 further includes: S3.

1. Take all frames and calculate the numerical entropy or variance at each byte position. Positions with high entropy values ​​belong to data fields that change frequently, while positions with low entropy values ​​belong to fixed values. Fixed value fields are temporarily excluded. S3.2 Starting from the first byte of the frame, try different window widths using the sliding window method and observe the correlation between the values ​​of adjacent windows: if the value in a window shows an obvious linear or monotonic relationship with the change of the frame number, and is highly independent of the value changes of its adjacent windows, then it is determined to be an independent field. S3.3, DSSAD: Each record must contain a trigger timestamp. Scan all frames and look for a 4-byte or 8-byte field. If the increment of the corresponding value between frames is basically constant and has an approximately linear relationship with the frame sequence number, then mark the field as a candidate timestamp.

5. The DSSAD data parsing method as described in claim 1, characterized in that, In step S4, the path A-laboratory bench injection method is suitable for scenarios where input signals can be applied, and the implementation steps include: S4.A.

1. For the physical quantity to be analyzed, design a set of test sequences covering the entire range. Each time a value is injected, maintain the state for at least 1 second to ensure that DSSAD can record the state at that moment. At the same time, the bench control software records each injected value and the corresponding precise timestamp. S4.A.2 During the injection process, ensure that the DSSAD is in recording mode. The DSSAD will automatically record the state corresponding to the injection signal. S4.A.

3. Read the original DSSAD data within the injection time window through the diagnostic interface, and segment the frame sequence according to S2. Since the injection time is known, locate the frame at the injection time through the candidate timestamp field. S4.A.

4. For each frame located, extract the binary values ​​of each candidate field and pair them with the corresponding injected physical quantities to obtain several mapping pairs (P). i V ij ), where P i V is the physical quantity injected. ij The binary value of candidate field j in the frame.

6. The DSSAD data parsing method as described in claim 1, characterized in that, In S4, the path B-vehicle bus monitoring method is suitable for scenarios where signals cannot be injected, and the implementation steps include: S4.B.1 Connect the high-bandwidth bus monitoring tool to the Ethernet diagnostic port of the target vehicle or directly connect it to the vehicle's Ethernet bus. The monitoring tool should support precise time synchronization to ensure the accuracy of the timestamps of the collected data. S4.B.

2. Based on the physical quantities to be analyzed, design a sequence of driving operations that includes significant changes; S4.B.

3. Start the monitoring tool to begin recording all messages on the bus. At the same time, read the raw binary data of DSSAD through another diagnostic interface and record the extraction time window. S4.B.4 During driving, a characteristic event is artificially created that can be recorded simultaneously in both the monitoring data stream and the DSSAD data frame. Based on the timestamp deviation of the characteristic event in the two data streams, the two timelines are aligned. S4.B.

5. In the monitored data stream, the actual physical quantity at each moment is calculated using the publicly available sensor protocols or physical relationships. S4.B.

6. In the DSSAD data frame, based on the aligned time axis, find the frame that is close to the time of the monitored physical quantity, extract the binary values ​​of each candidate field in the frame, and form a mapping pair with the physical quantity.

7. The DSSAD data parsing method as described in claim 1, characterized in that, S5 further includes: S5.

1. For the binary value sequence in the mapping relationship between physical quantity P and binary value V, analyze its statistical characteristics: If the binary value sequence takes only a few discrete states and has a non-linear relationship with physical quantities, but rather a state-to-state relationship, then it is determined to be an enumeration type. If the sequence of binary values ​​is continuous and the mapping relationship shows that the physical quantity ≈ constant × binary value, then it is determined to be an integer. If the binary value is 4 or 8 bytes long, and after attempting to interpret it according to IEEE 754 floating-point, the error with the physical quantity is within a reasonable range, then it is determined to be a floating-point type. S5.2 For 2 / 4 / 8 byte fields, interpret the value V in big-endian and little-endian order respectively. big and V little Calculate V respectively big The linear correlation coefficient with physical quantity P, and V little The byte order is determined by the linear correlation coefficient with the physical quantity P, with the byte order being the one whose correlation coefficient is closer to 1. S5.3 For the linear model P = k * V + b, use the least squares method to perform linear regression on all mapping pairs to obtain the optimal linear correlation coefficients k and b. At the same time, calculate the fitting error. If the error is within the allowable range, the linear model is confirmed to be effective. S5.4 Check if there are any binary values ​​in the mapping relationship whose corresponding physical quantities are obviously abnormal or do not conform to physical laws, and mark such values ​​as "invalid values" or "fault flags"; S5.

5. Verify the correctness of the analysis using vehicle kinematics relationships. If the physical quantities obtained after independent analysis of multiple fields can corroborate each other, the confidence level of the analysis is increased.

8. The DSSAD data parsing method as described in claim 1, characterized in that, The additional processing content of S7 for different architecture scenarios includes: Separate storage: Before the splitting in S2, the original data of each storage is read. After the splitting, the frames of different storages are merged into a unified frame sequence in ascending order of timestamp using the timestamp field identified in S4. During the merging, the timestamp interval is checked. If the time difference between adjacent frames of two storages is less than 100ms, they are inserted in chronological order. If the time difference exceeds 100ms, there is a possibility of data loss. Event-triggered system: The frame content of the event-triggered system changes depending on the event type. When identifying the candidate field in S3, the frames should first be classified according to a certain flag bit, and then the field boundary identification should be performed for each type of frame. Data compression: A few DSSADs may perform simple data compression. If the mapping relationship in S4 cannot be fitted, compression is suspected. The compression mode is detected by using the positions of a large number of repeated values ​​in the injected sequence. Decompression is performed before subsequent steps.

9. A computer-readable storage medium, characterized in that, The system contains a computer program that, when executed by a processor, causes the processor to perform the steps of the DSSAD data parsing method as described in any one of claims 1-8.

10. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the DSSAD data parsing method as described in any one of claims 1-8.

Images