Identity-based autonomous vehicle data processing system

Abstract

This application relates to the field of data processing technology, and in particular to an autonomous vehicle data processing system based on identity recognition, comprising: a data acquisition module outputting a raw data stream carrying a digital signature; a preprocessing module filtering illegal data streams based on the digital signature and outputting time-aligned LiDAR data and visual data; a feature fusion module outputting six-DOF pose estimation results; a security processing module generating secure feature descriptors by constraining the range of cosine similarity, timestamp difference, and encryption seed; a verification and storage module storing Merkle trees, encrypted point clouds, and encrypted images into a candidate node group; and a calibration module calculating the correlation coefficient between the timestamps of the visual data and the LiDAR data, and reconfiguring the hardware triggering timing of the data acquisition module. This application solves the problem of cross-modal data fusion reliability caused by sensor clock deviation, thereby improving the accuracy of identity recognition.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an autonomous vehicle data processing system based on identity recognition. Background Technology

[0002] Autonomous vehicle data based on identity recognition is associated with specific vehicles through unique identifiers. The system collects multi-dimensional information, including driving trajectory, environmental perception results, and decision logs. It uses encrypted storage and distributed ledger technology to ensure data integrity. Machine learning models automatically filter noisy data when analyzing behavior patterns. Blockchain node verification mechanism prevents identity forgery. Digital certificate system enables cross-platform data interoperability. Multimodal sensor timestamps are aligned to generate structured datasets. Edge computing nodes perform real-time identity authentication while protecting privacy information. Under the federated learning framework, all participants share model updates instead of raw data, ultimately forming a traceable and tamper-proof digital vehicle profile.

[0003] In identity-based autonomous vehicle data systems, insufficient time synchronization accuracy of multi-source data reduces the reliability of cross-sensor data fusion, constituting a key technical pain point in data processing. Because sensor hardware clocks such as LiDAR, visual sensors, and millimeter-wave radar have microsecond-level deviations, and vehicle identity recognition relies on the spatiotemporal consistency of multimodal data, timestamp drift occurs when asynchronously acquired raw data undergoes feature alignment at edge computing nodes. For example, when a vehicle makes lane-changing decisions at complex intersections, the visual recognition result and point cloud matching result may have identity association errors due to time asynchrony, thus affecting the model aggregation accuracy under the federated learning framework. Ultimately, this leads to deviations in the autonomous driving system's trajectory prediction and behavior analysis of the same vehicle. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application provides an autonomous vehicle data processing system based on identity recognition. This application solves the technical problem that the reliability of cross-modal data fusion during the identity recognition process of autonomous vehicles is reduced due to microsecond-level deviations in the hardware clocks of multi-source heterogeneous sensors.

[0005] To address the aforementioned technical problems, this application provides an autonomous vehicle data processing system based on identity recognition, comprising:

[0006] The data acquisition module analyzes the second pulse signal through a satellite receiver to generate a trigger reference, thereby driving the LiDAR and vision sensor to start synchronously and acquire the raw data streams of the LiDAR and vision sensor; after binding the vehicle identification number with the hardware trigger time, satellite time synchronization time and local crystal oscillator time, a digital signature is generated, and the raw data stream carrying the digital signature is output to the preprocessing module.

[0007] The preprocessing module filters out illegal data streams based on digital signatures, allocates legitimate raw data streams to corresponding buffer queues according to sensor type, performs clock skew elimination on the data in the buffer queues, and outputs time-aligned data sequences, which include LiDAR data and visual data.

[0008] The feature fusion module maps the data sequence to the vehicle coordinate system, fills in the time missing points of the LiDAR data, and parses the six-degree-of-freedom pose estimation results of the vehicle body based on the data sequence; it outputs the six-degree-of-freedom pose estimation results bound to the vehicle identification number and carrying the time stamp difference constraint range.

[0009] The security processing module generates an encryption seed based on the vehicle identification number to encrypt the LiDAR data and visual data respectively, resulting in encrypted point cloud and encrypted image. It calculates the cosine similarity between the feature vector of the encrypted point cloud and the feature vector of the encrypted image that satisfy the timestamp difference constraint range, and generates a security feature descriptor by combining the cosine similarity, timestamp difference constraint range and encryption seed.

[0010] The verification storage module constructs a complete Merkle tree based on the security feature descriptor and elects multiple roadside nodes as candidate node groups; the Merkle tree, encrypted point cloud, and encrypted image are stored in the candidate node groups;

[0011] In the model training module, each candidate node group acts as a client for federated learning. It uses decrypted LiDAR data and visual data as training samples to obtain the gradient update amount of the model and uploads the gradient update amount to the cloud. The cloud filters out abnormal gradient update vectors and obtains model parameters based on the gradient update amount. The model parameters are then distributed to each client for the next round of iteration.

[0012] The calibration module backtracks the training samples and timestamps of the training samples to the model based on the abnormal gradient update vector; calculates the correlation coefficient between the timestamps of the visual data and the timestamps of the LiDAR data; and reconfigures the hardware triggering timing of the data acquisition module based on the correlation coefficient.

[0013] Optionally, the data acquisition module is used for:

[0014] The second pulse signal output by the Global Positioning System is analyzed by a satellite signal receiver to generate a trigger reference with nanosecond-level accuracy to the hardware synchronization circuit.

[0015] The hardware synchronization circuit triggers the synchronization startup mechanism of the LiDAR and vision sensor through the interrupt pin, and collects the raw data streams of the LiDAR and vision sensor.

[0016] A digital signature is generated by binding the vehicle identification number (VIN) with the hardware trigger time, satellite time synchronization time, and local crystal oscillator time.

[0017] The original data stream is encrypted to obtain ciphertext;

[0018] The output data packet is sent to the preprocessing module. The data packet includes: triple-time stamp plaintext, ciphertext obtained by encrypting the original data stream, and the digital signature.

[0019] Optionally, the preprocessing module is used for:

[0020] The public key is obtained by parsing the digital signature using the elliptic curve digital signature algorithm and then matched with the certificate public key in the whitelist for verification. If they do not match, the ciphertext is considered invalid and filtered out. The valid ciphertext is then decrypted to obtain the original data stream.

[0021] A sliding window is used to dynamically allocate the raw data stream to the respective buffer queues based on the sensor type;

[0022] Clock skew elimination is performed on the data in the buffer queue, and the time-aligned data sequence is output.

[0023] Optionally, the feature fusion module is used for:

[0024] Map the LiDAR data and visual data in the data sequence to the vehicle coordinate system respectively;

[0025] The time gaps in the lidar data are detected, and the time gaps are filled in using a cubic spline interpolation algorithm.

[0026] The displacement of a fixed point on the ground is determined by continuous frames of LiDAR data, thereby obtaining the displacement of the vehicle. The outline changes of the building are analyzed by continuous frames of visual data, thereby obtaining the rotation angle of the vehicle. Finally, a six-degree-of-freedom pose estimation result is obtained, which is bound to the vehicle identification number and carries the time stamp difference constraint range.

[0027] Optionally, the security processing module is used for:

[0028] The vehicle identification number is processed by a secure hash algorithm to generate a vehicle identification number hash value, which serves as an encryption seed;

[0029] A lattice cryptography scheme supporting homomorphic operations and an encryption seed are used to encrypt the LiDAR data, preserving the vector norm and angle characteristics to obtain an encrypted point cloud;

[0030] Based on the chaotic mapping algorithm, the spatial distribution of pixels in the visual data is rearranged using a cryptographic seed to generate an encrypted image that cannot be reverse-analyzed.

[0031] Calculate the cosine similarity between the feature vectors of the encrypted point cloud and the feature vectors of the encrypted image, which satisfy the timestamp difference constraint range.

[0032] By constraining the range of cosine similarity, timestamp difference, and encryption seed, a secure feature descriptor is generated.

[0033] Optionally, the verification storage module is used for:

[0034] Extract cosine similarity, timestamp difference constraint range, and encryption seed from the security feature descriptor, and write the extracted data into the leaf node of the Merkle tree;

[0035] Starting from the leaf nodes, the hash values ​​of the parent nodes are recursively calculated until the hash value of the root node is generated, forming a complete Merkle tree;

[0036] Based on the geographical location of the storage nodes and the current network latency indicators, a consensus protocol is used to elect a candidate node group from the storage nodes;

[0037] The Merkle tree, encrypted point cloud, and encrypted image are stored in the candidate node group, and the storage operation of the timestamp of the LiDAR data being greater than or equal to the timestamp of the visual data is not performed; the encrypted point cloud and encrypted image that satisfy the causal order are written into a new block of the distributed ledger.

[0038] Optionally, the model training module is used for:

[0039] Each candidate node group is used as a client for federated learning. Encrypted point clouds and encrypted images are obtained as training samples, and federated training with differential privacy is performed. During the forward propagation phase, Gaussian noise is injected into the training samples to generate gradient update values ​​that satisfy differential privacy protection. A mapping relationship is established between the timestamp of the gradient update value and the timestamp of the currently input training sample to obtain a mapping relationship table including multiple sets of timestamps. The gradient update value and the mapping relationship table are uploaded to the cloud.

[0040] The cloud verifies the linear correlation between the training sample timestamps and gradient update timestamps in the mapping table; filters out abnormal gradient update amounts with linear correlation below a set threshold; the retained gradient update amounts are weighted and summed to obtain model parameters; the model parameters are then distributed to each client for the next iteration.

[0041] Optionally, the calibration module is used for:

[0042] The abnormal gradient update amount is used to backtrack the training samples input to the model and the timestamps of the training samples;

[0043] The correlation coefficient between the timestamps of the visual data and the timestamps of the LiDAR data was calculated.

[0044] If the correlation coefficient is lower than a set threshold, the Marzullo algorithm is used to calculate the optimal interval between the timestamps of the LiDAR data and the visual data; the optimal interval includes the start time point and the end time point.

[0045] The difference between the start time point and the earliest timestamp of the lidar data is used as the first clock deviation compensation amount of the lidar, and the difference between the start time point and the earliest timestamp of the visual data is used as the second clock deviation compensation amount of the visual sensor.

[0046] The first clock offset compensation amount and the second clock offset compensation amount are sent to the hardware synchronization circuit in the data acquisition module.

[0047] After receiving the trigger reference, the hardware synchronization circuit triggers the LiDAR to start after the first clock deviation compensation amount and the vision sensor to start after the second clock deviation compensation amount, thereby achieving hardware synchronization calibration.

[0048] Optionally, the calibration module is used for:

[0049] Record the clock deviation values ​​of the LiDAR and the vision sensor before and after hardware synchronization calibration, the first clock deviation compensation amount and the second clock deviation compensation amount output by the Marzullo algorithm, and the correlation coefficient;

[0050] The recorded data is packaged into a blockchain transaction structure and written into a distributed ledger to form a traceable audit trail chain.

[0051] The technical effects of this application's embodiments are as follows: The nanosecond-level synchronization accuracy provided by the data acquisition module provides a reliable time reference for the preprocessing module to distinguish between crystal oscillator errors and transmission delays. The six-degree-of-freedom pose estimation results output by the feature fusion module carry timestamp difference constraints, providing the maximum time window for encrypted computation for the security processing module. The verification and storage module filters out noise data that violates physical laws for the model training module through spatiotemporal causality checks, ensuring the reliability of the training data. The calibration module converts the abnormal gradient update amounts detected during model training into clock deviation compensation amounts and feeds them back to the hardware synchronization circuit of the data acquisition module, continuously optimizing the hardware trigger timing and forming a closed-loop optimization chain from data application to the acquisition end. This interlocking design enables the system to form a complete and reliable processing chain from data acquisition to model optimization when dealing with sensor clock deviations. Attached Figure Description

[0052] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on the drawings without creative effort.

[0053] Figure 1 This is a structural diagram of an autonomous vehicle data processing system based on identity recognition, provided in an embodiment of this application. Detailed Implementation

[0054] To make the technical solutions of this application clearer, the application will be clearly and completely described below with reference to specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. The application provided by various embodiments will be described in detail below with reference to the accompanying drawings. To better understand the purpose of this application, it will be described in further detail below.

[0055] See Figure 1 The identity recognition-based autonomous vehicle data processing system provided in this application includes:

[0056] The data acquisition module generates a trigger reference by parsing the second pulse signal through a satellite receiver to drive the LiDAR and vision sensor to start synchronously and acquire the raw data streams of the LiDAR and vision sensor; after binding the vehicle identification number with the hardware trigger time, satellite time synchronization time and local crystal oscillator time, it generates a digital signature and outputs the raw data stream carrying the digital signature to the preprocessing module.

[0057] The preprocessing module filters out illegal data streams based on digital signatures, allocates legitimate raw data streams to corresponding buffer queues according to sensor type, performs clock skew elimination on the data in the buffer queues, and outputs time-aligned data sequences, which include LiDAR data and visual data.

[0058] The feature fusion module maps the data sequence to the vehicle coordinate system, fills in the missing time points of the LiDAR data, and parses the six-degree-of-freedom pose estimation results of the vehicle body based on the data sequence; it outputs the six-degree-of-freedom pose estimation results bound to the vehicle identification number and carrying the time stamp difference constraint range; wherein, the origin of the vehicle coordinate system is set as the center point of the rear axle of the vehicle, the Z axis is perpendicular to the ground, and the X and Y axes are located on the horizontal plane.

[0059] The security processing module generates an encryption seed based on the vehicle identification number to encrypt the LiDAR data and visual data respectively, resulting in encrypted point cloud and encrypted image. It calculates the cosine similarity between the feature vector of the encrypted point cloud and the feature vector of the encrypted image that satisfy the timestamp difference constraint range, and generates a security feature descriptor using the cosine similarity, timestamp difference constraint range and encryption seed.

[0060] The verification storage module constructs a complete Merkle tree based on the security feature descriptor and elects multiple roadside nodes as candidate node groups; the Merkle tree, encrypted point cloud, and encrypted image are stored in the candidate node groups.

[0061] In the model training module, each candidate node group acts as a client for federated learning. It uses decrypted LiDAR data and visual data as training samples to obtain the gradient update amount of the model and uploads the gradient update amount to the cloud. The cloud filters out abnormal gradient update vectors and obtains model parameters based on the gradient update amount. The model parameters are then distributed to each client for the next round of iteration.

[0062] The calibration module backtracks the training samples and timestamps of the training samples to the model based on the abnormal gradient update vector; calculates the correlation coefficient between the timestamps of the visual data and the timestamps of the LiDAR data; and reconfigures the hardware triggering timing of the data acquisition module based on the correlation coefficient.

[0063] It should be noted that the data acquisition module, preprocessing module, and feature fusion module need to be deployed locally on the vehicle controller to meet the requirements of sensor hardware driving, millisecond-level real-time processing, and direct connection to the vehicle bus; the security processing module, verification and storage module, model training module, and calibration module are deployed on a cloud server cluster / roadside node to achieve encrypted computation and global optimization. Calibration parameters and encrypted data streams are transmitted between the vehicle and the cloud via 5G-V2X to form a closed-loop control.

[0064] The technical effects of this application's embodiments are as follows: The nanosecond-level synchronization accuracy provided by the data acquisition module provides a reliable time reference for the preprocessing module to distinguish between crystal oscillator errors and transmission delays. The six-degree-of-freedom pose estimation results output by the feature fusion module carry timestamp difference constraints, providing the maximum time window for encrypted computation for the security processing module. The verification and storage module filters out noise data that violates physical laws for the model training module through spatiotemporal causality checks, ensuring the reliability of the training data. The calibration module converts the abnormal gradient update amounts detected during model training into clock deviation compensation amounts and feeds them back to the hardware synchronization circuit of the data acquisition module, continuously optimizing the hardware trigger timing and forming a closed-loop optimization chain from data application to the acquisition end. This interlocking design enables the system to form a complete and reliable processing chain from data acquisition to model optimization when dealing with sensor clock deviations.

[0065] The structure and function of each module are described in detail below:

[0066] The data acquisition module is used to: analyze the pulse-per-second (PPS) signal output by the Global Positioning System (GPS) through a satellite signal receiver to generate a trigger reference (signal) with nanosecond-level accuracy. After receiving this trigger reference signal, the hardware synchronization circuit triggers the synchronization startup mechanism of the LiDAR and vision sensor via an interrupt pin, acquires the raw data streams from the LiDAR and vision sensor, and generates a digital signature by binding the Vehicle Identification Number (VIN) with the hardware trigger time, satellite time synchronization time, and local crystal oscillator time. Specifically, the hardware trigger time refers to the precise moment when the FPGA (Field-Programmable Gate Array) chip in the hardware synchronization circuit detects the rising edge of the PPS pulse generated by the satellite signal receiver; this moment is recorded by the FPGA's internal counter. The satellite time synchronization time refers to the UTC (Coordinated Universal Time) integer second corresponding to the PPS pulse, obtained by the satellite receiver from the GPS signal; each pulse is associated with only one specific UTC time point. The local crystal oscillator time refers to the sampling completion moment recorded by the internal clock of the LiDAR or vision sensor when data acquisition is completed; although this time may drift, it is recorded as the device's reference time. Three time stamps are bound to the same data packet, forming a verifiable time reference chain: the hardware trigger time determines the synchronization start point, the satellite time synchronization provides an absolute time reference, and the local crystal oscillator time records the actual sampling time of the device. This triple time stamp enables downstream modules to distinguish the essential difference between crystal oscillator error and transmission delay. Specifically, the plaintext time data group consisting of the hardware trigger time, satellite time synchronization time, and local crystal oscillator time is bound to the vehicle identification number in a structured data format; the bound data is then processed using an elliptic curve digital signature algorithm to generate a digital signature. The original data stream is then encrypted using AES-GCM to obtain ciphertext. The final output data packet includes: the plaintext with the triple time stamps, the ciphertext obtained from the AES-GCM encryption of the original data stream, and the digital signature.

[0067] In this embodiment, the interrupt pin response mechanism of the hardware synchronization circuit enables hardware-level synchronous startup of the LiDAR and vision sensor, eliminating differences in sensor physical trigger delay. The vehicle identification number is bound to a triple time stamp, forming a verifiable time reference chain. An elliptic curve digital signature algorithm encrypts the structured data packet, generating a digital signature carrying authentication and time synchronization information. The subsequent preprocessing module can directly verify the timing logic using the plaintext time stamp, while simultaneously using the digital signature to verify data integrity and identity authenticity, achieving synchronous and reliable transmission of time synchronization and authentication information.

[0068] The preprocessing module first uses an elliptic curve digital signature algorithm to parse the digital signature to obtain the public key, and then matches and verifies it against the certificate public keys in the whitelist. If they do not match, the ciphertext is considered invalid and filtered, leaving only the valid ciphertext. The valid ciphertext is then decrypted to obtain the raw data stream. A sliding window is then used to dynamically allocate the raw data stream to its respective buffer queue based on the sensor type. Specifically, a 20ms fixed window is allocated to the LiDAR data based on its 50Hz sampling rate, and a 33ms frame rate adaptive window is allocated to the visual data based on its 30Hz sampling rate. The data within each window is sorted by timestamp and then allocated to its respective buffer queue. Next, clock skew elimination is performed on the data in the buffer queue: using satellite time synchronization as the reference time and the local crystal oscillator time as the time to be calibrated, a sequence of differences between the satellite time synchronization time and the local crystal oscillator time (i.e., the difference between the satellite time synchronization time and the local crystal oscillator time) is calculated. Assuming that the crystal oscillator error changes very little between adjacent times, the difference between the two at time t-1 is added to the local crystal oscillator time at time t to achieve local crystal oscillator time error compensation, and so on. The Lucas-Kanade optical flow method is used to calculate inter-frame motion vectors to compensate for transmission delays, ultimately outputting a time-aligned data sequence (including LiDAR and visual data). A description of the Lucas-Kanade optical flow method can be found in existing technologies and will not be repeated here. It should be noted that other methods can also be used to eliminate clock skew in the buffer queue. If the average difference between the satellite timing and the local crystal oscillator time is within a set range, clock skew elimination is not required.

[0069] In this embodiment, the whitelist certificate mechanism of the preprocessing module limits the range of trusted sensor devices, reducing the risk of illegal data injection. A fixed window corresponds to the fixed sampling period of the LiDAR, while an adaptive window matches the variable frame rate characteristics of the visual sensor. By processing the cumulative error of the LiDAR crystal oscillator and the visual data transmission delay, multi-sensor temporal alignment is achieved, with timestamp alignment accuracy reaching the microsecond level, establishing a unified time coordinate system for the feature fusion module.

[0070] In the feature fusion module, a three-dimensional rotation transformation matrix (e.g., SO(3) Lie group transformation matrix) is first used to map the LiDAR data (in the LiDAR body coordinate system) in the time-aligned data sequence to the vehicle coordinate system. The visual data (in the visual sensor coordinate system) is also mapped to the vehicle coordinate system to establish a unified reference coordinate system for multiple sensors (i.e., the vehicle coordinate system). Time gaps in the LiDAR data are detected and filled in using a cubic spline interpolation algorithm. Considering that the external environment changes with the vehicle's driving state as the vehicle moves and turns, the displacement of a fixed point on the ground (e.g., a traffic light base) is determined using continuous frames of LiDAR data, thus obtaining the vehicle's (three-dimensional) displacement. The two values ​​are consistent and opposite in direction. The outline changes of buildings (e.g., residential buildings) are analyzed using continuous frames of visual data, thus obtaining the vehicle's (three-dimensional) rotation angle. For example, if the outline of a residential building rotates 15 degrees to the right, it means that the vehicle rotates 15 degrees to the left. Finally, the feature fusion module outputs a six-DOF pose estimation result bound to the Vehicle Identification Number (VIN) and carrying a timestamp difference constraint range, and transmits it to the security processing module. The timestamp difference constraint range is a predefined maximum time window, such as twice the maximum sampling period, providing a time-related benchmark for the security processing module to generate encrypted feature descriptors.

[0071] After receiving the six-DOF pose estimation results from the feature fusion module, the security processing module first processes the Vehicle Identification Number (VIN) using a secure hash algorithm to generate a VIN hash value, which serves as the encryption seed. Then, it encrypts the LiDAR data using a lattice cipher scheme supporting homomorphic operations and the encryption seed, preserving the vector norm (i.e., the Euclidean distance between encrypted vectors) and angular characteristics to obtain an encrypted point cloud. Specifically, the encryption seed is used to perform a linear transformation encryption on the LiDAR data, maintaining the vector dot product and norm characteristics unchanged during the encryption process. This supports the calculation of cosine similarity and Euclidean distance in the ciphertext space, satisfying the mathematical conditions for calculating the correlation degree of secure features. Simultaneously, a chaotic mapping algorithm is used to rearrange the spatial distribution of pixels in the visual data using the encryption seed, generating an encrypted image that cannot be reverse-analyzed. Specifically, a pixel scrambling sequence is generated iteratively using the Logistic mapping function, nonlinearly rearranging the spatial positions of pixels in the visual data. The chaotic mapping algorithm utilizes the sensitive dependency of initial parameters to generate an irreversible pixel distribution, eliminating the correlation of spatial adjacency features in the original image space, achieving irreversible encryption of visual features. Then, the cosine similarity between the feature vectors of the encrypted point cloud and the feature vectors of the encrypted image, which satisfy the timestamp difference constraint, is calculated. The timestamp difference constraint limits the encrypted point cloud and encrypted image participating in the cosine similarity calculation to a maximum time window requirement, for example, forcing the timestamp difference between the point cloud and visual data to not exceed twice the maximum sampling period, thus enforcing spatiotemporal correlation. The cosine similarity, timestamp difference constraint, and encryption seed are used to generate a secure feature descriptor, for example, by concatenating the cosine similarity, timestamp difference constraint, and encryption seed. This secure feature descriptor satisfies the spatiotemporal correlation requirement without exposing the original data features, forming an encrypted feature expression that combines spatial correlation and temporal consistency.

[0072] In this embodiment, the encryption seed provides a unified key source for point cloud and image encryption. The lattice cipher scheme preserves the vector space relationship characteristics, making the calculation of ciphertext spatial distance feasible. The chaotic mapping algorithm destroys the original pixel spatial adjacency relationship, eliminating the recognizability of visual features.

[0073] After the verification storage module obtains the security feature descriptor from the security processing module, it first extracts the cosine similarity, timestamp difference constraint range, and encryption seed from the security feature descriptor, and writes the extracted data into the leaf nodes of the Merkle tree. Starting from the leaf nodes, the hash values ​​of the parent nodes are recursively calculated until the root node hash value is generated, forming a complete Merkle tree as proof of data integrity. The leaf nodes include a cosine similarity field, a vehicle identification number hash value field, and a timestamp difference constraint range field. After completing the layer-by-layer hash calculation to generate the root node hash value, the root node hash value is written as a data integrity identifier into the block header of the blockchain, providing an anti-tampering verification mechanism. Next, based on the geographical location of the storage node (e.g., a roadside node) and the current network latency indicators, a consensus protocol is used to elect a candidate node group from the storage nodes. Specifically, the network latency indicators and geographical coordinates of the storage nodes are monitored in real time, the response time and bandwidth parameters of the storage nodes are calculated, and a candidate node group (including multiple candidate nodes) is selected according to the Raft protocol majority consensus rules (e.g., the geographical location of the storage node should be as close as possible to the vehicle location, with fast response time and high bandwidth; rules can be manually defined). Then, the Merkle tree, encrypted point cloud, and encrypted image are stored in the candidate node group. The timestamps of the LiDAR data and the visual data within the candidate node group are checked to ensure causal order. Storage operations are performed where the timestamp of the LiDAR data is less than the timestamp of the visual data; storage operations are not performed where the timestamp of the LiDAR data is greater than or equal to the timestamp of the visual data. Encrypted point clouds and encrypted images that satisfy the causal order are written into a new block in the distributed ledger, forming an immutable storage record. The block header includes the block hash value and the root node hash value.

[0074] This embodiment writes cosine similarity, timestamp difference constraints, and vehicle identification number hash values ​​into Merkle tree nodes, cryptographically binding data integrity with vehicle identity. The storage node mapping strategy is dynamically adjusted based on network latency to adapt to dynamic topology changes in the vehicle-to-everything (V2X) environment. The physical basis for performing timestamp causality checks is the light propagation speed constraint; the time interval between the laser radar's emitted pulse reaching an object and its reflection and reception is necessarily earlier than the time interval between the visual sensor's acquisition of visible light imaging, which helps filter logical anomalies caused by sensor clock deviations. The distributed ledger storage structure adopts a timestamp-sorted chain structure. Data blocks within each block are arranged in ascending order of timestamps, and blocks form a time-related chain through preceding hash values, ensuring data immutability.

[0075] In the model training module, each candidate node group acts as a client for federated learning, acquiring encrypted point clouds and encrypted images. The encrypted point clouds and images are loaded into a trusted execution environment, decrypted, and used as training samples to obtain LiDAR data and visual data. Federated training with differential privacy is then performed. The trained model is a convolutional neural network used to identify vehicle behavior patterns / driving scenarios, such as highway driving and traffic congestion, based on the decrypted LiDAR and visual data. Specifically, during the forward propagation phase, Gaussian noise is injected into the training samples to generate gradient updates that satisfy differential privacy protection. Simultaneously, the timestamp of gradient update generation is recorded (referred to as the gradient update timestamp), and a mapping relationship is established between the gradient update timestamp and the timestamp of the currently input training sample. As the model iterates, it requires input training samples with different timestamps, resulting in gradient updates generated at different timestamps. The timestamps obtained in each iteration are then mapped to obtain a mapping table containing multiple sets of timestamps, including two timestamp sequences. The gradient updates and the mapping table are then uploaded to the cloud. The cloud platform first verifies the linear correlation between the training sample timestamps and gradient update timestamps in the mapping table. For example, it calculates the Pearson correlation coefficient between two timestamp sequences, filters out abnormal gradient updates with linear correlation below a set threshold, and retains gradient updates with high linear correlation. Assuming the cloud platform receives 100 sets of gradient updates from 100 clients, these sets are treated as a population, and the Mahalanobis distance of each set relative to the population is calculated. Abnormal gradient updates deviating from the population distribution by three standard deviations are subject to weight decay adjustment, for example, reducing the weight to 0.01. The remaining gradient updates are given normal weights, for example, 0.02. All weights are summed to 1. The gradient updates are then weighted and summed to obtain the model parameters. These model parameters are then distributed to each client for the next iteration.

[0076] In this embodiment, gradient updates with linear correlation below a set threshold are filtered out, while gradient updates with high linear correlation are retained. This ensures the alignment of model updates with the physical timeline and avoids data corruption. The Mahalanobis distance filtering mechanism effectively suppresses abnormal gradient interference. The weight decay adjustment mechanism dynamically suppresses abnormal clients, preventing the asynchronous data from a single client from affecting the global model convergence performance.

[0077] The calibration module continuously collects abnormal gradient updates (i.e., abnormal gradient updates where the Mahalanobis distance deviates from the population distribution by three standard deviations) from the model training module. Based on these abnormal gradient updates, it back-inputs the training samples (e.g., 50 sets of training samples) and the timestamps of the training samples (including the timestamps of the 50 sets of LiDAR data and the timestamps of the visual data). Then, it calculates the correlation coefficient between the timestamps of the visual data and the LiDAR data. This embodiment considers that the occurrence of abnormal gradient updates generally indicates a time misalignment of the training samples. Therefore, this embodiment uses the correlation coefficient as the basis for calibration triggering; the lower the value, the greater the time synchronization error between the sensors. If the correlation coefficient is lower than a set threshold, it is considered that the timestamps of the visual data and the LiDAR data are misaligned. The Marzullo algorithm is used to calculate the optimal interval between the timestamps of the (50 sets) LiDAR data and the timestamps of the visual data, denoted as [start time point, end time point]. The difference between the start time point and the earliest timestamp of the LiDAR data is used as the first clock skew compensation amount for the LiDAR, and the difference between the start time point and the earliest timestamp of the visual data is used as the second clock skew compensation amount for the visual sensor. The Marzullo algorithm is used to solve the clock synchronization problem in the system. It is a robust estimation algorithm based on interval mathematics, capable of finding the interval most likely to contain the true value even in the presence of partially erroneous or abnormal data sources. The first and second clock skew compensation values ​​are sent to the hardware synchronization circuit in the data acquisition module.

[0078] Upon receiving the trigger reference (signal), the hardware synchronization circuit triggers the LiDAR to start after the first clock deviation compensation amount and the vision sensor to start after the second clock deviation compensation amount, thus achieving hardware synchronization calibration. In this embodiment, the calibration module implements cross-validation of multi-source data, improving the reliability of clock offset detection. The Marzullo algorithm is used to eliminate clock deviation, achieving sub-microsecond hardware trigger timing alignment.

[0079] Finally, the calibration module records the clock deviation values ​​of the LiDAR and vision sensor before and after hardware synchronization calibration, the first and second clock deviation compensation amounts output by the Marzullo algorithm, and the correlation coefficient. These recorded data are packaged into a blockchain transaction structure and written into a distributed ledger to form a traceable audit trail. The blockchain transaction structure includes: the hash value of the preceding block, the hash value of the vehicle identification number, and the timestamp of the hardware synchronization calibration.

Claims

1. An identity recognition based autonomous vehicle data processing system, characterized in that, The application relates to a vehicle six-degree-of-freedom pose estimation method based on federated learning, which comprises the following steps: A data acquisition module generates a trigger reference by analyzing a second pulse signal through a satellite receiver to drive a laser radar and a visual sensor to start synchronously and collect original data streams of the laser radar and the visual sensor; A digital signature is generated by binding a vehicle identification number with a hardware trigger time, a satellite time and a local crystal oscillator time, and original data streams carrying the digital signature are output to a preprocessing module; A preprocessing module filters illegal data streams according to the digital signature, distributes legal original data streams to corresponding cache queues according to sensor types, eliminates clock deviation of data in the cache queues, and outputs time-aligned data sequences, wherein the data sequences comprise laser radar data and visual data; A feature fusion module maps the data sequences to a vehicle body coordinate system, fills in time missing points of the laser radar data, and analyzes a six-degree-of-freedom pose estimation result according to the data sequences; A six-degree-of-freedom pose estimation result bound with a vehicle identification number and carrying a timestamp difference value range is output; A security processing module generates an encryption seed according to the vehicle identification number to encrypt the laser radar data and the visual data respectively, obtains encrypted point clouds and encrypted images, calculates a cosine similarity of feature vectors of the encrypted point clouds and feature vectors of the encrypted images which satisfy the timestamp difference value range, and generates a security feature descriptor according to the cosine similarity, the timestamp difference value range and the encryption seed; A verification storage module constructs a complete Merkle tree according to the security feature descriptor, elects multiple roadside nodes as a candidate node group, and stores the Merkle tree, the encrypted point clouds and the encrypted images to the candidate node group; A model training module takes the decrypted laser radar data and visual data as training samples, obtains a gradient update amount of a model, and uploads the gradient update amount to a cloud end as a client of federated learning of each candidate node group; The cloud end filters an abnormal gradient update vector, obtains a model parameter according to the gradient update amount, and distributes the model parameter to each client for next round iteration; A calibration module traces back training samples input to a model and timestamps of the training samples according to the abnormal gradient update vector, and calculates a correlation coefficient between the timestamps of the visual data and the timestamps of the laser radar data; and the hardware trigger timing of the data acquisition module is reconfigured according to the correlation coefficient. The data acquisition module is used for:

2. The identity recognition based autonomous vehicle data processing system of claim 1, wherein, Analyzing a second pulse signal output by a global positioning system through a satellite signal receiver to generate a trigger reference with nanosecond-level precision to a hardware synchronization circuit; The hardware synchronization circuit triggers a synchronous starting mechanism of a laser radar and a visual sensor through an interrupt pin to collect original data streams of the laser radar and the visual sensor; A digital signature is generated by binding a vehicle identification number (VIN) with a hardware trigger time, a satellite time and a local crystal oscillator time; Original data streams are encrypted to obtain ciphertext; Data packets are output to a preprocessing module, wherein the data packets comprise triple time mark plaintext, ciphertext obtained by encrypting the original data streams and the digital signature. The preprocessing module is used for:

3. The identity recognition based autonomous vehicle data processing system of claim 2, wherein, ​ The digital signature is analyzed by using an elliptic curve digital signature algorithm to obtain a public key, which is matched with a certificate public key in a whitelist; if the two do not match, the ciphertext is considered illegal and filtered, and the original data stream is decrypted to obtain legal ciphertext. A sliding window is used to dynamically allocate the original data stream to respective cache queues according to the sensor types. Clock bias elimination is performed on the data in the cache queues, and a time-aligned data sequence is output.

4. The identity recognition based autonomous vehicle data processing system of claim 3, wherein, The feature fusion module is configured to: map the laser radar data and the visual data in the data sequence to a vehicle body coordinate system, respectively; detect time missing points of the laser radar data, and use a cubic spline interpolation algorithm to fill in the time missing points; determine the displacement of the ground fixed point through the laser radar data of consecutive frames, thereby obtaining the displacement of the vehicle, and analyze the profile change of the building through the visual data of consecutive frames, thereby obtaining the rotation angle of the vehicle, and finally obtain a six-degree-of-freedom pose estimation result that is bound to the vehicle identification number and carries a timestamp difference value constraint range.

5. The identity recognition based autonomous vehicle data processing system of claim 4, wherein, The security processing module is configured to: perform a secure hash algorithm on the vehicle identification number to generate a vehicle identification number hash value as an encryption seed; encrypt the laser radar data using a lattice cryptography scheme supporting homomorphic operations and the encryption seed, while retaining the vector norm and angle characteristics, to obtain encrypted point clouds; rearrange the pixel space position distribution in the visual data using the encryption seed according to a chaotic mapping algorithm to generate an encrypted image that cannot be inversely analyzed; calculate the cosine similarity of the feature vectors of the encrypted point clouds and the feature vectors of the encrypted image that satisfy the timestamp difference value constraint range; generate a secure feature descriptor from the cosine similarity, the timestamp difference value constraint range, and the encryption seed.

6. The identity recognition based autonomous vehicle data processing system of claim 5, wherein, The verification storage module is configured to: extract the cosine similarity, the timestamp difference value constraint range, and the encryption seed from the secure feature descriptor, and write the extracted data to the leaf nodes of a Merkle tree; starting from the leaf nodes, recursively calculate the hash values of the parent nodes until the root node hash value is generated, forming a complete Merkle tree; based on the geographical locations and current network delay indicators of the storage nodes, use a consensus protocol to elect a candidate node group from the storage nodes; store the Merkle tree, the encrypted point clouds, and the encrypted image to the candidate node group, and do not perform storage operations on encrypted point clouds whose timestamps are greater than or equal to the timestamps of the visual data; write the encrypted point clouds and the encrypted image that satisfy the causal order to a new block of a distributed ledger.

7. The identity recognition based autonomous vehicle data processing system of claim 6, wherein, The model training module is configured to: treat each candidate node group as a client of federated learning, obtain the encrypted point clouds and the encrypted image as training samples, and perform federated training with differential privacy; in the forward propagation stage, inject Gaussian distribution noise into the training samples to generate gradient updates that satisfy differential privacy protection; establish a mapping relationship between the gradient update timestamp and the training sample timestamp of the current input, and obtain a mapping relationship table including multiple groups of timestamps; upload the gradient updates and the mapping relationship table to the cloud; verify the linear correlation between the training sample timestamps and the gradient update timestamps in the mapping relationship table at the cloud end; and Filtering the abnormal gradient update amount with a linear correlation lower than a set threshold; performing weighted summation on the retained gradient update amount to obtain a model parameter; and issuing the model parameter to each client for the next round of iteration. 8.The identity recognition based autonomous vehicle data processing system of claim 7, wherein, The calibration module is used to: Backtracking the training sample input to the model and the timestamp of the training sample based on the abnormal gradient update amount; Calculating a correlation coefficient between the timestamp of the visual data and the timestamp of the lidar data; If the correlation coefficient is lower than a set threshold, using the Marzullo algorithm to calculate an optimal interval of the timestamp of the lidar data and the timestamp of the visual data; the optimal interval includes a start time point and an end time point; Taking the difference between the start time point and the earliest timestamp of the lidar data as a first clock bias compensation amount of the lidar, and taking the difference between the start time point and the earliest timestamp of the visual data as a second clock bias compensation amount of the visual sensor; Sending the first clock bias compensation amount and the second clock bias compensation amount to a hardware synchronization circuit in the data acquisition module; After receiving a trigger reference, the hardware synchronization circuit triggers the lidar to start after the first clock bias compensation amount, and triggers the visual sensor to start after the second clock bias compensation amount, to realize hardware synchronization calibration. 9.The identity recognition based autonomous vehicle data processing system of claim 8, wherein, The calibration module is used to: Recording the clock bias values of the lidar and the visual sensor before and after the hardware synchronization calibration, the first clock bias compensation amount and the second clock bias compensation amount output by the Marzullo algorithm, and the correlation coefficient; Packing the recorded data into a blockchain transaction structure, and writing into a distributed ledger to form a traceable audit tracking chain.

Images