Intelligent networking driving recording system based on multi-mode perception and data tamper-proofing method

Abstract

The invention discloses an intelligent networking driving recording system based on multi-mode perception. The intelligent networking driving recording system comprises a perception layer, a processing layer, an application layer and a block chain evidence storage module. The invention further discloses a data tamper-proofing method of the intelligent networked driving recording system based on multi-mode perception, which comprises the following steps: S1, synchronously acquiring thermal radiation data, radar point cloud data and video image data in a driving environment through the infrared thermal imaging sensor, the 4D millimeter wave radar and the high-definition optical camera; s2, infrared thermal imaging data are optimized, radar point cloud clustering is optimized through a DBSCAN algorithm, and target tracking is achieved based on UKF. The system can cover the whole temperature range of-30 DEG C to 85 DEG C and severe weather such as rain, snow and haze, effectively reduces the false alarm rate of infrared thermal imaging and the missed detection rate of a millimeter wave radar static target, avoids the loss of accident key evidence caused by environmental factors, greatly improves the evidence obtaining reliability at night and in severe weather, and reduces traffic accident disputes.

Description

TECHNICAL FIELD

[0001] The application relates to a car recorder, in particular to an intelligent networking car recording system based on multi-modal perception and a data tamper-proofing method. BACKGROUND

[0002] With the deep evolution of intelligent transportation systems (ITS) towards networking and automation, car recording devices have evolved from traditional single video recording tools to core terminals supporting traffic incident evidence collection, accident responsibility determination and intelligent risk warning. Currently, the mainstream intelligent networking car recording system generally adopts a multi-modal sensor fusion architecture, which combines optical cameras (such as Sony IMX678), 4D millimeter wave radars (TI AWR1843 chip) and infrared thermal imaging (FLIR A65) to realize all-weather environmental perception. On the data processing level, it relies on edge computing platforms such as NVIDIA Jetson AGX Orin to achieve a real-time inference speed of 45FPS, meeting the basic dynamic target recognition needs. In terms of data security, it introduces blockchain storage technology, realizes a throughput of 120 transactions per second (TPS) through Hyperledger Fabric and other consortium chains, and preliminarily builds data tamper-proofing capabilities, providing important technical support for the development of intelligent transportation.

[0003] However, the existing technology still has three major core bottlenecks in actual application, which seriously restrict its reliability and practicality in complex scenarios: first, the multi-modal perception fusion performance is insufficient. The target recognition false alarm rate of infrared thermal imaging in bad weather such as rain, snow and fog is as high as 15%-20%, making it difficult to effectively distinguish between real targets and environmental interference. Although 4D millimeter wave radar has strong penetration ability, its recognition accuracy for static obstacles (such as stationary vehicles and roadside facilities) is less than 85%, which easily leads to missed detection in critical scenarios. At the same time, the data collection of different modal sensors is asynchronous in time and space, and they do not form an efficient collaboration, further reducing the credibility of the perception results. Second, the data tamper-proofing mechanism has delay and security risks. The average verification delay of the existing blockchain storage solution is 5 seconds, which cannot meet the needs of real-time evidence collection and immediate responsibility determination at the traffic accident scene. Moreover, the security system has a single point failure risk. If the edge computing node is physically damaged or subjected to malicious attacks, it may result in permanent loss of local critical data. In addition, the cross-chain verification mechanism has not been standardized, and the evidence mutual recognition between different platforms is difficult, which affects the judicial acceptance efficiency of data. Third, the functional positioning is limited to "post-recording" and lacks active warning capability. Traditional systems can only provide retrospective evidence after an accident occurs and cannot make risk predictions based on real-time perception data. At the same time, the extreme environment adaptability is insufficient. In low temperature below-30℃ or high temperature above 85℃ scenarios, the sensor and processing unit are prone to performance degradation, resulting in data collection interruption or distortion.

[0004] According to the Global Road Safety Report 2024 of the World Health Organization, 37% of traffic accident disputes are caused by incomplete evidence chain, and the proportion of evidence collection difficulties in night accidents and bad weather conditions is as high as 63%; and the "hit and run" disputes and insurance claim disputes are also difficult to be efficiently solved due to the technical defects of the existing equipment. Therefore, it is an urgent demand in the field of intelligent transportation to develop an intelligent networking driving record system with all-weather accurate perception, millisecond-level data tamper-proofing and real-time risk warning capability. SUMMARY

[0005] The purpose of the present application is to overcome the defects of the current driving recorders, such as insufficient multi-modal perception fusion performance, delayed and security risks in data tamper-proofing mechanism, and lack of active warning capability, and to provide an intelligent networking driving record system based on multi-modal perception and a data tamper-proofing method.

[0006] The purpose of the present application is achieved by the following technical solution: an intelligent networking driving record system based on multi-modal perception, comprising a perception layer, a processing layer, an application layer and a blockchain storage module; the perception layer is used for synchronously collecting multi-modal environment data, which includes an infrared thermal imaging sensor, a 4D millimeter wave radar and a high-definition optical camera; the processing layer is based on a heterogeneous computing architecture to realize time-space calibration, feature fusion and intelligent analysis of multi-modal data; the application layer supports API docking with a traffic management platform and real-time uploading of evidence packages; the blockchain storage module is used to realize non-tamperable storage and verification of data from collection to judicial adoption.

[0007] Further, the model of the infrared thermal imaging sensor in the perception layer is FLIR A65, the model of the 4D millimeter wave radar is TI AWR1843, and the model of the high-definition optical camera is Sony IMX678.

[0008] The processing layer includes a main control chip, an AI accelerator and a memory, the main control chip is NVIDIA Jetson AGX Orin, the AI accelerator is configured with at least 2 Tensor Cores, and the memory is 32GB LPDDR5 supporting end-to-end data processing delay <50ms.

[0009] The processing layer adopts a time-space joint calibration architecture, which at least includes: time synchronization, realized by hardware triggering PTP protocol, with synchronization accuracy <1us; space alignment, the coordinate unification of multi-modal sensors is completed by 9-point calibration method; feature fusion, multi-source features are mapped to the same perspective for fusion through BEV transformation.

[0010] The underlying platform of the blockchain storage module adopts Hyperledger Fabric 2.5, the consensus mechanism is a PBFT variant algorithm, and 120 transactions per second are supported.

[0011] The blockchain storage module further comprises a verification mechanism, which at least comprises: integrity checking, verifying that the data has not been tampered with through hash comparison; timestamp verification, used to access the national time center to obtain a trusted timestamp; cross-chain verification, using the Cosmos IBC protocol to realize mutual recognition of evidence between different blockchain platforms.

[0012] As a preferred mode, the system further comprises a security protection system, which at least comprises: transmission encryption, using the TLS1.3 protocol, with a delay of <50ms; storage encryption, using the AES-256-GCM algorithm, with a throughput of 800MB / s; access control, using a mixed permission model of RBAC+ABAC, and identity authentication conforming to the FIDO2 standard.

[0013] A data tamper-proofing method of an intelligent networking driving record system based on multi-modal perception, comprising the following steps: S1, through an infrared thermal imaging sensor, a 4D millimeter wave radar and a high-definition optical camera, thermal radiation data, radar point cloud data and video image data in a driving environment are synchronously collected; S2, the infrared thermal imaging data is optimized, the radar point cloud clustering is optimized through a DBSCAN algorithm, and target tracking is realized based on a UKF; S3, the multi-modal data after preprocessing is stored and encrypted, and a data hash value is calculated through a SHA-3 algorithm, and then a digital signature is carried out based on an ECDSA; S4, the encrypted data, the hash value and the digital signature are written into a Hyperledger Fabric 2.5 blockchain through a PBFT variant consensus mechanism, and a trusted timestamp of a national time center is acquired; S5, when data needs to be retrieved or verified, integrity checking is carried out through hash comparison, and the data reliability is confirmed in combination with timestamp verification and cross-chain verification, and a verification result is output.

[0014] Compared with the prior art, the present application has the following advantages and beneficial effects: (1) The system perception layer of the present application is combined with a FLIR A65 infrared thermal imaging sensor, a TI AWR1843 4D millimeter wave radar and a Sony IMX678 high-definition optical camera, and when the time-space joint calibration architecture of the processing layer is matched, the full temperature range of-30 DEG C to 85 DEG C and adverse weather such as rain, snow and haze can be covered, the false alarm rate of infrared thermal imaging and the static target missing rate of millimeter wave radar are effectively reduced, the missing of key evidence caused by environmental factors is avoided, the reliability of evidence collection at night and in adverse weather is greatly improved, and traffic accident disputes are reduced.

[0015] (2) The processing layer of the application is based on NVIDIA Jetson AGX Orin master chip, 2 or more TensorCore AI accelerators and 32GB LPDDR5 memory to build a heterogeneous computing architecture, the end-to-end data processing delay is less than 50ms, and the blockchain storage module adopts a PBFT variant consensus mechanism, supports 120 transactions per second (TPS), and the data chaining verification delay is less than 5s, which can quickly complete the collection, analysis and storage of multi-modal data, solve the problem of high delay of traditional system blockchain storage, meet the needs of real-time evidence collection and immediate responsibility determination of traffic accident scene, and improve the efficiency of event processing.

[0016] (3) The blockchain storage module of the application takes Hyperledger Fabric 2.5 as the underlying platform, and through the whole link protection of "SHA-3 hash calculation + ECDSA digital signature + national time center timestamp + Cosmos IBC cross-chain verification", combined with AES-256-GCM storage encryption (throughput 800MB / s) and TLS 1.3 transmission encryption (delay <50ms), realizes the tamper-proof storage and verification of the whole process from data collection to judicial adoption; When calling data, the credibility is confirmed through the three mechanisms of hash comparison, timestamp verification and cross-chain verification, to ensure that the data meets the GA / T 947-2015 standard, greatly improves the legal effect of driving data, and provides reliable evidence support for insurance claims and judicial determination.

[0017] (4) The application layer supports API docking with the traffic management platform, can output standardized evidence packages, and is suitable for multiple scenes such as traffic accident evidence collection, fleet intelligent management, UBI insurance pricing, etc., shortens the responsibility determination time in accident handling, realizes real-time risk monitoring in fleet management, and provides reliable data support for accurate pricing in UBI insurance, fully embodies its empowerment value to related industries such as intelligent transportation and insurance, and helps to improve the efficiency of the industry and ecological synergy. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 It is a flowchart of data tamper-proofing of the application. DETAILED DESCRIPTION

[0019] The application will be further described in detail below in conjunction with the embodiments, but the embodiments of the application are not limited thereto.

[0020] EMBODIMENT

[0021] As Figure 1As shown, the intelligent networking driving record system based on multi-modal perception described in this embodiment mainly includes four parts: a perception layer, a processing layer, an application layer, and a blockchain storage module. The perception layer is used for synchronous collection of multi-modal environment data, which is the "input end" of the system for obtaining driving environment information. Through the cooperative work of multiple types of sensors, it realizes the synchronous collection of all-weather, multi-dimensional environment data, providing raw data support for subsequent data processing and analysis. At the same time, it cooperates with the time synchronization mechanism of the processing layer to ensure the consistency of the data collection timing of the sensors in the perception layer, avoiding the perception deviation caused by asynchronous data, and providing high-quality raw data for subsequent space-time calibration and feature fusion.

[0022] The multi-modal data of the perception layer is collected by three types of sensors, including an infrared thermal imaging sensor, a 4D millimeter wave radar, and a high-definition optical camera. Among them, the FLIR A65 is preferred for the infrared thermal imaging sensor, with parameters of 640x512@30Hz, NETD<50mK, which can identify the thermal radiation of a vehicle that has been extinguished 150 meters away, and solve the target perception problem in low-light, fog, and other scenes.

[0023] The 640x512@30Hz means that the infrared thermal imaging sensor has 640 pixels in the horizontal direction and 512 pixels in the vertical direction, with a total of about 320,000 pixels; "Hz" (Hertz) here represents the number of times the infrared thermal imaging sensor outputs a complete image per unit time, and "30Hz" means that the infrared thermal imaging sensor can generate 30 complete infrared images per second.

[0024] NETD refers to "Noise Equivalent Temperature Difference", which is the abbreviation of Noise Equivalent Temperature Difference, and is the core indicator for evaluating the performance of an infrared thermal imaging sensor, used to quantify the smallest temperature difference that the infrared thermal imaging sensor can distinguish; the smaller the NETD value, the stronger the detection ability of the infrared thermal imaging sensor to small temperature changes in the environment. NETD<50mK means that the noise equivalent temperature difference of the FLIR A65 sensor is less than 50 millikelvin, which can distinguish temperature differences as low as 0.05°C.

[0025] The TI AWR1843 is preferred for the 4D millimeter wave radar. The working frequency band of the 4D millimeter wave radar is 76-81GHz, with a horizontal field of view of 120°, and it has the ability to penetrate rain and fog, and can accurately capture the distance, speed, and other dynamic information of the target. The Sony IMX678 is preferred for the high-definition optical camera, which realizes detail capture with 48MP high pixels and records visual information such as vehicles, pedestrians, and traffic signs.

[0026] The processing layer adopts a heterogeneous computing architecture to process, fuse and analyze the multi-modal data collected by the perception layer, and converts the raw data into effective information that can be used for early warning and detection. The heterogeneous computing architecture includes an NVIDIA Jetson AGX Orin master control chip, 2x Tensor Core AI accelerators and 32GB LPDDR5 memory. Among them, 2x Tensor Core AI accelerator refers to that 2 Tensor Core AI accelerators are used in the processing layer. The Tensor Core AI accelerator is a special processing unit integrated in the GPU launched by NVIDIA, which is designed to accelerate AI and high-performance computing tasks, can realize mixed precision calculation, dynamically adjust computing power, and thus improve the computing throughput. According to actual needs, the number of Tensor Core AI accelerators can also be 3 or more.

[0027] The above processing and fusion of multi-modal data, i.e. data optimization is completed through a space-time joint calibration architecture (ST-JCA). Specifically, in time, hardware-triggered PTP protocol (synchronization accuracy <1μs) is used to realize data timing unification; in space, 9-point calibration method is used to complete sensor coordinate unification, and then BEV (Bird's Eye View) transformation is used to map multi-source features to the same perspective, solving the "spatial and temporal dislocation" problem of different modal data, improving the consistency and accuracy of perception results; and the data is optimized. In this embodiment, for infrared thermal imaging data, background difference method + morphological processing is used for optimization, and for radar point cloud clustering, DBSCAN algorithm is used for optimization, and UKF (Unscented Kalman Filter) is used to realize target tracking.

[0028] The risk warning model and the accident automatic detection algorithm are also integrated in the processing layer. The risk warning model inputs a 12-dimensional feature vector (speed, acceleration, etc. driving parameters) and outputs a 5-level risk warning (accuracy 92.3%), which can identify potential risks such as collision and lane change danger in advance; the accident automatic detection module judges the collision event based on the impact force threshold, and identifies the rollover accident through the attitude angle analysis, realizes the rapid and automatic identification of the accident, and provides decision basis for the subsequent evidence preservation and early warning response; and the end-to-end data processing delay is <50ms, which meets the real-time analysis requirements.

[0029] The application layer is the "output and interaction port" of the system, responsible for converting the analysis results of the processing layer and the evidence data into practical application value, interfacing with external platforms and scenarios, and the specific roles are as follows: 1. Standardized data output, that is, encapsulating the processed effective data (such as multi-modal evidence at the accident scene, risk warning information) into evidence packages conforming to the GA / T 947-2015 standard, ensuring that the data meet the compliance requirements of traffic management, judicial determination and other scenarios. 2. Interface with external platforms, which supports interfacing with external platforms such as traffic management platforms, insurance systems, etc. through API to upload evidence packages and analysis results in real time. For example, timely push accident scene data to the traffic management platform to assist the police in quickly determining responsibility; provide reliable driving data to the insurance system to support UBI insurance pricing and claims auditing, realizing the deep integration of the system and industry scenarios.

[0030] The blockchain evidence storage module is the "security core" of the system, building a full-process trusted system from data collection to judicial adoption, solving the problems of data tampering and insufficient legal effectiveness.

[0031] The blockchain evidence storage module is based on the Hyperledger Fabric 2.5 underlying platform, adopts the PBFT variant consensus mechanism (supports 120 TPS throughput), writes the multi-modal data output by the processing layer into the blockchain according to the process of "collection -> SHA-3 hash calculation -> ECDSA digital signature -> chain storage", combines AES-256-GCM storage encryption (throughput 800MB / s) and TLS1.3 transmission encryption (delay <50ms), and ensures that the data cannot be tampered once it is on the chain.

[0032] At the same time, the blockchain evidence storage module also establishes a multi-dimensional verification mechanism, which verifies the integrity by comparing the hash values to confirm that the data has not been tampered with; the timestamp verification accesses the national time service center to obtain a trusted timestamp to prove the data generation time; the cross-chain verification adopts the Cosmos IBC protocol to realize mutual recognition of evidence between different blockchain platforms, ensuring the credibility and legal effectiveness of data in traffic management, judicial, insurance and other scenarios, and solving the problems of "difficulty in obtaining evidence and difficulty in adoption" of traditional driving records data.

[0033] Among them, storage encryption, transmission encryption and access control together constitute the security protection system of the system, that is, transmission encryption adopts the TLS1.3 protocol, its delay is <50ms; storage encryption adopts the AES-256-GCM algorithm, the throughput is 800MB / s; access control adopts the RBAC+ABAC hybrid permission model, and the identity authentication conforms to the FIDO2 standard.

[0034] Transmission encryption is for the transmission scenario of system data, TLS (Transport Layer Security) is an international standard for network transmission security, which is used to establish an encrypted communication link between the client and the server to prevent data from being stolen, tampered with or monitored during transmission. The TLS 1.3 protocol in this embodiment is the latest version of the protocol, which simplifies the handshake process and optimizes the encryption algorithm suite, while improving security and significantly reducing transmission delay, preventing data from being stolen or monitored during network transmission, and ensuring data confidentiality. At the same time, the TLS 1.3 protocol has a built-in data integrity verification mechanism that generates a verification value for the transmitted data using a hash algorithm (SHA-256), and the receiving party confirms whether the data has been tampered with during transmission by comparing the verification value, avoiding malicious tampering with accident evidence or driving data. The transmission delay of the TLS 1.3 protocol is <50ms, which matches the performance indicators of the processing layer "end-to-end data processing delay <50ms", ensuring that the system does not cause transmission lag when uploading real-time data (such as accident scene evidence packages) due to encryption, meeting the needs of real-time evidence collection and immediate liability determination for traffic accidents.

[0035] The storage encryption is used to ensure the long-term secure storage of local and on-chain data. The system storage encryption uses the AES-256-GCM algorithm to achieve an encryption throughput of 800MB / s, covering all data storage scenarios.

[0036] Access control is used to implement compliance and permission control of data access. The system access control uses a mixed permission model of RBAC+ABAC, and the identity authentication conforms to the FIDO2 standard. For different subjects (traffic management departments, insurance agencies, vehicle owners, etc.), the system solves the core problem of "who can access the data and which data can be accessed".

[0037] Based on the above, the data tamper-proofing method of the intelligent networked driving record system based on multi-modal perception of the embodiment specifically includes the following steps: S1, through the infrared thermal imaging sensor, 4D millimeter wave radar and high-definition optical camera, the thermal radiation data, radar point cloud data and video image data in the driving environment are synchronously collected.

[0038] The 4D millimeter wave radar can output four-dimensional data of distance, azimuth, height and speed, and can clearly depict the contour features of the target object. In terms of function implementation, the 4D millimeter wave radar can not only complete the tasks of traditional 3D millimeter wave radar such as ACC (adaptive cruise control), blind area monitoring, etc., but also better cope with small target recognition in complex traffic environments, such as fallen tires, low obstacles, etc.

[0039] S2, optimizing infrared thermal imaging data, optimizing radar point cloud clustering through DBSCAN algorithm and realizing target tracking based on UKF.

[0040] The optimization of infrared thermal imaging data in this step refers to optimizing thermal imaging data through background difference method and morphological processing, so as to quickly strip static background interference from complex infrared images, focus on dynamic targets, avoid target misjudgment caused by background noise, and narrow the range for subsequent target feature extraction.

[0041] The background difference method is to establish a "static background model" (such as the infrared image of the road surface and roadside facilities when there are no vehicles and pedestrians), calculate the pixel-level difference between the real-time collected infrared data and the background model, and determine the "dynamic target area" when the temperature difference (thermal radiation difference) of a certain area exceeds the set threshold. If the difference is below the threshold, it is determined as "static background".

[0042] The morphological processing is an image optimization technique based on mathematical morphology, which further processes the "noise points" and "incomplete target outline" that may exist after background difference. The processing of noise points specifically includes the following steps: first, erode the image to eliminate small noise points; then, perform dilation processing to restore the integrity of the target outline and avoid the target becoming smaller due to erosion; finally, remove scattered noise in the image while preserving the core outline of the target.

[0043] The processing of incomplete target outline specifically includes the following steps: first, fill in the small holes in the target outline, such as the gaps in the thermal outline of the vehicle caused by fog and haze; then, correct the roughness of the target edge after dilation to ensure the continuity and integrity of the infrared outline of the target, facilitating subsequent feature matching with 4D millimeter wave radar point cloud data and high-definition optical images.

[0044] The radar point cloud clustering optimized by the DBSCAN algorithm is a key operation of the processing layer for processing and analyzing the point cloud data collected by the 4D millimeter wave radar. The purpose is to extract valuable information from complex radar data, accurately identify and track target objects on the road, and provide reliable basis for subsequent risk warning, accident detection and other functions. Among them, the radar point cloud data is a discrete point set collected by the 4D millimeter wave radar, which reflects the position and distance of objects in the surrounding environment. Due to the complexity of the actual environment, the point cloud data may exist noise, uneven distribution and other situations, which is difficult to be directly used for target identification and tracking. The DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is a density-based clustering algorithm, which divides the points with density connection into the same cluster (category), and regards the points in the low-density area as noise points. When processing the radar point cloud data, the DBSCAN algorithm can automatically identify different target objects according to the density distribution of the point cloud. At the same time, it can effectively identify and remove noise points, avoid the interference of noise to the subsequent analysis, improve the quality and accuracy of the point cloud data, and optimize the radar point cloud clustering effect.

[0045] The target tracking based on UKF refers to determining the position, velocity and other state information of the target object in a continuous time sequence, and predicting its future motion trajectory. UKF (Unscented Kalman Filter) is an unscented Kalman filter. In the vehicle driving scene, the motion state of the target object (such as other vehicles, pedestrians) often presents nonlinear change, and UKF can more accurately process the nonlinear transmission of mean and covariance through unscented transformation.

[0046] When implementing target tracking based on UKF, first, a state space model is established according to the motion characteristics of the target, the state variables and observation model are determined, and the initial value of the state vector and the covariance matrix are given in the initialization stage, then entering the prediction stage, using the dynamic equation in the state space model to predict the state at the next time combined with the current state estimation; finally, the predicted state is corrected through the observation data, and the estimation of the target state is continuously adjusted, so as to realize the accurate tracking of the motion state of the target object.

[0047] S3, store and encrypt the preprocessed multi-modal data, calculate the data hash value through the SHA-3 algorithm, and then perform digital signature based on ECDSA.

[0048] This step is the data tamper-proofing process of the system, which connects "data preprocessing" and "blockchain storage". Through the combination of "encryption-hash-signature" three technologies, it provides protection for multi-modal driving data from three dimensions of data security, integrity and traceability.

[0049] The "pre-processed multi-modal data" refers to the results of the original data collected by the perception layer after optimization by the processing layer. For infrared thermal imaging data, it is data after noise reduction and target contour enhancement by "background difference method + morphological processing"; for radar point cloud data, it is target point cloud data after optimization clustering and environmental clutter elimination by DBSCAN algorithm; for video image data, it is image data after clarity optimization and key frame extraction (complying with GA / T947-2015 standard evidence package requirements). These data are the basis for subsequent safe operation and are also the core information for accident evidence and risk warning of the system, and the security of which in storage and transmission needs to be ensured by multiple technologies.

[0050] The storage encryption is for the scenario of "local storage of pre-processed data", and its core functions include the following two aspects: first, high-strength encryption to prevent theft. The encryption algorithm used in this embodiment is AES-256-GCM, which is a symmetric encryption algorithm complying with international encryption standards. The "256-bit" key length means that there are a large number of key combinations, and it is difficult for malicious attackers to obtain data content through brute force cracking, which can effectively prevent high-value pre-processed data (such as accident key frames and target tracking trajectories) from being illegally stolen during local storage. Second, taking into account security and storage efficiency. GCM (Galois / Counter Mode) is an "authenticated encryption mode" that can generate a message authentication code (MAC) while encrypting data, without the need for additional verification algorithms to verify data integrity. With a high throughput of 800 MB / s, it can meet the high-speed storage needs of multi-modal data, and avoid slowing down the data processing process due to encryption, matching the performance indicators of the processing layer "end-to-end data processing delay < 50 ms".

[0051] The SHA-3 algorithm calculates the data hash value, which is used to ensure the integrity of the data.

[0052] SHA-3 (Secure Hash Algorithm 3, secure hash algorithm 3) is a hash calculation standard, which generates a unique hash value for "encrypted multi-modal data". The core functions are as follows: first, generate data "digital fingerprint". The SHA-3 algorithm can compress input data of any size into a fixed length hash value (such as 256 bits, 512 bits), and has "uniqueness" and "irreversibility". Even if the input data changes by 1 bit, the generated hash value will be completely different. At the same time, it is impossible to reverse the original data from the hash value, ensuring that the data integrity is verifiable and the original data is not leaked. Second, provide the basis for subsequent verification. The calculated hash value will be written into the blockchain together with the encrypted data and digital signature (step S4). When the data is verified later (step S5), the hash value of the data can be recalculated and compared with the hash value stored on the chain. If they are consistent, it proves that the data has not been tampered with. If they are not consistent, it means that the data has been modified. This directly supports the "integrity check" mechanism of the blockchain evidence module.

[0053] The digital signature based on ECDSA is used to ensure data traceability and anti-repudiation. The ECDSA (Elliptic Curve Digital Signature Algorithm) is a data signing technology. This operation signs the combined information of "encrypted data + SHA-3 hash value" by the system. The core functions are as follows: first, confirm the legitimacy of data source. The signature process requires the use of the system's exclusive "private key" (only the system itself holds it and cannot be leaked), and the generated digital signature corresponds one-to-one with the system's "public key" (publicly available). During subsequent verification (step S5), the digital signature is decrypted by the public key. If the decryption is successful and matches the data hash value, it can be confirmed that the data comes from the system and not from other illegal entities. This solves the problem of "unknown data source". Second, achieve anti-repudiation. Since the private key is only held by the system, once the digital signature is completed, the system cannot deny that the data was generated or uploaded by it, avoiding disputes in subsequent accident investigation and judicial determination, and providing a traceability basis for the judicial acceptance of driving data. This is consistent with the core goal of the document "to achieve unalterable storage and verification of data from collection to judicial acceptance of the whole process".

[0054] The storage encryption, SHA-3 hash calculation, and ECDSA digital signature in this step are not independent operations, but form a progressive cooperative relationship, that is, the storage encryption is the "first line of defense", which ensures that the data is not stolen when stored locally and solves the "data confidentiality" problem; the SHA-3 hash calculation is the "second line of defense", which generates a unique identifier for the data to provide a basis for subsequent integrity verification and solves the problem of whether the data has been tampered with; the ECDSA digital signature is the "third line of defense", which confirms the source of the data and prevents repudiation, solving the problem of whether the data source is legal. The above three work together to ensure that the preprocessed multi-modal data has the characteristics of "unstealable, tamper-proof, and traceable" before entering the blockchain storage (step S4), laying a secure foundation for subsequent writing to the Hyperledger Fabric 2.5 blockchain and obtaining a national timing center timestamp, and ultimately realizing "data from collection to judicial adoption of the whole process of trust".

[0055] S4, write the encrypted data, hash value, and digital signature to the Hyperledger Fabric 2.5 blockchain through the PBFT variant consensus mechanism to obtain a trusted timestamp from the national timing center.

[0056] This step is a data tamper-proofing process that takes over the "data encryption-hash calculation-digital signature" of step S3, and combines blockchain storage with trusted timestamps to build a "tamper-proof + time-trustworthy" dual protection for data from processing to judicial adoption.

[0057] The three types of core data written to the Hyperledger Fabric 2.5 blockchain in this step are the result of "preprocessing + security enhancement", ensuring that the data itself has basic security and integrity. For encrypted data, i.e., multi-modal data encrypted by the AES-256-GCM algorithm in step S3, the data has been secured by symmetric encryption to avoid being illegally interpreted during the writing process to the blockchain or after storage.

[0058] For the SHA-3 hash value, i.e., the unique hash value calculated for the encrypted data in step S3, it serves as a "digital fingerprint" for the data, which can be used to verify whether the data has been tampered with in the future.

[0059] For the ECDSA digital signature, i.e., the digital signature generated based on the system private key in step S3, it corresponds to the system public key and is used to confirm the legality of the source of the written data and prevent data from being forged or repudiated.

[0060] The PBFT variant consensus mechanism is the key to achieving safe and fast writing of data to the Hyperledger Fabric 2.5 blockchain.

[0061] PBFT (Practical Byzantine Fault Tolerance) is a consensus mechanism commonly used in consortium chains. Its core is to ensure transaction legitimacy through "multi-round voting verification between nodes". It conducts multi-round verification on transaction proposals composed of encrypted data, hash values, and digital signatures. Only when more than 2 / 3 of the nodes confirm that the proposal is legitimate (no tampering and trusted source) can data be written to the blockchain. It can tolerate partial node failures or malicious attacks and solves the problem of "single point failure leading to data tampering".

[0062] "Variety" refers to simplifying part of the non-key verification process, optimizing node communication efficiency, maintaining Byzantine fault tolerance capability, achieving "120TPS" throughput and low latency (matching the performance indicators of the processing layer "end-to-end delay < 50ms"), ensuring that accident data can be quickly chained, and meeting the needs of "real-time evidence collection and immediate responsibility determination".

[0063] The Hyperledger Fabric 2.5 blockchain as the underlying blockchain platform is essentially adapted to the needs of driving data "controllable authority, privacy protection, and judicial compliance". Its core role in the data writing process is: 1. Authorized node management: Hyperledger Fabric 2.5 as a consortium chain only allows authorized nodes (such as traffic management departments, cooperative insurance institutions, and system operation nodes) to join the network and participate in consensus, avoiding unauthorized nodes from obtaining or tampering with data. It forms a synergy with the "RBAC + ABAC hybrid permission model" of the security protection system in this system to ensure that the authority of data storage is controllable. 2. Channel isolation and privacy protection. The platform supports the "channel" mechanism, which can put data in different scenarios into different dedicated channels, and only authorized nodes in the channel can access the corresponding data. For example, accident data is only open to traffic police, judicial, and involved insurance institution nodes, avoiding irrelevant subjects from obtaining sensitive information and solving the problem of "balancing multi-subject data sharing and privacy protection". 3. Modular architecture adaptation. The platform supports flexible extension of consensus modules and encryption modules, can seamlessly integrate "PBFT variety consensus", "SHA-3 hash verification", "ECDSA signature verification", and other technologies, while being compatible with subsequent cross-chain verification needs, providing an extensible framework for "data from collection to judicial adoption" of the whole process of storage.

[0064] The trusted timestamp of the national time service center is obtained, that is, the data is given "time legitimacy", which is a key enhancement to the blockchain storage, solves the risk of "time tampering", and has the following effects: 1. Authority of time source. The national time service center is the legal time issuing agency in China, and the timestamp provided by the national time service center has the authority of judicial approval, which is different from the local time of the blockchain node (which may be tampered with or have deviation), and ensures that the "generation time" of the written data cannot be forged. 2. Binding of timestamp and data. The timestamp will be written into the blockchain together with "encrypted data + hash value + digital signature", forming a complete association chain of "data content-data fingerprint-source signature-trusted time", and when verifying later, the generation order of the data can be confirmed through the timestamp, and a trusted basis in the time dimension is provided for accident responsibility determination (such as "who first violated the lane change"). 3. Core support for judicial acceptance. According to the "Electronic Data Forensics Rules" of China, electronic data with trusted timestamp has higher evidential value in judicial determination, and this operation makes the blockchain storage data meet the requirements of judicial compliance, directly supports the core goal of "data judicial acceptance", and solves the defect of "low efficiency of evidence judicial acceptance" in the prior art.

[0065] S5, when the data needs to be called or verified, the integrity is checked through hash comparison, the data credibility is confirmed through timestamp verification and cross-chain verification, and the verification result is output.

[0066] This step uses the "three verification mechanisms" of integrity check through hash comparison, combination of timestamp verification and cross-chain verification to confirm the credibility of the data, to ensure that the retrieved multi-modal data (encrypted data, hash value, digital signature) is not tampered with, the time is trusted and the cross-platform is mutually recognized, and finally provides judicial-level trusted evidence for traffic accident responsibility determination, insurance claim and other scenes.

[0067] As described above, the present application can be well implemented.

Claims

1. A smart networked vehicle recording system based on multimodal perception, characterized in that, It includes a perception layer, a processing layer, an application layer, and a blockchain evidence storage module. The perception layer is used to synchronously collect multimodal environmental data, including infrared thermal imaging sensors, 4D millimeter-wave radar, and high-definition optical cameras. The processing layer is based on a heterogeneous computing architecture to realize spatiotemporal calibration, feature fusion, and intelligent analysis of multimodal data. The application layer supports API integration with traffic management platforms and uploads evidence packages in real time. The blockchain evidence storage module is used to achieve tamper-proof storage and verification of data from collection to judicial acceptance.

2. The intelligent networked vehicle recording system based on multimodal perception according to claim 1, characterized in that, The infrared thermal imaging sensor in the sensing layer is model FLIR A65, the 4D millimeter-wave radar is model TIAWR1843, and the high-definition optical camera is model Sony IMX678.

3. The intelligent networked vehicle recording system based on multimodal perception according to claim 2, characterized in that, The processing layer includes a main control chip, an AI accelerator, and memory. The main control chip is an NVIDIA Jetson AGX Orin; the AI ​​accelerator is configured with at least two Tensor Cores; the memory is 32GB LPDDR5, supporting end-to-end data processing latency of <50ms.

4. The intelligent networked vehicle recording system based on multimodal perception according to claim 1, characterized in that, The processing layer adopts a spatiotemporal joint calibration architecture, which includes at least: time synchronization, achieved by hardware triggering the PTP protocol with a synchronization accuracy of <1μs; spatial alignment, using a 9-point calibration method to unify the coordinates of multimodal sensors; and feature fusion, which uses BEV transformation to map multi-source features to the same viewpoint for fusion.

5. The intelligent networked vehicle recording system based on multimodal perception according to claim 1, characterized in that, The underlying platform of the blockchain evidence storage module adopts Hyperledger Fabric 2.5, and the consensus mechanism is a variant of the PBFT algorithm, supporting 120 transactions per second.

6. The intelligent networked vehicle recording system based on multimodal perception according to claim 5, characterized in that, The blockchain evidence storage module also includes a verification mechanism, which includes at least: integrity verification, which verifies that the data has not been tampered with through hash comparison; timestamp verification, which is used to access the National Time Service Center to obtain a trusted timestamp; and cross-chain verification, which uses the CosmosIBC protocol to achieve mutual recognition of evidence between different blockchain platforms.

7. The intelligent networked vehicle recording system based on multimodal perception according to claim 1, characterized in that, The system also includes a security protection system, which includes at least: transmission encryption, using the TLS1.3 protocol with a latency of <50ms; storage encryption, using the AES-256-GCM algorithm with a throughput of 800MB / s; and access control, using a hybrid RBAC+ABAC permission model with authentication conforming to the FIDO2 standard.

8. A data anti-tampering method for an intelligent networked vehicle recording system based on multimodal perception according to any one of claims 1 to 7, characterized in that, Includes the following steps: S1. Simultaneously collect thermal radiation data, radar point cloud data and video image data in the driving environment through infrared thermal imaging sensors, 4D millimeter-wave radar and high-definition optical cameras. S2. Optimize infrared thermal imaging data, optimize radar point cloud clustering through DBSCAN algorithm, and achieve target tracking based on UKF; S3. Store and encrypt the preprocessed multimodal data, calculate the data hash value using the SHA-3 algorithm, and then perform digital signature based on ECDSA; S4. Write the encrypted data, hash value, and digital signature into the Hyperledger Fabric 2.5 blockchain through the PBFT variant consensus mechanism to obtain the trusted timestamp from the National Time Service Center; S5. When data needs to be retrieved or verified, integrity is checked by hash comparison, and the data credibility is confirmed by combining timestamp verification and cross-chain verification, and the verification result is output.

Images