AI-powered vehicle intelligent terminal driving recording methods and control systems

Abstract

The present invention provides a driving recording method and control system for an AI-powered vehicle intelligent terminal, comprising: acquiring three-dimensional road image data, radar detection data, vehicle dynamic state data, and real-time traffic event data to obtain a multimodal driving monitoring dataset; inputting the multimodal driving monitoring dataset into a preset cross-modal dynamic fusion model to output driving hazard warning levels and core recording areas; the cross-modal dynamic fusion model uses a hybrid architecture of a Transformer encoder and a lightweight convolutional decoder, with its feature interaction layer embedding a multi-source feature alignment module based on attention transfer, and adding an intelligent judgment mechanism composed of a driving scene dynamic weight function and a spatiotemporal feature correlation function before the output decision layer; and controlling the AI-powered vehicle intelligent terminal to perform scenario-based driving recording and encryption based on the driving hazard warning level and core recording area. This invention overcomes the current deficiency of being unable to accurately locate the core recording area.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a driving recording method and control system for an AI-powered in-vehicle smart terminal. Background Technology

[0002] With the rapid development of intelligent connected vehicle technology, the driving recording function of in-vehicle smart terminals has gradually evolved from traditional video loop recording to intelligent and scenario-based safety monitoring and recording. Currently, most AI in-vehicle smart terminal driving recording solutions on the market integrate multiple sensors such as cameras and radar to collect driving data, and then analyze the data through basic AI models to trigger corresponding recording operations.

[0003] However, in real-world driving scenarios, existing technical solutions still face numerous technical bottlenecks:

[0004] First, the adaptability of multimodal data fusion is insufficient. Existing solutions mostly involve simple splicing or weighting of features without differentiating the characteristics of different modal data in vehicle scenarios. This results in low feature correlation among multi-source data, making it easy to misjudge or miss risks, which in turn leads to unreasonable triggering of recording strategies.

[0005] Secondly, the model architecture lacks the ability to dynamically adapt to different scenarios. Most solutions use a single convolutional neural network, which has limitations in capturing global spatiotemporal features. At the same time, the feature interaction process of the model does not adjust the feature weights according to real-time changes in the driving scenario (such as vehicle speed, weather, and road conditions), making it impossible to accurately locate the core recording area.

[0006] Third, the encrypted storage of driving record data mostly adopts a single local encryption method, which poses risks of data tampering and difficulty in tracing the source, and is difficult to meet the data security requirements in the Internet of Vehicles environment. Summary of the Invention

[0007] The main objective of this invention is to provide a driving recording method and control system for an AI-powered vehicle smart terminal, aiming to overcome the current deficiency of being unable to accurately locate the core recording area.

[0008] To achieve the above objectives, the present invention provides a driving recording method for an AI-powered in-vehicle smart terminal, comprising the following steps:

[0009] Collect 3D road image data, radar detection data, vehicle dynamic status data, and real-time traffic event data to obtain a multimodal driving monitoring dataset;

[0010] The multimodal driving monitoring dataset is input into a preset cross-modal dynamic fusion model, which outputs the driving hazard warning level and the core recording area. The cross-modal dynamic fusion model uses a hybrid architecture of Transformer encoder and lightweight convolutional decoder. Its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of driving scene dynamic weight function and spatiotemporal feature correlation function is added before the output decision layer.

[0011] Based on the driving hazard warning level and core recording area, the AI ​​in-vehicle smart terminal is controlled to perform scenario-based driving recording operations, and the driving recording data is encrypted and synchronized to the vehicle-cloud collaborative storage system.

[0012] Furthermore, before obtaining the multimodal vehicle monitoring dataset, the following steps are included:

[0013] Dynamic noise suppression based on the driving scene is performed on the three-dimensional road image data;

[0014] Spatiotemporal deviation compensation is performed on the radar detection data when stitching together multiple radar point clouds.

[0015] Furthermore, the multimodal driving monitoring dataset is input into a preset cross-modal dynamic fusion model, which outputs the driving hazard warning level and core recording area, including:

[0016] Modal adaptation preprocessing is performed on the 3D road image data, radar detection data, vehicle dynamic state data, and real-time traffic event data in the multimodal driving monitoring dataset.

[0017] The preprocessed modal data are input into the Transformer encoder of the cross-modal dynamic fusion model, and the global correlation features of each modal data are extracted through a multi-head attention mechanism. Specifically, a spatial attention head is used for 3D road image data, a motion attention head is used for radar detection data, and an attribute attention head is used for vehicle dynamic state and real-time traffic event data.

[0018] Each globally associated feature is input into the feature interaction layer. The similarity between different globally associated features is calculated and a feature association matrix is ​​constructed through the attention transfer-based multi-source feature alignment module.

[0019] The feature correlation matrix is ​​input into a lightweight convolutional decoder, and local key features are extracted through multi-layer depthwise separable convolution. The local key features are then input into the intelligent judgment mechanism before the decision layer.

[0020] Based on the output of the intelligent judgment mechanism, the softmax classifier of the output decision layer outputs a three-level driving hazard warning level, and the bounding box regression algorithm locates the coordinate range of the core recording area.

[0021] Furthermore, modal adaptation preprocessing includes:

[0022] The 3D road image data is converted into a standardized feature map and then normalized for mean and variance.

[0023] Convert the target range and velocity parameters in the radar detection data into standardized vectors and remove anomalous jump values;

[0024] Map the speed and braking pressure parameters in the vehicle dynamic status data to the 0-1 range;

[0025] Convert real-time traffic event data into encoded vectors.

[0026] Furthermore, after calculating the similarity between different globally associated features and constructing the feature association matrix, the process also includes:

[0027] The mechanism for dynamically adjusting the feature resolution associated with vehicle speed is initiated, including:

[0028] When the real-time vehicle speed exceeds the threshold, the resolution of the feature map corresponding to the radar detection data will be increased to 512×512.

[0029] When the real-time vehicle speed is not greater than the threshold, the feature map resolution corresponding to the radar detection data is kept at 256×256 to improve the feature weight of pedestrians and non-motorized vehicles in the three-dimensional road image data.

[0030] Furthermore, the intelligent decision-making mechanism before the output decision layer includes:

[0031] The dynamic weighting function for driving scenarios is activated to identify the environmental scene through 3D road image data and dynamically adjust the feature weights of each modal data based on the environmental scene.

[0032] The spatiotemporal feature association function is activated, which calls the historical motion data of the target in the same area within the previous preset seconds to construct a target motion trend model. If it is determined that the target has a sudden change of direction, or a sudden acceleration / deceleration, the area where the target is located is marked as a high-priority core recording area.

[0033] Furthermore, each globally correlated feature is input into the feature interaction layer. Through an attention-transfer-based multi-source feature alignment module, the similarity between different globally correlated features is calculated, and a feature correlation matrix is ​​constructed, including:

[0034] Convolution operations are used to map all globally correlated features to the same number of feature channels.

[0035] Through an attention-transfer-based multi-source feature alignment module, the spatial global features corresponding to the 3D road image data are divided into three groups of sub-features according to the actual distance range in the driving scene, corresponding to the near-range monitoring area, the medium-range monitoring area, and the long-range monitoring area, respectively; the motion global features corresponding to the radar detection data are divided into four groups of sub-features according to the target type in the driving environment, corresponding to motor vehicle targets, non-motor vehicle targets, pedestrian targets, and fixed obstacle targets, respectively; the spatial global features corresponding to the vehicle dynamic state data and real-time traffic event data are saved in their global form to obtain multiple sub-features to be aligned;

[0036] Using a pre-trained in-vehicle multimodal model as the teacher model, its attention distribution map is extracted as a template. The attention parameters of the current model are optimized through parameter distillation so that the initial attention distribution of the multiple sub-features to be aligned does not deviate from the teacher model by a preset value.

[0037] Calculate the similarity between multiple sub-features to be aligned and construct a feature association matrix, where the matrix elements are weighted similarity values ​​between sub-features.

[0038] The present invention also provides a control system for an AI-powered in-vehicle smart terminal, comprising:

[0039] The data acquisition module is used to acquire 3D road image data, radar detection data, vehicle dynamic status data, and real-time traffic event data to obtain a multimodal driving monitoring dataset.

[0040] The output module is used to input the multimodal driving monitoring dataset into a preset cross-modal dynamic fusion model and output the driving hazard warning level and core recording area. The cross-modal dynamic fusion model uses a hybrid architecture of Transformer encoder and lightweight convolutional decoder. Its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of driving scene dynamic weight function and spatiotemporal feature correlation function is added before the output decision layer.

[0041] The control module is used to control the AI ​​vehicle-mounted smart terminal to perform scenario-based driving recording operations according to the driving hazard warning level and core recording area, and to encrypt the driving recording data before synchronizing it to the vehicle-cloud collaborative storage system.

[0042] The present invention provides a driving recording method and control system for an AI-powered vehicle intelligent terminal, comprising: acquiring three-dimensional road image data, radar detection data, vehicle dynamic state data, and real-time traffic event data to obtain a multimodal driving monitoring dataset; inputting the multimodal driving monitoring dataset into a preset cross-modal dynamic fusion model to output driving hazard warning levels and core recording areas; wherein, the cross-modal dynamic fusion model uses a hybrid architecture of a Transformer encoder and a lightweight convolutional decoder, its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of a driving scene dynamic weight function and a spatiotemporal feature correlation function is added before the output decision layer; according to the driving hazard warning level and core recording area, the AI-powered vehicle intelligent terminal is controlled to perform scenario-based driving recording operations, and the driving recording data is encrypted and synchronized to a vehicle-cloud collaborative storage system. In this invention, the cross-modal dynamic fusion model uses a hybrid architecture of Transformer encoder and lightweight convolutional decoder to capture global spatiotemporal features. Its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of a driving scene dynamic weight function and a spatiotemporal feature correlation function is added before the output decision layer to adjust the feature weights according to the real-time changes of the driving scene, overcoming the current defect of not being able to accurately locate the core recording area. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the driving recording method steps of an AI-powered vehicle smart terminal in one embodiment of the present invention;

[0044] Figure 2 This is a block diagram of the control system structure of an AI vehicle-mounted smart terminal in one embodiment of the present invention;

[0045] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0046] The implementation, functional features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] It is particularly important to note that all technical steps, algorithm applications, and parameter settings in the technical solution of this application have clear technical objectives and application value. They do not utilize complex steps and algorithmic formulas to achieve simple functions. To provide detailed explanations of each step and avoid ambiguity, some conventional algorithms are used for illustration. However, this does not mean that the algorithms and technical features listed herein are the only way to implement the technical solution of this application, nor is it intended to limit the scope of protection of this application. This application is not a combination or stacking of the listed algorithms and technical features; its essence is to exemplify the implementation methods of this application to fully explain it. It does not pursue formal complexity by adding meaningless technical steps, nor does it involve the accumulation of technologies divorced from practical needs; it conforms to the conventional logic of technical improvement and design.

[0049] Reference Figure 1 One embodiment of the present invention provides a driving recording method for an AI-powered in-vehicle smart terminal, comprising the following steps:

[0050] Step S1: Collect 3D road image data, radar detection data, vehicle dynamic status data and real-time traffic event data to obtain a multimodal driving monitoring dataset;

[0051] Step S2: Input the multimodal driving monitoring dataset into a preset cross-modal dynamic fusion model and output the driving hazard warning level and core recording area; wherein, the cross-modal dynamic fusion model uses a hybrid architecture of Transformer encoder and lightweight convolutional decoder, its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of driving scene dynamic weight function and spatiotemporal feature correlation function is added before the output decision layer;

[0052] Step S3: Based on the driving hazard warning level and core recording area, control the AI ​​vehicle smart terminal to perform scenario-based driving recording operations, encrypt the driving recording data, and synchronize it to the vehicle-cloud collaborative storage system.

[0053] In this embodiment, as described in step S1 above, the core objective is to acquire comprehensive driving data across all dimensions through multi-source sensors and data interaction channels, avoiding the information limitations of a single data source in complex driving scenarios. Specifically, three-dimensional road image data is acquired using an onboard binocular camera. Compared to the two-dimensional images from a traditional monocular camera, a binocular camera can generate three-dimensional images containing depth information using the principle of parallax, which can more accurately restore the spatial positional relationships of targets (such as pedestrians, vehicles, and obstacles) in the road scene. For example, it can accurately distinguish the lateral distance between vehicles in adjacent lanes and vehicles in the current lane, laying a spatial foundation for the subsequent location of the core recording area. At the same time, radar detection data is acquired through millimeter-wave radar. This data can capture the dynamic parameters of the target in real time (such as the relative distance, relative speed, and direction of movement between the target and the vehicle), and has anti-interference characteristics unaffected by adverse environments such as rain, fog, and strong light at night. It can compensate for the shortcomings of image data recognition in low visibility scenarios, such as the ability to stably detect the deceleration status of vehicles ahead in heavy rain.

[0054] Secondly, regarding the vehicle's own operating status, vehicle power status data is transmitted via the onboard CAN bus (Controller Area Network). This data covers key parameters such as engine speed, throttle opening, brake pressure, and steering angle, directly reflecting the vehicle's current control intentions and operating status. For example, a sudden increase in brake pressure can indicate whether the driver has initiated emergency braking, providing dynamic evidence from the vehicle side for determining the level of driving risk. Finally, regarding the macro-traffic environment, real-time traffic event data is obtained through V2X (Vehicle-to-Everything) interaction technology. This data includes macro-traffic information such as congestion conditions on the road ahead, traffic accident warnings, and road construction information. This allows the driving recorder system to perceive potential risks outside the visual range in advance, such as adjusting the recording strategy before entering a construction zone to ensure complete monitoring and recording of the construction area.

[0055] After collecting the four types of data, preliminary formatting and validity screening are required. Abnormal jump values ​​in radar detection data (such as false target data caused by electromagnetic interference) are removed, and lens distortion in 3D road image data is corrected. Finally, the four types of data are integrated into a multimodal driving monitoring dataset with a unified structure and complete information, ensuring that the data can be directly adapted to the input requirements of subsequent cross-modal dynamic fusion models.

[0056] As described in step S2 above, by constructing a hybrid architecture model that combines global feature capture capability with lightweight computing power adaptability, the accurate fusion and intelligent analysis of multimodal data are achieved. The final output is a driving hazard warning level and core recording area that can directly guide the recording operation, thus solving the problems of inaccurate multimodal data fusion and poor model adaptability in the existing technology.

[0057] The key to this step lies in the architecture design and functional implementation of the cross-modal dynamic fusion model: the model adopts a hybrid architecture of Transformer encoder + lightweight convolutional decoder. The Transformer encoder is responsible for extracting global correlation features of multimodal data. Compared with the limitation of traditional convolutional neural networks that can only capture local features, the Transformer encoder can simultaneously correlate spatial features in 3D road image data, motion features in radar detection data, attribute features in vehicle dynamic state data, and event features in real-time traffic event data through a multi-head attention mechanism. For example, it can globally correlate radar data of sudden decrease in distance to vehicles ahead, vehicle data of sudden increase in braking pressure, and traffic event data of accident warning ahead to accurately identify high-risk states in emergency braking and avoidance scenarios. The lightweight convolutional decoder, while ensuring computational adaptability, extracts and refines local key features of global correlation features. It replaces traditional convolution with depthwise separable convolution, reducing the number of model parameters and computational load (adapting to the limited computing power of vehicle terminals) while accurately locating the area range of key targets such as pedestrians crossing the road and obstacles falling, providing detailed support for the determination of the core recording area.

[0058] In the feature interaction layer of the model, a multi-source feature alignment module based on attention transfer is embedded. The core function of this module is to solve the problems of inconsistent feature dimensions and low correlation between different modal data. By introducing a pre-trained in-vehicle multimodal alignment model as a teacher model, its feature attention distribution in typical driving scenarios is extracted. This distribution is used as a transfer template to optimize the attention weight allocation of the current model, enabling accurate alignment of different modal features such as spatial features of 3D road image data and motion features of radar detection data. For example, in high-speed driving scenarios, the attention weight of distant target features in radar detection data is automatically increased to ensure effective capture of distant risk targets in high-speed scenarios. At the same time, this module can dynamically adjust the feature resolution according to the real-time vehicle speed. For example, when the vehicle speed exceeds 60km / h, the feature resolution of radar data is increased, and when driving at low speeds, the resolution of pedestrian features in image data is increased, further improving the scenario adaptability of feature alignment.

[0059] Before the model's output decision layer, an intelligent judgment mechanism consisting of a dynamic weighting function for driving scenarios and a spatiotemporal feature correlation function is added. The dynamic weighting function for driving scenarios can adjust the feature weights of each modality data according to the real-time identified driving scenarios (such as foggy days, nighttime, and rainy days). For example, in foggy scenarios, because image data is greatly interfered with by fog, the feature weights of radar detection data are automatically elevated to the dominant position to ensure the accuracy of risk judgment. The spatiotemporal feature correlation function constructs a target motion trend model by associating the historical motion data of targets in the same area within the previous 3 seconds (such as the target's distance change rate and speed fluctuation amplitude). For example, by continuously monitoring the speed decay trend of vehicles ahead, it can predict whether they will stop suddenly, avoiding warning delays caused by misjudgments of instantaneous data.

[0060] Finally, the model combines the above feature fusion and intelligent judgment results, outputs a three-level driving hazard warning level through a classification algorithm, and locates the specific range of the core recording area (accurate to the pixel level) through a bounding box regression algorithm.

[0061] As described in step S3 above, based on the driving hazard warning level and the core recording area, a layered and precise recording strategy is formulated to ensure complete recording of key risk scenarios while avoiding waste of storage resources in risk-free scenarios: When a hazard warning level is determined (such as a vehicle suddenly stopping or a pedestrian crossing the road), the AI ​​vehicle-mounted smart terminal is controlled to start the ultra-high-definition 4K full frame rate recording mode, and at the same time, the collision pre-recording function is activated (recalling the driving data of the previous 5 seconds) to ensure high-definition recording of the complete process before and after the dangerous event, providing clear and complete evidence for subsequent accident liability determination; when a warning level is determined (such as traffic congestion ahead or proximity to a construction area), the terminal is controlled to execute the 1080P key area tracking and recording mode. High-definition recording is performed only on the core recording areas located in step S2 (such as congested vehicle clusters or construction enclosure areas), while the bitrate of non-core areas (such as open roads) is reduced to reduce storage usage while ensuring the integrity of key information. When the condition is determined to be normal (such as a smooth road section in sunny weather), a 720P low bitrate cyclic recording mode is adopted, and redundant segments without events are automatically cleaned up every 30 minutes to further optimize the utilization efficiency of storage resources.

[0062] Regarding the encryption and storage of driving recorder data, this step first encrypts the recorded video data and associated metadata (such as recording time, GPS location, and risk level tags) to ensure data privacy during transmission and storage, preventing unauthorized theft or tampering. Then, a vehicle-cloud collaborative storage mechanism is activated: on one hand, the encrypted complete data is synchronized to the cloud storage system, prioritizing the 5G-V2X low-latency transmission channel to ensure fast and stable data upload to the cloud, achieving off-site backup and long-term storage, avoiding data loss due to local storage device failure; on the other hand, only the hash value of the encrypted data and the record index mapping table are stored locally on the vehicle terminal. The hash value serves as a basis for verifying data integrity (if it is necessary to verify whether the cloud data has been tampered with, only the local hash value needs to be compared with the hash value calculated from the cloud data). The record index mapping table contains key information such as the recorded event type, timestamp, and GPS 3D coordinates, allowing users to quickly locate and retrieve the corresponding complete record data in the cloud through the index, achieving millisecond-level record tracing response.

[0063] In one embodiment, before obtaining the multimodal vehicle monitoring dataset, the following steps are included:

[0064] Dynamic noise suppression based on the driving scene is performed on the three-dimensional road image data;

[0065] Spatiotemporal deviation compensation is performed on the radar detection data when stitching together multiple radar point clouds.

[0066] In one embodiment, the multimodal driving monitoring dataset is input into a preset cross-modal dynamic fusion model, which outputs driving hazard warning levels and core recording areas, including:

[0067] Modal adaptation preprocessing is performed on the 3D road image data, radar detection data, vehicle dynamic state data, and real-time traffic event data in the multimodal driving monitoring dataset.

[0068] The preprocessed modal data are input into the Transformer encoder of the cross-modal dynamic fusion model, and the global correlation features of each modal data are extracted through a multi-head attention mechanism. Specifically, a spatial attention head is used for 3D road image data, a motion attention head is used for radar detection data, and an attribute attention head is used for vehicle dynamic state and real-time traffic event data.

[0069] Each globally associated feature is input into the feature interaction layer. The similarity between different globally associated features is calculated and a feature association matrix is ​​constructed through the attention transfer-based multi-source feature alignment module.

[0070] The feature correlation matrix is ​​input into a lightweight convolutional decoder, and local key features are extracted through multi-layer depthwise separable convolution. The local key features are then input into the intelligent judgment mechanism before the decision layer.

[0071] Based on the output of the intelligent judgment mechanism, the softmax classifier of the output decision layer outputs a three-level driving hazard warning level, and the bounding box regression algorithm locates the coordinate range of the core recording area.

[0072] In this embodiment, modal adaptation preprocessing is first performed on the four types of data in the multimodal driving monitoring dataset. This is the basic premise to ensure that data from different sources and in different formats can be effectively identified and processed by the cross-modal dynamic fusion model. The core objective is to eliminate data format differences and filter out invalid interference information, so as to lay a unified data foundation for subsequent feature extraction and fusion.

[0073] Specifically, for 3D road image data, preprocessing focuses on standardization and image quality optimization: First, the image size is uniformly adjusted to the model's preset fixed resolution (e.g., 224×224 pixels) to ensure consistent image feature dimensions in the input model; then, mean-variance normalization is performed, by subtracting the mean pixel value and dividing by the pixel variance to eliminate the interference of pixel value fluctuations under different lighting conditions on feature extraction, such as avoiding the influence of high pixel values ​​under bright sunlight and low pixel values ​​under dim light at night on model judgment; at the same time, for possible lens distortion in the image (such as stretching deformation in edge areas), distortion correction is performed through camera calibration parameters to ensure that the spatial position relationship of targets in the image is consistent with the actual driving scene, providing an accurate image basis for subsequent spatial feature extraction.

[0074] For radar detection data, the core of preprocessing is parameter standardization and outlier removal: Parameters such as relative distance and relative speed of targets collected by radar are converted into standardized vectors within a fixed range (e.g., the 0-1 interval) through linear mapping, eliminating the influence of parameter value differences under different radar models and detection distances on the model. Simultaneously, reasonable data thresholds (e.g., distance range 0-500m) are set based on the physical logic of the driving scenario, and abnormal jump values ​​exceeding the threshold are removed. These outliers are mostly caused by electromagnetic interference and radar false detections (e.g., misidentifying roadside trees as moving targets). Failure to remove them will lead to deviations in subsequent feature fusion, affecting the accuracy of risk assessment.

[0075] For vehicle dynamic state data, the preprocessing focuses on parameter normalization and dynamic smoothing: parameters such as engine speed, throttle opening, and brake pressure are mapped to the 0-1 range, allowing parameters of different magnitudes to participate in feature calculations on the same dimension (e.g., avoiding the numerical difference between engine speed of several thousand rpm and brake pressure of several megapascals from masking the parameter change trend); at the same time, a sliding window averaging method is used to dynamically smooth the parameters, eliminating instantaneous fluctuations during sensor acquisition (e.g., slight jitters in brake pressure), and more accurately reflecting the true change trend of vehicle dynamic state. For example, by using the smoothed brake pressure change, it is possible to accurately determine whether the driver is slowly decelerating or braking urgently.

[0076] For real-time traffic event data, the preprocessing operation mainly involves format structuring conversion: Since traffic event data acquired through V2X interaction is mostly in text or encoded form (such as "construction ahead 500m" or "road congestion ahead"), it needs to be converted into One-Hot encoded vectors that the model can recognize. For example, different vector codes are assigned to the four types of events: accidents, congestion, construction, and normal events. This transforms non-numerical event information into structured numerical features, ensuring that the data can collaborate with numerical features from other modalities in subsequent global feature extraction.

[0077] Inputting the preprocessed modal data into the Transformer encoder and extracting globally correlated features using a specially designed multi-head attention head is a crucial step in achieving global information linkage among multimodal data. The core lies in utilizing the Transformer's attention mechanism to accurately capture the potential correlations between different modalities, while simultaneously strengthening the core feature representation of each modality through differentiated attention head design. The core advantage of the Transformer encoder is that it breaks through the limitations of local features in traditional convolutional neural networks through a multi-head attention mechanism, enabling the simultaneous association of data features from different modalities and dimensions. For the four types of preprocessed modal data, this step designs three types of differentiated attention heads to achieve feature capture on demand:

[0078] For 3D road image data, a spatial attention head is employed. This attention head's weight allocation focuses on the spatial relationships and target morphological features within the image. By calculating the attention weights for different regions of the image, it automatically enhances feature extraction for key targets for driving safety (such as pedestrians, non-motorized vehicles, and obstacles). For example, it assigns higher attention weights to pedestrians crossing the road and weakens the feature proportion of background areas (such as open road surfaces). Simultaneously, the spatial attention head can capture the spatial relationships between targets, providing a spatial dimension for subsequent assessments of whether targets pose a risk.

[0079] For radar detection data, a motion attention head is employed. This attention head focuses on the dynamic motion characteristics of targets, assigning attention weights by calculating the rate of change of target distance and speed, thus enhancing the feature extraction of targets with abnormal motion trends (such as a vehicle suddenly decelerating ahead or an oncoming vehicle deviating from its lane). For example, when the radar detects that the distance to a vehicle ahead is decreasing rapidly at a rate of 10 m / s, the motion attention head will assign a high weight to the target's speed characteristic, highlighting its risk attribute of rapidly approaching. At the same time, the motion attention head can correlate the motion relationships of multiple targets (such as whether vehicles in adjacent lanes intend to overtake), providing a dynamic basis for global risk assessment.

[0080] For vehicle dynamics status data and real-time traffic event data, an attribute attention head is employed. Both types of data are attribute-based features (the former reflects the vehicle's own state attributes, and the latter reflects external traffic environment attributes). The attribute attention head strengthens the feature expression of key attributes by calculating the correlation between the two types of data and driving risks and assigning weights accordingly. For example, when a sudden increase in braking pressure occurs in the vehicle dynamics status data, the attribute attention head will increase the weight of this parameter, highlighting the vehicle's emergency braking state attribute. When real-time traffic event data shows a forward accident warning, the attention head will strengthen the weight of this event feature, associating external environmental risks with the vehicle's own state. At the same time, the attribute attention head can capture the synergistic relationship between the two types of data (such as the simultaneous occurrence of vehicle emergency braking and forward accident warnings), further enhancing the risk orientation of the globally correlated features.

[0081] In actual computation, the three types of attention heads work in parallel, each extracting the core features of its corresponding modality data. Then, the feature fusion layer of the Transformer encoder integrates the three types of features into a unified global correlation feature. This feature is no longer isolated information of a single modality, but rather contains globally linked information including image spatial target, radar motion trend, vehicle status, and traffic events. For example, by integrating global features such as pedestrian crossing the image, radar detecting pedestrian distance of 20m, speed of 5km / h, vehicle not braking, and no traffic event, high-risk scenarios of pedestrian crossing and vehicle not slowing down can be accurately identified.

[0082] Next, the globally correlated features are input into the feature interaction layer. A multi-source feature alignment module based on attention transfer is used to calculate similarity and construct a feature correlation matrix. This is a core step in addressing the differences in feature dimensions and loose correlations between different modalities. The aim is to achieve deep fusion of multi-modal data through feature alignment and quantify the correlation strength between features using the correlation matrix, providing a precise fusion basis for subsequent local feature extraction. Although the globally correlated features of different modalities have been initially correlated through the Transformer encoder, inconsistencies in feature dimensions and uneven correlation remain (e.g., image features are two-dimensional matrices, while radar features are one-dimensional vectors). Further calibration and enhancement are needed through the multi-source feature alignment module. The similarity between different globally correlated features is calculated, and a feature correlation matrix is ​​constructed.

[0083] Then, the feature correlation matrix is ​​input into a lightweight convolutional decoder, and local key features are extracted through multi-layer depthwise separable convolution. These features are then input into an intelligent judgment mechanism, which ensures that the model is adapted to the limited computing power of the vehicle terminal and enhances the scene-specific adaptability of the features through intelligent judgment.

[0084] The core design principle of the lightweight convolutional decoder is to achieve a balance between computational power adaptation and feature extraction accuracy: it replaces traditional convolution with depthwise separable convolution, splitting the traditional spatial convolution + channel convolution into two independent operations. This reduces the number of model parameters and computational cost by more than 70% while ensuring feature extraction performance. This optimization is crucial for in-vehicle terminals, as their computing resources are limited (far lower than cloud servers). The high computational cost of traditional convolution leads to model latency, failing to meet the real-time requirements of driving scenarios.

[0085] In the specific computation, local key features are extracted step by step through three layers of depthwise separable convolutions: The first layer uses a small-sized convolution kernel (e.g., 3×3) to perform preliminary filtering on the global correlation features in the feature correlation matrix, selecting basic features with risk orientation; the second layer uses a medium-sized convolution kernel (e.g., 5×5) to expand the regional correlation of the basic features, capturing the collaborative relationships of multiple features within a local region; the third layer regresses a small-sized convolution kernel (e.g., 3×3) to refine the collaborative features, ultimately outputting local key features focused on high-risk target areas and key parameter changes. For example, in the scenario of pedestrians crossing the road, the output is a combination of local key features including pedestrian position coordinates, pedestrian speed, distance between the vehicle and the pedestrian, and vehicle braking status, providing accurate and detailed support for subsequent intelligent judgment.

[0086] After local key feature extraction, an intelligent judgment mechanism is implemented before the input-output decision layer. This mechanism achieves scenario-based dynamic adaptation through two types of functions, solving the problem of the one-size-fits-all judgment defect of traditional models:

[0087] Firstly, the dynamic weighting function for driving scenarios adjusts feature weights based on the real-time scene: it identifies the current driving scenario (such as foggy, nighttime, rainy, or sunny) through image features (e.g., brightness, contrast) and radar features (e.g., target recognition stability) among local key features, and dynamically adjusts the weights of local key features for each modality based on scene characteristics. For example, in foggy scenarios, image features are affected by fog, leading to a decrease in recognition accuracy. The function automatically increases the weight of radar features to 60%, decreases the weight of image features to 25%, and allocates the weights of the remaining features to 15%, ensuring that risk assessment is primarily based on radar features with strong anti-interference capabilities. In nighttime scenarios, after compensating for insufficient light through the image enhancement submodule, the weight of image features is increased to 50%, focusing on image recognition of pedestrians, non-motorized vehicles, and other targets to improve the accuracy of risk assessment in nighttime scenarios.

[0088] Secondly, the spatiotemporal feature correlation function strengthens trend prediction through historical data: it calls upon historical local key features of targets in the same area within the previous 3 seconds (such as the target's distance change rate and speed fluctuation amplitude) to construct a target motion trend model. For example, by continuously monitoring the distance changes of vehicles ahead (from 100m to 50m, then to 20m) and speed changes (from 60km / h to 40km / h, then to 10km / h), it predicts that the vehicle may perform an emergency stop; at the same time, it correlates with the vehicle's own historical dynamic state (such as the throttle opening remaining stable and no braking operation within the previous 3 seconds) to comprehensively determine the high-risk trend of the vehicle ahead suddenly stopping + the vehicle itself not slowing down, avoiding the warning lag caused by relying solely on instantaneous data (such as only detecting the current distance of 20m before determining the risk, which may miss the best braking opportunity).

[0089] Finally, based on the output of the intelligent judgment mechanism, the softmax classifier and bounding box regression algorithm of the output decision layer are used to output the three-level driving hazard warning level and the coordinate range of the core recording area.

[0090] The output decision layer first quantifies the driving hazard warning level using a softmax classifier: the scenario-based feature weights output by the intelligent judgment mechanism, plus the target movement trend result, are converted into a high-dimensional feature vector and input into the softmax classifier. This classifier calculates the probability similarity between the feature vector and three preset levels: hazard, warning, and normal, outputs the probability value of each category (the sum of probabilities is 1), and takes the category with the highest probability as the final warning level.

[0091] Simultaneously, the bounding box regression algorithm is used to locate the coordinate range of the core recording region. This algorithm is based on local key features extracted by a lightweight convolutional decoder, focusing on the spatial region where high-risk targets are located (such as pedestrians, vehicles that stop suddenly, and obstacles), and calculates the bounding box coordinates of the targets through regression analysis. Specifically, the algorithm first preliminarily determines the approximate boundary of the target based on image features (such as the rectangular bounding box of a pedestrian), and then combines the distance and position information from radar detection data to accurately correct the bounding box coordinates (such as adjusting the left and right positions of the bounding box in the image based on the lateral distance of the pedestrian detected by radar), finally outputting the coordinate range of the core recording region with pixel-level precision.

[0092] In addition, the output decision layer adds priority labels (high, medium, low) to the core recording area. The priority is determined based on the risk level and distance of the target. These priority labels can further guide scenario-based recording operations (such as using higher frame rates for high-priority areas) to ensure that recording resources are allocated to the most critical risk areas. Finally, the output decision layer simultaneously outputs three types of results: driving hazard warning level, coordinate range of the core recording area, and area priority label. These three types of results are interconnected and mutually supportive.

[0093] In one embodiment, the modal adaptation preprocessing includes:

[0094] The 3D road image data is converted into a standardized feature map and then normalized for mean and variance.

[0095] Convert the target range and velocity parameters in the radar detection data into standardized vectors and remove anomalous jump values;

[0096] Map the speed and braking pressure parameters in the vehicle dynamic status data to the 0-1 range;

[0097] Convert real-time traffic event data into encoded vectors.

[0098] In one embodiment, after calculating the similarity between different globally associated features and constructing a feature association matrix, the method further includes:

[0099] The mechanism for dynamically adjusting the feature resolution associated with vehicle speed is initiated, including:

[0100] When the real-time vehicle speed exceeds the threshold, the resolution of the feature map corresponding to the radar detection data will be increased to 512×512.

[0101] When the real-time vehicle speed is not greater than the threshold, the feature map resolution corresponding to the radar detection data is kept at 256×256 to improve the feature weight of pedestrians and non-motorized vehicles in the three-dimensional road image data.

[0102] In this embodiment, after completing the global correlation feature similarity calculation and feature correlation matrix construction, a dynamic adjustment mechanism for the feature resolution of vehicle speed correlation is initiated. The core of this mechanism is to adapt multimodal features based on vehicle speed differences. For high-speed driving, it is necessary to ensure the accuracy of long-distance target monitoring, while for low-speed driving, it is necessary to strengthen the identification of vulnerable traffic participants, thus addressing the shortcomings of fixed feature adaptation in certain scenarios.

[0103] Based on the real-time vehicle power status data transmitted via the vehicle's CAN bus, the system first extracts the real-time vehicle speed, calls a preset speed threshold (usually calibrated to 60 km / h to match high-speed / low-speed scenario division and vehicle braking characteristics), and then performs differentiated adjustments based on the comparison between the vehicle speed and the threshold.

[0104] When the real-time vehicle speed is greater than 60 km / h, it is classified as a high-speed scenario (targets move quickly, braking distances are long, and accurate identification of distant targets is required). In this case, the resolution of the feature map corresponding to the radar detection data is increased from 256×256 to 512×512, a fourfold increase in pixel density. This allows for the detailed representation of motion parameters of distant targets (such as minute changes in relative distance and speed fluctuation trends), avoiding delays in risk prediction. Simultaneously, the radar data sampling interval is shortened from 50 ms / time to 20 ms / time, ensuring the dynamic continuity of the high-resolution feature map. While the resolution of the 3D road image features remains unchanged, the subsequent fusion weight of radar features is increased based on the feature correlation matrix (e.g., from 35% to 45%), prioritizing the focus on risks from distant targets.

[0105] When the real-time vehicle speed is ≤60km / h, it is determined to be a low-speed scene (complex target types, close distance, and many sudden risks, requiring enhanced pedestrian / non-motorized vehicle recognition). At this time, the radar feature map resolution is maintained at 256×256 (to meet the low-speed detection requirements and avoid wasting computing power) and the sampling interval is 50ms / sample. The focus is on optimizing image features: the feature association matrix is ​​used to locate pedestrian / non-motorized vehicle related feature dimensions (such as contour features), and their weight is increased from 25% to 40%, while the weight of the background area is reduced (from 20% to 10%), allowing the model to focus on weak target details (such as body movements and turning intentions); the image detail enhancement submodule is launched simultaneously to enhance the contours of pedestrians / non-motorized vehicles with edge detection algorithms, improve the recognition of targets in low-resolution or slightly blurred images, and avoid missed or false judgments.

[0106] This mechanism works in conjunction with a feature association matrix, which provides the basis for feature association strength (such as pedestrian-related image feature localization), while the vehicle speed threshold provides the basis for scene segmentation, achieving scene-data-feature adaptation. The adjustment results directly affect the subsequent lightweight convolutional decoder, enabling it to prioritize the extraction of core risk point features (high-speed long-distance targets, low-speed pedestrians), providing accurate input for the intelligent judgment and output decision layer, ultimately improving the accuracy of hazard warnings and core recording area determination, and ensuring that the recording strategy at different vehicle speeds meets safety requirements.

[0107] In one embodiment, the intelligent decision-making mechanism before the output decision layer includes:

[0108] The dynamic weighting function for driving scenarios is activated to identify the environmental scene through 3D road image data and dynamically adjust the feature weights of each modal data based on the environmental scene.

[0109] The spatiotemporal feature association function is activated, which calls the historical motion data of the target in the same area within the previous preset seconds to construct a target motion trend model. If it is determined that the target has a sudden change of direction, or a sudden acceleration / deceleration, the area where the target is located is marked as a high-priority core recording area.

[0110] In one embodiment, each globally correlated feature is input into a feature interaction layer. A multi-source feature alignment module based on attention transfer is used to calculate the similarity between different globally correlated features and construct a feature correlation matrix, including:

[0111] Convolution operations are used to map all globally correlated features to the same number of feature channels.

[0112] Through an attention-transfer-based multi-source feature alignment module, the spatial global features corresponding to the 3D road image data are divided into three groups of sub-features according to the actual distance range in the driving scene, corresponding to the near-range monitoring area, the medium-range monitoring area, and the long-range monitoring area, respectively; the motion global features corresponding to the radar detection data are divided into four groups of sub-features according to the target type in the driving environment, corresponding to motor vehicle targets, non-motor vehicle targets, pedestrian targets, and fixed obstacle targets, respectively; the spatial global features corresponding to the vehicle dynamic state data and real-time traffic event data are saved in their global form to obtain multiple sub-features to be aligned;

[0113] Using a pre-trained in-vehicle multimodal model as the teacher model, its attention distribution map is extracted as a template. The attention parameters of the current model are optimized through parameter distillation so that the initial attention distribution of the multiple sub-features to be aligned does not deviate from the teacher model by a preset value.

[0114] Calculate the similarity between multiple sub-features to be aligned and construct a feature association matrix, where the matrix elements are weighted similarity values ​​between sub-features.

[0115] In this embodiment, the initial number of channels for the global association features extracted from different modal data by the Transformer encoder varies significantly. For example, the spatial global features of 3D road image data typically have a large number of channels because they need to carry rich spatial detail information; while the attribute global features of vehicle dynamic state data have a relatively small number of channels because they have fewer parameter dimensions. If the original channel dimensions are used directly for feature interaction, the features with fewer channels will be diluted during fusion and cannot play an effective role. Therefore, a 1×1 convolution operation is needed to unify the channels of all global association features: a 1×1 convolution can linearly transform the feature channel dimensions without changing the feature spatial dimensions, mapping the global association features corresponding to 3D road image data, radar detection data, vehicle dynamic state data, and real-time traffic event data to the same preset number of channels. This operation not only preserves the core information of each modal feature (such as the spatial texture of the image and the motion trend of the radar), but also gives all features a unified dimensional benchmark, laying a consistent data foundation for subsequent sub-feature grouping and similarity calculation.

[0116] Based on the spatial global features corresponding to 3D road image data, the data is grouped according to the actual distance range in the driving scenario: the monitoring area covered by the image is divided into a near-range monitoring area (usually corresponding to the range 0-50m in front of the vehicle, which is a high-risk area for sudden events, such as pedestrians crossing or falling obstacles), a mid-range monitoring area (corresponding to the range 50-150m, which is the area for vehicles to make regular following and lane-changing decisions), and a far-range monitoring area (corresponding to the range 150-500m, which is the area for early perception of macroscopic road conditions). Accordingly, the spatial global features are further subdivided into three corresponding local sub-features. This grouping method allows subsequent processing of image features to better align with the risk characteristics of different distance areas; for example, near-range sub-features focus on detailed targets, while far-range sub-features emphasize overall road condition trends.

[0117] Based on the global motion features corresponding to radar detection data, the targets are grouped according to their type in the driving environment: Combining the core monitoring objects in the vehicle scenario, the moving targets captured by radar are categorized into motor vehicle targets (such as cars and trucks, affecting following and overtaking safety), non-motor vehicle targets (such as electric bikes and bicycles, highly agile and prone to sudden changes in direction), pedestrian targets (vulnerable road users, requiring the highest priority), and fixed obstacle targets (such as guardrails and construction barriers, affecting path planning). The global motion features are then further broken down into four corresponding local sub-features. This grouping accurately distinguishes the risk attributes of different moving targets, avoiding feature association bias caused by mixed target types. For example, pedestrian sub-features are processed separately from other target sub-features, ensuring priority identification of pedestrian risks.

[0118] For the global features corresponding to vehicle dynamic state data and real-time traffic event data, since both types of data are global attribute features, vehicle dynamic state data reflects the overall vehicle operating status (such as speed and braking) and needs to be correlated with the driving scenario as a whole; real-time traffic event data reflects the macro traffic environment (such as congestion and accidents) and needs to be applied globally to risk assessment. Therefore, its global form is maintained without splitting, and together with the above seven sets of local sub-features, it constitutes multiple sub-feature sets to be aligned, ensuring that feature grouping meets the requirements of refinement without losing global attribute information.

[0119] The pre-trained in-vehicle multimodal model (teacher model) has been trained on a large number of typical driving scenarios (such as rainy days, nighttime, highways, and urban areas), and possesses mature feature attention allocation logic. For example, in rainy scenarios, the teacher model automatically increases the attention weight of radar features and reduces the weight of image features affected by rain; in low-speed urban scenarios, it increases the attention weight of pedestrian and non-motorized vehicle sub-features. An attention distribution map of the teacher model for the current driving scenario (e.g., a highway in clear weather) is extracted. This map quantifies the optimal attention ratio of various sub-features to be aligned (such as radar vehicle sub-features and long-range image sub-features) in the current scenario, serving as a template for parameter transfer.

[0120] Subsequently, using parametric distillation, the weight allocation logic in the teacher model's attention distribution map is transferred to the attention parameters of the current model. By calculating the deviation between the initial attention distribution of the sub-features to be aligned in the current model and the template distribution map of the teacher model, and minimizing the deviation, the attention parameters of the current model are iteratively optimized (e.g., adjusting the weight coefficients of each sub-feature). Ultimately, the deviation rate between the initial attention distribution of the sub-features to be aligned and the teacher model does not exceed a preset value (usually set to 10%). This process allows the current model to quickly acquire the scene adaptability of the teacher model, avoiding the problem of unreasonable attention allocation caused by training from scratch. For example, it ensures that the current model automatically adjusts the attention weights of the radar long-range vehicle sub-features to a reasonable range in high-speed scenarios.

[0121] Similarity calculation between sub-features: For different types of sub-features to be aligned, the cosine similarity algorithm is used to quantify the association strength. For example, the similarity between near-range sub-features in the image and radar pedestrian sub-features is calculated (reflecting the degree of matching between pedestrians in the near-range image and pedestrians detected by radar), the similarity between global features of vehicle dynamic state and radar vehicle sub-features (reflecting the degree of association between vehicle braking state and changes in distance to vehicles ahead), and the similarity between global features of real-time traffic events and far-range sub-features in the image (reflecting the degree of matching between traffic congestion events ahead and road conditions in far-range images). Cosine similarity quantifies the directional consistency by calculating the cosine value of the angle between two sub-feature vectors (value range 0-1). The closer the value is to 1, the stronger the association between the sub-features; the closer it is to 0, the weaker the association. It can accurately reflect the collaborative relationship between different modalities of sub-features in the current scene.

[0122] Subsequently, a feature association matrix is ​​constructed: the rows and columns of the matrix correspond to all sub-features to be aligned (e.g., rows / columns represent near / mid / long-range sub-features of the image, radar sub-features of motor vehicles / non-motor vehicles / pedestrians / fixed obstacles, global features of vehicle dynamic state, and global features of real-time traffic events, respectively). Each element in the matrix is ​​the similarity weighted value of two corresponding sub-features. This weighted value consists of two parts: first, the sub-feature's own weight optimized based on attention transfer (ensuring that key sub-features such as radar pedestrian sub-features have a higher influence in the association); and second, the similarity value between the sub-features, which is calculated by multiplying the two to obtain the final weighted value. For example, the similarity weighted value between radar pedestrian sub-features and near-range sub-features of the image needs to be calculated by combining the high weight of the pedestrian sub-feature with the high similarity between the two to ensure that the association has a strong impact on subsequent risk assessment. The final constructed feature association matrix can clearly present the distribution of association strength among all sub-features to be aligned, providing a quantitative basis for the subsequent lightweight convolutional decoder to extract local key features and for the intelligent judgment mechanism to adjust feature weights.

[0123] In one embodiment, encrypting the driving record data includes:

[0124] Obtain the protocol characteristics, business data characteristics, and transmission timing characteristics of the camera, radar, CAN bus, and IMU on the AI-powered vehicle smart terminal, and construct a three-dimensional curve;

[0125] Obtain the IP / hardware address topology of the camera, radar, CAN bus, and IMU, assign edge weights according to the data interaction frequency between nodes, and simulate a directed graph;

[0126] Obtain a preset initial array including vehicle history encrypted fingerprint fragments and automotive-grade random salt values. Adjust the array element arrangement order of the initial array according to the edge weights of the directed graph, and offset the array element values ​​according to the edge weight features to generate the first data array.

[0127] The fluctuation value of the three-dimensional curve is used as a correction coefficient to correct the first data array, resulting in the second data array.

[0128] Based on the second data array and the unique hardware identifier of the vehicle terminal, an encryption key is generated to encrypt the driving record data.

[0129] In this embodiment, firstly, the protocol characteristics, business data characteristics, and transmission timing characteristics of the camera, radar, CAN bus, and IMU on the AI ​​vehicle smart terminal are acquired and a three-dimensional curve is constructed. The core purpose is to capture the real-time status during driving through multi-dimensional features, so that the encryption process is deeply bound to the driving scenario, thereby improving the uniqueness and dynamism of the key.

[0130] Specifically, three types of features are extracted for the four types of core vehicle-mounted equipment: For cameras and radar, protocol features include the baud rate of data transmission (e.g., radar data transmission baud rate is usually higher than that of cameras because high-frequency feedback of target dynamics is required) and data frame structure (e.g., the resolution encoding format of camera image frames and the arrangement of parameter fields in radar target frames); business data features cover changes in road target size / position collected by cameras, target distance / speed fluctuations detected by radar, vehicle speed / braking pressure / throttle opening parameters transmitted by CAN bus, and vehicle angular velocity / acceleration recorded by IMU; transmission timing features refer to the data sampling interval of the four types of equipment (e.g., cameras typically sample at 15-30fps, and radar samples at 10-20 times / second) and data synchronization delay (e.g., the transmission time difference between CAN bus and IMU data).

[0131] After extracting the three types of features, time-series data processing methods (such as sliding window smoothing) are used to preprocess the feature data to eliminate transient interference (such as baud rate jumps caused by electromagnetic interference in radar). Then, with time as the horizontal axis and the numerical changes of protocol features, service data features, and transmission timing features as the vertical axis, three independent feature curves are constructed, which together form a three-dimensional feature curve. This three-dimensional curve can reflect the dynamic changes of equipment interaction and vehicle operation in real time during driving.

[0132] Next, the IP / hardware address topology of the camera, radar, CAN bus, and IMU is obtained. Edge weights are assigned based on the frequency of data interaction between nodes, and a directed graph is simulated. By quantifying the strength of the association between devices, a differentiated adjustment basis is provided for data array variations. First, the unique identifiers of the four types of sensing devices are obtained through the device management module of the vehicle terminal. For devices with IP addresses (such as cameras and radar), their local area network IPs are extracted; for hardware devices without IPs (such as CAN bus modules and IMUs), their hardware serial numbers (such as chip IDs) are extracted. This establishes a mapping between device identity and connection relationships, clarifies the data flow direction between each device (such as the CAN bus transmitting vehicle parameters to the terminal, and the camera transmitting image data to the terminal), and forms the initial topology. Subsequently, the frequency of data interaction between each device within a preset time window is statistically analyzed. For example, the number of times the camera transmits image frames to the terminal, the number of times the radar transmits target data to the terminal, and the number of times the CAN bus transmits power parameters to the terminal are counted. The interaction frequency is used as the weight of the edges in the topology graph (the higher the interaction frequency, the larger the edge weight). Meanwhile, the topology graph is defined as a directed graph according to the data flow direction, with arrows indicating the data transmission direction and the node size visually representing the core status of the devices. The resulting directed graph can quantitatively reflect the real-time interaction intensity between onboard devices, providing a weighting basis for subsequent differentiated adjustments to the initial array.

[0133] The system obtains a preset initial array containing vehicle historical encrypted fingerprint fragments and automotive-grade random salt values. The vehicle historical encrypted fingerprint fragments are key fragments generated during the encryption of the previous driving record, which can ensure the correlation between the current encryption and the historical encryption, forming an encryption chain. The automotive-grade random salt values ​​are generated in real time by the vehicle security chip (such as the SE chip), which is unpredictable and can avoid encryption vulnerabilities caused by fixed initial data. The two are combined in a fixed length (such as each occupying 128 bits) to form the initial array.

[0134] When adjusting the order of array elements, the edge weights of the directed graph are used as the basis: sort the edge weights from largest to smallest to obtain a weight sequence, and then rearrange the elements of the initial array according to the offset rules corresponding to the weight sequence (e.g., the array element corresponding to the edge with the largest weight is shifted 3 positions to the left, the edge with the second largest weight is shifted 2 positions to the left, and so on), to ensure that the arrangement order is strongly correlated with the device interaction strength.

[0135] When offsetting the array element values, feature parameters of the edge weights (such as the units and tens digits of the edge weights) are extracted and used as the offset of the element values ​​(e.g., if the units digit of the edge weight is 5, the corresponding array element value increases by 5; if the tens digit is 3, the corresponding element value decreases by 3). If the value exceeds the preset range after offset, it is adjusted to a reasonable range through modulo operation. Through the adjustment of the arrangement order and the offset of values, the initial array is transformed into a first data array with device interaction characteristics, initially realizing the binding of encrypted data with the status of the vehicle-mounted device.

[0136] The fluctuation values ​​of the three-dimensional curves (protocol feature curve, service data feature curve, and transmission timing feature curve) at the current moment are extracted. The fluctuation value is the difference between the feature value at the current moment and the average value within the time window. The fluctuation values ​​of the three types of features are normalized to the 0-1 interval respectively, and used as correction coefficients (protocol feature correction coefficient K1, service data feature correction coefficient K2, and transmission timing feature correction coefficient K3). Subsequently, the elements of the first data array are corrected dimensionally: elements related to protocol features are multiplied by K1, elements related to service data features are multiplied by K2, and elements related to transmission timing features are multiplied by K3. If the corrected element value is a decimal, it is rounded to the nearest integer. The correction process can eliminate the deviation of the first data array caused by temporary equipment interference (such as radar instantaneous baud rate fluctuations), making the array elements more consistent with the real characteristics of the current driving scenario, and finally generating a second data array with scene dynamic adaptability.

[0137] The unique hardware identifier of the vehicle terminal serves as its immutable identity, possessing device-specific attributes to prevent key cracking across devices. During key generation, the second data array and the unique hardware identifier of the vehicle terminal are first concatenated byte-by-byte. Then, a hash algorithm is used to process the concatenated data, generating a 256-bit fixed-length hash value, which becomes the final encryption key. Complete driving record data (including video stream data and metadata, such as driving time, GPS location, and hazard warning level) is grouped and encrypted (e.g., encrypted in 128-bit groups). After encryption, an encrypted data packet is generated. This packet contains not only the encrypted record data but also a key index (linking the generation information of the second data array and the hardware identifier). This facilitates tracing the key generation basis during subsequent decryption and ensures that only terminals / cloud devices with the corresponding hardware identifier and decryption permissions can reconstruct the driving record data using the key index and decryption algorithm, thus achieving data security protection.

[0138] In the above embodiments, this application incorporates some existing algorithms and technical features for explanation and description to make the specification more detailed, clear, and complete, thus complying with the provisions of the Patent Law. However, this is not achieved by using a series of complex steps and algorithmic formulas, nor by complicating the technical solution, nor by combining or stacking conventional or simple features. The existing algorithms and technical features listed are for the purpose of disclosing the specific implementation methods of each step of this application (not to limit this application) and to avoid situations where this application cannot be implemented.

[0139] Reference Figure 2 In another embodiment of the present invention, a control system for an AI-powered vehicle smart terminal is also provided, comprising:

[0140] The data acquisition module is used to acquire 3D road image data, radar detection data, vehicle dynamic status data, and real-time traffic event data to obtain a multimodal driving monitoring dataset.

[0141] The output module is used to input the multimodal driving monitoring dataset into a preset cross-modal dynamic fusion model and output the driving hazard warning level and core recording area. The cross-modal dynamic fusion model uses a hybrid architecture of Transformer encoder and lightweight convolutional decoder. Its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of driving scene dynamic weight function and spatiotemporal feature correlation function is added before the output decision layer.

[0142] The control module is used to control the AI ​​vehicle-mounted smart terminal to perform scenario-based driving recording operations according to the driving hazard warning level and core recording area, and to encrypt the driving recording data before synchronizing it to the vehicle-cloud collaborative storage system.

[0143] In this embodiment, the specific implementation of each module in the above system embodiment is described in the above method embodiment, and will not be repeated here.

[0144] Reference Figure 3 This invention also provides a computer device, which can be a server, and its internal structure can be as follows: Figure 3As shown, the computer device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores the data corresponding to this embodiment. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0145] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of some structures related to the present invention and do not constitute a limitation on the computer devices on which the present invention is applied.

[0146] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0147] In summary, the driving recording method and control system for the AI-powered vehicle intelligent terminal provided in this embodiment of the invention includes: acquiring three-dimensional road image data, radar detection data, vehicle dynamic state data, and real-time traffic event data to obtain a multimodal driving monitoring dataset; inputting the multimodal driving monitoring dataset into a preset cross-modal dynamic fusion model to output driving hazard warning levels and core recording areas; wherein, the cross-modal dynamic fusion model uses a hybrid architecture of a Transformer encoder and a lightweight convolutional decoder, its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of a driving scene dynamic weight function and a spatiotemporal feature correlation function is added before the output decision layer; according to the driving hazard warning level and core recording area, the AI-powered vehicle intelligent terminal is controlled to perform scenario-based driving recording operations, and the driving recording data is encrypted and synchronized to the vehicle-cloud collaborative storage system. In this invention, the cross-modal dynamic fusion model uses a hybrid architecture of Transformer encoder and lightweight convolutional decoder to capture global spatiotemporal features. Its feature interaction layer embeds a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of a driving scene dynamic weight function and a spatiotemporal feature correlation function is added before the output decision layer to adjust the feature weights according to the real-time changes of the driving scene, overcoming the current defect of not being able to accurately locate the core recording area.

[0148] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the present invention and embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0150] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A driving record method of an AI vehicle-mounted intelligent terminal, characterized by, The method comprises the following steps: Collecting three-dimensional road image data, radar detection data, vehicle power state data and real-time traffic event data to obtain a multi-modal driving monitoring data set; Inputting the multi-modal driving monitoring data set into a preset cross-modal dynamic fusion model to output a driving danger warning level and a core recording area; wherein the cross-modal dynamic fusion model has a hybrid architecture with a Transformer encoder and a lightweight convolution decoder, and a feature interaction layer embedded with a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of a driving scene dynamic weight function and a spatiotemporal feature correlation function added before the output decision layer; inputting the multi-modal driving monitoring data set into the preset cross-modal dynamic fusion model to output the driving danger warning level and the core recording area, comprising: extracting global correlation features of each modal data through a multi-head attention mechanism; using a spatial attention head for three-dimensional road image data, a motion attention head for radar detection data, and an attribute attention head for vehicle power state and real-time traffic event data; inputting each global correlation feature into a feature interaction layer to calculate the similarity between different global correlation features and construct a feature correlation matrix through a multi-source feature alignment module based on attention transfer; inputting the feature correlation matrix into a lightweight convolution decoder to extract local key features through multiple layers of depth separable convolution, and inputting the local key features into the intelligent judgment mechanism before the output decision layer; based on the output result of the intelligent judgment mechanism, outputting a three-level driving danger warning level through a softmax classifier of the output decision layer, and positioning the coordinate range of the core recording area through a bounding box regression algorithm; According to the driving danger warning level and the core recording area, controlling the AI vehicle-mounted intelligent terminal to perform a scenario-based driving recording operation, and synchronizing the driving recording data to a vehicle-cloud collaborative storage system after encryption. 2.The driving record method of the AI vehicle intelligent terminal according to claim 1, characterized in that, Before obtaining the multi-modal driving monitoring data set, comprising: Performing dynamic noise suppression based on the driving scene on the three-dimensional road image data; Performing spatiotemporal bias compensation when multi-radar point cloud splicing is performed on the radar detection data. 3.The driving record method of the AI vehicle intelligent terminal according to claim 1, characterized in that, Inputting the multi-modal driving monitoring data set into the preset cross-modal dynamic fusion model to output the driving danger warning level and the core recording area, further comprising: Performing modal adaptation preprocessing on the three-dimensional road image data, radar detection data, vehicle power state data and real-time traffic event data in the multi-modal driving monitoring data set respectively; Inputting each modal data after preprocessing into the Transformer encoder of the cross-modal dynamic fusion model. 4.The driving record method of the AI vehicle intelligent terminal according to claim 3, characterized in that, The modal adaptation preprocessing comprises: Converting the three-dimensional road image data into a standardized feature map and performing mean-variance normalization; Converting the target distance and speed parameters in the radar detection data into a standardized vector and removing abnormal jump values; Mapping the rotation speed and brake pressure parameters in the vehicle power state data to the 0-1 interval; Converting the real-time traffic event data into an encoded vector. 5.The driving record method of the AI vehicle intelligent terminal according to claim 3, characterized in that, After calculating the similarity between different global correlation features and constructing a feature correlation matrix, further comprising: Starting a feature resolution dynamic adjustment mechanism associated with vehicle speed, comprising: When the real-time vehicle speed is greater than the threshold value, the feature map resolution corresponding to the radar detection data is improved to 512*512; When the real-time vehicle speed is not greater than the threshold value, the feature map resolution corresponding to the radar detection data is maintained as 256*256, and the feature weight of the pedestrian and non-motor vehicle target in the three-dimensional road image data is improved. 6.The driving record method of the AI vehicle intelligent terminal according to claim 3, characterized in that, The intelligent judgment mechanism before the output decision layer comprises: Starting the driving scene dynamic weight function, identifying the environment scene through the three-dimensional road image data, and dynamically adjusting the feature weight of each modal data based on the environment scene; Starting the space-time feature correlation function, calling the historical motion data of the target in the same region within the preset seconds, constructing a target motion trend model, and if it is determined that the target has a sudden change in direction or rapid acceleration / rapid deceleration, marking the region where the target is located as a high-priority core recording region. 7.The driving record method of the AI vehicle intelligent terminal according to claim 3, characterized in that, Input each global correlation feature into the feature interaction layer, calculate the similarity between different global correlation features through the multi-source feature alignment module based on attention transfer, and construct a feature correlation matrix, comprising: mapping each global correlation feature to the same number of feature channels through convolution operation; Through the multi-source feature alignment module based on attention transfer, the spatial global feature corresponding to the three-dimensional road image data is divided into three groups of sub-features according to the actual distance range in the driving scene, respectively corresponding to the near-distance monitoring area, the middle-distance monitoring area and the long-distance monitoring area; the motion global feature corresponding to the radar detection data is divided into four groups of sub-features according to the target type in the driving environment, respectively corresponding to the motor vehicle target, the non-motor vehicle target, the pedestrian target and the fixed obstacle target; the spatial global feature corresponding to the vehicle power state data and the real-time traffic event data is saved in the global form, obtaining a plurality of sub-features to be aligned; Taking the pre-trained vehicle-mounted multi-modal model as a teacher model, extracting its attention distribution map as a template, optimizing the attention parameters of the current model through parameter distillation, so that the initial attention distribution of the plurality of sub-features to be aligned and the deviation rate of the teacher model do not exceed a preset value; Calculate the similarity between the plurality of sub-features to be aligned, and construct a feature correlation matrix, the matrix elements being the similarity weighted values between the sub-features.

8. A control system of an AI vehicle-mounted intelligent terminal, characterized in that, Comprise: The acquisition module is used for acquiring three-dimensional road image data, radar detection data, vehicle power state data and real-time traffic event data, and obtaining a multi-modal driving monitoring data set; The output module is configured to input the multi-modal driving monitoring data set into a preset cross-modal dynamic fusion model, and output a driving danger warning level and a core recording area; wherein the cross-modal dynamic fusion model has a hybrid architecture of a Transformer encoder and a lightweight convolution decoder, and a feature interaction layer thereof is embedded with a multi-source feature alignment module based on attention transfer, and an intelligent judgment mechanism composed of a driving scene dynamic weight function and a space-time feature association function is additionally arranged before an output decision layer; the multi-modal driving monitoring data set is input into the preset cross-modal dynamic fusion model, and the driving danger warning level and the core recording area are output, including: global correlation features of each modal data are extracted through a multi-head attention mechanism; a spatial attention head is used for three-dimensional road image data, a motion attention head is used for radar detection data, and an attribute attention head is used for vehicle dynamic state and real-time traffic event data; each global correlation feature is input into a feature interaction layer, the similarity between different global correlation features is calculated and a feature correlation matrix is constructed through the multi-source feature alignment module based on attention transfer; the feature correlation matrix is input into a lightweight convolution decoder, local key features are extracted through multi-layer depth separable convolution, and the local key features are input into the intelligent judgment mechanism before the output decision layer; based on the output result of the intelligent judgment mechanism, a softmax classifier of the output decision layer is used to output a three-level driving danger warning level, and a bounding box regression algorithm is used to locate the coordinate range of the core recording area; The control module is configured to control the AI vehicle-mounted intelligent terminal to perform a scenario-based driving recording operation according to the driving danger warning level and the core recording area, encrypt the driving recording data, and synchronize the encrypted driving recording data to a vehicle-cloud collaborative storage system.

Images