A method and device for sharing information based on driver cognitive guidance

Abstract

This invention relates to the fields of intelligent transportation and human-computer interaction technology, specifically to a human-computer co-driving method and device based on driver cognitive guidance and information sharing. It includes potential hazard target identification and attention acquisition, objective driving risk field construction and mapping, perception difference identification and supplementary prompts, and a dual-modal collaborative decision-making process. Through the synergistic effect of potential hazard target identification and attention acquisition, driving risk field construction and mapping, and perception difference identification and supplementary prompts, the accuracy of environmental perception, decision robustness, and safety performance in complex traffic scenarios are significantly improved. By dynamically adjusting the decision-making mode, this invention is applicable to various driving scenarios and different driving styles.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and human-machine interaction technology, specifically to a human-machine co-driving method and device based on driver cognitive guidance and information sharing. Background Technology

[0002] In recent years, with the development of deep learning and high-precision sensor technologies, autonomous driving technology has made significant progress. The academic and industrial communities frequently publish cutting-edge findings at top international academic conferences such as the IEEE Intelligent Vehicles Conference and CVPR, and end-to-end autonomous driving solutions such as Tesla's FSD and autonomous taxis have emerged. Currently, autonomous driving technology routes are mainly divided into two categories: one is the end-to-end deep learning method, which directly achieves global optimization from raw sensor input to control command output through a single neural network model; the other is the modular cooperative control method, which modularizes functions such as environmental perception, intent prediction, trajectory planning, and motion control, while combining driver state perception technology to achieve shared control between humans and machines. The former can directly learn driving strategies in specific scenarios such as closed highways, but still faces challenges in generalization ability and safety; the latter focuses more on human-machine collaboration, and in complex road environments, driver participation in the decision-making process is still required.

[0003] However, existing technical solutions still have many shortcomings, as follows:

[0004] 1) Low recognition rate of blind spots: Bird's-eye view perception methods based on monocular or limited number of cameras, such as LSS algorithm, have limited coverage of the environment and are prone to forming perception blind spots in practical application scenarios. The average crossover ratio of risk areas is often less than 0.35, which leads to the problem of missed detection of dangerous targets.

[0005] 2) Difficulty in aligning perception and decision-making: Data collected by different sensors and features fed back by the driver have issues of inconsistent coordinate systems and asynchronous timing. Simply using a channel stitching method will cause feature mismatch and will not provide effective information for accurate alignment in the decision-making process.

[0006] 3) Excessive reliance on high-precision maps and data annotations: Current autonomous driving systems rely heavily on high-precision maps and large-scale driver behavior annotation data, which increases the deployment cost of the system and the difficulty of data acquisition.

[0007] The aforementioned defects severely restrict the robustness and scalability of human-machine co-driving systems in real-world, complex environments. Summary of the Invention

[0008] The purpose of this invention is to provide a human-machine co-driving method and device based on driver cognitive guidance and information sharing. Through potential dangerous target identification and attention acquisition, objective driving risk field construction and mapping, perception difference identification and supplementary prompts, eye-tracking data, image data, point cloud data and self-defined state information are fully integrated to achieve two-way information sharing and collaborative control between the driver and the automatic system.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] A human-machine co-driving method based on driver cognitive guidance and information sharing includes the following steps:

[0011] Step 1: Identification of Potentially Hazardous Targets and Acquisition of Attention:

[0012] Collect driver's eye movement data and vehicle navigation status information;

[0013] Based on eye-tracking data and the vehicle's navigation status information, a Transformer model with an encoder-only architecture is used to identify dangerous scenarios perceived by the driver, and based on the dangerous scenarios, potential traffic participants that are obscured are determined.

[0014] By integrating image data, point cloud data, and eye-tracking data collected by the autonomous driving system to generate an attention heatmap, potential traffic participants can be detected and located to obtain potential target information, enabling the driver's attention to be shared with the autonomous driving system.

[0015] Step 2: Construction and mapping of objective driving risk field:

[0016] A three-dimensional driving risk field model centered on the vehicle is constructed. The three-dimensional risk field model is used to quantify the driving risk level at each location point in three-dimensional space.

[0017] The 3D driving risk field model is projected onto the image plane of the vehicle camera to generate an attention map for the autonomous driving system.

[0018] Step 3: Perceptual Difference Identification and Completion Hints:

[0019] Obtain a driver eye-tracking heatmap generated from driver eye-tracking data, and a system attention map generated by the perception module of the autonomous driving system;

[0020] Using the system attention map as the baseline ontology, the difference between the system attention map and the driver's eye movement heatmap is calculated to obtain a two-dimensional risk perception difference map.

[0021] The two-dimensional risk perception difference map is back-projected into three-dimensional space to obtain the driver's three-dimensional attention blind spot space domain; in the three-dimensional driving risk field model constructed in step two, all risk entities falling into the three-dimensional attention blind spot space domain are queried and extracted, thereby identifying traffic participants that exist in the three-dimensional driving risk field but are not reflected in the driver's eye movement heat map, and judging them as targets that the driver has not paid attention to.

[0022] Step 4, Dual-modal collaborative decision-making: Identify the driver's real-time subjective decision-making patterns, which are used to characterize the driver's personal driving style preferences;

[0023] Based on a three-dimensional driving risk field model, an objective decision-making state index representing the current overall driving risk pressure is calculated.

[0024] Based on the subjective decision-making mode and objective decision-making status indicators, the final collaborative decision-making parameters are determined by fusion, and the control commands of the autonomous driving system are adjusted accordingly.

[0025] Furthermore, in step four, the objective decision-making state index is calculated, specifically by calculating the driving perceived stress level, which is achieved by coupling the event cost field and the driver risk field, and is mathematically represented as:

[0026] ;

[0027] in, To assess driving stress levels. Let be the event cost field value at position i, representing the inherent severity weight of a collision occurring at that position; Let be the driver risk field value at position i, representing the probability density of the vehicle's trajectory intruding into that position.

[0028] Furthermore, in step four, the collaborative decision-making parameters are determined by integrating subjective decision-making modes and objective decision-making state indicators. Specifically, this includes: pre-setting system basic decision-making modes corresponding to different PSI value ranges, wherein the system basic decision-making modes include conservative modes and aggressive modes.

[0029] The identified driver subjective decision-making pattern is weighted and fused with the system basic decision-making pattern indicated by the current PSI value to generate the collaborative decision-making parameters; the weights of the weighted fusion are dynamically adjusted according to the driving scenario type and / or driver trust level.

[0030] Furthermore, the step four of identifying the driver's real-time subjective decision-making mode includes: acquiring the driver's real-time acceleration data under preset operating commands;

[0031] The real-time acceleration data is compared with a preset threshold, and the subjective decision-making mode is classified into a conservative mode, an aggressive mode, or a no-change mode based on the comparison result.

[0032] Furthermore, step four, adjusting the control commands of the autonomous driving system, specifically includes:

[0033] Based on a preset decision parameter mapping table, the collaborative decision parameters are mapped to the corresponding autonomous driving system control parameter group, which includes at least one of following distance, target vehicle speed deviation, and deceleration ratio.

[0034] Furthermore, before adjusting the control commands of the autonomous driving system in step four, time-domain smoothing is performed on the identified subjective decision-making patterns and / or the calculated objective decision-making state indicators.

[0035] Furthermore, the specific steps in step one of detecting and locating potential traffic participants by integrating image data, point cloud data, and eye-tracking data collected by the autonomous driving system to generate an attention heatmap include:

[0036] Extracting two-dimensional multi-view features from image data;

[0037] Two-dimensional attention features are extracted based on eye-tracking heatmaps, which are attention heatmaps acquired by an eye tracker and spatiotemporally aligned with image data;

[0038] The two-dimensional multi-view features are concatenated with the two-dimensional attention features to obtain the concatenated features;

[0039] The stitched features are converted into a three-dimensional feature tensor from a bird's-eye view through camera parameter coordinate transformation;

[0040] The point cloud data is rasterized and pseudo-image converted, and structured point cloud features are extracted through a two-dimensional convolutional network.

[0041] An adaptive channel attention mechanism is used to fuse the three-dimensional feature tensor with the structured point cloud features to obtain the first fused feature;

[0042] Based on the first fusion feature, cross-modal interaction modeling is performed through the Transformer decoder to achieve real-time detection and positioning of traffic participants and output the potential target information.

[0043] Furthermore, the specific steps in step one of identifying the dangerous scenarios perceived by the driver using the Transformer model with only encoder architecture include: preprocessing eye-tracking data and navigation information;

[0044] The preprocessed time series data is input into the Transformer model of the encoder-only architecture to extract high-dimensional time series features;

[0045] After mapping the high-dimensional temporal features with random convolution kernels, the probability of the current moment belonging to a cognitively dangerous scene is output through a regression operation.

[0046] Furthermore, the specific steps in step two of projecting the three-dimensional driving risk field onto the image plane of the vehicle-mounted camera include:

[0047] Obtain the intrinsic and extrinsic parameter matrices of the vehicle-mounted camera;

[0048] The 3D driving risk field is transformed from the world coordinate system to the camera coordinate system using the extrinsic parameter matrix;

[0049] Using the intrinsic parameter matrix and a specific planar projection formula, the risk field in the camera coordinate system is projected onto the image plane. The projection formula is as follows:

[0050] ;

[0051] ;

[0052] Where f is the focal length, h is a preset parameter; u and v are the horizontal and vertical pixel coordinates in the pixel coordinate system, respectively, and X and Y are the horizontal and depth coordinates of the three-dimensional point in the camera coordinate system, respectively.

[0053] A driver-co-driving information-sharing device based on driver cognitive guidance includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned information-sharing human-machine co-driving method.

[0054] By employing the above-described technology, the present invention has the following advantages:

[0055] 1. This invention improves the perception of potential dangerous areas from both spatial and temporal dimensions by combining the objective driving risk field construction and mapping steps with the driver's real-time subjective decision-making mode, and the coverage of dangerous areas is increased by 22.40%.

[0056] 2. In the process of identifying potential dangerous targets and acquiring attention, this invention adopts a Transformer-based multimodal fusion method to perform spatiotemporal alignment and weighted fusion of image data collected by camera, point cloud data collected by lidar, vehicle navigation status information acquired by radar, and eye-tracking data collected by eye tracker; and introduces a cross-attention mechanism in the fusion process, which improves the accuracy of obstacle detection and recognition in the surrounding environment by 15.00% while maintaining high timeliness during fusion.

[0057] 3. By aligning driver intent features with environmental perception features through an attention gating mechanism, the channel-level feature mismatch problem is alleviated. The key feature is the addition of channel attention to the Transformer encoding layer, adaptively adjusting the weights of important features, reducing alignment errors, and keeping the output error of subsequent decision networks within ±0.40m.

[0058] 4. In the perception difference recognition and supplementary prompting steps, this invention designs a two-way feedback discrimination mechanism. Specifically, the system employs an interactive feedback method, projecting a risk heatmap to the driver at the decision-making level while simultaneously feeding back the driver's attention distribution to the controller. This mechanism improves the human-machine interaction response speed, enhances driving stability by 30.00%, and strengthens the collaborative control effect between the autonomous driving system and the driver.

[0059] In summary, compared with existing technologies, this invention significantly improves environmental perception accuracy and decision robustness through the synergistic effects of potential hazard target identification and attention acquisition, driving risk field construction and mapping, and perception difference identification and supplementary prompting steps. This results in an information fusion accuracy rate exceeding 95.00%, significantly enhancing safety performance in complex traffic scenarios. Economically, it reduces reliance on high-precision maps and extensive driver data annotation, lowering system costs and maintenance expenses. In terms of application adaptability, by dynamically adjusting the decision-making mode, this invention is applicable to various driving scenarios (such as urban roads and highways) and different driving styles, making it more suitable for widespread application. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of the information sharing human-machine co-driving method based on driver cognition guidance in an embodiment.

[0062] Figure 2 This is a diagram of the hazardous scene recognition network structure used in the embodiment;

[0063] Figure 3 This is a diagram illustrating the overall structure of the hazardous scene cognition sharing network used in this embodiment.

[0064] Figure 4 This is a schematic diagram illustrating the projection principle of a pinhole camera in an embodiment.

[0065] Figure 5 Here is a flowchart of the process for perceptual difference recognition and supplementary prompts in an embodiment;

[0066] Figure 6 The diagram shows the structural block of the human-machine co-driving system driving simulation platform used in this invention for verification purposes. Detailed Implementation

[0067] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] This invention provides a human-machine co-driving method based on driver cognitive guidance and information sharing, implemented according to the following hardware configuration:

[0069] The system includes three cameras: a front-facing camera, a left-facing camera, and a right-facing camera. Each camera has a resolution of at least 800×320 and is used for panoramic visual perception. A driver eye tracker with a sampling frequency of ≥90.00Hz is used to collect information such as the driver's gaze and pupil changes. A LiDAR (Light Detection and Ranging) sensor is used to generate high-precision 3D point clouds in real time. An Inertial Measurement Unit (IMU) is used to acquire dynamic information such as vehicle acceleration and angular velocity. A GPS integrated navigation unit is also included, used for global positioning and attitude calibration. All of these sensors are connected to a central computing unit to ensure synchronized data acquisition and calibration. Figure 1 As shown, the information-sharing human-machine co-driving method based on driver cognitive guidance includes the following steps:

[0070] Step 1: Identification and Attention Acquisition of Potentially Hazardous Targets. The specific implementation process of this step is as follows:

[0071] Collect driver's eye movement data and the vehicle's navigation status information. The eye-tracking data includes pupil diameter. and gaze point The vehicle's navigation status information includes vehicle status and driving status information such as going straight, turning left, and turning right. In this embodiment, the navigation status information of the purple vehicle is provided by the simulator.

[0072] Data preprocessing: The eye-tracking data is processed sequentially for outlier data processing, gaze point filtering, and removal of invalid or interfering features; outliers include null values ​​and illegal values.

[0073] Based on preprocessed eye-tracking data and the vehicle's navigation status information, a hazardous scene recognition network identifies hazardous scenes perceived by the driver and determines the presence of occluded potential traffic participants based on these hazardous scenes. The hazardous scene recognition network in this implementation is a Transformer model with an encoder-only architecture, and its structure is as follows: Figure 2 As shown, the model includes an input layer, an attention layer based on transposed lexical units, a random convolutional kernel layer, a classifier, and a driver attention result output layer. Each type of preprocessed temporal data is treated as a lexical unit. The multimodal data composed of these transposed lexical units is input into a Transformer model with an encoder-only architecture. After extracting the temporal data, high-dimensional temporal features far exceeding 4 dimensions are obtained. For high-dimensional time series features After performing random convolution kernel mapping, the probability of the current time belonging to a cognitively dangerous scenario is output through regression operation.

[0074] An attention heatmap generated by integrating image data, point cloud data, and eye-tracking data collected by the autonomous driving system is used to detect and locate potential traffic participants, obtaining potential target information. These potential targets refer to occluded traffic participants whose visual information is displayed but cannot be directly identified by the current autonomous driving system. In this embodiment, this process is implemented using a hazardous scene cognitive sharing network. Figure 3 As shown, the hazardous scene cognition sharing network includes an image feature extraction network, an attention feature extraction network, a point cloud branch network, a feature fusion module, a BEV encoder, and an output layer. The input of the feature fusion module is connected to the image feature extraction network, the attention feature extraction network, and the point cloud branch network, respectively, and the output is connected to the output layer via the BEV encoder. Its processing flow is as follows:

[0075] Images from the driver's perspective, i.e., images captured by the surround-view camera. As input, an image feature extraction network is used to extract two-dimensional multi-view features from the image data. ;

[0076] Attention heatmap As input, an attention feature extraction network is used to extract two-dimensional attention features based on eye-tracking heatmaps. The eye-tracking heatmap is an attention heatmap acquired by an eye tracker in an autonomous driving system and corresponds to the surround view image data.

[0077] Two-dimensional multi-view features With two-dimensional attention features The pieces are then stitched together to obtain the stitched features.

[0078] The camera's field of view is discretized into a 3D mesh, and the mesh coordinates are transformed into 3D coordinates using camera parameters. Based on the 3D coordinates and the stitched 2D multi-view feature map, a 3D feature tensor with spatial awareness is generated. This converts two-dimensional features into a three-dimensional feature tensor under the bird's-eye view of BEV;

[0079] The point cloud data is rasterized and pseudo-image transformed, and structured point cloud features are extracted using a two-dimensional convolutional network; the rasterized LiDAR point cloud dataset is then processed. It is transformed into a pseudo-image form, and structured point cloud features are extracted using a two-dimensional convolutional network. ;

[0080] For three-dimensional feature tensors With structured point cloud features After splicing, an adaptive channel attention mechanism is used for fusion to obtain the first fused feature;

[0081] Based on the features obtained from the first fusion, cross-modal interaction modeling is carried out through the Transformer decoder to achieve real-time detection and localization of traffic participants and output potential target information.

[0082] Step 2: Construction and mapping of objective driving risk field:

[0083] 1) Construct a three-dimensional driving risk field model centered on the vehicle. This risk field model is used to quantify the driving risk level at each location point in three-dimensional space.

[0084] In this embodiment, the driving risk field model consists of the vehicle kinetic energy field formed by dynamic traffic participants in the driving scenario and the environmental potential field influenced by environmental factors. Its data expression is as follows:

[0085] ;

[0086] in, For the quantitative characterization of driving risk field, For the quantitative characterization of the vehicle's kinetic energy field, This is a quantitative characterization of the environmental potential field.

[0087] Real-time changes in road environmental parameters are strongly correlated with traffic accident risk. Visibility, as a crucial indicator of environmental perception, directly impacts driving safety boundaries. When weather conditions cause visibility to fall below a safe threshold, the driver's situational awareness of surrounding traffic units is significantly suppressed, leading to a sharp increase in collision risk and seriously threatening driving safety. Standard road visibility is 200 meters, at which point the environmental potential field is negligible. If road visibility drops below 50 meters, the environmental potential field reaches its peak. Adapting to changes in the driving environment and paying attention to visibility are crucial for ensuring safety. This requires enhancing sensitivity to weather changes during actual driving to reduce collision risk and improve road safety. Therefore, based on the above theory, the environmental potential field... The expression is:

[0088] ;

[0089] in, For weather visibility; The standard road visibility is 200; k is a constant and k < 0.

[0090] The driving risk field is used to represent the distribution of attention of the autonomous driving system to nearby vehicles. The driving risk field is mainly affected by the following four factors: (1) The influence of speed: At the same distance, as the vehicle speed increases, the intensity of its risk field gradually increases, indicating that high-speed vehicles have a more significant influence in the risk field. (2) The influence of acceleration: When the vehicle is traveling at a constant speed, the distribution of its risk field in front and behind is basically symmetrical; when the acceleration is negative, the intensity of the risk field behind is less than that in front, and when the acceleration is positive, the opposite is true. This phenomenon shows that acceleration not only affects the dynamic characteristics of the vehicle itself, but also forms a complex interaction relationship with surrounding vehicles. (3) The influence of vehicle size: As the size of the vehicle changes, the intensity and distribution range of the risk field should also change, and the influence in the longitudinal direction is more obvious, which shows that larger vehicles have a more obvious effect on surrounding vehicles in the longitudinal direction. (4) The influence of heading angle: The risk field does not strictly extend along the longitudinal direction of the road, but deviates when the vehicle turns. This dynamic adjustment characteristic with the change of heading angle makes the risk field more in line with the actual driving scenario, which helps to more accurately simulate the interaction between vehicles. Therefore, the driving risk field in this embodiment is represented by the following function:

[0091] ;

[0092] ;

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] Where R is the risk field coefficient, which is determined based on the fitted vehicle speed-accident number relationship curve; To represent the vehicle's position coordinates; The coordinates of his vehicle's location; and This is a factor influencing acceleration and velocity in the potential energy field; The angle between the direction of the line connecting the center of the vehicle and the target vehicle and the horizontal axis of the road coordinate system; Represents the risk field elliptic domain influence factor; The risk field elliptic domain distribution factor is represented by L; vehicle length is represented by W; vehicle width is represented by W. Represents the risk level factor; φ is the vehicle's heading angle; Vertical; For lateral velocity; For longitudinal acceleration, denoted as lateral acceleration; v is the average speed of vehicles on the road.

[0099] Since the dimensions of different risk fields differ, to ensure consistency in the final driving risk field calculation, this paper normalizes the field strength of each risk field and combines them according to weights to generate the final risk field. The weight coefficients of each risk field are determined experimentally: first, initial weights are set based on expert experience, then feedback optimization is performed using experimental data, and finally, the weight coefficients are adjusted. The calculation results under this weight ratio can more accurately reflect the actual risk of the vehicle, i.e., the attention distribution of the autonomous driving system. Therefore, the final evaluation formula for the driving risk field is:

[0100] ;

[0101] A three-dimensional driving risk field model was calculated based on the driving risk field model.

[0102] 2) Generate a driving risk attention map:

[0103] To achieve this goal, a common camera projection model is required. This embodiment uses a pinhole camera as the imaging device, and its imaging principle is as follows: Figure 4 As shown in Table 1, the process of projecting points in three-dimensional space onto the camera's image space requires the use of the camera's intrinsic and extrinsic parameters. The camera's intrinsic parameters include focal length and optical center coordinates. The specific imaging attributes of the camera used in this experiment are shown in Table 1.

[0104] Table 1 Simulator Camera Attribute Table

[0105]

[0106] Based on the imaging attributes in Table 1, the corresponding camera intrinsic parameter matrix is ​​calculated using the following formula:

[0107] ;

[0108] ;

[0109] ;

[0110] in, Focal length and K represents the coordinates of the principal point, and K is the intrinsic parameter matrix.

[0111] Projection calculations are performed based on the intrinsic parameter matrix: After obtaining the camera's intrinsic parameter matrix, the camera's rotation matrix R and translation vector T are read through the simulator to construct the camera's extrinsic parameter matrix. Using this extrinsic parameter matrix, the 3D points in the driving risk field are transformed from the world coordinate system to the camera coordinate system.

[0112] It should be noted that the driving risk field in this embodiment is a two-dimensional planar structure within a three-dimensional space. The pixel values ​​at each point on this plane represent the driving risk level at that location; that is, maxima in the pixel matrix represent high driving risk areas, and minima represent low driving risk areas. Its imaging logic differs somewhat from the conventional three-dimensional point imaging principle. Based on this characteristic, the final formula for calculating the projection from the camera coordinate system to the pixel coordinate system is as follows: ;

[0113] ;

[0114] Where f is the focal length, h is a preset parameter; u and v are the horizontal and vertical pixel coordinates in the pixel coordinate system, respectively, and X and Y are the horizontal and depth coordinates of the three-dimensional point in the camera coordinate system, respectively.

[0115] Step 3: Perception Difference Recognition and Supplementary Prompt Algorithm: This algorithm aims to achieve information coordination between the autonomous driving system and the driver at the perception level, thereby accurately identifying traffic participants that the driver has not noticed and issuing prompts. The specific process is as follows:

[0116] Obtain the driver's eye movement heatmap generated based on the driver's eye movement data, and the system attention map generated by the perception module of the autonomous driving system in step three;

[0117] Using the system attention map as the baseline ontology, the difference between the system attention map and the driver's eye movement heatmap is calculated to obtain a risk perception difference map.

[0118] The risk perception difference map is back-projected into three-dimensional space and compared with the three-dimensional driving risk field. Based on the comparison results, traffic participants that exist in the three-dimensional driving risk field but are not reflected in the driver's eye-tracking heat map are identified and judged as targets that the driver does not pay attention to.

[0119] For targets that the driver is not paying attention to, the system will display visual indicators and / or issue warnings on the human-machine interface of the autonomous driving system.

[0120] For the complete algorithm flow of the perceptual difference recognition and supplementary prompting algorithm in this embodiment, please refer to [link / reference]. Figure 5 For details on the specific implementation and parameters of the relevant algorithms, please refer to Table 3.

[0121] Table 3. Execution table of the perceptual difference recognition and supplementary prompt algorithm code.

[0122]

[0123] Step 4: Constructing a Dual-Modal Collaborative Decision-Making Mechanism: This embodiment uses the CARLA driving simulator as the testing platform. Based on an adjustable autonomous driving algorithm, it achieves precise matching between the driver's driving style and the autonomous driving system's decisions, thereby achieving human-vehicle decision-level information sharing. Specifically, this includes:

[0124] The system identifies the driver's real-time subjective decision-making pattern, which characterizes the driver's personal driving style preference. This embodiment utilizes the CARLA Traffic Manager module to support parameter adjustments to adapt to different driving styles. Based on actual scenario requirements, three typical driving styles—conservative, standard, and aggressive—are defined. Detailed parameter configurations for each style are shown in Table 2.

[0125] Table 2: Driving Style Parameters Table for Driving Simulator

[0126]

[0127] The real-time subjective decision-making patterns of the driver shown include:

[0128] Acquire real-time acceleration data from the driver under preset operating commands;

[0129] The real-time acceleration scalar data is compared with the industry safety and comfort preset threshold of 0.3g. Based on the comparison results, the subjective decision-making mode is classified into conservative mode (negative acceleration), aggressive mode (positive acceleration), or no change mode (scalar is below the preset threshold).

[0130] Based on a three-dimensional driving risk field model, an objective decision-making state index representing the current overall driving risk pressure is calculated. In this embodiment, the objective decision-making state index is the perceived driving pressure level, which is realized by coupling the event cost field and the driver risk field, and is mathematically represented as follows:

[0131] ;

[0132] in, To assess driving stress levels. Let be the event cost field value at position i, representing the inherent severity weight of a collision occurring at that position; Let be the driver risk field value at position i, representing the probability density of the vehicle's trajectory intruding into that position.

[0133] Based on subjective decision-making models and objective decision-making state indicators, the final collaborative decision-making parameters are determined through fusion, and the control commands of the autonomous driving system are adjusted accordingly. The specific implementation process includes:

[0134] The system has a pre-defined basic decision-making mode corresponding to different PSI value ranges, which includes a conservative mode and an aggressive mode.

[0135] The identified driver subjective decision-making pattern is weighted and fused with the system basic decision-making pattern indicated by the current PSI value to generate the collaborative decision-making parameters; the weights of the weighted fusion are dynamically adjusted according to the driving scenario type and / or driver trust level.

[0136] The identified subjective decision-making patterns and / or calculated objective decision-making state indicators are subjected to time-domain smoothing.

[0137] To ensure a smooth transition in the decision-making style of autonomous driving systems, thereby improving driving comfort and safety.

[0138] Based on the preset decision parameter mapping table shown in Table 4, the collaborative decision parameters are mapped to the autonomous driving system control parameter group corresponding to Table 2. The control parameter group includes at least one of following distance, target vehicle speed deviation, and deceleration ratio.

[0139] Table 4: Decision Parameter Mapping Table

[0140]

[0141] The dual-modal collaborative decision-making mechanism implemented in this paper does not require the reconstruction of the core algorithms of autonomous driving. It can achieve decision-level information sharing simply by adapting parameters, and has strong practicality and scalability.

[0142] Based on a six-degree-of-freedom driving simulation platform, the human-machine co-driving method based on driver cognitive guidance in this embodiment was experimentally verified:

[0143] like Figure 6 As shown, in the comparative experiment, the method of this invention was compared with the traditional human-machine co-driving method based on preset switching rules. The results show that in complex urban driving simulation scenarios, the method of this invention can reduce the vehicle path tracking error to within ±0.40m; the driver comfort index (comprehensive acceleration change rate) is improved by approximately 41.10% compared with the traditional method; and the driver's subjective psychological load (assessed by the NASA-TLX scale) is reduced by approximately 15.00%. Meanwhile, this method shows consistent advantages in both simulation testing (quantitative evaluation of performance indicators) and driver subjective evaluation, indicating that the proposed strategy can effectively improve driving safety and ride comfort.

Claims

1. A human-machine co-driving method based on driver cognitive guidance and information sharing, characterized in that, Includes the following steps: Step 1: Identification of Potentially Hazardous Targets and Acquisition of Attention: Collect driver's eye movement data and vehicle navigation status information; Based on eye-tracking data and the vehicle's navigation status information, a Transformer model with an encoder-only architecture is used to identify dangerous scenarios perceived by the driver, and based on the dangerous scenarios, potential traffic participants that are obscured are determined. By integrating image data, point cloud data, and eye-tracking data collected by the autonomous driving system to generate an attention heatmap, potential traffic participants can be detected and located to obtain potential target information, enabling the driver's attention to be shared with the autonomous driving system. Step 2: Construction and mapping of objective driving risk field: A three-dimensional driving risk field model centered on the vehicle is constructed. The three-dimensional risk field model is used to quantify the driving risk level at each location point in three-dimensional space. The 3D driving risk field model is projected onto the image plane of the vehicle camera to generate an attention map for the autonomous driving system. Step 3: Perceptual Difference Identification and Completion Hints: Obtain a driver eye-tracking heatmap generated from driver eye-tracking data, and a system attention map generated by the perception module of the autonomous driving system; Using the system attention map as the baseline ontology, the difference between the system attention map and the driver's eye movement heatmap is calculated to obtain a two-dimensional risk perception difference map; By back-projecting the two-dimensional risk perception difference map into three-dimensional space, the driver's three-dimensional blind spot spatial domain is obtained; In the three-dimensional driving risk field model constructed in step two, all risk entities falling into the three-dimensional attention blind spot spatial domain are queried and extracted, thereby identifying traffic participants that exist in the three-dimensional driving risk field but are not reflected in the driver's eye movement heatmap, and judging them as targets that the driver has not paid attention to. For targets that the driver is not paying attention to, visual identification and / or warning prompts are issued on the human-machine interface of the autonomous driving system to achieve the sharing of the autonomous driving system's attention with the driver; Step 4: Dual-modal collaborative decision-making: Identify the driver's real-time subjective decision-making pattern, which is used to characterize the driver's personal driving style preferences; Based on a three-dimensional driving risk field model, an objective decision-making state index representing the current overall driving risk pressure is calculated. Based on the subjective decision-making mode and objective decision-making status indicators, the final collaborative decision-making parameters are determined by fusion, and the control commands of the autonomous driving system are adjusted accordingly.

2. The method according to claim 1, characterized in that, Step four involves calculating the objective decision-making state index, specifically: calculating the perceived driving stress level, which is achieved by coupling the event cost field and the driver risk field, mathematically represented as: ； in, To assess driving stress levels. Let be the event cost field value at position i, representing the inherent severity weight of a collision occurring at that position; Let be the driver risk field value at position i, representing the probability density of the vehicle's trajectory intruding into that position.

3. The method according to claim 2, characterized in that, Step four, which involves determining collaborative decision-making parameters based on the fusion of subjective decision-making models and objective decision-making state indicators, specifically includes: The system has a pre-defined basic decision-making mode corresponding to different PSI value ranges, which includes a conservative mode and an aggressive mode. The identified driver subjective decision-making pattern is weighted and fused with the system basic decision-making pattern indicated by the current PSI value to generate the collaborative decision-making parameters; the weights of the weighted fusion are dynamically adjusted according to the driving scenario type and / or driver trust level.

4. The method according to claim 1, characterized in that, The step four, which identifies the driver's real-time subjective decision-making pattern, includes: Acquire real-time acceleration data from the driver under preset operating commands; The real-time acceleration data is compared with a preset threshold, and the subjective decision-making mode is classified into a conservative mode, an aggressive mode, or a no-change mode based on the comparison result.

5. The method according to claim 1, characterized in that, The adjustment of the control commands of the autonomous driving system in step four specifically includes: mapping the collaborative decision parameters to the corresponding autonomous driving system control parameter group according to the preset decision parameter mapping table, wherein the control parameter group includes at least one of following distance, target vehicle speed deviation, and deceleration ratio.

6. The method according to claim 1, characterized in that, Before adjusting the control commands of the autonomous driving system in step four, time-domain smoothing is performed on the identified subjective decision-making patterns and / or the calculated objective decision-making state indicators.

7. The method according to claim 1, characterized in that, The specific steps in step one of detecting and locating potential traffic participants by integrating image data, point cloud data, and eye-tracking data collected by the autonomous driving system to generate an attention heatmap include: Extracting two-dimensional multi-view features from image data; Two-dimensional attention features are extracted based on eye-tracking heatmaps, which are attention heatmaps acquired by an eye tracker and spatiotemporally aligned with image data; The two-dimensional multi-view features are concatenated with the two-dimensional attention features to obtain the concatenated features; The stitched features are converted into a three-dimensional feature tensor from a bird's-eye view through camera parameter coordinate transformation; The point cloud data is rasterized and pseudo-image converted, and structured point cloud features are extracted through a two-dimensional convolutional network. An adaptive channel attention mechanism is used to fuse the three-dimensional feature tensor with the structured point cloud features to obtain the first fused feature; Based on the first fusion feature, cross-modal interaction modeling is performed through the Transformer decoder to achieve real-time detection and positioning of traffic participants and output the potential target information.

8. The method according to claim 1, characterized in that, The specific steps in step one of identifying the dangerous scenarios perceived by the driver using the Transformer model with only an encoder architecture include: Preprocessing of eye-tracking data and navigation information; The preprocessed time series data is input into the Transformer model of the encoder-only architecture to extract high-dimensional time series features; After mapping the high-dimensional temporal features with random convolution kernels, the probability of the current moment belonging to a cognitively dangerous scene is output through a regression operation.

9. The method according to claim 1, characterized in that, The specific steps in step two of projecting the three-dimensional driving risk field onto the image plane of the vehicle-mounted camera include: Obtain the intrinsic and extrinsic parameter matrices of the vehicle-mounted camera; The 3D driving risk field is transformed from the world coordinate system to the camera coordinate system using the extrinsic parameter matrix; Using the intrinsic parameter matrix and a specific planar projection formula, the risk field in the camera coordinate system is projected onto the image plane. The projection formula is as follows: ； ； Where f is the focal length, h is a preset parameter; u and v are the horizontal and vertical pixel coordinates in the pixel coordinate system, respectively, and X and Y are the horizontal and depth coordinates of the three-dimensional point in the camera coordinate system, respectively.

10. A driver-cognitive guided information-sharing human-machine co-driving device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the information sharing human-machine co-driving method as described in any one of claims 1 to 9.

Images