Decision generation method and device for man-machine co-driving

Abstract

The invention provides a man-machine co-driving decision generation method and device. The method comprises the following steps: acquiring multi-source data in a driving process of a vehicle; based on the multi-source data, a fuzzy reasoning algorithm and a multi-modal model are used for identifying the driving group type of the driver, and obtained identification results are fused through a linear weighted fusion algorithm to obtain the driving group type; obtaining a driving condition based on the multi-source data, determining a corresponding look-up table based on the driving condition, and determining a man-machine cooperation factor estimation value by using the look-up table and the driving group type of the driver; determining a man-machine cooperation factor of the decision through a cooperation equation; and the decision of the intelligent driving system and the decision of the driver are fused to obtain a comprehensive driving decision. Dynamic cooperation of the intelligent driving system and human drivers is achieved, the scientificity and adaptability of driving decisions are improved, and the safety and reliability of vehicle driving are enhanced.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and more particularly to a method and device for generating decisions for human-machine co-driving. Background Art

[0002] Despite the rapid development of autonomous driving technology, its commercialization is hampered by multiple factors. In complex traffic scenarios, faced with emergencies, extreme weather, and irregular road conditions, the decision-making algorithms of existing autonomous driving systems struggle to accurately and promptly make optimal driving decisions, resulting in insufficient system reliability. Furthermore, the new infrastructure for the Internet of Vehicles (IoV) is still incomplete, and communications between vehicles, infrastructure, and other vehicles suffer from delays and instability, making it impossible to provide a stable information exchange environment for autonomous driving. Furthermore, the lack of legislation defining the rights and responsibilities of autonomous driving, and the ambiguity surrounding accident liability determination, further exacerbate the uncertainty of industry development and create numerous obstacles to the large-scale application of fully autonomous driving.

[0003] To address these issues, an innovative human-machine collaborative control strategy is urgently needed. By anthropomorphizing the human-machine collaborative lateral control system, we can effectively reduce potential conflicts between humans and machines while ensuring trajectory tracking accuracy and significantly improving the shared driving experience, providing effective support for the development and application of human-machine collaborative driving technology. Summary of the Invention

[0004] The embodiments of the present disclosure provide a method and device for generating decisions for human-machine co-driving, to solve the related problems existing in existing technical solutions.

[0005] Based on the above problems, in a first aspect, a decision-making method for human-machine co-driving is provided, comprising:

[0006] Acquire multi-source data during vehicle driving;

[0007] Based on the multi-source data, the driving group type of the driver is identified using a fuzzy inference algorithm and a multimodal model respectively, and the obtained identification results are fused using a linear weighted fusion algorithm to obtain the driving group type;

[0008] Determining a driving condition based on the multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driver's driving group type;

[0009] Combining the estimated value of the human-machine collaboration factor and the human-machine collaboration factor determined in the previous decision, the human-machine collaboration factor of this decision is determined by a collaboration formula;

[0010] Based on the human-machine collaboration factor determined in this decision, the decisions of the intelligent driving system and the driver are integrated to obtain a comprehensive driving decision.

[0011] In conjunction with the first aspect, in a possible implementation, the driving group types include: a general driving group, an expert driving group, and a novice driving group;

[0012] The method of identifying the driving group type of the driver by using a fuzzy inference algorithm based on the multi-source data includes:

[0013] Setting a width and a step size of a sliding window, and using the sliding window to intercept the multi-source data within a first preset time period multiple times; wherein the starting point of the sliding window is the start time of the first preset time period, and the end point is the end time of the first preset time period;

[0014] Determining corresponding values ​​of key features for the multi-source data intercepted each time by the sliding window;

[0015] A fuzzy inference engine constructed using a fuzzy logic system is used to fuzzify the corresponding values ​​of the key features, and fuzzy inference is performed based on a priori fuzzy rules to output a corresponding driving group label and a timestamp; wherein the driving group label corresponds to the driving group type of the driver;

[0016] The number of clusters is set, and cluster analysis is performed on the driving group types within the second preset time period based on a clustering algorithm to obtain a cluster with the largest total number of the same driving group labels within the cluster, and the driving group type corresponding to the driving group label with the largest total number in the cluster is used as the identification result.

[0017] In conjunction with the first aspect, in one possible implementation, identifying the driving group type of the driver based on the multi-source data using a multimodal model includes:

[0018] Normalizing the data corresponding to the key features in the multi-source data, and enhancing the data corresponding to the key features using a preset data enhancement algorithm;

[0019] Determining corresponding advanced features based on the corresponding data of the enhanced key features; wherein the advanced features are extracted from the corresponding data of the key features using a preset deep learning model;

[0020] The corresponding data of the enhanced key features and the advanced features are input into a preset multimodal model to obtain the corresponding driving group type.

[0021] In conjunction with the first aspect, in one possible implementation, the preset multimodal model includes:

[0022] Differentiate the types of input data and match the corresponding preset neural network models to obtain corresponding dynamic features and / or spatial features for different types of data;

[0023] Performing feature fusion on the dynamic features and / or spatial features to obtain comprehensive features;

[0024] The comprehensive features are used to calculate the probability distribution of different group types through the classification layer, and the driving group type with the highest probability is output.

[0025] In combination with the first aspect, in a possible implementation manner, the driving condition includes: a large curvature driving condition and a small curvature driving condition;

[0026] The obtaining of a driving condition based on the multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driving group type of the driver include:

[0027] determining, based on the road curvature in the multi-source data, whether the driving condition is a large curvature driving condition or a small curvature driving condition;

[0028] When the driving condition is a high curvature driving condition, obtaining an estimated value of a human-machine cooperation factor using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position;

[0029] When the driving condition is a small curvature driving condition, an estimated value of the human-machine cooperation factor is obtained using a second lookup table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position.

[0030] In conjunction with the first aspect, in one possible implementation, the driving conditions include: a high-speed, high-curvature condition, a low-speed, low-curvature condition, a transition condition, and an emergency condition;

[0031] The obtaining of a driving condition based on the multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driving group type of the driver include:

[0032] Determining, based on the vehicle longitudinal speed and road curvature in the multi-source data, whether the driving condition is a high-speed, large-curvature condition, a low-speed, small-curvature condition, a transition condition, or an emergency condition;

[0033] When the driving condition is a high-speed and high-curvature condition, a first lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the preview point position;

[0034] When the driving condition is a low-speed, small-curvature condition, a second lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass.

[0035] When the driving condition is a transition condition, a first human-machine cooperation factor estimate is obtained using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position; a second human-machine cooperation factor estimate is obtained using a second lookup table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position; and the human-machine cooperation factor estimate is determined based on the first human-machine cooperation factor estimate, the second human-machine cooperation factor estimate, and preset parameters.

[0036] When the driving condition is an emergency condition, a multi-dimensional warning is initiated.

[0037] In conjunction with the first aspect, in one possible implementation, the collaborative formula is as follows:

[0038] β * =ηβ cur +(1-η)β pre

[0039] Among them, η represents the preset parameters related to the driving group type, β pre Characterizes the human-machine collaboration factor determined by the last decision, β * The human-machine collaboration factor that characterizes this decision, β cur An estimated value of the human-machine collaboration factor that characterizes this decision; and / or

[0040] The human-machine collaboration factor determined based on the current decision integrates the decisions of the intelligent driving system and the driver to obtain a comprehensive driving decision, including:

[0041] Obtaining respective decisions of the intelligent driving system and the driver;

[0042] Combining the human-machine collaboration factor determined in the current decision and the decisions of the intelligent driving system and the driver, a comprehensive driving decision is obtained through a fusion formula;

[0043] The respective decisions of the intelligent driving system and the driver, as well as the comprehensive driving decision, include: torque;

[0044] The fusion formula is as follows:

[0045] T total =T d +β * T a

[0046] Among them, T total Characterizing comprehensive driving decisions, T a Characterize the decision of the intelligent driving system, T d Characterizes the driver's decision, β * Characterize the human-machine collaboration factor of this decision.

[0047] In combination with the first aspect, in a possible implementation, the key features include at least one of the following: vehicle side deviation, vehicle lateral acceleration, vehicle longitudinal acceleration, positive vehicle longitudinal acceleration, negative vehicle longitudinal acceleration, vehicle lateral shock wave, vehicle longitudinal shock wave, steering wheel angle, steering wheel angular velocity, following vehicle distance, driver's reaction time, frequency of driver observing left and right rearview mirrors, average grip strength of driver operating the steering wheel, driver's facial expression tension factor, driver's annual number of traffic violations, total number of sampling points, peak value of side deviation, mean value of side deviation, standard deviation of side deviation, root mean square error of side deviation, peak value of lateral acceleration, mean value of lateral acceleration, standard deviation of lateral acceleration, peak value of longitudinal acceleration, peak value of longitudinal deceleration, frequency of sudden acceleration, frequency of sudden deceleration, peak value of lateral impact degree, peak value of longitudinal impact degree, peak value of steering angle, mean value of steering angle, standard deviation of steering angle, peak value of steering angular velocity, mean value of steering angular velocity, and standard deviation of steering angular velocity; wherein, the total number of sampling points is consistent with the width of the sliding window.

[0048] In combination with the first aspect, in a possible implementation, the method further includes: after acquiring multi-source data during the driving process of the vehicle, performing time calibration, time synchronization, and filtering on the multi-source data.

[0049] In a second aspect, a decision-making device for human-machine co-driving is provided, comprising:

[0050] A data acquisition module is used to acquire multi-source data during the vehicle's driving process;

[0051] A driving group module is used to identify the driving group type of the driver using a fuzzy inference algorithm and a multimodal model based on the multi-source data, and to fuse the obtained identification results using a linear weighted fusion algorithm to obtain the driving group type;

[0052] a query estimation module, configured to obtain a driving condition based on the multi-source data, determine a corresponding query table based on the driving condition, and determine an estimated value of a human-machine collaboration factor using the query table and the driver's driving group type;

[0053] A synergy factor module is used to combine the estimated value of the human-machine synergy factor and the human-machine synergy factor determined in the previous decision to determine the human-machine synergy factor for this decision through a synergy formula;

[0054] The comprehensive decision-making module is used to integrate the decisions of the intelligent driving system and the driver based on the human-machine collaboration factor determined by the current decision to obtain a comprehensive driving decision.

[0055] In a third aspect, a computer device is provided, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of a decision-making method for human-machine co-driving as described in the first aspect, or in combination with any possible implementation manner of the first aspect are performed.

[0056] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a decision-making method for human-machine co-driving as described in the first aspect, or in combination with any possible implementation of the first aspect are executed.

[0057] The beneficial effects of the embodiments of the present disclosure include:

[0058] The embodiments of the present disclosure provide a method and device for generating decisions for human-machine co-driving, which are applied to the field of intelligent connected vehicles. During vehicle driving, collaborative decision-making between the intelligent driving system and the human driver is achieved, thereby improving driving safety and comfort.

[0059] First, multi-source data is collected during vehicle operation to provide a foundation for subsequent analysis. This comprehensive data, encompassing vehicle status and environmental information, more accurately reflects actual driving conditions. Fuzzy inference algorithms and multimodal models are then used to identify the driver group type and fuse the results. This multi-method approach analyzes driver characteristics from different perspectives, mitigates the limitations of a single algorithm, improves identification accuracy, and provides more accurate driver profile information for subsequent decision fusion. Based on the multi-source data, driving conditions and corresponding lookup tables are determined. The human-machine collaboration factor is then estimated based on the driving conditions and driver type. This dual factorization of driving conditions and driver type allows the intelligent driving system to flexibly adjust its level of collaboration with the driver based on varying circumstances. This estimated value is combined with the human-machine collaboration factor determined from the previous decision, and a collaborative formula is used to determine the human-machine collaboration factor for the current decision, ensuring consistent and dynamic decision-making. The human-machine collaboration factor determined for the current decision is then used to fuse the intelligent driving system and driver decisions, leveraging the strengths of both. The intelligent system can assist the driver in complex driving conditions while retaining control in scenarios where the driver is more skilled.

[0060] This human-machine co-driving decision-making method achieves dynamic collaboration between the intelligent driving system and the human driver through multi-source data collection and processing, accurate driver type identification, reasonable human-machine collaboration factor determination and effective decision fusion, thereby improving the scientific nature and adaptability of driving decisions and enhancing the safety and reliability of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of a method for generating a decision for human-machine co-driving provided by an embodiment of the present disclosure;

[0062] Figure 2 A design block diagram of a decision-making method for human-machine co-driving provided in an embodiment of the present disclosure;

[0063] Figure 3 The first query expression provided in the embodiment of the present disclosure is intended;

[0064] Figure 4 A second query representation provided in an embodiment of the present disclosure;

[0065] Figure 5 A structural diagram of a decision-making device for human-machine co-driving provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0066] The present disclosure provides a method and apparatus for making decisions for human-machine co-driving. Preferred embodiments of the present disclosure are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are intended only to illustrate and explain the present disclosure and are not intended to limit the present disclosure. Furthermore, the embodiments and features within the embodiments of the present disclosure may be combined with one another unless there is a conflict.

[0067] The embodiment of the present disclosure provides a decision-making method for human-machine co-driving, such as Figure 1 As shown, including:

[0068] S101, acquiring multi-source data during the vehicle's driving process;

[0069] S102: Based on multi-source data, the fuzzy inference algorithm and the multimodal model are used to identify the driving group type of the driver, and the obtained identification results are fused using a linear weighted fusion algorithm to obtain the driving group type;

[0070] S103: Obtaining a driving condition based on multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driver's driving group type;

[0071] S104. Determine the human-machine collaboration factor for this decision by using a collaboration formula, combining the estimated human-machine collaboration factor and the human-machine collaboration factor determined in the previous decision.

[0072] S105. Based on the human-machine collaboration factor determined in this decision, the decisions of the intelligent driving system and the driver are integrated to obtain a comprehensive driving decision.

[0073] In the current field of human-machine co-driving in intelligent connected vehicles, existing technologies have shortcomings in driver group identification, operating condition adaptation, and decision fusion. Relying on a single algorithm or a small number of features to identify driver group types makes it difficult to capture the complex characteristics of drivers, and the ability to handle fuzzy and multimodal data is insufficient, leading to biased identification results. Driving condition classification is crude, failing to comprehensively consider factors such as curve curvature. There is a lack of detailed differentiation between the intelligent system and the driver's coordination method under different operating conditions, which affects driving safety and comfort. Human-machine decision fusion is fixed and cannot be dynamically adjusted based on operating conditions and driver characteristics, making it difficult to leverage the strengths of both parties and reducing the scientific nature of decision-making.

[0074] In response to the above problems, this solution proposes a decision-making method for human-machine co-driving, such as Figure 2 As shown in the figure, in the identification of driving group types, the fuzzy reasoning algorithm and the multimodal model are combined, and the linear weighted fusion algorithm is used to integrate the results to improve the identification accuracy from multiple dimensions; in the processing of driving conditions, based on the refined classification of multi-source data, by setting up the corresponding query table, the estimated value of the human-machine collaboration factor is determined in combination with the driver group type, so as to realize the flexible adjustment of the intelligent system collaboration strategy; in the human-machine decision fusion link, the estimated value of the human-machine collaboration factor is used as the fusion weight, and it is dynamically adjusted according to the working conditions and driver characteristics, so that the advantages of both parties complement each other, generate scientific and reasonable comprehensive driving decisions, and enhance the adaptability and safety of the human-machine co-driving system.

[0075] Specifically, this method is applied to intelligent connected vehicle scenarios, aiming to achieve collaborative decision-making between the intelligent driving system and human drivers to ensure driving safety and efficiency.

[0076] In the embodiment of the present disclosure, first, multi-source data is obtained during the vehicle's driving process. These data are collected in real time through multiple devices, thereby providing a basis for subsequent analysis.

[0077] Then, based on multi-source data, a fuzzy inference algorithm and a multimodal model are used to identify driver group types. The fuzzy inference algorithm addresses the ambiguity of the data and categorizes drivers' driving styles into different types. The multimodal model, on the other hand, integrates multi-dimensional information such as drivers' operating habits and physiological characteristics for classification. Furthermore, a linear weighted fusion algorithm is used to combine the identification results of the two methods to improve classification accuracy.

[0078] Next, the system determines the driving condition based on multi-source data. Each condition corresponds to a pre-stored lookup table that records the estimated human-machine collaboration factor for each driver type. Therefore, in a curved road, the estimated human-machine collaboration factor for a conservative driver will lead the intelligent driving system to intervene more in steering assistance. The lookup table, combined with the driver type, accurately determines the estimated human-machine collaboration factor.

[0079] Subsequently, the human-machine collaboration factor for this decision is determined using a collaborative formula, combining the estimated value with the human-machine collaboration factor determined in the previous decision. This dynamic optimization and adjustment of the current collaborative relationship is then performed using historical decision information. This ensures the temporal consistency of the decision-making process while flexibly adapting to changing driving scenarios based on real-time operating conditions. This ensures that the human-machine collaboration strategy output by the system is more aligned with actual driving needs, improving the dynamic adaptability and reliability of decision-making.

[0080] Finally, the human-machine synergy factor determined by this decision is used to integrate the decisions of the intelligent driving system and the driver. When the synergy factor favors the intelligent system, such as at complex intersections, the intelligent driving system's decision-making weight increases, assisting the driver with steering and avoidance. Conversely, on simple straight sections, the driver's decision-making dominates.

[0081] In summary, this method determines the synergy factor through multi-source data collection, multi-algorithm fusion identification, and matching of working conditions and driving groups, thereby realizing dynamic collaborative decision-making between the intelligent driving system and the driver. It not only improves the scientific nature and adaptability of driving decisions, but also enhances the safety and reliability of vehicle driving, and has innovative and practical value.

[0082] In one possible implementation, the data acquisition equipment may include millimeter-wave radar, lidar, visual sensor, inertial navigation unit, RSU roadside unit, cloud platform, etc.

[0083] In another embodiment provided by the present disclosure, the types of driving groups include: a general driving group, an expert driving group, and a novice driving group;

[0084] In the above step S102, based on multi-source data, the fuzzy inference algorithm is used to identify the driver's driving group type, which can be implemented as follows:

[0085] Step 1: Set the width and step size of the sliding window, and use the sliding window to intercept multi-source data within a first preset time period multiple times; wherein the starting point of the sliding window is the start time of the first preset time period, and the end point is the end time of the first preset time period;

[0086] Step 2: Determine the corresponding value of the key feature for each interception of the multi-source data by the sliding window;

[0087] Step 3: Using the fuzzy inference engine constructed by the fuzzy logic system, the corresponding values ​​of the key features are fuzzified, and fuzzy inference is performed based on the prior fuzzy rules to output the corresponding driving group label and timestamp; wherein the driving group label corresponds to the driver's driving group type;

[0088] Step 4: Set the number of clusters and perform cluster analysis on the driving group types within the second preset time period based on the clustering algorithm to obtain the cluster with the largest number of the same driving group labels within the cluster, and use the driving group type corresponding to the driving group label with the largest number in the cluster as the identification result.

[0089] In the embodiment of the present disclosure, a specific implementation process of using a fuzzy inference algorithm to identify the types of driving groups (general driving group, expert driving group, and novice driving group) is provided, including:

[0090] Step 1 sets the sliding window width and step size, and repeatedly captures multi-source data within the first preset time period. The sliding window acts like a data acquisition frame, starting at the start of the first preset time period and moving in set steps until the end, capturing a segment of data with each movement. For example, if the first preset time period is 10 minutes of vehicle travel, with a window width of 1 minute and a step size of 30 seconds, multiple 1-minute data sets will be captured within the 10-minute period. This step is used to segment continuous multi-source data into easily processable segments, preventing excessive amounts of data from being difficult to analyze.

[0091] Step 2: Determine the corresponding values ​​of key features for each intercepted multi-source data. Key features may include average vehicle speed, steering frequency, braking force, etc. For example, in one intercepted data, the average speed was 60 km / h and the steering frequency was 5 times per minute. By determining these key feature values, we can extract the core information from the data and provide a basis for subsequent analysis.

[0092] In step three, a fuzzy inference engine is used to fuzzify the key eigenvalues. Based on a priori fuzzy rules, the engine outputs a driver group label and a timestamp. For example, if the average speed is high and the drivers frequently make turns, the fuzzy inference engine might output an "expert driver group" label based on a priori rules and record the time. Fuzzification converts precise numerical values ​​into fuzzy concepts, which aligns with the fuzzy nature of human driving style. A priori rules are derived from extensive driving data to ensure accurate inference.

[0093] Step 4 sets the number of clusters and performs cluster analysis on the driver group types within the second preset time period using a clustering algorithm. For example, setting the number of clusters to 3 will divide the driver group labels within the second preset time period into three categories. The cluster with the largest number of labels of the same type will be identified, and the driver group type corresponding to the most common label in that cluster will be used as the identification result. This step eliminates data fluctuations through cluster analysis and produces a stable identification of driver group types.

[0094] In summary, the method provided by the present invention forms a complete driver driving group type identification process through sliding window segmentation processing, key feature extraction, fuzzy reasoning and cluster analysis. It has rigorous logic and can accurately distinguish different driving group types. It provides a reliable basis for adjusting the collaborative strategy according to the driver's characteristics in subsequent human-machine co-driving decision-making, and enhances the pertinence and effectiveness of decision-making.

[0095] In another embodiment provided by the present disclosure, in the above step S102, based on multi-source data, the multimodal model is used to identify the driving group type of the driver, which can be implemented as follows:

[0096] Step 1: normalize the corresponding data of key features in the multi-source data, and enhance the corresponding data of key features using a preset data enhancement algorithm;

[0097] Step 2: Determine corresponding advanced features based on the corresponding data of the enhanced key features; wherein the advanced features are extracted from the corresponding data of the key features using a preset deep learning model;

[0098] Step 3: Input the corresponding data of the enhanced key features and the advanced features into the preset multimodal model to obtain the corresponding driving group type.

[0099] In the disclosed embodiment, specific implementation steps are provided for identifying the driving group type of a driver using a multimodal model. By processing multi-source data, an accurate judgment of the group type to which the driver belongs (general driving group, expert driving group and novice driving group) is achieved.

[0100] Step 1: Normalize the data corresponding to key features in the multi-source data and enhance the data using a pre-set data augmentation algorithm. Normalization converts key feature data of different scales and distributions to the same interval, facilitating subsequent calculations. The data augmentation algorithm expands the data volume through methods such as translation, rotation, and noise addition. For example, normalizing vehicle speed data to the interval [0, 1] and adding slight noise to brake force data to simulate different driving environments increase data diversity and improve model generalization.

[0101] Step 2 determines high-level features based on the enhanced key feature corresponding data and extracts them using a preset deep learning model. Deep learning models (such as convolutional neural networks and recurrent neural networks) can automatically mine deep features of the data. For example, a convolutional neural network can be used to process the enhanced vehicle steering angle data to extract high-level features such as continuous steering patterns and steering amplitude change trends. These features can more accurately reflect the driver's driving behavior characteristics.

[0102] In step three, the enhanced key feature data and advanced features are fed into a pre-set multimodal model to determine the driver group type. The multimodal model integrates multidimensional information, such as vehicle motion data and driver operation data, to make a comprehensive judgment and output the driver's group type. If the input data indicates stable driving and accurate decision-making, the multimodal model may determine that the driver is an expert driver.

[0103] In summary, the method provided by this disclosure builds a complete system for identifying driver group types through data preprocessing, advanced feature extraction, and multimodal fusion judgment. Its unique features include fully tapping into the value of multi-source data, leveraging deep learning and multimodal models to automatically extract features and make decisions, reducing manual intervention; creatively combining data enhancement with multimodal fusion to improve identification accuracy; and achieving significant results, enabling rapid and accurate identification of driver group types, providing a reliable basis for subsequent human-machine collaborative decision-making, and enhancing the adaptability of the human-machine co-driving system to different drivers and the scientific nature of its decision-making.

[0104] In another embodiment provided by the present disclosure, the preset multimodal model includes:

[0105] Differentiate the types of input data and match the corresponding preset neural network models to obtain corresponding dynamic features and / or spatial features for different types of data;

[0106] Performing feature fusion on dynamic features and / or spatial features to obtain comprehensive features;

[0107] The comprehensive features are used to calculate the probability distribution of different group types through the classification layer, and the driving group type with the highest probability is output.

[0108] In the embodiment of the present disclosure, the specific working mechanism of the preset multimodal model in the human-machine co-driving decision-making method is as follows:

[0109] First, the type of input data is distinguished, and the corresponding preset neural network model is matched for different types of data to obtain the corresponding dynamic features and / or spatial features. The input data includes different types of data such as vehicle motion data, environmental perception data, and driver operation data. For example, for the sequence data (dynamic data) of vehicle speed changing over time, the recurrent neural network (RNN) is matched to extract its dynamic features in the time series; for the road image data (spatial data) taken by the camera, the convolutional neural network (CNN) is matched to extract spatial features such as road markings and obstacle positions. In this step, the appropriate model is selected according to the characteristics of the data, giving full play to the advantages of different neural networks and accurately extracting data features.

[0110] Next, dynamic and / or spatial features are fused to generate comprehensive features. The extracted features of different types are integrated, such as combining the dynamic features of vehicle speed changes with the spatial features of road images. Fusion methods can include concatenation and weighted summation. For example, the dynamic feature vector extracted by the RNN and the spatial feature vector extracted by the CNN can be concatenated into a new feature vector, forming a comprehensive feature containing multi-dimensional information, enabling the model to analyze data from multiple perspectives.

[0111] Finally, the comprehensive features are passed through the classification layer to calculate the probability distribution of different group types and output the driver group type with the highest probability. The classification layer, typically consisting of a fully connected layer and a softmax function, maps the comprehensive features into the probability space of different driver group types. For example, the calculated probabilities of belonging to the general driver group, the expert driver group, and the novice driver group are 0.2, 0.7, and 0.1, respectively. The expert driver group with the highest probability is ultimately output as the identification result, thus determining the driver group type.

[0112] In summary, the characteristics of this preset multimodal model are that it customizes the selection of processing models according to the data type, which is highly targeted; it integrates multiple types of features to break the limitations of a single data type; it can comprehensively analyze multi-source data and accurately identify the type of group to which the driver belongs, providing a precise basis for subsequent human-machine collaborative decision-making, and effectively improving the scientificity and reliability of the decision-making of the human-machine co-driving system.

[0113] In another embodiment provided by the present disclosure, the driving conditions include: a large curvature driving condition and a small curvature driving condition;

[0114] The above-mentioned step S103, obtaining a driving condition based on multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of the human-machine cooperation factor using the lookup table and the driver's driving group type, can be implemented as follows: determining whether the driving condition is a large curvature driving condition or a small curvature driving condition based on the road curvature in the multi-source data;

[0115] When the driving condition is a high curvature driving condition, an estimated value of the human-machine cooperation factor is obtained using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position;

[0116] When the driving condition is a small curvature driving condition, the estimated value of the human-machine cooperation factor is obtained using the second query table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position.

[0117] In the embodiment of the present disclosure, the specific implementation process of determining the estimated value of the human-machine cooperation factor based on the driving conditions (high curvature driving conditions and low curvature driving conditions) and the driver group type includes:

[0118] First, the driving condition type is determined based on road curvature from multi-source data. Road curvature is acquired in real time by onboard sensors (such as cameras and radar). When the curvature value exceeds a preset threshold, it is determined to be a high-curvature driving condition (such as a sharp turn); otherwise, it is determined to be a low-curvature driving condition (such as a straight road or a gentle curve). For example, frequent sharp turns on mountain roads would be considered a high-curvature condition, while long straight-line driving on a highway would be considered a low-curvature condition. This step provides the basis for selecting the appropriate lookup table.

[0119] Under high-curvature driving conditions, the first query table is used to obtain an estimated value of the human-machine collaboration factor based on the driving group type and the comprehensive lateral deviation at the preview point position. The preview point position is usually a certain distance in front of the vehicle (such as 10-20 meters). The comprehensive lateral deviation refers to the lateral distance deviation between the vehicle's current driving trajectory and the desired trajectory processed by the preview point. Different driving group types (general, expert, novice) correspond to different collaboration factors. For example, a novice may not be precise enough when making a sharp turn, and the absolute value of the lateral deviation of the preview point is large. At this time, the first query table will give a higher intelligent driving system intervention weight (larger collaboration factor) to enhance the assistance strength. In addition, when the absolute value of the lateral deviation of the preview point is small, the first query table will obtain an estimated value of 0 for the human-machine collaboration factor, that is, the intelligent driving system will not intervene in the user's driving behavior at this time. The same applies to the following small-curvature driving conditions.

[0120] Under low-curvature driving conditions, a second lookup table is used to estimate the human-machine collaboration factor based on the driver group type and the combined lateral deviation at the vehicle's center of mass. The lateral deviation at the vehicle's center of mass reflects the degree of deviation between the vehicle's current actual position and the lane center. For example, when driving on a straight road, the absolute value of the lateral deviation of the center of mass is typically small. Therefore, the second lookup table will assign a lower intelligent system intervention weight (e.g., an estimated human-machine collaboration factor of 0 or 0.1), giving the driver more control. However, a novice driver may frequently deviate from the lane center, in which case the collaboration factor increases, and the intelligent system provides more corrective assistance.

[0121] In summary, the method provided by the present invention adopts different lateral deviation reference points (pre-aiming points and center of mass positions) for different curvature working conditions, and dynamically adjusts the estimated value of the human-machine collaboration factor in combination with the type of driver group. A dual query table mechanism is established to accurately match different working conditions and driving groups, thereby improving the adaptability of collaborative decision-making. In large curvature working conditions, early intervention is made through the pre-aiming point deviation, and in small curvature working conditions, real-time correction is made based on the center of mass deviation, which not only ensures the safety of complex road conditions, but also respects the driver's operating habits. In addition, through the setting of the query table, the intervention timing and degree of the intelligent driving system corresponding to different driving groups are refined, thereby achieving refined control of the human-machine co-driving system.

[0122] In another embodiment provided by the present disclosure, the driving conditions include: a high-speed large-curvature condition, a low-speed small-curvature condition, a transition condition, and an emergency condition;

[0123] The above-mentioned step S103, obtaining a driving condition based on multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of the human-machine cooperation factor using the lookup table and the driver's driving group type, can be implemented as follows: determining whether the driving condition is a high-speed, high-curvature condition, a low-speed, low-curvature condition, a transitional condition, or an emergency condition based on the vehicle longitudinal speed and road curvature in the multi-source data;

[0124] In the case of high-speed and high-curvature driving conditions, the first lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the preview point position;

[0125] In the case of low-speed and small-curvature driving conditions, the second lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the vehicle's center of mass.

[0126] When the driving condition is a transition condition, a first human-machine cooperation factor estimate is obtained using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position; a second human-machine cooperation factor estimate is obtained using a second lookup table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position; and the human-machine cooperation factor estimate is determined based on the first human-machine cooperation factor estimate, the second human-machine cooperation factor estimate, and preset parameters.

[0127] When the driving condition is an emergency condition, a multi-dimensional warning is activated.

[0128] In the embodiment of the present disclosure, another specific implementation method for determining the estimated value of the human-machine collaboration factor based on driving conditions and driver group types is provided. In addition to the original large curvature and small curvature conditions, transition conditions and emergency conditions are added, and corresponding collaboration factor calculation logic and early warning mechanism are provided.

[0129] First, the driving condition type is determined based on the vehicle's longitudinal speed and road curvature from multiple data sources. For example, when the vehicle speed exceeds 80 km / h and the road curvature radius is less than 100 meters, it is considered a high-speed, high-curvature condition (such as a curve on a mountain highway); when the vehicle speed is less than 30 km / h and the curvature radius is greater than 500 meters, it is considered a low-speed, low-curvature condition (such as a straight road in the city); when the vehicle speed or curvature falls between the above ranges, it is considered a transition condition; and when emergency braking or collision risk is detected, it is considered an emergency condition.

[0130] Under high-speed, high-curvature conditions, the first lookup table is used to determine the synergy factor based on the lateral deviation of the preview point and the type of driver. For example, for an expert driver with a small deviation from the preview point on a high-speed curve, the synergy factor will reduce the level of intelligent system intervention, giving the driver more control. However, for a novice driver with a large deviation, the system will increase the weight of assistance.

[0131] In low-speed, low-curvature conditions, the second lookup table is used to determine the synergy factor based on lateral center of mass deviation and the type of driver. For example, if the average driver exhibits minimal center of mass deviation during slow, straight driving, the system reduces intervention. However, if frequent deviations are detected (e.g., driver distraction), the synergy factor is increased to enhance assistance.

[0132] A dual-factor fusion strategy is employed for transition conditions. The lateral deviations of the preview point and the center of mass are simultaneously calculated, and two synergy factors are derived using the first and second lookup tables, respectively. The estimated human-machine synergy factor for this decision is then determined based on preset parameters. For example, on a section with a gradual curvature change, the preset parameter is 0.6, with the preview point factor weighted at 0.6 and the center of mass factor weighted at 1-0.6 = 0.4. This combined estimate of the human-machine synergy factor allows for smooth transitions between different operating conditions.

[0133] Multi-dimensional warnings are activated in emergency situations. For example, when the system detects a sudden obstacle ahead, in addition to automatic braking, it will also use visual (dashboard warning light), auditory (buzzer), and tactile (steering wheel vibration) multi-dimensional warnings to remind the driver to take over the vehicle.

[0134] In summary, the disclosed embodiments refine the classification of working conditions and design differentiated collaborative strategies for different scenarios; the dual-factor fusion mechanism for transition working conditions solves the decision-making jump problem of a single reference point during working condition conversion; and the multi-dimensional early warning mechanism improves safety in emergency situations.

[0135] In another embodiment provided by the present disclosure, the collaborative formula is as follows:

[0136] β * =ηβ cur +(1-η)β pre

[0137] Among them, η represents the preset parameters related to the driving group type, β pre Characterizes the human-machine collaboration factor determined by the last decision, β * The human-machine collaboration factor that characterizes this decision, β cur An estimated value of the human-machine collaboration factor that characterizes this decision; and / or

[0138] The above step S104, which integrates the decisions of the intelligent driving system and the driver based on the human-machine collaboration factor determined in this decision to obtain a comprehensive driving decision, can be implemented as follows:

[0139] Step 1: Obtain the decisions of the intelligent driving system and the driver;

[0140] Step 2: Combine the human-machine collaboration factor determined in this decision and the decisions of the intelligent driving system and the driver to obtain a comprehensive driving decision through a fusion formula;

[0141] Among them, the individual decisions of the intelligent driving system and the driver as well as the comprehensive driving decision include: torque;

[0142] The above fusion formula is as follows:

[0143] T total =T d +β * T a

[0144] Among them, T total Characterizing comprehensive driving decisions, T a Characterize the decision of the intelligent driving system, T d Characterizes the driver's decision, β * Characterize the human-machine collaboration factor of this decision.

[0145] In the embodiment of the present disclosure, the estimated value of the human-machine collaboration factor β is weighted fused. cur Human-machine collaboration factor β with the previous decision pre Get the human-machine collaboration factor β for this decision * , where η can be obtained through empirical design, reinforcement learning, and other methods, and is not limited here. This collaborative formula combines historical decision-making experience with real-time estimation results to dynamically adjust the collaborative relationship, preserving decision consistency while adapting to real-time operating conditions. This ensures that the output collaborative factor better aligns with driver characteristics and scenario requirements, improving dynamic decision-making adaptability.

[0146] In the disclosed embodiment, the specific implementation process of integrating the intelligent driving system and the driver's decisions based on the human-machine collaboration factor determined in this decision to obtain a comprehensive driving decision includes:

[0147] Step 1: Obtain the respective decisions of the intelligent driving system and the driver. During driving, the intelligent driving system uses sensor data and algorithm analysis to generate torque decisions based on the current driving conditions, such as outputting torque to control steering in a curve. The driver then makes corresponding torque decisions through the steering wheel based on their own judgment and operating habits, such as manually adjusting the steering force.

[0148] Step 2 combines the human-machine collaboration factor determined in this decision with the decisions of the intelligent driving system and the driver, and obtains a comprehensive driving decision through a fusion formula. Taking the torque decision as an example, the human-machine collaboration factor determined in this decision is used as the weight coefficient. When the value is close to 1, it indicates that the intelligent driving system is the dominant decision-maker. At this time, the comprehensive driving decision is closer to the decision of the intelligent driving system. For example, in an emergency, the intelligent driving system outputs a large braking torque to avoid a collision. When the value is close to 1, the comprehensive decision basically adopts this braking torque. When the value is close to 0, the driver's decision is dominant. For example, in low-speed and small-curvature conditions, the driver is experienced, the intelligent driving system is close to 0, and the comprehensive decision is mainly based on the torque generated by the driver's operation. When the value is between 0 and 1, such as 0.5 in transition conditions, the torque decisions of the intelligent driving system and the driver are fused with equal weights to form a comprehensive torque decision that takes into account the advantages of both, realizing human-machine collaborative control.

[0149] In summary, the disclosed embodiment establishes a dynamic balance mechanism between the intelligent driving system and the driver's decision-making through a quantified human-machine collaborative factor. A unified mathematical formula is used to perform weighted fusion of decisions from different sources, making the decision-making process standardized and adjustable. The estimated value of the human-machine collaborative factor based on the driving conditions and driving group types determined above can accurately match the decision-making needs in different scenarios, fully exerting the stability and accuracy of the intelligent driving system under complex conditions, while retaining the driver's operational dominance in appropriate scenarios, effectively improving the scientific nature, adaptability and driving safety of the human-machine co-driving system's decision-making, and realizing a complete closed loop from condition judgment to decision output.

[0150] In another embodiment provided by the present disclosure, the key features include at least one of the following: vehicle lateral deviation, vehicle lateral acceleration, vehicle longitudinal acceleration, positive vehicle longitudinal acceleration, negative vehicle longitudinal acceleration, vehicle lateral shock wave, vehicle longitudinal shock wave, steering wheel angle, steering wheel angular velocity, following vehicle distance, driver's reaction time, frequency of driver checking left and right rearview mirrors, average grip strength of driver operating the steering wheel, driver's facial expression tension factor, driver's annual number of traffic violations, total number of sampling points, peak value of lateral deviation, mean value of lateral deviation, standard deviation of lateral deviation, root mean square error of lateral deviation, peak value of lateral acceleration, mean value of lateral acceleration, standard deviation of lateral acceleration, peak value of longitudinal acceleration, peak value of longitudinal deceleration, frequency of sudden acceleration, frequency of sudden deceleration, peak value of lateral impact degree, peak value of longitudinal impact degree, peak value of steering angle, mean value of steering angle, standard deviation of steering angle, peak value of steering angular velocity, mean value of steering angular velocity, and standard deviation of steering angular velocity; wherein the total number of sampling points is consistent with the width of the sliding window.

[0151] In the disclosed embodiments, the key features used to identify driver group types are closely related to the aforementioned fuzzy inference algorithm, multimodal model, and driving condition determination steps, providing a rich data dimension for comprehensive and accurate analysis of driver behavior and driving environment. The key features are represented and their specific meanings are as follows:

[0152] y CG Indicates the lateral deviation (i.e., side deviation) of the vehicle's center of mass in the vehicle coordinate system;

[0153] a y Indicates the vehicle's lateral acceleration;

[0154] a x Indicates the vehicle's longitudinal acceleration;

[0155] Indicates the vehicle's longitudinal acceleration (used to describe the characteristics when the driver steps on the accelerator pedal);

[0156] Indicates the vehicle's longitudinal deceleration (used to describe the characteristics when the driver steps on the brake pedal);

[0157] Indicates the lateral impact of the vehicle (used to describe lateral comfort);

[0158] Indicates the longitudinal impact of the vehicle (used to describe longitudinal comfort);

[0159] δ sw Indicates the steering wheel angle (used to describe the characteristics of the driver's steering wheel operation);

[0160] Indicates the steering wheel angular velocity (used to describe the characteristics of the driver's steering wheel operation);

[0161] d s Indicates the following distance;

[0162] τ rsp Indicates the driver's reaction time (the time it takes to take actions such as steering and braking in the event of an emergency);

[0163] λ view Indicates the frequency of the driver checking the left and right rearview mirrors;

[0164] μ grip Indicates the average grip strength of the driver operating the steering wheel;

[0165] κ face Indicates the driver's facial expression stress factor;

[0166] φ ruleIndicates the number of traffic violations per year by the driver;

[0167] N represents the total number of sampling points;

[0168] Indicates the peak value of lateral deviation;

[0169] represents the mean of the lateral deviation;

[0170] Indicates the standard deviation of the lateral deviation, which is used to quantify the fluctuation of the lateral deviation curve;

[0171] It represents the root mean square error of the lateral deviation, which is used to quantify the trajectory tracking accuracy;

[0172] Indicates the peak value of lateral acceleration;

[0173] represents the mean value of lateral acceleration;

[0174] represents the standard deviation of lateral acceleration;

[0175] Indicates the peak value of longitudinal acceleration;

[0176] Indicates the peak value of longitudinal deceleration;

[0177] represents the frequency of rapid acceleration, where is the design threshold;

[0178] Indicates the frequency of sudden deceleration, where is the design threshold;

[0179] Indicates the peak value of the lateral impact intensity;

[0180] Indicates the peak value of longitudinal impact intensity;

[0181] Indicates the peak value of the steering angle;

[0182] represents the mean value of the steering angle;

[0183] Indicates the standard deviation of the steering angle, which is used to quantify the fluctuation of the steering angle curve;

[0184] Indicates the peak value of the steering angular velocity;

[0185] represents the mean value of the steering angular velocity;

[0186] Indicates the standard deviation of the steering angular velocity, which is used to quantify the fluctuation of the steering angular velocity curve.

[0187] In summary, the method provided by the present disclosure covers a wide variety of key features, comprehensively characterizing the driving situation from the vehicle's motion state to the driver's operating behavior and physiological and psychological state. Comprehensive analysis of these features can more accurately identify the driver's driving group type. In terms of effect, by extracting and analyzing these key features, a solid data foundation is provided for the subsequent fuzzy reasoning algorithm, multimodal model, and determination of the estimated value of the human-machine collaborative factor, thereby improving the accuracy and reliability of the human-machine co-driving decision-making method and further enhancing the safety and comfort of driving.

[0188] In another embodiment provided by the present disclosure, the above method further includes: after acquiring multi-source data during the driving process of the vehicle, performing time calibration, time synchronization and filtering on the multi-source data.

[0189] In the disclosed embodiment, in the data preprocessing phase after obtaining multi-source vehicle driving data, time calibration, time synchronization and filtering operations are clarified to provide high-quality data support for subsequent steps such as driver group type identification, driving condition determination and decision fusion, and are closely related to the entire method process.

[0190] Time calibration corrects the timestamps of data collected by different sensors. Timing errors in various sensors (such as cameras, radar, and steering wheel angle sensors) can lead to inconsistent data times. For example, a camera might record the appearance of an obstacle as 10:00:05, while a radar might record it as 10:00:05.2. Using a time calibration algorithm, the timestamps of both are adjusted to ensure data accuracy in the temporal dimension, allowing subsequent analysis to be based on a consistent time base.

[0191] Time synchronization involves aligning calibrated multi-source data in a time series. Multi-source data can be collected at different frequencies, such as 10 times per second for a speed sensor and 30 times per second for an image sensor. Through interpolation or resampling, data at different frequencies can be aligned at the same time point, forming a complete time series. For example, interpolating low-frequency speed data to the same time interval as the image data facilitates comprehensive analysis of vehicle motion and environmental information.

[0192] Filtering removes noise from data. Data collection and transmission are susceptible to noise caused by electromagnetic interference, sensor errors, and other factors. For example, vehicle acceleration data can experience unusual fluctuations due to electromagnetic interference. Methods such as mean filtering, Kalman filtering, and particle filtering can be used to smooth raw data. For example, Kalman filtering can predict the next moment's data based on the vehicle's motion model and, combined with measurement corrections, effectively filter noise from acceleration data, ensuring that the data more accurately reflects the vehicle's actual motion.

[0193] In summary, the method provided by the present disclosure systematically solves the time and data quality issues of multi-source data in the data preprocessing stage, and builds an accurate and consistent data foundation through the combined operations of time calibration, synchronization and filtering. The processing flow designed for the characteristics of multi-source heterogeneous data ensures the collaborative availability of data. It provides reliable data for subsequent steps such as using key features to identify driver group types and determine the estimated value of the human-machine collaboration factor based on working conditions, avoids decision-making bias caused by data errors, enhances the stability and accuracy of the human-machine co-driving decision-making method, and improves the reliability of the entire system in driving scenario analysis and decision-making.

[0194] Example 1

[0195] When using fuzzy inference algorithms to identify the type of driving group, the specific implementation steps include:

[0196] Step 1: Dynamic sliding window design. According to the needs of different driving scenarios, the sliding window length N is dynamically adjusted to adapt to driving behavior analysis at different time scales.

[0197] Step 2: Calculate key indicators. Perform statistical analysis on the pre-processed data within the sliding window and record the values ​​of key indicators.

[0198] Step 3: Fuzzy reasoning. Design a priori fuzzy rules (see Table 1). Use advanced fuzzy logic systems such as the Mamdani model or the Sugeno model to build a fuzzy inference engine and output the driving group label and timestamp.

[0199] Step 4 Cluster analysis: Use K-means or other clustering algorithms to perform cluster analysis on group labels, select the cluster with the largest number of labels within the cluster, and use this label as the label of the driving group.

[0200] Step 5 dynamically adjusts driving group labels. Taking into account the evolution and volatility of the group, the size of the group label dataset is dynamically adjusted according to the timestamp to improve the rapid identification capability under complex conditions such as group evolution or driving state changes. At the same time, a feedback mechanism is introduced to continuously optimize the method based on actual application results.

[0201] Table 1 Prior rules

[0202]

[0203] Example 2

[0204] When using a multimodal model to identify the type of driving group, the specific implementation steps include:

[0205] Step 1: Data preprocessing: normalize the artificial features mentioned above. Use data augmentation techniques such as rotation, cropping, scaling, and time shifting to increase data diversity and improve the generalization ability of the model.

[0206] Step 2: Advanced feature extraction: Use deep learning models such as convolutional neural networks or recurrent neural networks (GRU, Gated Recurrent Unit) to learn artificial feature data and extract advanced features.

[0207] Step 3: Build a multimodal model. For time series data, choose LSTM (Long Short-Term Memory), SRU (Simple Recurrent Unit), or GRU networks; for image data, choose CNN networks. Train the model using the training set data and evaluate the performance of the multimodal model using methods such as cross-validation. Use grid search, random search, or Bayesian optimization to find the optimal hyperparameter combination to improve model performance.

[0208] Step 4: Model iteration. As data samples continue to grow, the model is updated in real time using online learning algorithms to adapt to new driving environments and demographics. Model performance is regularly evaluated and adjusted and optimized based on the results. User feedback is collected to further improve the model's accuracy and reliability.

[0209] Step 5 involves online driver group identification, using a trained multimodal model to classify drivers into groups. A continuous judgment or scoring mechanism is introduced to improve the accuracy and reliability of group classification. Also, given the evolving and volatile nature of groups, the group classification threshold and judgment strategy need to be dynamically adjusted to adapt to changes in time periods and driving environments.

[0210] Example 3

[0211] The specific implementation steps of fusing the obtained identification results through the linear weighted fusion algorithm to obtain the driving group type include:

[0212] Step 1 adopts a linear weighted fusion strategy to fuse the identification results of the fuzzy inference algorithm and the multimodal model.

[0213] Step 2 introduces a feedback mechanism to adjust the weight parameters of the fusion strategy according to the actual application effect.

[0214] Example 4

[0215] A method for generating the first query table and the second query table includes the following steps:

[0216] Step 1: Determine the human-machine collaboration factor estimation model under different working conditions;

[0217] Human-machine collaboration factor estimation model for large-curvature road conditions

[0218]

[0219] Human-machine collaboration factor estimation model for small-curvature road conditions

[0220]

[0221] Where, β L , β CG They represent the estimated values ​​of the human-machine cooperation factor for large curvature and small curvature road conditions, β0, β s are all design parameters, α is the characteristic parameter of the driving group, which indicates the driving group's acceptance of the degree of intervention of the intelligent driving system, 2b w Indicates the width of the comfortable driving area. The selection of this value is related to the lane keeping habits of the driving group. Represents the comprehensive lateral deviation at the vehicle's center of mass, Indicates the comprehensive lateral deviation at the preview point. Respectively represent the normalized values ​​of the vehicle lateral deviation and yaw angle deviation at the vehicle center of mass. denote the normalized values ​​of the vehicle lateral deviation and yaw angle deviation at the preview point, respectively. λ1 and λ2 are both design parameters, and satisfy λ1+λ2=1.

[0222] Step 2: Set corresponding driving group characteristic parameters for different driving groups;

[0223] (a) For road conditions with large curvature

[0224] ① Expert driving group, characteristic parameters satisfy: α=0.45, b w =0.4,β s =-0.2, β0=0;

[0225] ② General driving group, characteristic parameters meet: α=0.26, b w =0.3,β s =-0.2, β0=0;

[0226] ③ Novice driver group, characteristic parameters satisfy: α=0.12, b w =0.2,β s =-0.2, β0=0.05;

[0227] (b) For road conditions with small curvature

[0228] ① Expert driving group, characteristic parameters satisfy: α=0.67, b w =0.5,β s =-0.15, β0=0;

[0229] ② General driving group, characteristic parameters meet: α=0.52, b w =0.4,β s =-0.15, β0=0;

[0230] ③ Novice driver group, characteristic parameters meet: α=0.35, b w =0.3,β s =-0.15, β0=0.05.

[0231] Step 3 will bring the characteristic parameters of different driving groups into the human-machine collaboration factor estimation model respectively, obtain the corresponding relationship and generate the first query table and the second query table.

[0232] The human-machine collaboration factor estimation model for different driving groups, the first query table is as follows Figure 3 As shown, the second query table is as follows Figure 4 As shown. Among them, from Figure 2 It can be seen that the comprehensive lateral deviation ξ at the preview point position L The absolute value of determines the degree of intervention of the intelligent driving system. For different driving groups, the intervention standards of the intelligent driving system are different. When the absolute value of the deviation is small, the estimated value of the human-machine collaboration factor approaches 0, which means that the intelligent driving system does not intervene in user operations and the driver can control the vehicle independently. As the absolute value of the deviation increases, the intelligent driving system begins to intervene, and the larger the absolute value of the deviation, the larger the estimated value of the human-machine collaboration factor, and the higher the degree of intervention of the intelligent driving system. Taking the expert group as an example, its intervention standards are relatively stricter, and the intelligent driving system will intervene deeply when the absolute value of the deviation is larger; the intervention point of the general group is relatively moderate; and the novice group, due to insufficient driving experience, will have the intelligent driving system intervene earlier and more when the absolute value of the deviation is slightly larger, in order to ensure driving safety. Similarly Figure 3 The comprehensive lateral deviation ξ at the vehicle's center of mass CGThe value of also affects the value of the estimated value of the human-machine collaborative factor. When the absolute value of the comprehensive lateral deviation at the center of mass of the vehicle is small, the intelligent driving system does not intervene in the user operation. However, when the vehicle produces a large deviation during driving, the intelligent driving system intervenes to provide assisted driving. The greater the deviation, the higher the degree of intervention of the intelligent driving system. At the same time, different driving groups correspond to different value strategies, which in turn produce different human-machine collaborative driving decisions.

[0233] Example 5

[0234] According to the size of the road curvature, the road conditions are divided into large curvature conditions and small curvature conditions, and different human-machine collaborative factor estimation models are corresponding. In practical applications, in addition to road conditions, vehicle conditions (i.e., the longitudinal speed of the vehicle) and factors such as frequent switching between conditions need to be considered. Therefore, in order to further improve the friendliness, comfort and stability of human-machine collaborative lateral control, based on Example 4, an implementation method for generating the human-machine collaborative factor determined by this decision is provided:

[0235] (1) Initialization

[0236] The initial estimated value of the human-machine collaboration factor satisfies β=β CG (ξ CG ,α,b w ,β s );

[0237] (2) Working condition identification

[0238] Step 1: Key data collection, obtain the longitudinal speed v of the vehicle x , road curvature ρ.

[0239] Step 2 is the judgment of the turning condition. When the turning condition is not met, that is, when the conditions for changing lanes, overtaking, turning around a curve, etc. are not met, it is determined to be an emergency condition and jumps back to Step 1. Otherwise, it enters Step 3.

[0240] Step 3 Working condition identification, when the vehicle speed v x >v H And the road curvature ρ>ρ H When the vehicle speed v x <v L And the road curvature ρ<ρ L When , it is determined to be a low-speed and small-curvature operating condition; in other cases, it is determined to be a transitional operating condition. H 、v L , ρ H , ρ L All are design parameters.

[0241] (3) Determination of estimated value of human-machine collaboration factor

[0242] Under high-speed and high-curvature working conditions, the human-machine collaboration factor estimation model satisfies:

[0243] β cur =β L ;

[0244] Under low-speed and small-curvature working conditions, the human-machine collaboration factor estimation model satisfies:

[0245] β cur =β CG ;

[0246] Under transition conditions, the human-machine collaboration factor estimation model satisfies:

[0247] β cur =γβ L +(1-γ)β CG ;

[0248] Where γ is the design parameter.

[0249] (4) Determination of human-machine collaboration factors

[0250] Get the human-machine collaboration factor β determined by the last decision pre ;

[0251] Based on the estimated value of the human-machine collaboration factor β determined above cur and the above β pre The human-machine collaboration factor of this decision is obtained through collaborative calculation;

[0252] Collaborative formula:

[0253] β * =ηβ cur +(1-η)β pre

[0254] Among them, η represents the preset parameters related to the driving group type, β pre Characterizes the human-machine collaboration factor determined by the last decision, β * The human-machine collaboration factor that characterizes this decision, β cur It is the estimated value of the human-machine collaboration factor for this decision.

[0255] In addition, under emergency conditions, there is no need to calculate the human-machine collaboration factor. Instead, the multi-dimensional audio-visual-tactile warning strategy is activated to enter the emergency response process.

[0256] Example 6

[0257] In a specific intelligent connected vehicle test scenario, the vehicle, equipped with millimeter-wave radar, cameras, steering wheel angle sensors, and other equipment, travels at 60 km / h on an urban road. During this process, the system generates a human-machine co-driving decision using the following steps.

[0258] First, multi-source data is acquired and pre-processed. The sensor collects 100 sets of data per second, covering key characteristic data such as vehicle side deviation, lateral acceleration, and steering wheel angle. For example, within a 10-second period, the initial value of the vehicle side deviation data is 0.1m, and the lateral acceleration is between 0.5-1.2m / s. 2 Fluctuations. These data are time-calibrated to correct the timestamp errors of each sensor, reducing the time difference between radar and camera data from 0.15s to 0.01s. Time synchronization is achieved through interpolation, unifying the sampling frequency of low-frequency vehicle longitudinal acceleration data (collected 10 times per second) and high-frequency steering wheel angle data (collected 50 times per second) to 50 times per second. Kalman filtering is used to remove noise, and the fluctuation range of lateral acceleration data is reduced from 0.5-1.2m / s. 2 Stable to 0.6-1.0m / s 2 .

[0259] Next, the driver's driving group type is identified. Using a fuzzy inference algorithm, the sliding window width is set to 2 seconds and the step length is 1 second. 10 seconds of data are intercepted to obtain 9 groups of segments. Taking one group as an example, the average vehicle speed is 58 km / h and the average steering wheel angular velocity is 15° / s. Based on the fuzzy inference engine and prior rules, the "general driving group" label is output. At the same time, using a multimodal model, after normalizing the key feature data, a convolutional neural network is used to extract the spatial features of the road image captured by the camera, and a recurrent neural network is used to extract the dynamic features of the vehicle movement. After multimodal model fusion judgment, the "general driving group" type is also output. Using a linear weighted fusion algorithm, the final driving group type is determined to be "general driving group."

[0260] Next, the driving condition and estimated human-machine cooperation factor were determined. Based on the vehicle's longitudinal speed of 60 km / h and the road curvature (radius of curvature 800 meters), the condition was determined to be low-speed, low-curvature. The combined lateral deviation at the vehicle's center of mass was calculated to be 0.15 meters. The second lookup table was consulted, yielding an estimated human-machine cooperation factor of 0.1.

[0261] Subsequently, η is preset to 0.9 and the human-machine collaboration factor determined in the last decision is 0.09. Then, through the collaboration formula, the human-machine collaboration factor of this decision is 0.099.

[0262] Finally, the fusion decision is made. Based on the current working conditions and environment, the intelligent driving system outputs a steering torque decision of 12N·m; the steering torque decision generated by the driver's operation is 10N·m. According to the fusion formula

[0263] T total =10N·m+0.099·12N·m=11.188N·m, and the steering torque for comprehensive driving decision is 11.188N·m, realizing human-machine collaborative driving.

[0264] Furthermore, the human-machine co-driving decision-making method provided herein only makes decisions regarding lateral torque, without interfering with other driving factors such as the vehicle's longitudinal speed control, power output, and braking, allowing these factors to remain in their original state or be determined independently by the driver. This significantly reduces the complexity of the decision-making system, alleviating the computational burden and risk of decision conflicts associated with the coupling of multiple factors. Furthermore, focusing on lateral torque decisions allows for more precise handling of lateral motion control issues such as steering and lane keeping, improving the vehicle's handling stability and safety in scenarios such as cornering and lane changes. Furthermore, not interfering with other driving factors helps preserve the driver's original control habits, reduces conflicts in human-machine interaction, and enhances the compatibility and practicality of the human-machine co-driving system.

[0265] Based on the same disclosed concept, the embodiments of the present disclosure also provide a decision-making device, electronic device and storage medium for human-machine co-driving. Since the principles of the problems solved by these devices, electronic devices and storage media are similar to those of the aforementioned decision-making method for human-machine co-driving, the implementation of the device, electronic device and storage medium can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0266] With the above Figure 1 Corresponding to the method shown, the embodiment of the present disclosure also provides a decision-making device for human-machine co-driving, such as Figure 5 As shown, including:

[0267] The data acquisition module 501 is used to acquire multi-source data during the vehicle's driving process;

[0268] The driving group module 502 is used to identify the driving group type of the driver based on multi-source data using a fuzzy inference algorithm and a multimodal model, and fuse the obtained identification results using a linear weighted fusion algorithm to obtain the driving group type;

[0269] A query estimation module 503 is configured to obtain a driving condition based on multi-source data, determine a corresponding query table based on the driving condition, and determine an estimated value of a human-machine collaboration factor using the query table and the driver's driving group type;

[0270] The collaborative factor module is used to combine the estimated value of the human-machine collaborative factor with the human-machine collaborative factor determined in the previous decision, and determine the human-machine collaborative factor of this decision through a collaborative formula;

[0271] The comprehensive decision-making module is used to integrate the decisions of the intelligent driving system and the driver based on the human-machine collaboration factors determined by this decision to obtain a comprehensive driving decision.

[0272] The driving group module 502 is used to set the width and step size of the sliding window, and use the sliding window to intercept multi-source data within the first preset time period multiple times; wherein, the starting point of the sliding window is the starting time of the first preset time period, and the end point is the end time of the first preset time period; determine the corresponding value of the key feature for the multi-source data intercepted each time by the sliding window; use the fuzzy inference engine constructed by the fuzzy logic system to fuzzify the corresponding value of the key feature, and perform fuzzy inference based on the prior fuzzy rules to output the corresponding driving group label and timestamp; wherein, the driving group label corresponds to the driving group type of the driver; set the number of clusters, and perform cluster analysis on the driving group type within the second preset time period based on the clustering algorithm to obtain the cluster with the largest total number of the same driving group label within the cluster, and use the driving group type corresponding to the driving group label with the largest total number in the cluster as the identification result.

[0273] In another embodiment provided by the present disclosure, the above-mentioned driving group module 502 is used to normalize the corresponding data of the key features in the multi-source data, and enhance the corresponding data of the key features using a preset data enhancement algorithm; determine the corresponding high-level features based on the corresponding data of the enhanced key features; wherein the high-level features are extracted from the corresponding data of the key features using a preset deep learning model; input the corresponding data of the enhanced key features and the high-level features into a preset multimodal model to obtain the corresponding driving group type.

[0274] In another embodiment provided by the present disclosure, the preset multimodal model includes:

[0275] Differentiate the types of input data and match the corresponding preset neural network models to obtain corresponding dynamic features and / or spatial features for different types of data;

[0276] Performing feature fusion on dynamic features and / or spatial features to obtain comprehensive features;

[0277] The comprehensive features are used to calculate the probability distribution of different group types through the classification layer, and the driving group type with the highest probability is output.

[0278] In another embodiment provided by the present disclosure, the driving conditions include: a large curvature driving condition and a small curvature driving condition;

[0279] The query estimation module 503 is configured to determine whether the driving condition is a large curvature driving condition or a small curvature driving condition based on the road curvature in the multi-source data;

[0280] When the driving condition is a high curvature driving condition, an estimated value of the human-machine cooperation factor is obtained using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position;

[0281] When the driving condition is a small curvature driving condition, the estimated value of the human-machine cooperation factor is obtained using the second query table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position.

[0282] In another embodiment provided by the present disclosure, the driving conditions include: a high-speed large-curvature condition, a low-speed small-curvature condition, a transition condition, and an emergency condition;

[0283] The query estimation module 503 is configured to determine whether the driving condition is a high-speed, high-curvature condition, a low-speed, low-curvature condition, a transition condition, or an emergency condition based on the vehicle longitudinal speed and road curvature in the multi-source data;

[0284] In the case of high-speed and high-curvature driving conditions, the first lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the preview point position;

[0285] In the case of low-speed and small-curvature driving conditions, the second lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the vehicle's center of mass.

[0286] When the driving condition is a transition condition, a first human-machine cooperation factor estimate is obtained using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position; a second human-machine cooperation factor estimate is obtained using a second lookup table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position; and the human-machine cooperation factor estimate is determined based on the first human-machine cooperation factor estimate, the second human-machine cooperation factor estimate, and preset parameters.

[0287] When the driving condition is an emergency condition, a multi-dimensional warning is activated.

[0288] In another embodiment provided by the present disclosure, in the synergy factor module 504, the synergy formula is as follows:

[0289] β * =ηβ cur +(1-η)β pre

[0290] Among them, η represents the preset parameters related to the driving group type, β pre Characterizes the human-machine collaboration factor determined by the last decision, β * The human-machine collaboration factor that characterizes this decision, β cur An estimated value of the human-machine collaboration factor that characterizes this decision; and / or

[0291] The comprehensive decision module 505 is used to obtain the decisions of the intelligent driving system and the driver; the human-machine cooperation factor determined in this decision and the decisions of the intelligent driving system and the driver are combined to obtain the comprehensive driving decision through a fusion formula;

[0292] Among them, the individual decisions of the intelligent driving system and the driver as well as the comprehensive driving decision include: torque;

[0293] The above fusion formula is as follows:

[0294] T total =T d +β * T a

[0295] Among them, T total Characterizing comprehensive driving decisions, T a Characterize the decision of the intelligent driving system, T d Characterizes the driver's decision, β * Characterize the human-machine collaboration factor of this decision.

[0296] In another embodiment provided by the present disclosure, the key features include at least one of the following: vehicle lateral deviation, vehicle lateral acceleration, vehicle longitudinal acceleration, positive vehicle longitudinal acceleration, negative vehicle longitudinal acceleration, vehicle lateral shock wave, vehicle longitudinal shock wave, steering wheel angle, steering wheel angular velocity, following vehicle distance, driver's reaction time, frequency of driver checking left and right rearview mirrors, average grip strength of driver operating the steering wheel, driver's facial expression tension factor, driver's annual number of traffic violations, total number of sampling points, peak value of lateral deviation, mean value of lateral deviation, standard deviation of lateral deviation, root mean square error of lateral deviation, peak value of lateral acceleration, mean value of lateral acceleration, standard deviation of lateral acceleration, peak value of longitudinal acceleration, peak value of longitudinal deceleration, frequency of sudden acceleration, frequency of sudden deceleration, peak value of lateral impact degree, peak value of longitudinal impact degree, peak value of steering angle, mean value of steering angle, standard deviation of steering angle, peak value of steering angular velocity, mean value of steering angular velocity, and standard deviation of steering angular velocity; wherein the total number of sampling points is consistent with the width of the sliding window.

[0297] In another embodiment provided by the present disclosure, the above-mentioned device further includes: a pre-processing module, which is used to perform time calibration, time synchronization and filtering on the multi-source data after acquiring the multi-source data during the driving process of the vehicle.

[0298] An embodiment of the present disclosure provides a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of a decision-making method for human-machine co-driving provided by any embodiment of the present disclosure are performed.

[0299] The computer device provided in the embodiments of the present disclosure includes a processor, a memory, and a bus. The memory is used to store and execute instructions, and includes internal memory and external memory. The internal memory is also called internal memory, which is used to temporarily store the calculation data in the processor and the data exchanged with external memory such as a hard disk. The processor exchanges data with the external memory through the internal memory. When the electronic device is running, the processor and the memory communicate through the bus, so that the processor executes the following instructions:

[0300] Acquire multi-source data during vehicle driving;

[0301] Based on multi-source data, the fuzzy inference algorithm and multimodal model are used to identify the driving group type of drivers, and the identification results are fused through the linear weighted fusion algorithm to obtain the driving group type;

[0302] Determining a driving condition based on multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driver's driving group type;

[0303] Combining the estimated value of the human-machine collaboration factor and the human-machine collaboration factor determined in the previous decision, the human-machine collaboration factor of this decision is determined by a collaboration formula;

[0304] Based on the human-machine collaboration factor determined in this decision, the decisions of the intelligent driving system and the driver are integrated to obtain a comprehensive driving decision.

[0305] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of a method for generating a decision for human-machine co-driving provided in any embodiment of the present disclosure. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0306] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented through hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present disclosure.

[0307] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present disclosure.

[0308] Those skilled in the art will appreciate that the modules in the devices of the embodiments may be distributed in the devices of the embodiments as described in the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module or further split into multiple submodules.

[0309] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.

[0310] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A decision-making method for human-machine co-driving, characterized in that: include: Acquire multi-source data during vehicle driving; Based on the multi-source data, the driving group type of the driver is identified using a fuzzy inference algorithm and a multimodal model respectively, and the obtained identification results are fused using a linear weighted fusion algorithm to obtain the driving group type; Determining a driving condition based on the multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driver's driving group type; Combining the estimated value of the human-machine collaboration factor and the human-machine collaboration factor determined in the previous decision, the human-machine collaboration factor of this decision is determined by a collaboration formula; Based on the human-machine collaboration factor determined in this decision, the decisions of the intelligent driving system and the driver are integrated to obtain a comprehensive driving decision.

2. The method according to claim 1, wherein The driving group types include: general driving group, expert driving group and novice driving group; The method of identifying the driving group type of the driver by using a fuzzy inference algorithm based on the multi-source data includes: Setting a width and a step size of a sliding window, and using the sliding window to intercept the multi-source data within a first preset time period multiple times; wherein the starting point of the sliding window is the start time of the first preset time period, and the end point is the end time of the first preset time period; Determining corresponding values ​​of key features for the multi-source data intercepted each time by the sliding window; A fuzzy inference engine constructed using a fuzzy logic system is used to fuzzify the corresponding values ​​of the key features, and fuzzy inference is performed based on a priori fuzzy rules to output a corresponding driving group label and a timestamp; wherein the driving group label corresponds to the driving group type of the driver; The number of clusters is set, and cluster analysis is performed on the driving group types within the second preset time period based on a clustering algorithm to obtain a cluster with the largest total number of the same driving group labels within the cluster, and the driving group type corresponding to the driving group label with the largest total number in the cluster is used as the identification result.

3. The method according to claim 1, wherein The identifying the driving group type of the driver using a multimodal model based on the multi-source data includes: Normalizing the data corresponding to the key features in the multi-source data, and enhancing the data corresponding to the key features using a preset data enhancement algorithm; Determining corresponding advanced features based on the corresponding data of the enhanced key features; wherein the advanced features are extracted from the corresponding data of the key features using a preset deep learning model; The corresponding data of the enhanced key features and the advanced features are input into a preset multimodal model to obtain the corresponding driving group type.

4. The method according to claim 3, wherein The preset multimodal model includes: Differentiate the types of input data and match the corresponding preset neural network models to obtain corresponding dynamic features and / or spatial features for different types of data; Performing feature fusion on the dynamic features and / or spatial features to obtain comprehensive features; The comprehensive features are used to calculate the probability distribution of different group types through the classification layer, and the driving group type with the highest probability is output.

5. The method according to claim 1, wherein The driving conditions include: large curvature driving conditions and small curvature driving conditions; The obtaining of a driving condition based on the multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driving group type of the driver include: determining, based on the road curvature in the multi-source data, whether the driving condition is a large curvature driving condition or a small curvature driving condition; When the driving condition is a high curvature driving condition, obtaining an estimated value of a human-machine cooperation factor using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position; When the driving condition is a small curvature driving condition, an estimated value of the human-machine cooperation factor is obtained using a second lookup table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position.

6. The method according to claim 1, wherein The driving conditions include: high-speed and large-curvature conditions, low-speed and small-curvature conditions, transition conditions, and emergency conditions; The obtaining of a driving condition based on the multi-source data, determining a corresponding lookup table based on the driving condition, and determining an estimated value of a human-machine collaboration factor using the lookup table and the driving group type of the driver include: Determining, based on the vehicle longitudinal speed and road curvature in the multi-source data, whether the driving condition is a high-speed, large-curvature condition, a low-speed, small-curvature condition, a transition condition, or an emergency condition; When the driving condition is a high-speed and high-curvature condition, a first lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the preview point position; When the driving condition is a low-speed, small-curvature condition, a second lookup table is used to obtain an estimated value of the human-machine cooperation factor based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass. When the driving condition is a transition condition, a first human-machine cooperation factor estimate is obtained using a first lookup table based on the driving group type and the comprehensive lateral deviation at the preview point position; a second human-machine cooperation factor estimate is obtained using a second lookup table based on the driving group type and the comprehensive lateral deviation at the vehicle center of mass position; and the human-machine cooperation factor estimate is determined based on the first human-machine cooperation factor estimate, the second human-machine cooperation factor estimate, and preset parameters. When the driving condition is an emergency condition, a multi-dimensional warning is initiated.

7. The method according to claim 1, wherein The collaborative formula is as follows: b * =ηβ cur +(1-n)b pre Among them, η represents the preset parameters related to the driving group type, β pre Characterizes the human-machine collaboration factor determined by the last decision, β * The human-machine collaboration factor that characterizes this decision, β cur An estimated value of the human-machine collaboration factor that characterizes this decision; and / or The human-machine collaboration factor determined based on the current decision integrates the decisions of the intelligent driving system and the driver to obtain a comprehensive driving decision, including: Obtaining respective decisions of the intelligent driving system and the driver; Combining the human-machine collaboration factor determined in the current decision and the decisions of the intelligent driving system and the driver, a comprehensive driving decision is obtained through a fusion formula; The respective decisions of the intelligent driving system and the driver, as well as the comprehensive driving decision, include: torque; The fusion formula is as follows: T total =T d +β * T a Among them, T total Characterizing comprehensive driving decisions, T a Characterize the decision of the intelligent driving system, T d Characterizes the driver's decision, β * Characterize the human-machine collaboration factor of this decision.

8. The method according to claim 2 or 3, wherein: The key features include at least one of the following: vehicle side deviation, vehicle lateral acceleration, vehicle longitudinal acceleration, positive vehicle longitudinal acceleration, negative vehicle longitudinal acceleration, vehicle lateral shock wave, vehicle longitudinal shock wave, steering wheel angle, steering wheel angular velocity, following vehicle distance, driver's reaction time, frequency of driver checking left and right rearview mirrors, average grip strength of driver operating the steering wheel, driver's facial expression tension factor, driver's annual number of traffic violations, total number of sampling points, peak value of side deviation, mean value of side deviation, standard deviation of side deviation, root mean square error of side deviation, peak value of lateral acceleration, mean value of lateral acceleration, standard deviation of lateral acceleration, peak value of longitudinal acceleration, peak value of longitudinal deceleration, frequency of sudden acceleration, frequency of sudden deceleration, peak value of lateral impact degree, peak value of longitudinal impact degree, peak value of steering angle, mean value of steering angle, standard deviation of steering angle, peak value of steering angular velocity, mean value of steering angular velocity, and standard deviation of steering angular velocity; wherein the total number of sampling points is consistent with the width of the sliding window.

9. The method according to claim 1, wherein The method further includes: after acquiring multi-source data during the driving process of the vehicle, performing time calibration, time synchronization and filtering on the multi-source data.

10. A decision-making device for human-machine co-driving, characterized in that: include: A data acquisition module is used to acquire multi-source data during the vehicle's driving process; A driving group module is used to identify the driving group type of the driver using a fuzzy inference algorithm and a multimodal model based on the multi-source data, and to fuse the obtained identification results using a linear weighted fusion algorithm to obtain the driving group type; a query estimation module, configured to obtain a driving condition based on the multi-source data, determine a corresponding query table based on the driving condition, and determine an estimated value of a human-machine collaboration factor using the query table and the driver's driving group type; A synergy factor module is used to combine the estimated value of the human-machine synergy factor and the human-machine synergy factor determined in the previous decision to determine the human-machine synergy factor for this decision through a synergy formula; The comprehensive decision-making module is used to integrate the decisions of the intelligent driving system and the driver based on the human-machine collaboration factor determined by the current decision to obtain a comprehensive driving decision.

Images