A human-machine collaborative steering control method, device, and storage medium

Abstract

The present disclosure provides a human-machine collaborative steering control method, device, and storage medium, including: extracting artificial features from vehicle operation data to obtain first key parameters characterizing the vehicle operation state and driving behavior features; inputting the vehicle operation data and the first key parameters into a driving group identification model to determine the first driving group to which the driver belongs; determining a first human-machine collaboration factor according to the vehicle operation data and the first driving group to which the driver belongs; and controlling the vehicle to steer according to the first human-machine collaboration factor, the first torque for controlling the vehicle steering by the vehicle driving system, and the second torque for controlling the vehicle steering by the driver. By using the driving group identification model to identify the driving group to which the driver belongs and participating in the steering control of the vehicle by the vehicle driving system and the driver, the present disclosure makes an anthropomorphic improvement to the steering control of the vehicle, effectively reducing potential conflicts between humans and machines and enhancing the co-driving experience.

Description

Technical Field

[0001] The present disclosure relates to the field of driverless technologies, and in particular, to a human-machine collaborative steering control method, apparatus, and storage medium. Background Art

[0002] The commercialization process of driverless technologies faces many problems, such as challenges in driving decisions in complex scenarios, imperfections in the new infrastructure of vehicle networking, and the lack of relevant legislation for defining rights and responsibilities. These factors have largely delayed the full implementation of driverless technologies. Therefore, before achieving full-domain driverless, intelligent vehicles will still be in a state of human-machine co-driving for a long time. In this state, during the steering driving process of the vehicle, the incorrect allocation of driving control rights can easily lead to safety accidents. Therefore, there is an urgent need to provide a control method that can efficiently balance the steering control rights between the driver and the vehicle driving system to ensure driving safety. Summary of the Invention

[0003] Embodiments of the present disclosure provide a human-machine collaborative steering control method, apparatus, and storage medium to solve the problem that the incorrect allocation of driving control rights can easily lead to safety accidents in the existing human-machine co-driving state.

[0004] Based on the above problems, in a first aspect, a human-machine collaborative steering control method provided by embodiments of the present disclosure includes:

[0005] Obtain the preprocessed vehicle operation data, and perform artificial feature extraction on the vehicle operation data to obtain first key parameters characterizing the vehicle operation state and driving behavior characteristics;

[0006] Input the vehicle operation data and the first key parameters into a driving group identification model to determine the first driving group to which the driver belongs; wherein, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model; the first driving groups include: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group;

[0007] Determine a first human-machine collaboration factor according to the vehicle operation data and the first driving group to which the driver belongs;

[0008] Determine a combined torque according to the first human-machine collaboration factor, the first torque for the vehicle driving system to control the vehicle steering, and the second torque for the driver to control the vehicle steering, and control the vehicle to steer according to the combined torque.

[0009] In combination with the first aspect, in a possible implementation manner, the inputting the vehicle operation data and the first key parameters into a driving group identification model to determine the first driving group to which the driver belongs includes:

[0010] Input the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs;

[0011] Input the first key parameter into the deep neural network model to determine the second driving group label to which the driver belongs;

[0012] Fuse the fuzzy inference model and the deep neural network model in a linear weighted manner, and determine the first driving group to which the driver belongs according to the first driving group label and the second driving group label.

[0013] Combined with the first aspect, in a possible implementation manner, the inputting the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs includes:

[0014] Perform statistical analysis on the vehicle operation data by using a dynamic sliding window to determine the second key parameter and the first key index calculated based on the second key parameter;

[0015] Pre - construct prior fuzzy rules and a fuzzy inference engine, and input the second key parameter and the first key index into the fuzzy inference engine according to the prior fuzzy rules to determine the first driving group label data set;

[0016] Use the K - means algorithm to cluster the first driving group label data set, and determine the driving group label with the largest number within the K - means algorithm cluster as the first driving group label to which the driver belongs.

[0017] Combined with the first aspect, in a possible implementation manner, the inputting the second key parameter and the first key index into the fuzzy inference engine according to the prior fuzzy rules to determine the first driving group label data set includes:

[0018] Input the second key parameter and the first key index into the fuzzy inference engine according to the prior fuzzy rules to determine the second driving group label data set and the time stamps corresponding to the elements in the second driving group label data set;

[0019] Adjust the data volume of the second driving group label data set according to the time stamps, and determine the adjusted second driving group label data set as the first driving group label data set.

[0020] Combined with the first aspect, in a possible implementation manner, the deep neural network model includes: a deep learning model and a multi - modal model;

[0021] Inputting the first key parameter into the deep neural network model to determine the second driving group label to which the driver belongs, includes:

[0022] After normalizing the first key parameter, input it into the deep learning model to extract the first high-level feature;

[0023] Input the first high-level feature and the first key parameter into the multi-modal model to determine the second driving group label to which the driver belongs; wherein, the multi-modal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model or an SRU network model or a GRU network model.

[0024] Combined with the first aspect, in a possible implementation manner, the determining the first human-machine collaboration factor according to the vehicle operation data and the first driving group to which the driver belongs, includes:

[0025] Extract a third key parameter from the vehicle operation data to determine the first real-time working condition of the vehicle driving;

[0026] Determine the first human-machine collaboration factor according to the first real-time working condition and the first relationship between the first real-time working condition and the human-machine collaboration factor model; wherein, the human-machine collaboration factor model includes: a second human-machine collaboration factor for large curvature road conditions and a third human-machine collaboration factor for small curvature road conditions;

[0027] The formula of the second human-machine collaboration factor is expressed as:

[0028]

[0029] The formula of the third human-machine collaboration factor is expressed as:

[0030]

[0031] Wherein, the characteristic parameter of the first driving group determined according to the first driving group to which the driver belongs is expressed as c 1、 c 2、 k 1、 k 2;

[0032] Determined according to the vehicle operation data:

[0033] The comprehensive lateral deviation at the vehicle centroid position is expressed as: ;

[0034] The comprehensive lateral deviation at the preview point position is expressed as: ;

[0035] The lateral deviation of the vehicle at the vehicle's center of mass position is expressed as ;

[0036] The normalized value of the yaw angle deviation at the vehicle's center of mass position is expressed as ;

[0037] The lateral deviation of the vehicle at the preview point position is expressed as ;

[0038] The normalized value of the yaw angle deviation at the preview point position is expressed as ;

[0039] The first preset value is expressed as , and the second preset value is expressed as , and it satisfies .

[0040] Combined with the first aspect, in a possible implementation manner, the third key parameter includes: a real-time vehicle speed parameter and a real-time road parameter;

[0041] The first real-time driving condition includes: a high-speed large-curvature condition, a low-speed small-curvature condition, a transition condition, and an emergency condition;

[0042] Extracting the third key parameter from the vehicle operation data and determining the first real-time driving condition of the vehicle includes:

[0043] When the real-time road parameter indicates that the road where the vehicle is located does not have steering conditions, determining the first real-time driving condition of the vehicle as an emergency condition;

[0044] When the real-time road parameter indicates that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameter is greater than a third preset value and the real-time vehicle speed parameter is greater than a fourth preset value, determining the first real-time driving condition of the vehicle as a high-speed large-curvature condition;

[0045] When the real-time road parameter indicates that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameter is less than a fifth preset value and the real-time vehicle speed parameter is less than a sixth preset value, determining the first real-time driving condition of the vehicle as a low-speed small-curvature condition;

[0046] When the real-time road parameter indicates that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameter is less than or equal to the third preset value and greater than or equal to the fifth preset value, or the real-time vehicle speed parameter is less than or equal to the fourth preset value and greater than or equal to the sixth preset value, determining the first real-time driving condition of the vehicle as a transition condition;

[0047] Determining a first human-machine collaboration factor according to the first real-time working condition and the first relationship between the first real-time working condition and the human-machine collaboration factor model includes:

[0048] When the first real-time working condition is a high-speed and large-curvature working condition, determining the second human-machine collaboration factor as the first human-machine collaboration factor;

[0049] When the first real-time working condition is a low-speed and small-curvature working condition, determining the third human-machine collaboration factor as the first human-machine collaboration factor;

[0050] When the first real-time working condition is a transition working condition, determining the first human-machine collaboration factor according to the second human-machine collaboration factor and the third human-machine collaboration factor;

[0051] When the first real-time working condition is an emergency working condition, starting a warning.

[0052] Combined with the first aspect, in a possible implementation manner, the driving group identification model is trained as follows:

[0053] Obtaining training data from the vehicle operation data;

[0054] Inputting the training data into the fuzzy inference model for training, and introducing a first feedback mechanism to optimize the fuzzy inference model to obtain a trained fuzzy inference model;

[0055] Performing artificial feature extraction on the training data to obtain first key parameters characterizing the vehicle operation state and driving behavior characteristics, inputting them into the deep neural network model for training, using a hyperparameter tuning method to optimize the performance of the deep neural network model, and using a cross-validation method to evaluate the performance of the deep neural network model to obtain a trained deep neural network model; wherein, the hyperparameter tuning method includes: grid search or random search or Bayesian optimization;

[0056] Fusing the trained fuzzy inference model and the trained deep neural network model in a linear weighted manner to obtain a driving group identification model, and introducing a second feedback mechanism to optimize the driving group identification model to obtain a trained driving group identification model.

[0057] In a second aspect, a human-machine collaborative steering control device is provided, including:

[0058] A key parameter extraction module, configured to obtain preprocessed vehicle operation data, and perform artificial feature extraction on the vehicle operation data to obtain first key parameters characterizing the vehicle operation state and driving behavior characteristics;

[0059] A driving group determination module, configured to input the vehicle operation data and the first key parameter into a driving group identification model to determine the first driving group to which the driver belongs; wherein, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model; the first driving group includes: a steady driving group, an aggressive driving group, a conservative driving group, and a transient driving group;

[0060] A human-machine cooperation factor determination module, configured to determine a first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs;

[0061] A steering control module, configured to determine a combined torque according to the first human-machine cooperation factor, a first torque for the vehicle driving system to control the vehicle steering, and a second torque for the driver to control the vehicle steering, and control the vehicle to steer according to the combined torque.

[0062] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the human-machine cooperation steering control method described in the first aspect or any possible implementation manner in combination with the first aspect.

[0063] The beneficial effects of the embodiments of the present disclosure include:

[0064] The human-machine cooperation steering control method, device, and storage medium provided by the present disclosure include: obtaining preprocessed vehicle operation data, and performing artificial feature extraction on the vehicle operation data to obtain a first key parameter characterizing the vehicle operation state and driving behavior characteristics; inputting the vehicle operation data and the first key parameter into a driving group identification model to determine the first driving group to which the driver belongs; wherein, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model; the first driving group includes: a steady driving group, an aggressive driving group, a conservative driving group, and a transient driving group; determining a first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs; determining a combined torque according to the first human-machine cooperation factor, a first torque for the vehicle driving system to control the vehicle steering, and a second torque for the driver to control the vehicle steering, and controlling the vehicle to steer according to the combined torque. The human-machine cooperation steering control method provided by the embodiments of the present disclosure collects vehicle operation data during the vehicle operation process, uses a driving group identification model to identify the driving group to which the driver belongs, determines the corresponding human-machine cooperation factor, and participates in the steering control of the vehicle by the vehicle driving system and the driver, thereby making an anthropomorphic improvement to the steering control of the vehicle, effectively reducing potential conflicts between humans and machines, and enhancing the co-driving experience. Description of the Drawings

[0065] Figure 1Flowchart of the human-machine collaborative steering control method provided by an embodiment of the present disclosure;

[0066] Figure 2 Flowchart of the synthetic torque control for the vehicle to steer and drive provided by an embodiment of the present disclosure;

[0067] Figure 3 Schematic diagram of the human-machine collaborative factor provided by an embodiment of the present disclosure;

[0068] Figure 4 Structural diagram of the human-machine collaborative steering control device provided by an embodiment of the present disclosure. Detailed implementation manners

[0069] Embodiments of the present disclosure provide a human-machine collaborative steering control method, device, and storage medium. The preferred embodiments of the present disclosure are described below with reference to the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure. And without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0070] Embodiments of the present disclosure provide a human-machine collaborative steering control method, as Figure 1 shown, including the following steps:

[0071] S101. Obtain the preprocessed vehicle operation data, and perform artificial feature extraction on the vehicle operation data to obtain the first key parameters characterizing the vehicle operation state and driving behavior characteristics;

[0072] S102. Input the vehicle operation data and the first key parameters into the driving group identification model to determine the first driving group to which the driver belongs; wherein, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model; the first driving groups include: a steady driving group, an aggressive driving group, a conservative driving group, and a transient driving group;

[0073] S103. Determine the first human-machine collaborative factor according to the vehicle operation data and the first driving group to which the driver belongs;

[0074] S104. Determine the synthetic torque according to the first human-machine collaborative factor, the first torque for the vehicle steering system to control the vehicle steering, and the second torque for the driver to control the vehicle steering, and control the vehicle to steer and drive according to the synthetic torque.

[0075] The disclosed embodiments can be applied to the field of unmanned driving technology, especially in human-machine collaborative driving scenarios. The vehicle driving system can collect vehicle operation data using data acquisition devices such as millimeter-wave radar, laser radar, visual sensor, inertial navigation unit, RSU roadside unit, cloud platform, etc. Using these vehicle operation data, the vehicle driving system can plan the vehicle driving strategy, control the torque of the vehicle steering, and then control the vehicle steering. For example, millimeter-wave radar uses electromagnetic waves in the millimeter-wave frequency band to detect information such as the distance, speed and angle of the target object. In severe weather conditions, such as heavy rain and heavy fog, the performance of millimeter-wave radar is less affected by severe weather, and it can still stably obtain information about surrounding vehicles and obstacles to ensure driving safety. The laser radar can construct a three-dimensional point cloud map of the surrounding environment by emitting a laser beam and measuring the time of reflected light. During the driving process of the vehicle, the laser radar continuously scans the surrounding environment with a high-frequency laser beam, and can monitor the dynamic changes of surrounding objects in real time. When a surrounding vehicle suddenly changes lanes and cuts in front, the LiDAR can capture this change in a very short time, and the vehicle driving system can make timely response strategies, such as adjusting the speed, maintaining a safe distance, or performing steering avoidance operations, effectively avoiding collision accidents, and greatly improving the safety and reliability of autonomous driving. Visual sensors, that is, cameras, can play important roles such as environmental perception, target recognition and tracking, auxiliary decision-making and control, map construction and positioning, and safety monitoring. The inertial navigation unit can provide real-time attitude information such as pitch, yaw and roll of the vehicle by measuring the acceleration and angular velocity of the vehicle. Help the vehicle driving system predict the vehicle's motion state and achieve more precise control. The RSU roadside unit can send road condition information, traffic signal status, etc. to the vehicle to provide assisted driving decision support for the vehicle driving system. The cloud platform can update maps and navigation information in real time, provide accurate route planning services for the vehicle driving system, and can also help vehicles analyze various situations and make decisions faster through the powerful algorithm processing capabilities of cloud computing. However, the commercialization of driverless technology has been hampered to a great extent due to the challenges of driving decisions in complex scenarios, the imperfect new infrastructure of the Internet of Vehicles, and the lack of relevant legislation on the definition of rights and responsibilities. Therefore, before achieving full-domain driverless driving, smart cars will still face a long period of human-machine co-driving. In the human-machine co-driving system, the reasonable allocation of driving control rights has become the key, that is, how to reasonably and efficiently balance the driving control rights between the driver and the vehicle driving system. Especially when the vehicle is turning, the incorrect allocation of driving control rights can easily lead to safety accidents.

[0076] In the embodiments of the present disclosure, the preprocessing of vehicle operation data may include: performing time synchronization, calibration, and filtering on the vehicle operation data. Due to differences in their working principles and operating mechanisms, the data collected by different data acquisition devices may have time deviations. Therefore, time synchronization and calibration can be performed on the vehicle operation data to ensure data accuracy and consistency. For example, millimeter-wave radar, lidar, and vision sensors work simultaneously to collect surrounding environment data. The data update frequency of millimeter-wave radar may be fifty times per second, the data update frequency of lidar is relatively low, and the frame rates of different vision sensors for collecting images also vary. Without time synchronization, these collected vehicle operation data cannot accurately correspond in time, resulting in deviations in the perception of the surrounding environment. By adopting high-precision clock synchronization technologies, such as the precise timekeeping function of the Global Positioning System (GPS) or the network-based time synchronization protocol, the data collected by different devices can be unified to the same time reference. During the process of collecting vehicle operation data, it is also necessary to calibrate the measurement errors of the devices. Taking the vision sensor as an example, due to factors such as lens distortion, the captured images may have geometric distortion. The vision sensor can be calibrated by using a standard calibration object to establish an accurate imaging model, eliminate the errors caused by lens distortion, and improve the accuracy of image data. Advanced filtering algorithms such as Kalman filtering or particle filtering can be used to filter the vehicle operation data after time synchronization and calibration to reduce the influence of noise and outliers. After obtaining the preprocessed vehicle operation data, artificial feature extraction is performed on the vehicle operation data to obtain the first key parameters characterizing the vehicle operation state and driving behavior characteristics. Artificial feature extraction can be a process of constructing physically meaningful and interpretable feature quantities from vehicle operation data through domain knowledge, and the experience of domain experts is required to design and select features to better capture the essence of the problem. For example, in the process of artificial feature extraction, the distribution of vehicle operation data is described through mean, variance, standard deviation, etc., the conversion of vehicle operation data from the time domain to the frequency domain is performed through Fourier transform, wavelet transform, etc., and the dimension of vehicle operation data is reduced through principal component analysis (PCA), independent component analysis (ICA), etc. The first key parameters characterizing the vehicle operation state and driving behavior characteristics may include: the lateral deviation at the vehicle centroid position in the vehicle coordinate system, that is, the side slip is expressed as ; the vehicle longitudinal speed is expressed as ; the vehicle lateral acceleration is expressed as ; the vehicle longitudinal acceleration is expressed as , the vehicle longitudinal acceleration, which is used to describe the characteristics when the driver steps on the accelerator pedal, is expressed as , the vehicle longitudinal deceleration, which is used to describe the characteristics when the driver steps on the brake pedal, is expressed as ; Lateral impact degree of the vehicle, used to describe lateral comfort, expressed as ; Longitudinal impact degree of the vehicle, used to describe longitudinal comfort, expressed as ; Steering wheel angle, used to describe the characteristics when the driver operates the steering wheel, expressed as ; Steering wheel angular speed, used to describe the characteristics when the driver operates the steering wheel, expressed as ; Following distance is expressed as ; Frequency of the driver observing the left and right rearview mirrors is expressed as ; Frequency of the driver turning on the turn signal is expressed as ; Average grip force of the driver operating the steering wheel is expressed as ; Tension factor of the driver's facial expression is expressed as ; Distraction factor of the driver is expressed as ; Fatigue factor of the driver is expressed as ; Duration from the need for emergency braking to the driver stepping on the pedal or a forward collision occurs is expressed as ; Continuous driving duration is expressed as ; Total number of sampling points is expressed as N;

[0077] Peak value of the vehicle's longitudinal speed is expressed as ;

[0078] Mean value of the vehicle's longitudinal speed is expressed as ;

[0079] Standard deviation of the vehicle's longitudinal speed, used to quantify the fluctuation of the vehicle speed curve, is expressed as ;

[0080] Peak value of the sideslip is expressed as ;

[0081] Mean value of the sideslip is expressed as ;

[0082] Standard deviation of the sideslip, used to quantify the fluctuation of the sideslip curve, is expressed as ;

[0083] Root mean square error of the sideslip, used to quantify the trajectory tracking accuracy, is expressed as ;

[0084] Peak value of the lateral acceleration is expressed as ;

[0085] Mean value of the lateral acceleration is expressed as ;

[0086] Standard deviation of the lateral acceleration is expressed as ;

[0087] The peak value of the longitudinal acceleration is expressed as ;

[0088] The peak value of the longitudinal deceleration is expressed as ;

[0089] The frequency of rapid acceleration is expressed as , where is a preset threshold;

[0090] The frequency of rapid deceleration is expressed as , where is a preset threshold;

[0091] The peak value of the lateral impact degree is expressed as ;

[0092] The peak value of the longitudinal impact degree is expressed as ;

[0093] The peak value of the steering angle is expressed as ;

[0094] The mean value of the steering angle is expressed as ;

[0095] The standard deviation of the steering angle, used to quantify the fluctuation of the steering angle curve, is expressed as ;

[0096] The peak value of the steering angular velocity is expressed as ;

[0097] The mean value of the steering angular velocity is expressed as ;

[0098] The standard deviation of the steering angular velocity, used to quantify the fluctuation of the steering angular velocity curve, is expressed as .

[0099] For different drivers, there are significant differences in their driving styles, driving skills, and operating preferences. The first driving groups thus divided can include: steady driving groups, aggressive driving groups, conservative driving groups, and transient driving groups. The transient driving groups can include driving groups in a road rage state, driving groups in a distracted state, and driving groups in a fatigued state. Inputting vehicle operation data and the first key parameters into the driving group identification model can determine the first driving group to which the driver belongs. Among them, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model. During the analysis of driving behavior characteristics, many factors, such as "aggressive" and "conservative", are fuzzy. The fuzzy inference model can be a reasoning method based on fuzzy logic and fuzzy set theory, capable of processing uncertain and fuzzy information. The deep neural network model can learn hierarchical features from low-level sensor data to high-level driving behavior representations through a deep network structure, and can process multi-modal data such as videos and radar point clouds to achieve information complementarity. By fusing the fuzzy inference model and the deep neural network model, the advantages of both can be fully utilized, retaining both the ability of the fuzzy inference model to handle uncertainty and the powerful feature learning ability of the deep neural network model.

[0100] Furthermore, as Figure 2 shown, the target path of the vehicle driving system is usually set as the center line of the lane, and different driving groups have different sensitivities to vehicle sideslip, that is, the vehicle driving system does not exactly coincide with the driver's target path. This also means that different driving groups have different behavioral habits in lane keeping and have different acceptances of the intervention timing and intervention degree of the vehicle driving system. In order to take into account the development trend of the vehicle's lateral deviation and the behavioral habits of the driving group in lane keeping and improve the human-machine co-driving experience, the results of the respective decisions of the driver and the vehicle driving system are fused to form a closed-loop control. The vehicle driving system makes a decision and planning based on the vehicle operation data to obtain the first torque for controlling the vehicle to turn . The vehicle operation data can also be used to measure the working conditions of the vehicle. According to the vehicle operation data and the first driving group to which the driver belongs, the first human-machine cooperation factor is determined. Under different working conditions, the first human-machine cooperation factors determined by different driving groups are different to ensure the safety of vehicle driving. Then, according to the first human-machine cooperation factor, the first torque of the vehicle driving system for controlling the vehicle to turn, and the second torque of the driver for controlling the vehicle to turn, the combined torque is determined, and its formula is expressed as: , where represents the combined torque, represents the first torque, represents the second torque, Indicates the first human-machine collaboration factor. Finally, based on the combined torque Control the vehicle to steer and drive.

[0101] In the embodiment of the present application, in order to meet the personalized operation needs of different driving groups, compared with the prior art, the human-machine collaborative steering control is anthropomorphically improved, and the driving group to which the driver belongs is assigned. Combining the real-time vehicle operation state effectively reduces the potential conflict between the driver and the vehicle driving system and improves the co-driving experience.

[0102] In another embodiment of the present disclosure, in the above step S102, the vehicle operation data and the first key parameter are input into the driving group identification model to determine the first driving group to which the driver belongs, including the following steps:

[0103] Step 1: Input the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs;

[0104] Step 2: Input the first key parameter into the deep neural network model to determine the second driving group label to which the driver belongs;

[0105] Step 3: Use the linear weighting method to fuse the fuzzy inference model and the deep neural network model, and determine the first driving group to which the driver belongs according to the first driving group label and the second driving group label.

[0106] In the embodiments of the present disclosure, a fuzzy inference model and a deep neural network model are fused in a linear weighted manner to construct a driving group identification model, which processes vehicle operation data and the first key parameters to determine the first driving group to which the driver belongs. For the above step 1, the first driving group labels correspond to the first driving groups and may include: a stable driving group label, an aggressive driving group label, a conservative driving group label, and a transient driving group label. The determined first driving group label of the driver may be one of the stable driving group label, the aggressive driving group label, the conservative driving group label, and the transient driving group label. The fuzzy inference model may be an inference model based on fuzzy logic and fuzzy set theory that processes uncertain and fuzzy information. During the determination process of the first driving group label of the driver, due to the large differences in the driving styles, driving skills, operation preferences, etc. of the driver, the vehicle operation data often has uncertainty. For example, data such as driving speed and acceleration may fluctuate due to various factors. Inputting the vehicle operation data into the fuzzy inference model, the fuzzy inference model can well handle this uncertainty, thereby performing online fuzzy identification of the first driving group label of the driver. For the above step 2, the second driving group labels correspond to the first driving groups and may include: a stable driving group label, an aggressive driving group label, a conservative driving group label, and a transient driving group label. The determined second driving group label of the driver may be one of the stable driving group label, the aggressive driving group label, the conservative driving group label, and the transient driving group label. The first key parameters can characterize the vehicle operation state and driving behavior characteristics. Inputting the first key parameters into the deep neural network model to extract high-level features. According to the characteristics of the driving behavior characteristics, a convolutional neural network (CNN) can be selected to extract image features, such as the facial expressions and head postures of the driver; or an SRU network model or a GRU network model, especially a long short-term memory network (LSTM), can be selected to analyze the driving behavior feature sequences, such as the movement trajectories and operation habits of the vehicle. Inputting the first key parameters into the trained deep neural network model can output the probability distribution of each driving group label, and the driving group label with the highest probability in the probability distribution is used as the second driving group label of the driver. For the above step 3, the fuzzy inference model and the deep neural network model are fused in a linear weighted manner. Different weights can be assigned to the fuzzy inference model and the deep neural network model according to different vehicle operation scenarios. For example, in the highway scenario, the weight of the fuzzy inference model is 0.4, and the weight of the deep neural network model is 0.6; in the urban congestion scenario, the weight of the fuzzy inference model is 0.6, and the weight of the deep neural network model is 0.4. Determine the first driving group to which the driver belongs according to the first driving group label and the second driving group label.Exemplarily, if the first driving group label to which the driver belongs determined by the fuzzy inference model is a stable driving group label, and the second driving group label to which the driver belongs determined by the deep neural network model is an aggressive driving group label, and the vehicle is traveling in a highway scenario, then the first driving group to which the driver belongs determined after fusion is the aggressive driving group. Adjusting the weight parameters of this fusion strategy according to the actual vehicle operation scenario can effectively fuse the fuzzy inference model and the deep neural network model, so as to more accurately determine the first driving group to which the driver belongs.

[0107] In another embodiment of the present disclosure, in the above step 1, inputting the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs includes:

[0108] Step 1: Use a dynamic sliding window to statistically analyze the vehicle operation data to determine the second key parameter and the first key index calculated based on the second key parameter;

[0109] Step 2: Pre-construct a priori fuzzy rules and a fuzzy inference engine. According to the a priori fuzzy rules, input the second key parameter and the first key index into the fuzzy inference engine to determine the first driving group label data set;

[0110] Step 3: Use the K-means algorithm to cluster the first driving group label data set, and determine the driving group label with the largest number within the K-means algorithm cluster as the first driving group label to which the driver belongs.

[0111] In the embodiment of the present disclosure, an online fuzzy identification of the first driving group to which the driver belongs is performed using a fuzzy inference model. For the above step 1, the dynamic sliding window technology can automatically adjust the window length N according to different driving scenarios to adapt to the analysis of driving behavior characteristics at different time scales. For example, in the highway cruise scenario, the window length range of the dynamic sliding window is 5 - 10s, in the urban congestion following scenario, the window length range of the dynamic sliding window is 0.5 - 2s, and in the mountain curve driving scenario, the window length range of the dynamic sliding window is 2 - 5s. Statistically analyze the vehicle operation data within the window, record the value of the second key parameter, and calculate the first key index based on the second key parameter. The statistical analysis method can be the same as the artificial feature extraction method. The second key parameter may include: the lateral deviation at the vehicle centroid position in the vehicle coordinate system, that is, the side slip is expressed as ; the vehicle longitudinal speed is expressed as ; the vehicle lateral acceleration is expressed as ; the vehicle longitudinal acceleration is expressed as , the vehicle longitudinal acceleration, which is used to describe the characteristics when the driver steps on the accelerator pedal, is expressed as , the longitudinal deceleration of the vehicle, which is used to describe the characteristics when the driver steps on the brake pedal, is expressed as ; the lateral impact degree of the vehicle, which is used to describe the lateral comfort, is expressed as ; the longitudinal impact degree of the vehicle, which is used to describe the longitudinal comfort, is expressed as ; the steering wheel angle, which is used to describe the characteristics when the driver operates the steering wheel, is expressed as ; the steering wheel angular velocity, which is used to describe the characteristics when the driver operates the steering wheel, is expressed as ; the following distance is expressed as ; the frequency of the driver observing the left and right rearview mirrors is expressed as ; the frequency of the driver turning on the turn signal is expressed as ; the average grip force of the driver operating the steering wheel is expressed as ; the facial expression tension factor of the driver is expressed as ; the distraction factor of the driver is expressed as ; the fatigue factor of the driver is expressed as ; the duration from the need for emergency braking to the driver stepping on the pedal or a forward collision occurs is expressed as ; the continuous driving duration is expressed as ; the total number of sampling points is expressed as N.

[0112] The first key indicators may include:

[0113] The peak value of the vehicle longitudinal speed is expressed as ;

[0114] The mean value of the vehicle longitudinal speed is expressed as ;

[0115] The standard deviation of the vehicle longitudinal speed, which is used to quantify the fluctuation of the vehicle speed curve, is expressed as ;

[0116] The peak value of the sideslip is expressed as ;

[0117] The mean value of the sideslip is expressed as ;

[0118] The standard deviation of the sideslip, which is used to quantify the fluctuation of the sideslip curve, is expressed as ;

[0119] The root mean square error of the sideslip, which is used to quantify the trajectory tracking accuracy, is expressed as ;

[0120] The peak value of the lateral acceleration is expressed as ;

[0121] The mean value of the lateral acceleration is expressed as ;

[0122] The standard deviation of the lateral acceleration is expressed as ;

[0123] The peak value of the longitudinal acceleration is expressed as ;

[0124] The peak value of the longitudinal deceleration is expressed as ;

[0125] The frequency of rapid acceleration is expressed as , where is a preset threshold;

[0126] The frequency of rapid deceleration is expressed as , where is a preset threshold;

[0127] The peak value of the lateral jerk is expressed as ;

[0128] The peak value of the longitudinal jerk is expressed as ;

[0129] The peak value of the steering angle is expressed as ;

[0130] The mean value of the steering angle is expressed as ;

[0131] The standard deviation of the steering angle, which is used to quantify the fluctuation of the steering angle curve, is expressed as ;

[0132] The peak value of the steering angular velocity is expressed as ;

[0133] The mean value of the steering angular velocity is expressed as ;

[0134] The standard deviation of the steering angular velocity, which is used to quantify the fluctuation of the steering angular velocity curve, is expressed as .

[0135] For the above-mentioned step 2, the fuzzy inference model may include: prior fuzzy rules and a fuzzy inference engine. The prior fuzzy rules can be established based on domain knowledge, such as the experience of driving behavior experts or historical data statistics. Exemplarily, the pre-constructed prior fuzzy rules shown in Table 1. The form of the fuzzy rule is "if the input variable satisfies certain conditions, then the output variable belongs to a certain fuzzy set". For example, "if the vehicle speed is over the speed limit, then the first driving group label is the label of the aggressive driving group".

[0136]

[0137] Table 1

[0138] An advanced fuzzy logic system such as the Mamdani model or the Sugeno model can be used to construct a fuzzy inference engine. The input variables of the fuzzy inference engine are the second key parameter and the first key indicator, and the output variable is the first driving group label dataset. The prior fuzzy rules define fuzzy sets for each input variable and output variable, such as moderate speed, low speed, speeding, etc., and appropriate membership functions are selected, such as triangular, trapezoidal or Gaussian membership functions, to describe the degree to which a variable belongs to different fuzzy sets. The fuzzy inference engine makes inferences based on the prior fuzzy rules and the input variables, obtains the membership degree or exact value of each driving group label, and stores the label in the dataset to obtain the first driving group label dataset.

[0139] For the above step three, the K-means algorithm is used to cluster the first driving group label dataset. The elbow method or the silhouette coefficient can be used to determine the number of clusters K. K data points are randomly selected from the first driving group label dataset as the initial cluster centers. The Euclidean distance from each data point to each cluster center is iteratively calculated, and the data point is assigned to the cluster corresponding to the nearest cluster center. For each cluster, its cluster center is recalculated, that is, the mean of all data points within the cluster, until the change in the cluster center is less than the preset threshold or the maximum number of iterations is reached, and the iteration stops. For each cluster, the number of each driving group label in it is counted, and the driving group label with the largest number within the K-means algorithm cluster is determined as the first driving group label to which the driver belongs. Exemplarily, as shown in Table 2, the number of conservative driving group labels in cluster ID2 is the largest, so the conservative driving group label corresponding to cluster ID2 is determined as the first driving group label to which the driver belongs.

[0140]

[0141] Table 2

[0142] Combined with the prior fuzzy rules, the fuzzy inference model enables the system to have the dual advantages of data-driven and expert knowledge. By using K-means clustering for label distribution statistics, drivers can be automatically classified into the most representative driving groups.

[0143] In another embodiment of the present disclosure, in the above step two, according to the prior fuzzy rules, the second key parameter and the first key indicator are input into the fuzzy inference engine to determine the first driving group label dataset, including:

[0144] Step (1): According to the prior fuzzy rules, input the second key parameter and the first key index into the fuzzy inference engine to determine the second driving group label dataset and the timestamps corresponding to the elements in the second driving group label dataset.

[0145] Step (2): Adjust the data volume of the second driving group label dataset according to the timestamps, and determine the adjusted second driving group label dataset as the first driving group label dataset.

[0146] In the embodiment of the present disclosure, the output result of the fuzzy inference engine also includes timestamps, and the data volume of the first driving group label dataset is determined according to the timestamps. For the above step (1), the input variables of the fuzzy inference engine are the second key parameter and the first key index, and the output variables are the second driving group label dataset and the timestamps corresponding to the elements in the second driving group label dataset. The fuzzy inference engine performs inference based on the prior fuzzy rules and the input variables to obtain the membership degree or exact value of each driving group label, and stores the label in the dataset to obtain the second driving group label dataset and the timestamps corresponding to the elements in the second driving group label dataset. For the above step (2), in order to improve the rapid identification ability under complex conditions such as group evolution or driving state changes, the data volume of the second driving group label dataset is adjusted according to the timestamps. For example, the data volume within the last timestamp interval of 2 s is retained. The adjusted second driving group label dataset can be determined as the first driving group label dataset. The second driving group label dataset is generated by processing real-time data through the fuzzy inference engine, and then the data volume is dynamically aligned and adjusted based on the timestamps, and finally the first driving group label dataset with high consistency is output, improving the rapid identification ability under complex conditions such as group evolution or driving state changes.

[0147] In another embodiment of the present disclosure, the deep neural network model includes: a deep learning model and a multi-modal model;

[0148] In the above step 2, inputting the first key parameter into the deep neural network model to determine the second driving group label to which the driver belongs includes the following steps:

[0149] Step 1: After normalizing the first key parameter, input it into the deep learning model to extract the first high-level feature;

[0150] Step 2: Input the first high-level feature and the first key parameter into the multi-modal model to determine the second driving group label to which the driver belongs; wherein, the multi-modal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model or an SRU network model or a GRU network model.

[0151] In the embodiments of the present disclosure, through the advanced feature extraction of the deep learning model and the multi-modal model fusion, the temporal and image data can be integrated to accurately determine the second driving group label to which the driver belongs. For the above-mentioned step one, the first key parameter is normalized, for example, the first key parameter is scaled to the range of [0, 1]. The normalized data is input into the deep learning model to extract the first advanced feature. The deep learning model can be a CNN convolutional neural network model or a GRU recurrent neural network model. The first key parameter after normalization is learned by using a deep learning model such as a CNN convolutional neural network model or a GRU recurrent neural network model to extract advanced features and output the first advanced feature. The CNN convolutional neural network model (Convolutional Neural Network) includes a convolutional layer, an activation function layer, a pooling layer, and a fully connected layer for feature extraction. The GRU recurrent neural network model (Gated Recurrent Unit) can be a variant of an RNN recurrent neural network model. The traditional RNN recurrent neural network model is prone to problems of gradient disappearance or gradient explosion when processing long sequence data, while the GRU recurrent neural network model solves these problems by introducing a gating mechanism.

[0152] For the above step 2, input the first high-level feature and the first key parameter into the multi-modal model. The multi-modal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data. The network model for processing time series data includes: an LSTM network model or an SRU network model or a GRU network model. For the time series data in the first high-level feature and the first key parameter, select an LSTM network model or an SRU network model or a GRU network model; for the image data in the first high-level feature and the first key parameter, select a CNN convolutional neural network model. For example, the time series data can be vehicle speed, acceleration, etc., and the image data can be the facial expression image of the driver. The LSTM network model (Long Short-Term Memory) can be an improved variant of the RNN recurrent neural network model, which solves the problem of gradient disappearance or explosion of the traditional RNN recurrent neural network model through a gating mechanism and cell state, and is good at modeling long sequence dependencies. The SRU network model can be a lightweight recurrent unit, which significantly improves the training speed through simplified gating calculations and highly parallel designs, and is suitable for processing ultra-long sequences. The deep neural network model can output the probability distribution of each driving group label. According to the output probability distribution, select the one with the highest probability among the stable driving group label, the aggressive driving group label, the conservative driving group label, and the transient driving group label as the second driving group label to which the driver belongs. The first key parameter can be obtained by manually extracting features from the vehicle operation data. Input the first high-level feature and the first key parameter into the multi-modal model to form a mixed input data of "manual feature + learning feature", which improves the representation ability while maintaining interpretability.

[0153] In another embodiment of the present disclosure, in the above step S103, determining the first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs includes the following steps:

[0154] Step 1, extract the third key parameter from the vehicle operation data and determine the first real-time working condition of the vehicle driving;

[0155] Step 2, determine the first human-machine cooperation factor according to the first real-time working condition and the first relationship between the first real-time working condition and the human-machine cooperation factor model; wherein, the human-machine cooperation factor model includes: a second human-machine cooperation factor for large curvature road conditions and a third human-machine cooperation factor for small curvature road conditions;

[0156] The formula for the second human-machine cooperation factor is:

[0157]

[0158] The formula for the third human-machine cooperation factor is:

[0159]

[0160] Among them, the characteristic parameters of the first driving group determined according to the first driving group to which the driver belongs are expressed as c 1. c 2. k 1. k 2;

[0161] Determined according to the vehicle operation data:

[0162] The comprehensive lateral deviation at the vehicle's center of mass position is expressed as: ;

[0163] The comprehensive lateral deviation at the preview point position is expressed as: ;

[0164] The vehicle lateral deviation at the vehicle's center of mass position is expressed as ;

[0165] The normalized value of the yaw angle deviation at the vehicle's center of mass position is expressed as ;

[0166] The vehicle lateral deviation at the preview point position is expressed as ;

[0167] The normalized value of the yaw angle deviation at the preview point position is expressed as ;

[0168] The first preset value is expressed as , and the second preset value is expressed as , and it satisfies .

[0169] In the embodiments of the present disclosure, a third key parameter is extracted from the vehicle operation data to determine the first real-time working condition, and according to the first relationship between the first real-time working condition and the human-machine cooperation factor model, the first human-machine cooperation factor is determined. For the above step 1, a third key parameter is extracted from the vehicle operation data to determine the first real-time working condition of the vehicle. The third key parameter may include a real-time vehicle speed parameter and a real-time road parameter. The third key parameter can be extracted from the vehicle operation data by means of artificial feature extraction. The first real-time working condition characterizes whether the vehicle has steering conditions on the current road. According to the real-time vehicle speed parameter and the real-time road parameter, the first real-time working condition of the vehicle can be determined. For the above step 2, according to the first real-time working condition and the first relationship between the first real-time working condition and the human-machine cooperation factor model, the first human-machine cooperation factor is determined. The human-machine cooperation factor model includes: a second human-machine cooperation factor for large curvature road conditions and a third human-machine cooperation factor for small curvature road conditions. The formula for the second human-machine cooperation factor is expressed as:

[0170]

[0171] The formula of the third human-machine collaboration factor is expressed as:

[0172]

[0173] Among them, the characteristic parameters of the first driving group determined according to the driving group to which the driver belongs are expressed as c 1. c 2. k 1. k 2; for example, the human-machine collaboration factor models for different driving groups, such as Figure 3 shown, where represents the width of the comfortable driving area, and the selection of this value is related to the lane-keeping habits of the driving group. The corresponding driving group characteristic parameters for different driving groups are as follows. A feedback mechanism can be introduced to adjust these characteristic parameters according to the actual application effect.

[0174] (a) For road conditions with large curvature

[0175] ① For the aggressive driving group, the characteristic parameters satisfy:

[0176] ② For the steady driving group, the characteristic parameters satisfy:

[0177] ③ For the conservative driving group, the characteristic parameters satisfy:

[0178] ④ For the transient driving group, the characteristic parameters are dynamically changing. In the road rage state, the characteristic parameters are the same as those of the aggressive driving group; in the distracted state, the characteristic parameters are the same as those of the steady driving group; in the fatigued state, the characteristic parameters are the same as those of the conservative driving group.

[0179] (b) For road conditions with small curvature

[0180] ① For the aggressive driving group, the characteristic parameters satisfy:

[0181] ② For the steady driving group, the characteristic parameters satisfy:

[0182] ③ For the conservative driving group, the characteristic parameters satisfy:

[0183] ④ For the transient driving group, the characteristic parameters are dynamically changing. In the road rage state, the characteristic parameters are the same as those of the aggressive driving group; in the distracted state, the characteristic parameters are the same as those of the steady driving group; in the fatigued state, the characteristic parameters are the same as those of the conservative driving group.

[0184] The following parameters can be determined from the vehicle operation data through manual feature extraction:

[0185] The comprehensive lateral deviation at the vehicle's center of mass position is expressed as: ;

[0186] The comprehensive lateral deviation at the preview point position is expressed as: ;

[0187] The vehicle lateral deviation at the vehicle's center of mass position is expressed as ;

[0188] The normalized value of the yaw angle deviation at the vehicle's center of mass position is expressed as ;

[0189] The vehicle lateral deviation at the preview point position is expressed as ;

[0190] The normalized value of the yaw angle deviation at the preview point position is expressed as ;

[0191] The first preset value is expressed as , and the second preset value is expressed as , and satisfies .

[0192] In another embodiment of the present disclosure,

[0193] In the embodiment of the present disclosure, when designing the human-machine cooperation factor model, in order to further improve the friendliness, comfort and stability of the human-machine cooperative steering control, in addition to classifying the road conditions into large curvature conditions and small curvature conditions according to the road curvature size and designing the corresponding human-machine cooperation factor models, it is also necessary to consider the vehicle's longitudinal speed and factors such as frequent switching between conditions. The third key parameter includes: real-time vehicle speed parameter and real-time road parameter. The first real-time condition includes: high-speed large curvature condition, low-speed small curvature condition, transition condition and emergency condition. For the above step one, when the real-time road parameter indicates that the road where the vehicle is located does not have steering conditions, for example, when conditions such as lane change, overtaking, U-turn or curve steering are not available, it is determined that the first real-time condition of the vehicle driving is an emergency condition. For the above step two, when the real-time vehicle speed parameter and the road curvature value in the real-time road parameter , it is determined as a high-speed large curvature condition; for the above step three, when the real-time vehicle speed parameter and the road curvature value in the real-time road parameter , it is determined as a low-speed small curvature condition; for the above step four, in other cases, it is determined as a transition condition. Wherein is the third preset value, is the fourth preset value, is the fifth preset value, is the sixth preset value. The initial first human-machine collaboration factor is the third human-machine collaboration factor, that is . For the above step 5, when the first real-time working condition is a high-speed and large-curvature working condition, the second human-machine collaboration factor is determined as the first human-machine collaboration factor, that is . For the above step 6, when the first real-time working condition is a low-speed and small-curvature working condition, the third human-machine collaboration factor is determined as the first human-machine collaboration factor, that is . For the above step 7, when the first real-time working condition is a transition working condition, the first human-machine collaboration factor is determined according to the second human-machine collaboration factor and the third human-machine collaboration factor. Exemplarily, the first human-machine collaboration factor , where , are all preset values, satisfying . For the above step 8, when the first real-time working condition is an emergency working condition, an alarm is activated, for example, an auditory-visual-tactile multi-dimensional alarm strategy is activated. Determining the first human-machine collaboration factor by combining the real-time vehicle speed parameter and the real-time road parameter can ensure the safety of vehicle operation.

[0194] In another embodiment of the present disclosure, the driving group identification model is trained in the following manner:

[0195] Step 1: Obtain the data to be trained from the vehicle operation data;

[0196] Step 2: Input the data to be trained into the fuzzy inference model for training, and introduce a first feedback mechanism to optimize the fuzzy inference model to obtain a trained fuzzy inference model;

[0197] Step 3: Manually extract features from the data to be trained to obtain the first key parameters characterizing the vehicle operation state and driving behavior features, input them into the deep neural network model for training, use the hyperparameter tuning method to optimize the performance of the deep neural network model, and use the cross-validation method to evaluate the performance of the deep neural network model to obtain a trained deep neural network model; among them, the hyperparameter tuning method includes: grid search or random search or Bayesian optimization;

[0198] Step 4: Use the linear weighting method to fuse the trained fuzzy inference model and the trained deep neural network model to obtain the driving group identification model, and introduce a second feedback mechanism to optimize the driving group identification model to obtain a trained driving group identification model.

[0199] In the embodiments of the present disclosure, training data is obtained from vehicle operation data, and a feedback mechanism is introduced to train the driving group identification model. For step 1 above, training data is obtained from vehicle operation data, and according to the characteristics of the driving behavior of drivers, the training data is labeled with corresponding driving groups, such as smooth driving groups, aggressive driving groups, conservative driving groups, and transient driving groups. The training data can be divided into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the model parameters and evaluate the model performance, and the test set is used to finally evaluate the generalization ability of the model. For step 2 above, the training data is input into the fuzzy inference model for training, and a first feedback mechanism is introduced to optimize the fuzzy inference model, and a trained fuzzy inference model is obtained. The first feedback mechanism can include optimizing the fuzzy inference model according to the actual application effect by means of expert review or scoring by end users.

[0200] For the above-mentioned step 3, artificial feature extraction is performed on the data to be trained to obtain the first key parameters characterizing the vehicle running state and driving behavior characteristics, and then input into the deep neural network model for training. After normalizing the first key parameters, data augmentation techniques such as rotation, cropping, scaling, and time translation can be used to increase data diversity, and then input into the deep neural network model for training to improve the generalization ability of the deep neural network model. Hyperparameter tuning methods are used to optimize the performance of the deep neural network model. The hyperparameter tuning methods include: grid search, random search, or Bayesian optimization. Grid search can be the most intuitive hyperparameter tuning method, which finds the optimal solution by exhaustively enumerating all possible hyperparameter combinations. Define a discrete value range for each hyperparameter, then traverse these combinations and evaluate each combination through cross-validation. Random search finds the optimal solution by randomly sampling hyperparameter combinations, randomly extracting a certain number of parameter combinations from the parameter distribution, and then evaluating each combination through cross-validation. Bayesian optimization is an optimization method based on a probability model, which constructs a probability model to predict the performance of hyperparameters and selects the hyperparameter combination most likely to improve performance for evaluation. The cross-validation method is used to evaluate the performance of the deep neural network model. The cross-validation method can divide the data set into multiple subsets, and then train and validate on different subsets to more accurately evaluate the generalization ability of the model. Finally, the trained deep neural network model is obtained. As the data to be trained continues to be enriched, an online learning algorithm can be used to update the deep neural network model in real time to adapt to new driving environments and driving group characteristics. Regularly evaluate the performance of the deep neural network model, and adjust and optimize the deep neural network model according to the evaluation results. Collect user feedback data to further improve the accuracy and reliability of the deep neural network model. When using the trained deep neural network model to classify driving groups, a continuous determination mechanism or a scoring mechanism can be introduced to improve the accuracy and reliability of driving group classification. Considering the evolution and variability of driving groups, the thresholds and determination strategies for driving group classification can be dynamically adjusted to adapt to changes in different time periods and driving environments.

[0201] For the above-mentioned step 4, the trained fuzzy inference model and the trained deep neural network model are fused using a linear weighting method to obtain a driving group identification model. A second feedback mechanism is introduced to optimize the driving group identification model to obtain the trained driving group identification model. For example, during the operation of the driving group identification model, feedback data of the driver, such as the operation data of the driver, etc., are collected. This data can be used to adjust the parameters of the model to improve the real-time performance and adaptability of the model, and obtain the trained driving group identification model. The data to be trained is obtained from the vehicle operation data, and a feedback mechanism is introduced to train the driving group identification model to improve the accuracy of the driving group identification model.

[0202] Based on the same inventive concept, embodiments of the present disclosure also provide a human-machine collaborative steering control device. Since the principles of the problems solved by these devices are similar to those of the aforementioned human-machine collaborative steering control method, the implementation of this device can refer to the implementation of the aforementioned method, and the repeated parts will not be elaborated again.

[0203] Embodiments of the present disclosure provide a human-machine collaborative steering control device, as Figure 4 shown, including:

[0204] A key parameter extraction module 401, configured to obtain preprocessed vehicle operation data, and perform artificial feature extraction on the vehicle operation data to obtain first key parameters characterizing the vehicle operation state and driving behavior characteristics;

[0205] A driving group determination module 402, configured to input the vehicle operation data and the first key parameters into a driving group identification model to determine the first driving group to which the driver belongs; wherein, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model; the first driving groups include: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group;

[0206] A human-machine collaboration factor determination module 403, configured to determine a first human-machine collaboration factor according to the vehicle operation data and the first driving group to which the driver belongs;

[0207] A steering control module 404, configured to determine a combined torque according to the first human-machine collaboration factor, the first torque for the vehicle driving system to control vehicle steering, and the second torque for the driver to control vehicle steering, and control the vehicle to steer and drive according to the combined torque.

[0208] In another embodiment of the present disclosure, the driving group determination module 402 is configured to input the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs;

[0209] Input the first key parameters into the deep neural network model to determine the second driving group label to which the driver belongs;

[0210] Fuse the fuzzy inference model and the deep neural network model in a linear weighted manner, and determine the first driving group to which the driver belongs according to the first driving group label and the second driving group label.

[0211] In another embodiment of the present disclosure, the driving group determination module 402 is configured to perform statistical analysis on the vehicle operation data by using a dynamic sliding window, determine a second key parameter and a first key index calculated based on the second key parameter;

[0212] A priori fuzzy rules and a fuzzy inference engine are pre-constructed. According to the a priori fuzzy rules, the second key parameter and the first key index are input into the fuzzy inference engine to determine a first driving group label data set;

[0213] The K-means algorithm is used to cluster the first driving group label data set, and the driving group label with the largest number within the K-means algorithm cluster is determined as the first driving group label to which the driver belongs.

[0214] In another embodiment of the present disclosure, the driving group determination module 402 is configured to input the second key parameter and the first key index into the fuzzy inference engine according to the a priori fuzzy rules to determine a second driving group label data set and the time stamps corresponding to the elements in the second driving group label data set;

[0215] The data volume of the second driving group label data set is adjusted according to the time stamps, and the adjusted second driving group label data set is determined as the first driving group label data set.

[0216] In another embodiment of the present disclosure, the deep neural network model includes: a deep learning model and a multi-modal model;

[0217] The driving group determination module 402 is configured to perform normalization processing on the first key parameter and then input it into the deep learning model to extract first-level advanced features;

[0218] The first-level advanced features and the first key parameter are input into the multi-modal model to determine the second driving group label to which the driver belongs; wherein, the multi-modal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model or an SRU network model or a GRU network model.

[0219] In another embodiment of the present disclosure, the human-machine collaboration factor determination module 403 is configured to extract a third key parameter from the vehicle operation data and determine a first real-time working condition of the vehicle driving;

[0220] Determine the first human-machine collaboration factor according to the first real-time working condition and the first relationship between the first real-time working condition and the human-machine collaboration factor model; wherein, the human-machine collaboration factor model includes: a second human-machine collaboration factor for large curvature road conditions and a third human-machine collaboration factor for small curvature road conditions;

[0221] The formula of the second human-machine collaboration factor is expressed as:

[0222]

[0223] The formula of the third human-machine collaboration factor is expressed as:

[0224]

[0225] Wherein, the characteristic parameters of the first driving group determined according to the first driving group to which the driver belongs are expressed as c 1、 c 2、 k 1、 k 2;

[0226] Determined according to the vehicle operation data:

[0227] The comprehensive lateral deviation at the vehicle centroid position is expressed as: ;

[0228] The comprehensive lateral deviation at the preview point position is expressed as: ;

[0229] The vehicle lateral deviation at the vehicle centroid position is expressed as ;

[0230] The normalized value of the yaw angle deviation at the vehicle centroid position is expressed as ;

[0231] The vehicle lateral deviation at the preview point position is expressed as ;

[0232] The normalized value of the yaw angle deviation at the preview point position is expressed as ;

[0233] The first preset value is expressed as , the second preset value is expressed as , and satisfies .

[0234] In another embodiment of the present disclosure, the third key parameter includes: a real-time vehicle speed parameter and a real-time road parameter;

[0235] The first real-time working condition includes: high-speed large curvature condition, low-speed small curvature condition, transition condition and emergency condition;

[0236] The human-machine collaboration factor determination module 403 is configured to

[0237] When the real-time road parameters indicate that the road where the vehicle is located does not have steering conditions, determine that the first real-time driving condition of the vehicle is an emergency condition;

[0238] When the real-time road parameters indicate that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameters is greater than a third preset value and the real-time vehicle speed parameter is greater than a fourth preset value, determine that the first real-time driving condition of the vehicle is a high-speed large-curvature condition;

[0239] When the real-time road parameters indicate that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameters is less than a fifth preset value and the real-time vehicle speed parameter is less than a sixth preset value, determine that the first real-time driving condition of the vehicle is a low-speed small-curvature condition;

[0240] When the real-time road parameters indicate that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameters is less than or equal to the third preset value and greater than or equal to the fifth preset value, or the real-time vehicle speed parameter is less than or equal to the fourth preset value and greater than or equal to the sixth preset value, determine that the first real-time driving condition of the vehicle is a transition condition;

[0241] The human-machine collaboration factor determination module 403 is configured to

[0242] When the first real-time driving condition is a high-speed large-curvature condition, determine the second human-machine collaboration factor as the first human-machine collaboration factor;

[0243] When the first real-time driving condition is a low-speed small-curvature condition, determine the third human-machine collaboration factor as the first human-machine collaboration factor;

[0244] When the first real-time driving condition is a transition condition, determine the first human-machine collaboration factor according to the second human-machine collaboration factor and the third human-machine collaboration factor;

[0245] When the first real-time driving condition is an emergency condition, start a warning.

[0246] In another embodiment of the present disclosure, the driving group determination module 402 is configured to train the driving group identification model in the following manner:

[0247] Obtain training data from the vehicle operation data;

[0248] Input the training data into the fuzzy inference model for training, and introduce a first feedback mechanism to optimize the fuzzy inference model to obtain a trained fuzzy inference model;

[0249] Artificial feature extraction is performed on the data to be trained to obtain first key parameters characterizing the vehicle running state and driving behavior characteristics, which are input into the deep neural network model for training. The performance of the deep neural network model is optimized by using a hyperparameter tuning method, and the performance of the deep neural network model is evaluated by using a cross-validation method to obtain a trained deep neural network model; wherein, the hyperparameter tuning method includes: grid search or random search or Bayesian optimization;

[0250] The trained fuzzy inference model and the trained deep neural network model are fused in a linear weighted manner to obtain a driving group identification model, and a second feedback mechanism is introduced to optimize the driving group identification model to obtain a trained driving group identification model.

[0251] Based on the same general inventive concept, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the human-machine collaborative steering control method described in any one of the above embodiments.

[0252] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by hardware or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present disclosure.

[0253] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present disclosure.

[0254] Those skilled in the art can understand that the modules in the device in the embodiments can be distributed in the device in the embodiments according to the description of the embodiments, or can be correspondingly changed and located in one or more devices different from the present embodiment. The modules of the above embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0255] The serial numbers of the above embodiments of the present disclosure are only for description and do not represent the advantages and disadvantages of the embodiments.

[0256] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure is also intended to include these modifications and variations.

Claims

1. A human-machine collaborative steering control method, characterized in that, Including: Obtain the preprocessed vehicle operation data, and perform manual feature extraction on the vehicle operation data to obtain the first key parameters characterizing the vehicle operation state and driving behavior characteristics; Input the vehicle operation data and the first key parameters into the driving group identification model to determine the first driving group to which the driver belongs; wherein, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model; the first driving group includes: a steady driving group, an aggressive driving group, a conservative driving group, and a transient driving group; Determine the first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs; The step of determining the first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs includes: Extract the third key parameter from the vehicle operation data to determine the first real-time working condition of the vehicle driving; Determine the first human-machine cooperation factor according to the first real-time working condition and the first relationship between the first real-time working condition and the human-machine cooperation factor model; wherein, the human-machine cooperation factor model includes: a second human-machine cooperation factor for large curvature road conditions and a third human-machine cooperation factor for small curvature road conditions; The formula of the second human-machine cooperation factor is expressed as: The formula of the third human-machine cooperation factor is expressed as: Among them, the characteristic parameters of the first driving group determined according to the first driving group to which the driver belongs are expressed as c 1、 c 2、 k 1、 k 2; Determined according to the vehicle operation data: The comprehensive lateral deviation at the vehicle's center of mass is expressed as: ; The comprehensive lateral deviation at the preview point position is expressed as: ; The lateral deviation of the vehicle at the vehicle's center of mass is expressed as ; The normalized value of the yaw angle deviation at the vehicle's center of mass is expressed as ; The lateral deviation of the vehicle at the preview point position is expressed as ; The normalized value of the yaw angle deviation at the preview point position is expressed as ; The first preset value is represented as , the second preset value is represented as , and it satisfies ; Determine the combined torque according to the first human-machine cooperation factor, the first torque for controlling the vehicle steering by the vehicle driving system, and the second torque for controlling the vehicle steering by the driver, and control the vehicle to steer and drive according to the combined torque.

2. The method according to claim 1, wherein The step of inputting the vehicle operation data and the first key parameters into the driving group identification model to determine the first driving group to which the driver belongs includes: Input the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs; Input the first key parameters into the deep neural network model to determine the second driving group label to which the driver belongs; Fuse the fuzzy inference model and the deep neural network model in a linear weighted manner, and determine the first driving group to which the driver belongs according to the first driving group label and the second driving group label.

3. The method according to claim 2, wherein The step of inputting the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs includes: Perform statistical analysis on the vehicle operation data using a dynamic sliding window to determine the second key parameter and the first key index calculated based on the second key parameter; Pre-construct prior fuzzy rules and a fuzzy inference engine, and input the second key parameter and the first key index into the fuzzy inference engine according to the prior fuzzy rules to determine the first driving group label data set; Cluster the first driving group label data set using the K-means algorithm, and determine the driving group label with the largest number within the K-means algorithm cluster as the first driving group label to which the driver belongs.

4. The method according to claim 3, characterized in that, According to the prior fuzzy rules, input the second key parameter and the first key index into the fuzzy inference machine to determine the first driver group label dataset, including: According to the prior fuzzy rules, input the second key parameter and the first key index into the fuzzy inference machine to determine the second driver group label dataset and the timestamps corresponding to the elements in the second driver group label dataset; Adjust the data volume of the second driver group label dataset according to the timestamps, and determine the adjusted second driver group label dataset as the first driver group label dataset.

5. The method according to claim 2, wherein The deep neural network model includes: a deep learning model and a multi-modal model; The step of inputting the first key parameter into the deep neural network model to determine the second driver group label to which the driver belongs includes: After normalizing the first key parameter, input it into the deep learning model to extract the first high-level feature; Input the first high-level feature and the first key parameter into the multi-modal model to determine the second driver group label to which the driver belongs; wherein, the multi-modal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model or an SRU network model or a GRU network model.

6. The method according to claim 1, wherein The third key parameter includes: a real-time vehicle speed parameter and a real-time road parameter; The first real-time driving condition includes: a high-speed large curvature condition, a low-speed small curvature condition, a transition condition, and an emergency condition; The step of extracting the third key parameter from the vehicle operation data to determine the first real-time driving condition of the vehicle includes: When the real-time road parameter indicates that the road where the vehicle is located does not have steering conditions, determine that the first real-time driving condition of the vehicle is an emergency condition; When the real-time road parameter indicates that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameter is greater than a third preset value and the real-time vehicle speed parameter is greater than a fourth preset value, determine that the first real-time driving condition of the vehicle is a high-speed large curvature condition; When the real-time road parameter indicates that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameter is less than a fifth preset value and the real-time vehicle speed parameter is less than a sixth preset value, determine that the first real-time driving condition of the vehicle is a low-speed small curvature condition; When the real-time road parameter indicates that the road where the vehicle is located has steering conditions, and the road curvature value in the real-time road parameter is less than or equal to the third preset value and greater than or equal to the fifth preset value, or the real-time vehicle speed parameter is less than or equal to the fourth preset value and greater than or equal to the sixth preset value, determine that the first real-time driving condition of the vehicle is a transition condition; The step of determining the first human-machine collaboration factor according to the first real-time driving condition and the first relationship between the first real-time driving condition and the human-machine collaboration factor model includes: When the first real-time driving condition is a high-speed large curvature condition, determine the second human-machine collaboration factor as the first human-machine collaboration factor; When the first real-time working condition is a low-speed and small-curvature working condition, the third human-machine cooperation factor is determined as the first human-machine cooperation factor; When the first real-time working condition is a transition working condition, the first human-machine cooperation factor is determined according to the second human-machine cooperation factor and the third human-machine cooperation factor; When the first real-time working condition is an emergency working condition, a warning is started.

7. The method according to claim 1, characterized in that, The driving group identification model is trained in the following manner: Obtain the data to be trained from the vehicle operation data; Input the data to be trained into the fuzzy inference model for training, and introduce a first feedback mechanism to optimize the fuzzy inference model, obtaining a trained fuzzy inference model; Perform artificial feature extraction on the data to be trained to obtain the first key parameters representing the vehicle operation state and driving behavior characteristics, input them into the deep neural network model for training, use the hyperparameter tuning method to optimize the performance of the deep neural network model, and use the cross-validation method to evaluate the performance of the deep neural network model, obtaining a trained deep neural network model; wherein, the hyperparameter tuning method includes: grid search or random search or Bayesian optimization; Fuse the trained fuzzy inference model and the trained deep neural network model in a linear weighted manner to obtain a driving group identification model, and introduce a second feedback mechanism to optimize the driving group identification model, obtaining a trained driving group identification model.

8. A human-machine collaborative steering control device, characterized in that, It includes: A key parameter extraction module, configured to obtain the preprocessed vehicle operation data, and perform artificial feature extraction on the vehicle operation data to obtain the first key parameters representing the vehicle operation state and driving behavior characteristics; A driving group determination module, configured to input the vehicle operation data and the first key parameters into the driving group identification model to determine the first driving group to which the driver belongs; wherein, the driving group identification model is constructed based on a fuzzy inference model and a deep neural network model; the first driving group includes: a steady driving group, an aggressive driving group, a conservative driving group, and a transient driving group; A human-machine cooperation factor determination module, configured to determine the first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs; The determination of the first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs includes: Extract the third key parameter from the vehicle operation data to determine the first real-time working condition of the vehicle driving; Determine the first human-machine cooperation factor according to the first real-time working condition and the first relationship between the first real-time working condition and the human-machine cooperation factor model; wherein, the human-machine cooperation factor model includes: a second human-machine cooperation factor for large-curvature road working conditions and a third human-machine cooperation factor for small-curvature road working conditions; The formula representation of the second human-machine cooperation factor is: The formula representation of the third human-machine cooperation factor is: Among them, the characteristic parameters of the first driving group determined according to the first driving group to which the driver belongs are expressed as c 1、 c 2、 k 1、 k 2; Determined according to the vehicle operation data: The comprehensive lateral deviation at the vehicle's center of mass is expressed as: ; The comprehensive lateral deviation at the preview point position is expressed as: ; The lateral deviation of the vehicle at the vehicle's center of mass is expressed as ; The normalized value of the yaw angle deviation at the vehicle's center of mass is expressed as ; The lateral deviation of the vehicle at the preview point position is expressed as ; The normalized value of the yaw angle deviation at the preview point position is expressed as ; The first preset value is represented as , the second preset value is represented as , and it satisfies ; A steering control module is configured to determine a combined torque based on the first human-machine collaboration factor, a first torque for controlling the vehicle steering by the vehicle driving system, and a second torque for controlling the vehicle steering by the driver, and control the vehicle to steer and travel according to the combined torque.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it executes the human-machine collaborative steering control method according to any one of claims 1 to 7.

Images