Man-machine cooperative steering control method and device and storage medium

Abstract

The invention provides a man-machine cooperative steering control method and device and a storage medium, and the method comprises the steps: carrying out the artificial feature extraction of vehicle operation data, and obtaining a first key parameter representing the vehicle operation state and driving behavior features; inputting the vehicle operation data and the first key parameter into a driving group identification model, and determining a first driving group to which the driver belongs; determining a first man-machine cooperation factor according to the vehicle operation data and a first driving group to which the driver belongs; and controlling the vehicle to steer according to the first man-machine cooperation factor, a first torque for controlling the vehicle to steer by a vehicle driving system and a second torque for controlling the vehicle to steer by a driver. According to the invention, the driving group identification model is adopted to identify the driving group to which the driver belongs, and participates in the steering control of the vehicle by the vehicle driving system and the driver, so that the steering control of the vehicle is subjected to anthropomorphic improvement, potential conflicts between a man and a machine are effectively reduced, and the co-driving experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of unmanned driving technology, and in particular to a human-machine collaborative steering control method, device and storage medium. Background Art

[0002] The commercialization process of driverless technology faces many difficulties, such as challenges in driving decisions in complex scenarios, imperfections in the new infrastructure of the Internet of Vehicles, and the lack of relevant legislation on the definition of rights and responsibilities. These factors have greatly delayed the full implementation of driverless technology. Therefore, before the realization of full-domain driverless driving, smart cars will still be in a state of human-machine co-driving for a long time. In this state, when the vehicle is turning, the incorrect allocation of driving control rights can easily lead to safety accidents. Therefore, it is urgent to provide a control method that can efficiently balance the steering control rights between the driver and the vehicle driving system to ensure driving safety. Summary of the invention

[0003] The embodiments of the present disclosure provide a human-machine collaborative steering control method, device and storage medium, which are used to solve the problem that in the existing human-machine co-driving state, the incorrect allocation of driving control rights can easily lead to safety accidents.

[0004] Based on the above problems, in a first aspect, an embodiment of the present disclosure provides a human-machine collaborative steering control method, comprising: Acquire the preprocessed vehicle operation data, and perform artificial feature extraction on the vehicle operation data to obtain a first key parameter characterizing the vehicle operation state and driving behavior characteristics; Inputting the vehicle operation data and the first key parameter into a driving group identification model to determine a first driving group to which the driver belongs; wherein the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model; the first driving group includes: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group; determining a first human-machine cooperation factor according to the vehicle operation data and a first driving group to which the driver belongs; A composite torque is determined based on the first human-machine cooperative factor, a first torque used by a vehicle driving system to control vehicle steering, and a second torque used by a driver to control vehicle steering, and the vehicle steering is controlled based on the composite torque.

[0005] In combination with the first aspect, in a possible implementation manner, inputting the vehicle operation data and the first key parameter into a driving group identification model to determine the first driving group to which the driver belongs includes: Inputting the vehicle operation data into the fuzzy inference model to determine a first driving group label to which the driver belongs; Inputting the first key parameter into the deep neural network model to determine a second driving group label to which the driver belongs; The fuzzy inference model and the deep neural network model are fused in a linear weighted manner, and the first driving group to which the driver belongs is determined according to the first driving group label and the second driving group label.

[0006] In combination with the first aspect, in a possible implementation manner, inputting the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs includes: Performing statistical analysis on the vehicle operation data using a dynamic sliding window to determine a second key parameter and a first key indicator calculated based on the second key parameter; Pre-constructing a priori fuzzy rules and a fuzzy inference engine, and inputting the second key parameter and the first key indicator into the fuzzy inference engine according to the a priori fuzzy rules to determine a first driving group label data set; The first driving group label data set is clustered using a K-means algorithm, and the driving group label with the largest number in the K-means algorithm cluster is determined as the first driving group label to which the driver belongs.

[0007] In combination with the first aspect, in a possible implementation manner, inputting the second key parameter and the first key indicator into the fuzzy inference engine according to the priori fuzzy rule to determine the first driving group label data set includes: According to the priori fuzzy rule, the second key parameter and the first key indicator are input into the fuzzy inference engine to determine a second driving group label data set and timestamps corresponding to elements in the second driving group label data set; The data size of the second driving group label data set is adjusted according to the timestamp, and the adjusted second driving group label data set is determined as the first driving group label data set.

[0008] In combination with the first aspect, in a possible implementation, the deep neural network model includes: a deep learning model and a multimodal model; The step of inputting the first key parameter into the deep neural network model to determine a second driving group label to which the driver belongs includes: After normalizing the first key parameter, the first key parameter is input into the deep learning model to extract the first high-level feature; The first high-level feature and the first key parameter are input into the multimodal model to determine the second driving group label to which the driver belongs; wherein the multimodal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model, an SRU network model, or a GRU network model.

[0009] In combination with the first aspect, in a possible implementation manner, determining a first human-machine cooperation factor according to the vehicle operation data and a first driving group to which the driver belongs includes: Extracting a third key parameter from the vehicle operation data to determine a first real-time operating condition of the vehicle; Determine a first human-machine collaboration factor according to the first real-time working condition and a first relationship between the first real-time working condition and a human-machine collaboration factor model; wherein the human-machine collaboration factor model includes: a second human-machine collaboration factor for a large curvature road working condition and a third human-machine collaboration factor for a small curvature road working condition; The formula of the second human-machine collaboration factor is expressed as:

[0010] The formula of the third human-machine collaboration factor is expressed as:

[0011] The characteristic parameter of the first driving group determined according to the first driving group to which the driver belongs is expressed as c 1. c 2. k 1. k 2; Determined based on the vehicle operation data: The comprehensive lateral deviation at the vehicle center of mass is expressed as: ; The comprehensive lateral deviation at the preview point is expressed as: ; The lateral deviation of the vehicle at the vehicle center of mass is expressed as ; The normalized value of the yaw angle deviation at the vehicle center of mass is expressed as ; The lateral deviation of the vehicle at the preview point is expressed as ; The normalized value of the yaw angle deviation at the preview point is expressed as ; The first preset value is expressed as , the second preset value indicates , and satisfies .

[0012] In combination with the first aspect, in a possible implementation manner, the third key parameter includes: a real-time vehicle speed parameter and a real-time road parameter; The first real-time working condition includes: high-speed and large-curvature working condition, low-speed and small-curvature working condition, transition working condition and emergency working condition; The extracting the third key parameter from the vehicle operation data to determine the first real-time operating condition of the vehicle includes: When the real-time road parameter indicates that the road where the vehicle is located does not have a turning condition, determining that the first real-time operating condition of the vehicle is an emergency operating condition; When the real-time road parameter indicates that the road where the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is greater than a third preset value, and the real-time vehicle speed parameter is greater than a fourth preset value, determining that the first real-time operating condition of the vehicle is a high-speed and large-curvature operating condition; When the real-time road parameter indicates that the road where the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is less than a fifth preset value, and the real-time vehicle speed parameter is less than a sixth preset value, determining that the first real-time operating condition of the vehicle is a low-speed and small-curvature operating condition; When the real-time road parameter indicates that the road on which the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is less than or equal to the third preset value and greater than or equal to the fifth preset value, or the real-time vehicle speed parameter is less than or equal to the fourth preset value and greater than or equal to the sixth preset value, determining that the first real-time operating condition of the vehicle is a transitional operating condition; The determining a first human-machine collaboration factor according to the first real-time operating condition and a first relationship between the first real-time operating condition and a human-machine collaboration factor model includes: When the first real-time working condition is a high-speed and large-curvature working condition, determining the second human-machine cooperation factor as the first human-machine cooperation factor; When the first real-time operating condition is a low-speed and small-curvature operating condition, determining the third human-machine cooperation factor as the first human-machine cooperation factor; When the first real-time operating condition is a transitional operating condition, determining a first human-machine cooperation factor according to the second human-machine cooperation factor and the third human-machine cooperation factor; When the first real-time operating condition is an emergency operating condition, an early warning is initiated.

[0013] In combination with the first aspect, in a possible implementation manner, the driving group identification model is trained in the following manner: Acquiring data to be trained from the vehicle operation data; Inputting the data to be trained into the fuzzy reasoning model for training, and introducing a first feedback mechanism to optimize the fuzzy reasoning model to obtain a trained fuzzy reasoning model; Performing artificial feature extraction on the data to be trained to obtain the first key parameter characterizing the vehicle running state and driving behavior characteristics, inputting the first key parameter into the deep neural network model for training, optimizing the performance of the deep neural network model by a hyperparameter tuning method, and evaluating the performance of the deep neural network model by a cross-validation method to obtain a trained deep neural network model; wherein the hyperparameter tuning method includes: grid search or random search or Bayesian optimization; The trained fuzzy inference model and the trained deep neural network model are fused in a linear weighted manner to obtain a driving group identification model, and a second feedback mechanism is introduced to optimize the driving group identification model to obtain a trained driving group identification model.

[0014] In a second aspect, a human-machine collaborative steering control device is provided, comprising: A key parameter extraction module is used to obtain the pre-processed vehicle operation data and perform artificial feature extraction on the vehicle operation data to obtain a first key parameter characterizing the vehicle operation state and driving behavior characteristics; A driving group determination module, configured to input the vehicle operation data and the first key parameter into a driving group identification model to determine a first driving group to which the driver belongs; wherein the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model; the first driving group includes: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group; a human-machine cooperation factor determination module, configured to determine a first human-machine cooperation factor according to the vehicle operation data and a first driving group to which the driver belongs; The steering control module is used to determine the composite torque based on the first human-machine cooperation factor, the first torque used by the vehicle driving system to control the vehicle steering, and the second torque used by the driver to control the vehicle steering, and control the vehicle steering according to the composite torque.

[0015] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the human-machine collaborative steering control method as described in the first aspect or in any possible implementation manner in combination with the first aspect are executed.

[0016] The beneficial effects of the embodiments of the present disclosure include: The human-machine collaborative steering control method, device and storage medium provided by the present disclosure include: obtaining vehicle operation data after preprocessing, and performing artificial feature extraction on the vehicle operation data to obtain a first key parameter characterizing the vehicle operation state and driving behavior characteristics; inputting the vehicle operation data and the first key parameter into a driving group identification model to determine a first driving group to which the driver belongs; wherein the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model; the first driving group includes: a stable driving group, an aggressive driving group, a conservative driving group and a transient driving group; determining a first human-machine collaborative factor based on the vehicle operation data and the first driving group to which the driver belongs; determining a composite torque based on the first human-machine collaborative factor, a first torque used by a vehicle driving system to control vehicle steering, and a second torque used by the driver to control vehicle steering, and controlling the vehicle steering according to the composite torque. The human-machine collaborative steering control method provided by the embodiment of the present disclosure collects vehicle operation data during the operation of the vehicle, adopts a driving group identification model to identify the driving group to which the driver belongs, determines the corresponding human-machine collaborative factors, and participates in the vehicle driving system and the driver's steering control of the vehicle, thereby performing anthropomorphic improvement on the vehicle's steering control, effectively reducing potential conflicts between humans and machines and improving the shared driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of a human-machine collaborative steering control method provided by an embodiment of the present disclosure; Figure 2 A flow chart of a vehicle steering driving controlled by synthetic torque provided in an embodiment of the present disclosure; Figure 3 A schematic diagram of a human-machine collaboration factor provided by an embodiment of the present disclosure; Figure 4 A structural diagram of the human-machine collaborative steering control device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The embodiments of the present disclosure provide a human-machine collaborative steering control method, device and storage medium. The preferred embodiments of the present disclosure are described below in conjunction with the drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0019] The present disclosure provides a human-machine collaborative steering control method. Figure 1 As shown, the following steps are included: S101, obtaining pre-processed vehicle operation data, and performing manual feature extraction on the vehicle operation data to obtain a first key parameter characterizing a vehicle operation state and a driving behavior feature; S102, inputting the vehicle operation data and the first key parameter into a driving group identification model to determine a first driving group to which the driver belongs; wherein the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model; the first driving group includes: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group; S103, determining a first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs; S104. Determine a composite torque based on the first human-machine cooperative factor, a first torque used by the vehicle driving system to control vehicle steering, and a second torque used by the driver to control vehicle steering, and control the vehicle steering according to the composite torque.

[0020] The disclosed embodiments can be applied to the field of unmanned driving technology, especially in human-machine collaborative driving scenarios. The vehicle driving system can collect vehicle operation data using data acquisition devices such as millimeter-wave radar, laser radar, visual sensor, inertial navigation unit, RSU roadside unit, cloud platform, etc. Using these vehicle operation data, the vehicle driving system can plan the vehicle driving strategy, control the torque of the vehicle steering, and then control the vehicle steering. For example, millimeter-wave radar uses electromagnetic waves in the millimeter-wave frequency band to detect information such as the distance, speed and angle of the target object. In severe weather conditions, such as heavy rain and heavy fog, the performance of millimeter-wave radar is less affected by severe weather, and it can still stably obtain information about surrounding vehicles and obstacles to ensure driving safety. The laser radar can construct a three-dimensional point cloud map of the surrounding environment by emitting a laser beam and measuring the time of reflected light. During the driving process of the vehicle, the laser radar continuously scans the surrounding environment with a high-frequency laser beam, and can monitor the dynamic changes of surrounding objects in real time. When a surrounding vehicle suddenly changes lanes and cuts in front, the LiDAR can capture this change in a very short time, and the vehicle driving system can make timely response strategies, such as adjusting the speed, maintaining a safe distance, or performing steering avoidance operations, effectively avoiding collision accidents, and greatly improving the safety and reliability of autonomous driving. Visual sensors, that is, cameras, can play important roles such as environmental perception, target recognition and tracking, auxiliary decision-making and control, map construction and positioning, and safety monitoring. The inertial navigation unit can provide real-time attitude information such as pitch, yaw and roll of the vehicle by measuring the acceleration and angular velocity of the vehicle. Help the vehicle driving system predict the vehicle's motion state and achieve more precise control. The RSU roadside unit can send road condition information, traffic signal status, etc. to the vehicle to provide assisted driving decision support for the vehicle driving system. The cloud platform can update maps and navigation information in real time, provide accurate route planning services for the vehicle driving system, and can also help vehicles analyze various situations and make decisions faster through the powerful algorithm processing capabilities of cloud computing. However, the commercialization of driverless technology has been hampered to a great extent due to the challenges of driving decisions in complex scenarios, the imperfect new infrastructure of the Internet of Vehicles, and the lack of relevant legislation on the definition of rights and responsibilities. Therefore, before achieving full-domain driverless driving, smart cars will still face a long period of human-machine co-driving. In the human-machine co-driving system, the reasonable allocation of driving control rights has become the key, that is, how to reasonably and efficiently balance the driving control rights between the driver and the vehicle driving system. Especially when the vehicle is turning, the incorrect allocation of driving control rights can easily lead to safety accidents.

[0021] In the disclosed embodiment, preprocessing the vehicle operation data may include: performing time synchronization, calibration and filtering on the vehicle operation data. Due to the differences in their working principles and operating mechanisms, different data acquisition devices may have deviations in the time of the collected data. Therefore, the vehicle operation data may be time synchronized and calibrated to ensure data accuracy and consistency. For example, millimeter wave radar, laser radar and visual sensor work simultaneously to collect surrounding environment data. The data update frequency of millimeter wave radar may be fifty times per second, the data update frequency of laser radar is relatively low, and the frame rate of different visual sensors to collect images is also different. If time synchronization is not performed, these collected vehicle operation data cannot accurately correspond in time, resulting in deviations in the perception of the surrounding environment. By adopting high-precision clock synchronization technology, such as the precise timing function of the global positioning system (GPS), or a network-based time synchronization protocol, the data collected by different devices can be unified to the same time reference. In the process of collecting vehicle operation data, the measurement error of the device also needs to be calibrated. Taking the visual sensor as an example, due to the influence of factors such as lens distortion, the captured image may have geometric distortion. The visual sensor can be calibrated by using standard calibration objects to establish an accurate imaging model, eliminate the error caused by lens distortion, and improve the accuracy of image data. Advanced filtering algorithms such as Kalman filtering or particle filtering can be used to filter the time-synchronized and calibrated vehicle operation data to reduce the influence of noise and outliers. After obtaining the pre-processed vehicle operation data, the vehicle operation data is manually extracted to obtain the first key parameter characterizing the vehicle operation state and driving behavior characteristics. Manual feature extraction can be a process of constructing physically meaningful and interpretable feature quantities from vehicle operation data through domain knowledge. The experience of domain experts is required to design and select features to better capture the essence of the problem. For example, the process of manual feature extraction describes the distribution of vehicle operation data through mean, variance, standard deviation, etc., converts vehicle operation data from time domain to frequency domain through Fourier transform, wavelet transform, etc., and reduces the dimension of vehicle operation data through principal component analysis (PCA), independent component analysis (ICA), etc. The first key parameter characterizing the vehicle operation state and driving behavior characteristics can include: the lateral deviation at the position of the vehicle's center of mass in the vehicle coordinate system, that is, the side deviation is expressed as ; The longitudinal velocity of the vehicle is expressed as ; The lateral acceleration of the vehicle is expressed as ; The longitudinal acceleration of the vehicle is expressed as , the longitudinal acceleration of the vehicle, is used to describe the characteristics when the driver steps on the accelerator pedal, and is expressed as , the vehicle longitudinal deceleration, used to describe the characteristics when the driver steps on the brake pedal, expressed as ; Vehicle lateral impact, used to describe lateral comfort, expressed as ; Vehicle longitudinal impact, used to describe longitudinal comfort, expressed as ; Steering wheel angle, used to describe the characteristics of the driver's steering wheel operation, expressed as ; Steering wheel angular velocity, used to describe the characteristics of the driver operating the steering wheel, expressed as ; The following distance is expressed as ; The frequency of the driver observing the left and right rearview mirrors is expressed as ; The frequency of the driver turning on the turn signal is expressed as ; The average grip force of the driver operating the steering wheel is expressed as ; The driver's facial expression stress factor is expressed as ; The driver distraction factor is expressed as ; Driver fatigue factor is expressed as ; The time from the need for emergency braking to the driver stepping on the pedal or a forward collision occurring is expressed as ; Continuous driving time is expressed as ;The total number of sampling points is denoted as N; The peak longitudinal velocity of the vehicle is expressed as ; The mean longitudinal velocity of the vehicle is expressed as ; The standard deviation of the vehicle longitudinal velocity is used to quantify the fluctuation of the vehicle speed curve and is expressed as ; The peak value of the lateral deflection is expressed as ; The mean of the lateral deviation is expressed as ; The standard deviation of the lateral movement is used to quantify the fluctuation of the lateral movement curve and is expressed as ; The root mean square error of the side deviation is used to quantify the trajectory tracking accuracy and is expressed as ; The peak value of the lateral acceleration is expressed as ; The mean value of the lateral acceleration is expressed as ; The standard deviation of the lateral acceleration is expressed as ; The peak value of longitudinal acceleration is expressed as ; The peak value of longitudinal deceleration is expressed as ; The frequency of rapid acceleration is expressed as ,in is the preset threshold; The frequency of rapid deceleration is expressed as ,in is the preset threshold; The peak value of the lateral impact strength is expressed as ; The peak value of longitudinal impact intensity is expressed as ; The peak value of the steering angle is expressed as ; The mean value of the steering angle is expressed as ; The standard deviation of the steering angle is used to quantify the fluctuation of the steering angle curve and is expressed as ; The peak value of the steering angular velocity is expressed as ; The mean value of the steering angular velocity is expressed as ; The standard deviation of the steering angular velocity is used to quantify the fluctuation of the steering angular velocity curve and is expressed as .

[0022] Different drivers have different driving styles, driving skills, and control preferences. The first driving group divided in this way may include: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group. The transient driving group may include a driving group in a road rage state, a driving group in a distracted state, and a driving group in a fatigued state. The vehicle operation data and the first key parameter are input into the driving group identification model to determine the first driving group to which the driver belongs. Among them, the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model. In the process of analyzing driving behavior characteristics, many factors, such as "aggressive" and "conservative", are fuzzy. The fuzzy reasoning model can be a reasoning method based on fuzzy logic and fuzzy set theory, which can handle uncertainty and fuzzy information. The deep neural network model can learn hierarchical features from low-level sensor data to high-level driving behavior representation through a deep network structure, and can process multimodal data such as video and radar point cloud to achieve information complementarity. By integrating the fuzzy reasoning model and the deep neural network model, we can fully utilize the advantages of both, retaining the ability of the fuzzy reasoning model to handle uncertainty while also having the powerful feature learning ability of the deep neural network model.

[0023] Furthermore, if Figure 2As shown, the target path of the vehicle driving system is usually set to the center line of the lane, and different driving groups have different sensitivities to vehicle side deviation, that is, the target path of the vehicle driving system and the driver are not completely consistent. This also means that different driving groups have different behavioral habits in lane keeping, and have different acceptance of the timing and degree of intervention of the vehicle driving system. In order to take into account the development trend of vehicle lateral deviation and the behavioral habits of driving groups in lane keeping, and to improve the human-machine co-driving experience, the results of the respective decisions of the driver and the vehicle driving system are integrated to form a closed-loop control. The vehicle driving system makes decisions and plans based on the vehicle operation data to obtain the first torque to control the steering of the vehicle. The vehicle operation data can also be used to measure the working condition of the vehicle. The first human-machine cooperation factor is determined based on the vehicle operation data and the first driving group to which the driver belongs. , under different working conditions, different driving groups determine different first human-machine cooperation factors to ensure vehicle driving safety. Then, according to the first human-machine cooperation factor, the first torque of the vehicle driving system to control the vehicle steering and the second torque of the driver to control the vehicle steering, the synthetic torque is determined, and its formula is expressed as: ,in represents the resultant torque, represents the first torque, represents the second torque, Represents the first human-machine synergy factor. Finally, according to the synthetic torque Control the vehicle's steering.

[0024] In order to meet the personalized operation needs of different driving groups, the embodiment of the present application performs an anthropomorphic improvement on the human-machine collaborative steering control compared with the prior art, and assigns the driver to the driving group to which he belongs. Combined with the real-time vehicle operating status, it effectively reduces the potential conflict between the driver and the vehicle driving system, and improves the shared driving experience.

[0025] In another embodiment of the present disclosure, in the above step S102, the vehicle operation data and the first key parameter are input into the driving group identification model to determine the first driving group to which the driver belongs, including the following steps: Step 1, inputting vehicle operation data into a fuzzy inference model to determine a first driving group label to which the driver belongs; Step 2, inputting the first key parameter into the deep neural network model to determine the second driving group label to which the driver belongs; Step 3: The fuzzy reasoning model and the deep neural network model are integrated in a linear weighted manner, and the first driving group to which the driver belongs is determined according to the first driving group label and the second driving group label.

[0026] In the disclosed embodiment, the fuzzy reasoning model and the deep neural network model are fused in a linear weighted manner to construct a driving group identification model, and the vehicle operation data and the first key parameter are processed to determine the first driving group to which the driver belongs. For the above step 1, the first driving group label corresponds to the first driving group, and may include: a stable driving group label, an aggressive driving group label, a conservative driving group label, and a transient driving group label. The first driving group label to which the determined driver belongs may be one of a stable driving group label, an aggressive driving group label, a conservative driving group label, and a transient driving group label. The fuzzy reasoning model may be a reasoning model based on fuzzy logic and fuzzy set theory that processes uncertainty and fuzzy information. In the process of determining the first driving group label to which the driver belongs, due to the large differences in the driving style, driving skills, and handling preferences of the drivers, the vehicle operation data often has uncertainty, such as driving speed, acceleration, and other data may fluctuate due to various factors. The vehicle operation data is input into the fuzzy reasoning model, and the fuzzy reasoning model can handle this uncertainty well, thereby performing online fuzzy identification of the first driving group label to which the driver belongs. For the above step 2, the second driving group label corresponds to the first driving group, and may include: a stable driving group label, an aggressive driving group label, a conservative driving group label, and a transient driving group label. The second driving group label to which the determined driver belongs may be one of a stable driving group label, an aggressive driving group label, a conservative driving group label, and a transient driving group label. The first key parameter may characterize the vehicle operation state and driving behavior characteristics. The first key parameter is input into the deep neural network model to extract high-level features. According to the characteristics of the driving behavior characteristics, a convolutional neural network (CNN) may be selected to extract image features, such as the driver's facial expression, head posture, etc.; an SRU network model or a GRU network model, especially a long short-term memory network (LSTM), may also be selected to analyze the driving behavior feature sequence, such as the vehicle's motion trajectory, operating habits, etc. The first key parameter is input into the trained deep neural network model, and the probability distribution of each driving group label may be output, and the driving group label with the highest probability in the probability distribution may be used as the second driving group label to which the driver belongs. For the above step 3, the fuzzy reasoning model and the deep neural network model are fused in a linear weighted manner. Different weights can be assigned to the fuzzy reasoning model and the deep neural network model according to different vehicle operation scenarios. For example, in the highway scenario, the weight of the fuzzy reasoning model is 0.4, and the weight of the deep neural network model is 0.6; in the urban congestion scenario, the weight of the fuzzy reasoning model is 0.6, and the weight of the deep neural network model is 0.4. According to the first driving group label and the second driving group label, the first driving group to which the driver belongs is determined.For example, if the first driving group label to which the driver belongs determined by the fuzzy inference model is a stable driving group label, and the second driving group label to which the driver belongs determined by the deep neural network model is an aggressive driving group label, and the vehicle is driving on a highway, then the first driving group to which the driver belongs after fusion is an aggressive driving group. By adjusting the weight parameters of the fusion strategy according to the actual vehicle operation scenario, the fuzzy inference model and the deep neural network model can be effectively fused, thereby more accurately determining the first driving group to which the driver belongs.

[0027] In another embodiment of the present disclosure, in the above step 1, inputting the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs includes: Step 1: Perform statistical analysis on the vehicle operation data using a dynamic sliding window to determine the second key parameter and the first key indicator calculated based on the second key parameter; Step 2: construct a priori fuzzy rules and a fuzzy inference engine in advance, input the second key parameter and the first key indicator into the fuzzy inference engine according to the priori fuzzy rules, and determine the first driving group label data set; Step 3: Use the K-means algorithm to cluster the first driving group label data set, and determine the driving group label with the largest number in the K-means algorithm cluster as the first driving group label to which the driver belongs.

[0028] In the disclosed embodiment, a fuzzy inference model is used to perform online fuzzy identification on the first driving group to which the driver belongs. For step one above, the dynamic sliding window technology can automatically adjust the window length N according to different driving scenarios to adapt to the analysis of driving behavior characteristics at different time scales. For example, in the highway cruising scenario, the window length range of the dynamic sliding window is 5-10s, in the city congestion following scenario, the window length range of the dynamic sliding window is 0.5-2s, and in the mountain curve driving scenario, the window length range of the dynamic sliding window is 2-5s. Perform statistical analysis on the vehicle operation data within the window, record the value of the second key parameter, and calculate the first key indicator based on the second key parameter. The statistical analysis method may be the same as the manual feature extraction method. The second key parameter may include: the lateral deviation at the center of mass position of the vehicle in the vehicle coordinate system, that is, the side deviation is expressed as ; The longitudinal velocity of the vehicle is expressed as ; The lateral acceleration of the vehicle is expressed as ; The longitudinal acceleration of the vehicle is expressed as , the longitudinal acceleration of the vehicle, is used to describe the characteristics when the driver steps on the accelerator pedal, and is expressed as , the vehicle longitudinal deceleration, used to describe the characteristics when the driver steps on the brake pedal, expressed as ; Vehicle lateral impact, used to describe lateral comfort, expressed as ; Vehicle longitudinal impact, used to describe longitudinal comfort, expressed as ; Steering wheel angle, used to describe the characteristics of the driver's steering wheel operation, expressed as ; Steering wheel angular velocity, used to describe the characteristics of the driver's steering wheel operation, expressed as ; The following distance is expressed as ; The frequency of the driver observing the left and right rearview mirrors is expressed as ; The frequency of the driver turning on the turn signal is expressed as ; The average grip force of the driver operating the steering wheel is expressed as ; The driver's facial expression stress factor is expressed as ; The driver distraction factor is expressed as ; Driver fatigue factor is expressed as ; The time from the need for emergency braking to the driver stepping on the pedal or a forward collision occurring is expressed as ; Continuous driving time is expressed as ; The total number of sampling points is denoted as N.

[0029] The first key indicators may include: The peak longitudinal velocity of the vehicle is expressed as ; The mean longitudinal velocity of the vehicle is expressed as ; The standard deviation of the vehicle longitudinal velocity is used to quantify the fluctuation of the vehicle speed curve and is expressed as ; The peak value of the lateral deflection is expressed as ; The mean of the lateral deviation is expressed as ; The standard deviation of the lateral movement is used to quantify the fluctuation of the lateral movement curve and is expressed as ; The root mean square error of the side deviation is used to quantify the trajectory tracking accuracy and is expressed as ; The peak value of the lateral acceleration is expressed as ; The mean value of the lateral acceleration is expressed as ; The standard deviation of the lateral acceleration is expressed as ; The peak value of longitudinal acceleration is expressed as ; The peak value of longitudinal deceleration is expressed as ; The frequency of rapid acceleration is expressed as ,in is the preset threshold; The frequency of rapid deceleration is expressed as ,in is the preset threshold; The peak value of the lateral impact strength is expressed as ; The peak value of longitudinal impact intensity is expressed as ; The peak value of the steering angle is expressed as ; The mean value of the steering angle is expressed as ; The standard deviation of the steering angle is used to quantify the fluctuation of the steering angle curve and is expressed as ; The peak value of the steering angular velocity is expressed as ; The mean value of the steering angular velocity is expressed as ; The standard deviation of the steering angular velocity is used to quantify the fluctuation of the steering angular velocity curve and is expressed as .

[0030] For the above step 2, the fuzzy reasoning model may include: a priori fuzzy rules and a fuzzy reasoning engine. A priori fuzzy rules can be established based on domain knowledge, such as driving behavior expert experience or historical data statistics. Exemplary, pre-built a priori fuzzy rules are shown in Table 1. The fuzzy rule form is "if the input variable meets certain conditions, the output variable belongs to a certain fuzzy set". For example, "if the vehicle speed is speeding, the first driving group label is the aggressive driving group label".

[0031]

[0032] Table 1 The fuzzy inference machine can be constructed by using advanced fuzzy logic systems such as the Mamdani model or the Sugeno model. The input variables of the fuzzy inference machine are the second key parameter and the first key indicator, and the output variable is the first driving group label data set. The prior fuzzy rules define fuzzy sets for each input variable and output variable, such as moderate speed, low speed, speeding, etc., and select appropriate membership functions, such as triangular, trapezoidal or Gaussian membership functions to describe the degree to which the variables belong to different fuzzy sets. The fuzzy inference machine performs reasoning based on the prior fuzzy rules and input variables to obtain the membership or exact value of each driving group label, and stores the label in the data set to obtain the first driving group label data set.

[0033] For the above step three, the K-means algorithm is used to cluster the first driving group label data set. The Elbow Method or the Silhouette Coefficient can be used to determine the number of clusters K. K data points are randomly selected from the first driving group label data set as the initial cluster centers. The Euclidean distance from each data point to each cluster center is iteratively calculated, and the data point is assigned to the cluster corresponding to the cluster center with the closest distance. For each cluster, its cluster center is recalculated, that is, the mean of all data points in the cluster until the change in the cluster center is less than the preset threshold or the maximum number of iterations is reached, and the iteration is stopped. For each cluster, the number of driving group labels in each cluster is counted, and the driving group label with the largest number in the K-means algorithm cluster is determined as the first driving group label to which the driver belongs. Exemplarily, as shown in Table 2, the number of conservative driving group labels in cluster ID2 is the largest, and the conservative driving group label corresponding to cluster ID2 is determined as the first driving group label to which the driver belongs.

[0034]

[0035] Table 2 The fuzzy reasoning model is combined with a priori fuzzy rules. The system has the dual advantages of data-driven and expert knowledge. Through K-means clustering for label distribution statistics, drivers can be automatically classified into the most representative driving groups.

[0036] In another embodiment of the present disclosure, in the above step 2, according to the prior fuzzy rule, the second key parameter and the first key indicator are input into the fuzzy inference engine to determine the first driving group label data set, including: Step (i), according to a priori fuzzy rules, inputting the second key parameter and the first key indicator into the fuzzy inference engine, determining the second driving group label data set and the timestamps corresponding to the elements in the second driving group label data set; Step (ii): adjusting the data size of the second driving group label data set according to the timestamp, and determining the adjusted second driving group label data set as the first driving group label data set.

[0037] In the disclosed embodiment, the output result of the fuzzy inference engine also includes a timestamp, and the size of the data volume of the first driving group label data set is determined according to the timestamp. For the above step (i), the input variables of the fuzzy inference engine are the second key parameter and the first key indicator, and the output variables are the second driving group label data set and the timestamp corresponding to the elements in the second driving group label data set. The fuzzy inference engine performs inference according to the prior fuzzy rules and the input variables to obtain the membership or exact value of each driving group label, and stores the label in the data set to obtain the second driving group label data set and the timestamp corresponding to the elements in the second driving group label data set. For the above step (ii), in order to improve the rapid identification capability under complex conditions such as group evolution or driving state changes, the data volume of the second driving group label data set is adjusted according to the timestamp, for example, the data volume within the last timestamp interval of 2s is retained. The adjusted second driving group label data set can be determined as the first driving group label data set. The second driving group label data set is generated by processing real-time data with the fuzzy inference engine, and then the data volume is dynamically aligned and adjusted based on the timestamp, and finally the first driving group label data set with high consistency is output, so as to improve the rapid identification capability under complex conditions such as group evolution or driving state changes.

[0038] In yet another embodiment of the present disclosure, the deep neural network model includes: a deep learning model and a multimodal model; In the above step 2, the first key parameter is input into the deep neural network model to determine the second driving group label to which the driver belongs, including the following steps: Step 1: After normalizing the first key parameter, input it into the deep learning model to extract the first high-level feature; Step 2: Input the first high-level feature and the first key parameter into the multimodal model to determine the second driving group label to which the driver belongs; wherein the multimodal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model, an SRU network model, or a GRU network model.

[0039] In the disclosed embodiment, through the high-level feature extraction of the deep learning model and the fusion of the multimodal model, the time series and image data can be integrated to accurately determine the second driving group label to which the driver belongs. For the above step one, the first key parameter is normalized, for example, the first key parameter is scaled to the range of [0, 1]. The normalized data is input into the deep learning model to extract the first high-level feature. The deep learning model can be a CNN convolutional neural network model or a GRU recurrent neural network model. The first key parameter after normalization is learned by using a deep learning model such as a CNN convolutional neural network model or a GRU recurrent neural network model, and the high-level features are extracted, and the first high-level features are output. The CNN convolutional neural network model (Convolutional Neural Network) includes a convolutional layer, an activation function layer, a pooling layer, and a fully connected layer for feature extraction. The GRU recurrent neural network model (Gated Recurrent Unit) can be a variant of the RNN recurrent neural network model. The traditional RNN recurrent neural network model is prone to gradient vanishing or gradient exploding problems when processing long sequence data, and the GRU recurrent neural network model solves these problems by introducing a gating mechanism.

[0040] For the above step 2, the first high-level feature and the first key parameter are input into the multimodal model. The multimodal model is constructed based on the CNN convolutional neural network model and the network model for processing time series data. The network model for processing time series data includes: LSTM network model or SRU network model or GRU network model. For the time series data in the first high-level feature and the first key parameter, select the LSTM network model or the SRU network model or the GRU network model; for the image data in the first high-level feature and the first key parameter, select the CNN convolutional neural network model. For example, the time series data can be vehicle speed, acceleration, etc., and the image data can be the facial expression image of the driver. The LSTM network model (Long Short-Term Memory) can be an improved variant of the RNN recurrent neural network model. It solves the gradient vanishing or explosion problem of the traditional RNN recurrent neural network model through the gating mechanism and cell state, and is good at modeling long sequence dependencies. The SRU network model can be a lightweight recurrent unit, which significantly improves the training speed by simplifying the gating calculation and highly parallel design, and is suitable for processing ultra-long sequences. The deep neural network model can output the probability distribution of each driving group label. According to the output probability distribution, the label with the highest probability among the stable driving group label, aggressive driving group label, conservative driving group label and transient driving group label is selected as the second driving group label to which the driver belongs. The first key parameter can be obtained by artificial feature extraction of vehicle operation data, and the first high-level feature and the first key parameter are input into the multimodal model to form mixed input data of "artificial feature + learning feature", which improves the representation ability while maintaining interpretability.

[0041] In another embodiment of the present disclosure, in the above step S103, determining the first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs includes the following steps: Step 1, extracting a third key parameter from vehicle operation data to determine a first real-time operating condition of the vehicle; Step 2: determining a first human-machine collaboration factor according to the first real-time working condition and a first relationship between the first real-time working condition and the human-machine collaboration factor model; wherein the human-machine collaboration factor model includes: a second human-machine collaboration factor for a large curvature road working condition and a third human-machine collaboration factor for a small curvature road working condition; The formula of the second human-machine synergy factor is expressed as:

[0042] The formula of the third human-machine synergy factor is expressed as:

[0043] Among them, the characteristic parameters of the first driving group determined according to the first driving group to which the driver belongs are expressed as c 1. c 2. k 1. k 2; Determined based on vehicle operating data: The comprehensive lateral deviation at the vehicle center of mass is expressed as: ; The comprehensive lateral deviation at the preview point is expressed as: ; The lateral deviation of the vehicle at the vehicle center of mass is expressed as ; The normalized value of the yaw angle deviation at the vehicle center of mass is expressed as ; The lateral deviation of the vehicle at the preview point is expressed as ; The normalized value of the yaw angle deviation at the preview point is expressed as ; The first preset value is expressed as , the second preset value indicates , and satisfies .

[0044] In the disclosed embodiment, a third key parameter is extracted from the vehicle operation data to determine the first real-time operating condition, and the first human-machine coordination factor is determined based on the first relationship between the first real-time operating condition and the human-machine coordination factor model. For the above step 1, the third key parameter is extracted from the vehicle operation data to determine the first real-time operating condition of the vehicle. The third key parameter may include a real-time vehicle speed parameter and a real-time road parameter. The third key parameter may be extracted from the vehicle operation data by artificial feature extraction. The first real-time operating condition characterizes whether the vehicle has the turning condition on the current road. The first real-time operating condition of the vehicle can be determined based on the real-time vehicle speed parameter and the real-time road parameter. For the above step 2, the first human-machine coordination factor is determined based on the first real-time operating condition and the first relationship between the first real-time operating condition and the human-machine coordination factor model. The human-machine coordination factor model includes: a second human-machine coordination factor for large curvature road conditions and a third human-machine coordination factor for small curvature road conditions. The formula of the second human-machine coordination factor is expressed as:

[0045] The formula of the third human-machine synergy factor is expressed as:

[0046] Among them, the characteristic parameters of the first driving group determined according to the first driving group to which the driver belongs are expressed as c 1.c 2. k 1. k 2; For example, human-machine collaboration factor models for different driving groups, such as Figure 3 As shown, Indicates the width of the comfortable driving area. The selection of this value is related to the lane keeping habits of the driving group. The corresponding driving group characteristic parameters for different driving groups are as follows. A feedback mechanism can be introduced to adjust the characteristic parameters according to the actual application effect.

[0047] (a) For roads with large curvature ① Aggressive driving group, characteristic parameters meet the following requirements:

[0048] ② Stable driving group, the characteristic parameters meet the following requirements:

[0049] ③ Conservative driving group, characteristic parameters meet:

[0050] ④ For the transient driving group, the characteristic parameters are dynamically changing. In the road rage state, the characteristic parameters are consistent with those of the aggressive driving group; in the distracted state, the characteristic parameters are consistent with those of the stable driving group; in the fatigue state, the characteristic parameters are consistent with those of the conservative driving group.

[0051] (b) For roads with small curvature ① Aggressive driving group, characteristic parameters meet the following requirements:

[0052] ② Stable driving group, the characteristic parameters meet the following requirements:

[0053] ③ Conservative driving group, characteristic parameters meet:

[0054] ④ For the transient driving group, the characteristic parameters are dynamically changing. In the road rage state, the characteristic parameters are consistent with those of the aggressive driving group; in the distracted state, the characteristic parameters are consistent with those of the stable driving group; in the fatigue state, the characteristic parameters are consistent with those of the conservative driving group.

[0055] The following parameters can be determined from vehicle operation data by manual feature extraction: The comprehensive lateral deviation at the vehicle center of mass is expressed as: ; The comprehensive lateral deviation at the preview point is expressed as: ; The lateral deviation of the vehicle at the vehicle center of mass is expressed as ; The normalized value of the yaw angle deviation at the vehicle center of mass is expressed as ; The lateral deviation of the vehicle at the preview point is expressed as ; The normalized value of the yaw angle deviation at the preview point is expressed as ; The first preset value is expressed as , the second preset value indicates , and satisfies .

[0056] In another embodiment of the present disclosure, In the disclosed embodiment, when designing the human-machine collaborative factor model, in order to further improve the friendliness, comfort and stability of the human-machine collaborative steering control, in addition to dividing the road conditions into large curvature conditions and small curvature conditions according to the size of the road curvature, the corresponding human-machine collaborative factor model should be designed, and the longitudinal speed of the vehicle, as well as factors such as frequent switching between conditions, should also be considered. The third key parameter includes: real-time vehicle speed parameters and real-time road parameters. The first real-time conditions include: high-speed large curvature conditions, low-speed small curvature conditions, transition conditions and emergency conditions. With respect to the above step one, when the real-time road parameters indicate that the road where the vehicle is located does not have the turning conditions, for example, when the conditions such as changing lanes or overtaking or turning around a curve are not met, it is determined that the first real-time condition of the vehicle is an emergency condition. With respect to the above step two, when the real-time vehicle speed parameters And the road curvature value in the real-time road parameters When the real-time vehicle speed parameter is And the road curvature value in the real-time road parameters When , it is determined to be a low-speed and small-curvature operating condition; for the above step 4, the rest of the conditions are determined to be transitional conditions. is the third preset value, is the fourth preset value, is the fifth preset value, is the sixth preset value. The initial first human-machine collaboration factor is the third human-machine collaboration factor, that is, For the above step 5, when the first real-time working condition is a high-speed and large-curvature working condition, the second human-machine cooperation factor is determined as the first human-machine cooperation factor, that is, For the above step 6, when the first real-time working condition is a low-speed and small-curvature working condition, the third human-machine cooperation factor is determined as the first human-machine cooperation factor, that is, With respect to the above step 7, when the first real-time working condition is a transition working condition, the first human-machine cooperation factor is determined according to the second human-machine cooperation factor and the third human-machine cooperation factor. For example, the first human-machine cooperation factor ,in , All are preset values, satisfying In step eight, when the first real-time working condition is an emergency working condition, an early warning is initiated, for example, an audio-visual-tactile multi-dimensional early warning strategy is initiated. The first human-machine cooperation factor is determined by combining the real-time vehicle speed parameter and the real-time road parameter to ensure the safety of vehicle operation.

[0057] In another embodiment of the present disclosure, the driving group identification model is trained in the following manner: Step 1: Obtain the data to be trained from the vehicle operation data; Step 2: input the data to be trained into the fuzzy reasoning model for training, and introduce a first feedback mechanism to optimize the fuzzy reasoning model to obtain a trained fuzzy reasoning model; Step 3: Performing artificial feature extraction on the training data to obtain the first key parameter characterizing the vehicle operation status and driving behavior characteristics, inputting the first key parameter into the deep neural network model for training, optimizing the performance of the deep neural network model by using a hyperparameter tuning method, and evaluating the performance of the deep neural network model by using a cross-validation method to obtain a trained deep neural network model; wherein the hyperparameter tuning method includes: grid search or random search or Bayesian optimization; Step 4: Use a linear weighted method to fuse the trained fuzzy inference model and the trained deep neural network model to obtain a driving group identification model, and introduce a second feedback mechanism to optimize the driving group identification model to obtain a trained driving group identification model.

[0058] In the disclosed embodiment, the data to be trained is obtained from the vehicle operation data, and a feedback mechanism is introduced to train the driving group identification model. For the above step 1, the data to be trained is obtained from the vehicle operation data, and the corresponding driving groups are labeled with the data to be trained according to the characteristics of the driving behavior characteristics of the drivers, such as a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group. The data to be trained can be divided into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the model parameters and evaluate the model performance, and the test set is used to finally evaluate the generalization ability of the model. For the above step 2, the data to be trained is input into the fuzzy reasoning model for training, and the first feedback mechanism is introduced to optimize the fuzzy reasoning model to obtain the trained fuzzy reasoning model. The first feedback mechanism may include optimizing the fuzzy reasoning model according to the actual application effect by adopting the method of expert review or the method of final user scoring.

[0059] For the above step 3, artificial feature extraction is performed on the training data to obtain the first key parameter that characterizes the vehicle operation status and driving behavior characteristics, and the first key parameter is input into the deep neural network model for training. After the first key parameter is normalized, data enhancement techniques such as rotation, cropping, scaling, and time translation are used to increase the diversity of the data, and the data is input into the deep neural network model for training to improve the generalization ability of the deep neural network model. The performance of the deep neural network model is optimized by using a hyperparameter tuning method, and the hyperparameter tuning method includes: grid search or random search or Bayesian optimization. Grid search can be the most intuitive hyperparameter tuning method. It searches for the optimal solution by exhaustively enumerating all possible hyperparameter combinations, defines a discrete value range for each hyperparameter, and then traverses these combinations, and cross-validates and evaluates each combination. Random search searches for the optimal solution by randomly sampling hyperparameter combinations, randomly extracts a certain number of parameter combinations from the parameter distribution, and then cross-validates and evaluates each combination. Bayesian optimization is an optimization method based on a probability model. It predicts the performance of hyperparameters by building a probability model and selects the hyperparameter combination that is most likely to improve the performance for evaluation. The cross-validation method is used to evaluate the performance of the deep neural network model. The cross-validation method can more accurately evaluate the generalization ability of the model by dividing the data set into multiple subsets and then training and validating on different subsets. Finally, the trained deep neural network model is obtained. As the training data continues to increase, the online learning algorithm can be used to update the deep neural network model in real time to adapt to new driving environments and driving group characteristics. The performance of the deep neural network model is regularly evaluated, and the deep neural network model is adjusted and optimized based on the evaluation results. User feedback data is collected to further improve the accuracy and reliability of the deep neural network model. Using the trained deep neural network model to classify drivers into driving groups, a continuous judgment mechanism or scoring mechanism can be introduced to improve the accuracy and reliability of driving group classification. Considering the evolution and volatility of driving groups, the threshold and judgment strategy of driving group classification can be dynamically adjusted to adapt to changes in different time periods and driving environments.

[0060] For step 4 above, a linear weighted method is used to fuse the trained fuzzy inference model and the trained deep neural network model to obtain a driving group identification model. A second feedback mechanism is introduced to optimize the driving group identification model to obtain a trained driving group identification model. For example, during the operation of the driving group identification model, feedback data from the driver is collected, such as the driver's operation data. These data can be used to adjust the parameters of the model, improve the real-time and adaptability of the model, and obtain a trained driving group identification model. The data to be trained is obtained from the vehicle operation data, and a feedback mechanism is introduced to train the driving group identification model to improve the accuracy of the driving group identification model.

[0061] Based on the same disclosed concept, the embodiments of the present disclosure also provide a human-machine collaborative steering control device. Since the principles of the problems solved by these devices are similar to those of the aforementioned human-machine collaborative steering control methods, the implementation of the device can refer to the implementation of the aforementioned methods, and the repeated parts will not be repeated.

[0062] The present disclosure provides a human-machine collaborative steering control device, such as Figure 4 As shown, including: The key parameter extraction module 401 is used to obtain the pre-processed vehicle operation data and perform artificial feature extraction on the vehicle operation data to obtain the first key parameter characterizing the vehicle operation state and driving behavior characteristics; A driving group determination module 402 is used to input the vehicle operation data and the first key parameter into a driving group identification model to determine a first driving group to which the driver belongs; wherein the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model; the first driving group includes: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group; A human-machine cooperation factor determination module 403, configured to determine a first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs; The steering control module 404 is used to determine the composite torque based on the first human-machine cooperation factor, the first torque of the vehicle driving system controlling the vehicle steering and the second torque of the driver controlling the vehicle steering, and control the vehicle steering according to the composite torque.

[0063] In another embodiment of the present disclosure, the driving group determination module 402 is used to input the vehicle operation data into the fuzzy reasoning model to determine the first driving group label to which the driver belongs; Inputting the first key parameter into the deep neural network model to determine a second driving group label to which the driver belongs; The fuzzy inference model and the deep neural network model are fused in a linear weighted manner, and the first driving group to which the driver belongs is determined according to the first driving group label and the second driving group label.

[0064] In another embodiment of the present disclosure, the driving group determination module 402 is used to perform statistical analysis on the vehicle operation data using a dynamic sliding window to determine a second key parameter and a first key indicator calculated based on the second key parameter; Pre-constructing a priori fuzzy rules and a fuzzy inference engine, and inputting the second key parameter and the first key indicator into the fuzzy inference engine according to the a priori fuzzy rules to determine a first driving group label data set; The first driving group label data set is clustered using a K-means algorithm, and the driving group label with the largest number in the K-means algorithm cluster is determined as the first driving group label to which the driver belongs.

[0065] In another embodiment of the present disclosure, the driving group determination module 402 is used to input the second key parameter and the first key indicator into the fuzzy inference engine according to the prior fuzzy rule, and determine the second driving group label data set and the timestamps corresponding to the elements in the second driving group label data set; The data size of the second driving group label data set is adjusted according to the timestamp, and the adjusted second driving group label data set is determined as the first driving group label data set.

[0066] In yet another embodiment of the present disclosure, the deep neural network model includes: a deep learning model and a multimodal model; The driving group determination module 402 is used to normalize the first key parameter and then input it into the deep learning model to extract the first high-level feature; The first high-level feature and the first key parameter are input into the multimodal model to determine the second driving group label to which the driver belongs; wherein the multimodal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model, an SRU network model, or a GRU network model.

[0067] In another embodiment of the present disclosure, the human-machine collaboration factor determination module 403 is used to extract a third key parameter from the vehicle operation data to determine a first real-time operating condition of the vehicle; Determine a first human-machine collaboration factor according to the first real-time working condition and a first relationship between the first real-time working condition and a human-machine collaboration factor model; wherein the human-machine collaboration factor model includes: a second human-machine collaboration factor for a large curvature road working condition and a third human-machine collaboration factor for a small curvature road working condition; The formula of the second human-machine collaboration factor is expressed as:

[0068] The formula of the third human-machine collaboration factor is expressed as:

[0069] The characteristic parameter of the first driving group determined according to the first driving group to which the driver belongs is expressed as c 1. c 2. k 1. k 2; Determined based on the vehicle operation data: The comprehensive lateral deviation at the vehicle center of mass is expressed as: ; The comprehensive lateral deviation at the preview point is expressed as: ; The lateral deviation of the vehicle at the vehicle center of mass is expressed as ; The normalized value of the yaw angle deviation at the vehicle center of mass is expressed as ; The lateral deviation of the vehicle at the preview point is expressed as ; The normalized value of the yaw angle deviation at the preview point is expressed as ; The first preset value is expressed as , the second preset value indicates , and satisfies .

[0070] In yet another embodiment of the present disclosure, the third key parameter includes: a real-time vehicle speed parameter and a real-time road parameter; The first real-time working condition includes: high-speed and large-curvature working condition, low-speed and small-curvature working condition, transition working condition and emergency working condition; The human-machine collaboration factor determination module 403 is used to When the real-time road parameter indicates that the road where the vehicle is located does not have a turning condition, determining that the first real-time operating condition of the vehicle is an emergency operating condition; When the real-time road parameter indicates that the road where the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is greater than a third preset value, and the real-time vehicle speed parameter is greater than a fourth preset value, determining that the first real-time operating condition of the vehicle is a high-speed and large-curvature operating condition; When the real-time road parameter indicates that the road where the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is less than a fifth preset value, and the real-time vehicle speed parameter is less than a sixth preset value, determining that the first real-time operating condition of the vehicle is a low-speed and small-curvature operating condition; When the real-time road parameter indicates that the road on which the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is less than or equal to the third preset value and greater than or equal to the fifth preset value, or the real-time vehicle speed parameter is less than or equal to the fourth preset value and greater than or equal to the sixth preset value, determining that the first real-time operating condition of the vehicle is a transitional operating condition; The human-machine collaboration factor determination module 403 is used to When the first real-time working condition is a high-speed and large-curvature working condition, determining the second human-machine cooperation factor as the first human-machine cooperation factor; When the first real-time operating condition is a low-speed and small-curvature operating condition, determining the third human-machine cooperation factor as the first human-machine cooperation factor; When the first real-time operating condition is a transitional operating condition, determining a first human-machine cooperation factor according to the second human-machine cooperation factor and the third human-machine cooperation factor; When the first real-time operating condition is an emergency operating condition, an early warning is initiated.

[0071] In another embodiment of the present disclosure, the driving group determination module 402 is used to train the driving group identification model in the following manner: Acquiring data to be trained from the vehicle operation data; Inputting the data to be trained into the fuzzy reasoning model for training, and introducing a first feedback mechanism to optimize the fuzzy reasoning model to obtain a trained fuzzy reasoning model; Performing artificial feature extraction on the data to be trained to obtain the first key parameter characterizing the vehicle running state and driving behavior characteristics, inputting the first key parameter into the deep neural network model for training, optimizing the performance of the deep neural network model by a hyperparameter tuning method, and evaluating the performance of the deep neural network model by a cross-validation method to obtain a trained deep neural network model; wherein the hyperparameter tuning method includes: grid search or random search or Bayesian optimization; The trained fuzzy inference model and the trained deep neural network model are fused in a linear weighted manner to obtain a driving group identification model, and a second feedback mechanism is introduced to optimize the driving group identification model to obtain a trained driving group identification model.

[0072] Based on the same disclosed concept, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the human-machine collaborative steering control method as described in any of the above embodiments are executed.

[0073] Through the description of the above implementation methods, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by hardware, or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0074] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present disclosure.

[0075] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the present embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0076] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0077] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A human-machine collaborative steering control method, characterized in that: include: Acquire the preprocessed vehicle operation data, and perform artificial feature extraction on the vehicle operation data to obtain a first key parameter characterizing the vehicle operation state and driving behavior characteristics; Inputting the vehicle operation data and the first key parameter into a driving group identification model to determine a first driving group to which the driver belongs; wherein the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model; the first driving group includes: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group; determining a first human-machine cooperation factor according to the vehicle operation data and a first driving group to which the driver belongs; A composite torque is determined based on the first human-machine cooperative factor, a first torque used by a vehicle driving system to control vehicle steering, and a second torque used by a driver to control vehicle steering, and the vehicle steering is controlled based on the composite torque.

2. The method according to claim 1, characterized in that The step of inputting the vehicle operation data and the first key parameter into a driving group identification model to determine the first driving group to which the driver belongs includes: Inputting the vehicle operation data into the fuzzy inference model to determine a first driving group label to which the driver belongs; Inputting the first key parameter into the deep neural network model to determine a second driving group label to which the driver belongs; The fuzzy inference model and the deep neural network model are fused in a linear weighted manner, and the first driving group to which the driver belongs is determined according to the first driving group label and the second driving group label.

3. The method according to claim 2, characterized in that The step of inputting the vehicle operation data into the fuzzy inference model to determine the first driving group label to which the driver belongs includes: Performing statistical analysis on the vehicle operation data using a dynamic sliding window to determine a second key parameter and a first key indicator calculated based on the second key parameter; Pre-constructing a priori fuzzy rules and a fuzzy inference engine, and inputting the second key parameter and the first key indicator into the fuzzy inference engine according to the a priori fuzzy rules to determine a first driving group label data set; The first driving group label data set is clustered using a K-means algorithm, and the driving group label with the largest number in the K-means algorithm cluster is determined as the first driving group label to which the driver belongs.

4. The method according to claim 3, characterized in that The step of inputting the second key parameter and the first key indicator into the fuzzy inference engine according to the priori fuzzy rule to determine a first driving group label data set includes: According to the priori fuzzy rule, the second key parameter and the first key indicator are input into the fuzzy inference engine to determine a second driving group label data set and timestamps corresponding to elements in the second driving group label data set; The data size of the second driving group label data set is adjusted according to the timestamp, and the adjusted second driving group label data set is determined as the first driving group label data set.

5. The method according to claim 2, characterized in that The deep neural network model includes: a deep learning model and a multimodal model; The step of inputting the first key parameter into the deep neural network model to determine a second driving group label to which the driver belongs includes: After normalizing the first key parameter, the first key parameter is input into the deep learning model to extract the first high-level feature; The first high-level feature and the first key parameter are input into the multimodal model to determine the second driving group label to which the driver belongs; wherein the multimodal model is constructed based on a CNN convolutional neural network model and a network model for processing time series data; the network model for processing time series data includes: an LSTM network model, an SRU network model, or a GRU network model.

6. The method according to claim 1, characterized in that The determining a first human-machine cooperation factor according to the vehicle operation data and the first driving group to which the driver belongs includes: Extracting a third key parameter from the vehicle operation data to determine a first real-time operating condition of the vehicle; Determine a first human-machine collaboration factor according to the first real-time working condition and a first relationship between the first real-time working condition and a human-machine collaboration factor model; wherein the human-machine collaboration factor model includes: a second human-machine collaboration factor for a large curvature road working condition and a third human-machine collaboration factor for a small curvature road working condition; The formula of the second human-machine collaboration factor is expressed as: The formula of the third human-machine collaboration factor is expressed as: The characteristic parameter of the first driving group determined according to the first driving group to which the driver belongs is expressed as c 1. c 2. k 1. k 2; Determined based on the vehicle operation data: The comprehensive lateral deviation at the vehicle center of mass is expressed as: ; The comprehensive lateral deviation at the preview point is expressed as: ; The lateral deviation of the vehicle at the vehicle center of mass is expressed as ; The normalized value of the yaw angle deviation at the vehicle center of mass is expressed as ; The lateral deviation of the vehicle at the preview point is expressed as ; The normalized value of the yaw angle deviation at the preview point is expressed as ; The first preset value is expressed as , the second preset value indicates , and satisfies .

7. The method according to claim 6, characterized in that The third key parameter includes: real-time vehicle speed parameter and real-time road parameter; The first real-time working condition includes: high-speed and large-curvature working condition, low-speed and small-curvature working condition, transition working condition and emergency working condition; The extracting the third key parameter from the vehicle operation data to determine the first real-time operating condition of the vehicle includes: When the real-time road parameter indicates that the road where the vehicle is located does not have a turning condition, determining that the first real-time operating condition of the vehicle is an emergency operating condition; When the real-time road parameter indicates that the road where the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is greater than a third preset value, and the real-time vehicle speed parameter is greater than a fourth preset value, determining that the first real-time operating condition of the vehicle is a high-speed and large-curvature operating condition; When the real-time road parameter indicates that the road where the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is less than a fifth preset value, and the real-time vehicle speed parameter is less than a sixth preset value, determining that the first real-time operating condition of the vehicle is a low-speed and small-curvature operating condition; When the real-time road parameter indicates that the road on which the vehicle is located has a turning condition, and the road curvature value in the real-time road parameter is less than or equal to the third preset value and greater than or equal to the fifth preset value, or the real-time vehicle speed parameter is less than or equal to the fourth preset value and greater than or equal to the sixth preset value, determining that the first real-time operating condition of the vehicle is a transitional operating condition; The determining a first human-machine collaboration factor according to the first real-time operating condition and a first relationship between the first real-time operating condition and a human-machine collaboration factor model includes: When the first real-time working condition is a high-speed and large-curvature working condition, determining the second human-machine cooperation factor as the first human-machine cooperation factor; When the first real-time operating condition is a low-speed and small-curvature operating condition, determining the third human-machine cooperation factor as the first human-machine cooperation factor; When the first real-time operating condition is a transitional operating condition, determining a first human-machine cooperation factor according to the second human-machine cooperation factor and the third human-machine cooperation factor; When the first real-time operating condition is an emergency operating condition, an early warning is initiated.

8. The method according to claim 1, characterized in that The driving group identification model is trained in the following manner: Acquiring data to be trained from the vehicle operation data; Inputting the data to be trained into the fuzzy reasoning model for training, and introducing a first feedback mechanism to optimize the fuzzy reasoning model to obtain a trained fuzzy reasoning model; Performing artificial feature extraction on the data to be trained to obtain the first key parameter characterizing the vehicle running state and driving behavior characteristics, inputting the first key parameter into the deep neural network model for training, optimizing the performance of the deep neural network model by a hyperparameter tuning method, and evaluating the performance of the deep neural network model by a cross-validation method to obtain a trained deep neural network model; wherein the hyperparameter tuning method includes: grid search or random search or Bayesian optimization; The trained fuzzy inference model and the trained deep neural network model are fused in a linear weighted manner to obtain a driving group identification model, and a second feedback mechanism is introduced to optimize the driving group identification model to obtain a trained driving group identification model.

9. A human-machine collaborative steering control device, characterized in that: include: A key parameter extraction module is used to obtain the pre-processed vehicle operation data and perform artificial feature extraction on the vehicle operation data to obtain a first key parameter characterizing the vehicle operation state and driving behavior characteristics; A driving group determination module, configured to input the vehicle operation data and the first key parameter into a driving group identification model to determine a first driving group to which the driver belongs; wherein the driving group identification model is constructed based on a fuzzy reasoning model and a deep neural network model; the first driving group includes: a stable driving group, an aggressive driving group, a conservative driving group, and a transient driving group; a human-machine cooperation factor determination module, configured to determine a first human-machine cooperation factor according to the vehicle operation data and a first driving group to which the driver belongs; The steering control module is used to determine the composite torque based on the first human-machine cooperation factor, the first torque used by the vehicle driving system to control the vehicle steering, and the second torque used by the driver to control the vehicle steering, and control the vehicle steering according to the composite torque.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the human-machine collaborative steering control method as described in any one of claims 1 to 8 is executed.

Images