Driving behavior correction method and device, electronic equipment and storage medium

Abstract

The invention provides a driving behavior correction method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining sensor data of a vehicle, obtaining initial eye movement data through an eye tracker, screening out target eye movement data from the initial eye movement data, obtaining a predicted driving behavior according to the sensor data and the target eye movement data, and correcting the driving behavior according to the predicted driving behavior. And determining a deviation value of the driving behavior according to the predicted driving behavior and a pre-acquired actual driving behavior of the driver, and if the deviation value of the driving behavior is greater than a preset first deviation threshold, correcting the predicted driving behavior. According to the method, the driving behavior is predicted by combining the sensor data and the eye movement data of the driver, the predicted driving behavior is obtained, the predicted driving behavior is further corrected, and the predicted driving behavior can be closer to the actual driving behavior of the driver.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of autonomous driving technology, and in particular to a driving behavior correction method, device, electronic device, and storage medium. Background Art

[0002] In the automotive field, shadow mode is a technology that helps improve autonomous driving technology. When shadow mode is turned on, the vehicle's various sensors continuously collect road information during normal driving. Without affecting the driver, the vehicle uses this information to generate simulated driving logic, and then iterates and optimizes the autonomous driving algorithm by comparing the simulated driving logic with the actual driving logic.

[0003] However, the existing shadow mode only relies on the road information collected by sensors to simulate the driving logic, which makes it difficult for the simulated driving logic to fully represent the driver's driving behavior and habits, and thus the simulated driving logic is even worse than the driver's actual driving behavior. Therefore, in order to improve the auxiliary effect of autonomous driving, it is particularly important to make the simulated driving logic consistent with the driver's driving behavior. Summary of the invention

[0004] In view of the above problems, a driving behavior correction method, device, electronic device and storage medium are proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] In a first aspect, an embodiment of the present application provides a driving behavior correction method, which is applied to a vehicle, wherein the vehicle includes an eye tracker, and the method includes:

[0006] Acquiring sensor data of the vehicle and acquiring initial eye movement data through the eye tracker;

[0007] Filtering target eye movement data from the initial eye movement data;

[0008] Obtaining a predicted driving behavior based on the sensor data and the target eye movement data;

[0009] Determining a deviation value of the driving behavior according to the predicted driving behavior and the actual driving behavior of the driver acquired in advance;

[0010] If the deviation value of the driving behavior is greater than a preset first deviation threshold, the predicted driving behavior is corrected.

[0011] Optionally, the initial eye movement data includes a fixation point and a fixation time of the fixation point;

[0012] The step of filtering out target eye movement data from the initial eye movement data comprises:

[0013] Acquire an image in front of the vehicle;

[0014] Mapping the gaze point to the image in front of the vehicle to obtain an image of the gaze point;

[0015] Classifying the image of the gaze point to obtain an image classification result;

[0016] Judging whether the initial eye movement data is valid according to the judgment result of the image classification result and the judgment result of the fixation time;

[0017] If the initial eye movement data is valid, the initial eye movement data is used as the target eye movement data.

[0018] Optionally, judging whether the initial eye movement data is valid according to the judgment result of the image classification result and the judgment result of the fixation time includes:

[0019] Determining whether the image classification result is a preset key target;

[0020] If the image classification result is the key target, determining whether the fixation time is greater than or equal to a preset time threshold;

[0021] If the gaze time is greater than or equal to the time threshold, it is determined that the initial eye movement data is valid.

[0022] Optionally, obtaining the predicted driving behavior according to the sensor data and the target eye movement data includes:

[0023] The sensor data and the target eye movement data are input into a pre-generated driving behavior output model to obtain a predicted driving behavior.

[0024] Optionally, the driving behavior output model includes an operator, wherein the operator is used to process the target eye movement data;

[0025] The modifying of the predicted driving behavior comprises:

[0026] Get the loss value and target threshold;

[0027] comparing the loss value to the target threshold;

[0028] If the loss value is greater than the target threshold, the weight of the operator is reduced;

[0029] If the loss value is less than or equal to the target threshold, the weight of the operator is increased.

[0030] Optionally, obtaining the loss value includes:

[0031] Obtain high-precision map data;

[0032] Determining a standard driving behavior based on the high-precision map data and the sensor data;

[0033] A loss value is determined based on the predicted driving behavior and the standard driving behavior.

[0034] Optionally, the acquiring target threshold includes:

[0035] Determining a data deviation value according to the high-precision map data and the sensor data;

[0036] Comparing the data deviation value with a preset second deviation threshold;

[0037] If the data deviation value is smaller than the second deviation threshold, the data deviation value is used as the target threshold.

[0038] In a second aspect, an embodiment of the present application provides a driving behavior correction device, which is applied to a vehicle, wherein the vehicle includes an eye tracker, and the device includes:

[0039] a data acquisition module, configured to acquire sensor data of the vehicle and to acquire initial eye movement data through the eye tracker;

[0040] A data screening module, configured to screen target eye movement data from the initial eye movement data;

[0041] a driving behavior prediction module, configured to obtain a predicted driving behavior based on the sensor data and the target eye movement data;

[0042] a driving behavior deviation determination module, configured to determine a driving behavior deviation value according to the predicted driving behavior and the pre-acquired actual driving behavior of the driver;

[0043] The driving behavior correction module is configured to correct the predicted driving behavior if the deviation value of the driving behavior is greater than a preset first deviation threshold.

[0044] Optionally, the initial eye movement data includes a fixation point and a fixation time of the fixation point;

[0045] The data screening module comprises:

[0046] An image acquisition submodule, configured to acquire an image in front of the vehicle;

[0047] A gaze point mapping submodule is configured to map the gaze point to the image in front of the vehicle to obtain an image of the gaze point;

[0048] An image classification submodule is configured to classify the image of the gaze point to obtain an image classification result;

[0049] A data validity judgment submodule is configured to judge whether the initial eye movement data is valid according to the judgment result of the image classification result and the judgment result of the fixation time;

[0050] The target eye movement data determination submodule is configured to use the initial eye movement data as the target eye movement data if the initial eye movement data is valid.

[0051] Optionally, the data validity judgment submodule includes:

[0052] An image classification result judging unit, configured to judge whether the image classification result is a preset key target;

[0053] a gaze time judgment unit, configured to judge whether the gaze time is greater than or equal to a preset time threshold if the image classification result is the key target;

[0054] The data validity determination unit is configured to determine that the initial eye movement data is valid if the gaze time is greater than or equal to the time threshold.

[0055] Optionally, the driving behavior prediction module includes:

[0056] The driving behavior output model processing submodule is configured to input the sensor data and the target eye movement data into a pre-generated driving behavior output model to obtain a predicted driving behavior.

[0057] Optionally, the driving behavior output model includes an operator, wherein the operator is used to process the target eye movement data;

[0058] The driving behavior correction module comprises:

[0059] A pre-data acquisition submodule, configured to obtain a loss value and a target threshold;

[0060] a data comparison submodule, configured to compare the loss value with the target threshold;

[0061] A first weight adjustment submodule, configured to reduce the weight of the operator if the loss value is greater than the target threshold;

[0062] The second weight adjustment submodule is configured to increase the weight of the operator if the loss value is less than or equal to the target threshold.

[0063] Optionally, the pre-acquisition data submodule includes:

[0064] A high-precision map data acquisition unit, configured to acquire high-precision map data;

[0065] a standard data determination unit, configured to determine a standard driving behavior according to the high-precision map data and the sensor data;

[0066] The loss value determining unit is configured to determine a loss value according to the predicted driving behavior and the standard driving behavior.

[0067] Optionally, the pre-acquisition data submodule includes:

[0068] a data deviation value determining unit, configured to determine a data deviation value according to the high-precision map data and the sensor data;

[0069] a data deviation value comparison unit, for comparing the data deviation value with a preset second deviation threshold;

[0070] The target threshold determination unit uses the data deviation value as the target threshold if the data deviation value is less than the second deviation threshold.

[0071] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the instructions to implement any of the driving behavior correction methods described above.

[0072] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute any of the driving behavior correction methods described above.

[0073] In an embodiment of the present application, the sensor data of the vehicle is obtained and the initial eye movement data is obtained through an eye tracker, the target eye movement data is screened out from the initial eye movement data, and the predicted driving behavior is obtained according to the sensor data and the target eye movement data. The deviation value of the driving behavior is determined according to the predicted driving behavior and the actual driving behavior of the driver obtained in advance. If the deviation value of the driving behavior is greater than the preset first deviation threshold, the predicted driving behavior is corrected. The present application predicts the driving behavior by combining the sensor data and the driver's eye movement data to obtain the predicted driving behavior. Since the eye movement data assists in characterizing the driver's driving intention, the predicted driving behavior is closer to the driver's actual driving behavior. In addition, the present application determines the deviation value of the driving behavior according to the predicted driving behavior and the actual driving behavior of the driver obtained in advance. When the deviation value of the driving behavior is greater than the first deviation threshold, the predicted driving behavior is corrected. By continuously correcting the predicted driving behavior, it is possible to avoid the predicted driving behavior from deviating from the actual driving behavior of the driver due to excessive deviation, so that the predicted driving behavior is closer and closer to the actual driving behavior of the driver, thereby achieving the effect of improving intelligent driving assistance.

[0074] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0076] Figure 1 is a flowchart of the steps of a driving behavior correction method provided by an embodiment of the present application;

[0077] Figure 2 It is a schematic diagram of a process for determining a deviation value of a driving behavior provided by an embodiment of the present application;

[0078] Figure 3 is a schematic diagram of a flow chart for correcting a predicted driving behavior provided by an embodiment of the present application;

[0079] Figure 4 is a device block diagram of a driving behavior correction device provided in an embodiment of the present application;

[0080] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0082] In conjunction with the accompanying drawings, a driving behavior correction method, device, storage medium and vehicle provided in an embodiment of the present application are described in detail through specific embodiments and their application scenarios.

[0083] Figure 1 is a flowchart of a driving behavior correction method provided by an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0084] Step 101, obtaining sensor data of the vehicle and obtaining initial eye movement data through an eye tracker.

[0085] Step 102: Filter out target eye movement data from the initial eye movement data.

[0086] In some embodiments of the present application, the vehicle includes a sensor, wherein the sensor may be a millimeter wave radar, a camera, a lidar, etc.

[0087] like Figure 2 As shown, when the shadow mode is turned on for the vehicle, data collected by the sensor is obtained during the normal driving of the vehicle. The data collected by the sensor is sensor data, wherein the sensor data may be road data.

[0088] In addition, during the driving process, the driver's eye gaze direction and gaze point are related to driving behavior, so the driving behavior can be predicted based on the driver's eye movement data. In order to obtain the driver's eye movement data, an eye tracker can be arranged on the vehicle in advance. The eye tracker is a special instrument for collecting eye movement data. The working principle of the eye tracker is: irradiate the user's eyes with infrared rays, and then use a camera to collect infrared rays reflected from the user's cornea and retina. Due to the physiological structure and physical characteristics of the eyeball, under the premise that the relative position of the light source and the head remains unchanged, the light spot formed by corneal reflection will not move, and the direction of the ray reflected from the retina indicates the direction of the pupil. Finally, according to the angle between the cornea and the light reflected from the pupil, the direction of eye movement can be calculated, and further eye movement data such as the gaze point, the trajectory diagram of the gaze point, and the dwell time of each gaze point can be obtained.

[0089] In order to ensure that the eye tracker can completely scan the position of the driver's eyes, the eye tracker can be placed on the A-pillar in the cabin or on the steering wheel. After the eye tracker is placed, the driver's initial eye movement data can be collected through the eye tracker.

[0090] When the initial eye movement data of the driver is obtained, the initial eye movement data may be screened, and the screened eye movement data is the target eye movement data.

[0091] Step 103, obtaining predicted driving behavior based on the sensor data and the target eye movement data.

[0092] Step 104 , determining a deviation value of the driving behavior according to the predicted driving behavior and the actual driving behavior of the driver acquired in advance.

[0093] Step 105: If the deviation value of the driving behavior is greater than a preset first deviation threshold, the predicted driving behavior is corrected.

[0094] In some embodiments of the present application, the vehicle includes an intelligent driving domain controller. The driving behavior may include vehicle longitudinal data and vehicle lateral data, wherein the vehicle longitudinal data may include vehicle speed, braking degree, etc., and the vehicle lateral data may include a steering wheel angle.

[0095] like Figure 2 As shown, after acquiring the sensor data and the target eye movement data, the intelligent driving domain controller can predict the driving behavior based on the sensor data and the target eye movement data to obtain the predicted driving behavior.

[0096] The intelligent driving domain controller can also obtain the actual driving behavior of the driver. When the intelligent driving domain controller obtains the actual driving behavior of the driver, the intelligent driving domain controller subtracts the predicted driving behavior from the actual driving behavior of the driver to obtain the deviation value of the driving behavior between the two.

[0097] The first deviation threshold is a preset threshold used to determine whether the predicted driving behavior is significantly different from the actual driving behavior of the driver. After obtaining the deviation value of the driving behavior, the deviation value of the driving behavior is compared with the preset first deviation threshold.

[0098] If the deviation value of the driving behavior is greater than the first deviation threshold, indicating that there is an obvious deviation between the predicted driving behavior and the driver's actual driving behavior, the predicted driving behavior and sensor data can be sent to the cloud, and the cloud corrects the predicted driving behavior based on the predicted driving behavior and sensor data.

[0099] If the deviation value of the driving behavior is less than or equal to the first deviation threshold, it means that the predicted driving behavior is consistent with the actual driving behavior of the driver. Then, the steps of obtaining sensor data and the driver's eye movement data, filtering the eye movement data, obtaining target eye movement data, and inputting the sensor data and the target eye movement data into the intelligent driving domain controller can be continued. The intelligent driving domain controller predicts the driving behavior based on the sensor data and the target eye movement data to obtain the predicted driving behavior.

[0100] The present application predicts driving behavior by combining sensor data and the driver's eye movement data to obtain predicted driving behavior. Since the eye movement data helps to characterize the driver's driving intention, the predicted driving behavior is closer to the driver's actual driving behavior. In addition, the present application determines the deviation value of the driving behavior based on the predicted driving behavior and the driver's actual driving behavior acquired in advance. When the deviation value of the driving behavior is greater than the first deviation threshold, the predicted driving behavior is corrected. By continuously correcting the predicted driving behavior, it is possible to avoid the predicted driving behavior deviating from the driver's actual driving behavior due to excessive deviation, so that the predicted driving behavior is closer and closer to the driver's actual driving behavior, thereby achieving the effect of improving intelligent driving assistance.

[0101] Furthermore, in some embodiments of the present application, step 102 may also include the following steps:

[0102] Step 1021, acquiring an image in front of the vehicle.

[0103] Step 1022, mapping the gaze point to the image in front of the vehicle to obtain an image of the gaze point.

[0104] Step 1023, classify the image of the gaze point to obtain an image classification result.

[0105] Step 1024, judging whether the initial eye movement data is valid according to the judgment result of the image classification result and the judgment result of the fixation time.

[0106] Step 1025: If the initial eye movement data is valid, the initial eye movement data is used as the target eye movement data.

[0107] In some embodiments of the present application, the initial eye movement data includes a fixation point and a fixation time of the fixation point.

[0108] The specific implementation process of filtering out the target eye movement data from the initial eye movement data is as follows: an image in front of the vehicle is obtained through an on-board camera (e.g., a front-view camera) on the vehicle. The fixation point included in the initial eye movement data is mapped to the image in front of the vehicle to obtain an image of the fixation point. The image of the fixation point is classified by an image processing module on the vehicle to obtain an image classification result, for example, the image classification result can be a road, a traffic sign, a pedestrian, a vehicle, etc. When the image classification result is obtained, the image classification result is judged to obtain a judgment result of the image classification result, and the fixation time included in the initial eye movement data is judged to obtain a judgment result of the fixation time. Further, based on the judgment result of the image classification result and the judgment result of the fixation time, it is judged whether the initial eye movement data is valid. When the initial eye movement data is valid, the initial eye movement data is directly used as the target eye movement data. When the initial eye movement data is invalid, the initial eye movement data is not used as the target eye movement data.

[0109] The present application determines the validity of the initial eye movement data by mapping the gaze point to the image in front of the vehicle, classifying the image of the gaze point, and combining the image classification results and the gaze time judgment results. It can screen out truly valid eye movement data as target eye movement data, which helps to improve the accuracy of the target eye movement data and avoid interference of invalid eye movement data on subsequent analysis or processing.

[0110] Furthermore, in some embodiments of the present application, step 1024 may also include the following steps:

[0111] Sub-step 11, determining whether the image classification result is a preset key target.

[0112] Sub-step 12: If the image classification result is a key target, determine whether the gaze time is greater than or equal to a preset time threshold.

[0113] Sub-step 13: If the fixation time is greater than or equal to the time threshold, it is determined that the initial eye movement data is valid.

[0114] In some embodiments of the present application, the preset key targets are a predefined set of targets or areas, which are usually closely related to the driver's attention distribution, driving safety, etc. when driving a vehicle, among which the preset key targets can be lane lines, traffic lights, etc.

[0115] The preset time threshold is a preset value used to determine whether the gaze time is long enough.

[0116] After obtaining the image classification result, it is determined whether the image classification result is a key target. If the image classification result is a key target, it means that the driver's gaze is focused on the key target, and then it is further determined whether the dwell time of the gaze point, that is, the gaze time is greater than or equal to the preset time threshold.

[0117] If the fixation time is greater than or equal to the time threshold, the initial eye movement data is determined to be valid.

[0118] If the image classification result is a non-critical target and the gaze time is less than the time threshold, or the image classification result is a critical target and the gaze time is less than the time threshold, or the image classification result is a non-critical target and the gaze time is greater than or equal to the time threshold, then the initial eye movement data is determined to be invalid and the initial eye movement data is discarded, or new initial eye movement data is reacquired.

[0119] This application determines whether the image classification result is a preset key target. If it is a key target, the gaze time is further compared with the preset time threshold, which can effectively screen out valid initial eye movement data and provide an accurate data basis for subsequent more in-depth analysis based on eye movement data (such as the construction of a driving behavior output model, etc.).

[0120] Furthermore, in some embodiments of the present application, step 103 may also include the following steps:

[0121] Step 1031 , input the sensor data and the target eye movement data into a pre-generated driving behavior output model to obtain a predicted driving behavior.

[0122] In some embodiments of the present application, the driving behavior output model is a prediction model that can predict sensor data and target eye movement data to determine the predicted driving behavior. The driving behavior output model is integrated in the intelligent driving domain controller.

[0123] like Figure 2 As shown, the acquired sensor data and target eye movement data are used as inputs of a pre-generated driving behavior output model, and are input into the driving behavior output model. The driving behavior output model can automatically output the predicted driving behavior.

[0124] This application obtains predicted driving behavior by simultaneously inputting sensor data and target eye movement data into the driving behavior output model, that is, adding the driver's eye movement data to predict driving behavior on the basis of sensor data. Since eye movement data reflects the driver's visual attention, the driver's eye movement data is fused on the basis of sensor data. This fusion of multi-source data can more comprehensively describe the driver's driving behavior, making the predicted driving behavior more consistent with the driver's driving behavior, that is, making the predicted driving behavior more and more like an "experienced driver", thereby improving the accuracy of the driver's driving behavior prediction.

[0125] Furthermore, in some embodiments of the present application, step 105 may also include the following steps:

[0126] Step 1051, obtain the loss value and target threshold.

[0127] Step 1052, compare the loss value with the target threshold.

[0128] Step 1053: If the loss value is greater than the target threshold, the weight of the operator is reduced.

[0129] Step 1054: If the loss value is less than or equal to the target threshold, the weight of the operator is increased.

[0130] In some embodiments of the present application, a driving behavior output model is integrated in the intelligent driving domain controller, wherein the driving behavior output model includes several operators, and different operators are used to process different data. For example, the driving behavior output model has corresponding operators for processing sensor data acquired by a lidar, corresponding operators for processing sensor data acquired by a millimeter-wave radar, corresponding operators for processing data acquired by a camera, and corresponding operators for processing target eye movement data.

[0131] like Figure 3 As shown, when the cloud receives the predicted driving behavior and sensor data sent by the intelligent driving domain controller, the cloud obtains the loss value and the target threshold based on the predicted driving behavior and sensor data.

[0132] When the loss value and the target threshold are obtained, the loss value and the target threshold are compared to determine whether the loss value is greater than the target threshold.

[0133] When the loss value is greater than the target threshold, it means that the target eye movement data causes a deviation between the predicted driving behavior and the driver's actual driving behavior. In this case, the weight of the operator used to process the target eye movement data is reduced to reduce the impact of the target eye movement data on the fitting results of the driving behavior output model.

[0134] When the loss value is less than or equal to the target threshold, it means that the target eye movement data has successfully guided the predicted driving behavior to shift towards the driver's actual driving behavior, and is even less influential than the high-precision map data. In this case, the weight of the operator used to process the target eye movement data is increased so that the predicted driving behavior obtained by the driving behavior output model is more in line with the driver's actual driving behavior.

[0135] This application reduces the weight of the operator that processes the target eye movement data when the loss value is greater than the target threshold. This helps to reduce the deviation caused by the target eye movement data, because if the target eye movement data causes the predicted driving behavior to deviate greatly from the driver's actual driving behavior, reducing the influence of the target eye movement data in the driving behavior output model can make the fitting result of the driving behavior output model more accurate. In addition, this application increases the weight of the operator that processes the target eye movement data when the loss value is less than or equal to the target threshold, because at this time the target eye movement data successfully guides the predicted driving behavior to deviate toward the driver's actual driving behavior, and increasing its weight can make the predicted driving behavior obtained by the driving behavior output model more in line with the driver's actual driving behavior. Finally, this application dynamically adjusts the weight of the operator that processes the target eye movement data in the driving behavior output model according to the comparison between the loss value and the target threshold, so that the driving behavior output model can be continuously optimized and better adapted to the influence of different data on the predicted driving behavior, thereby improving the accuracy of the driving behavior output model in predicting driving behavior.

[0136] Furthermore, in some embodiments of the present application, step 1051 may also include the following steps:

[0137] Sub-step 21, obtaining high-precision map data.

[0138] Sub-step 22, determining standard driving behavior based on high-precision map data and sensor data.

[0139] Sub-step 23, determining a loss value according to the predicted driving behavior and the standard driving behavior.

[0140] In some embodiments of the present application, high-precision map data is obtained through the cloud, wherein the high-precision map data is obtained by collecting road information using a map data collection vehicle.

[0141] like Figure 3 As shown in the figure, when high-precision map data is obtained, the cloud can obtain standard driving behavior based on the high-precision map data and sensor data. When the cloud obtains the standard driving behavior, the cloud subtracts the predicted driving behavior from the standard driving behavior to obtain the loss value, where the loss value is the key factor for the back propagation of the driving behavior output model and represents the error that the driving behavior output model needs to reduce.

[0142] The present application determines the loss value between the predicted driving behavior and the standard driving behavior. The loss value represents the error that the driving behavior output model needs to reduce. By continuously adjusting the parameters in the driving behavior output model according to the loss value, the predicted driving behavior output by the driving behavior output model can be corrected, so that the driving behavior predicted by the driving behavior output model is continuously close to the actual driving behavior of the driver, thereby improving the accuracy of driving behavior prediction.

[0143] Furthermore, in some embodiments of the present application, step 1051 may also include the following steps:

[0144] Sub-step 31, determining a data deviation value based on the high-precision map data and the sensor data.

[0145] Sub-step 32, comparing the data deviation value with a preset second deviation threshold.

[0146] Sub-step 33: If the data deviation value is less than the second deviation threshold, the data deviation value is used as the target threshold.

[0147] In some embodiments of the present application, Figure 3 As shown, when the cloud obtains high-precision map data and sensor data, it subtracts the high-precision map data and the sensor data to obtain a data deviation value.

[0148] After obtaining the data deviation value, the data deviation value is compared with a preset second deviation threshold value to determine whether the data deviation value is less than the second deviation threshold value. If the data deviation value is less than the second deviation threshold value, the data deviation value is used as the target threshold value. If the data deviation value is greater than or equal to the second deviation threshold value, the data deviation value is not used as the target threshold value.

[0149] The present application obtains a data deviation value by subtracting the high-precision map data and the sensor data, and compares the data deviation value with a preset second deviation threshold. It can screen out a data deviation value with a smaller deviation as the target threshold, thereby improving the credibility of the target threshold and reducing the impact of the data deviation value with a larger error on subsequent operations.

[0150] Corresponding to the method provided in the above-mentioned driving behavior correction method embodiment of the present application, see Figure 4 , the present application also provides a device block diagram of a driving behavior correction device. In this embodiment, the device includes:

[0151] A data acquisition module 401 is configured to acquire sensor data of the vehicle and acquire initial eye movement data through an eye tracker;

[0152] A data screening module 402 is configured to screen target eye movement data from the initial eye movement data;

[0153] The driving behavior prediction module 403 is configured to obtain a predicted driving behavior based on the sensor data and the target eye movement data;

[0154] The driving behavior deviation determination module 404 is configured to determine a deviation value of the driving behavior according to the predicted driving behavior and the actual driving behavior of the driver acquired in advance;

[0155] The driving behavior correction module 405 is configured to correct the predicted driving behavior if the deviation value of the driving behavior is greater than a preset first deviation threshold.

[0156] Optionally, the initial eye movement data includes a fixation point and a fixation time of the fixation point;

[0157] The data screening module 402 includes:

[0158] An image acquisition submodule, configured to acquire an image in front of the vehicle;

[0159] A gaze point mapping submodule is configured to map the gaze point to an image in front of the vehicle to obtain an image of the gaze point;

[0160] An image classification submodule is configured to classify the image of the gaze point to obtain an image classification result;

[0161] A data validity judgment submodule is configured to judge whether the initial eye movement data is valid according to the judgment result of the image classification result and the judgment result of the fixation time;

[0162] The target eye movement data determination submodule is configured to use the initial eye movement data as the target eye movement data if the initial eye movement data is valid.

[0163] Optionally, the data validity judgment submodule includes:

[0164] An image classification result judging unit, configured to judge whether the image classification result is a preset key target;

[0165] A gaze time judgment unit, configured to judge whether the gaze time is greater than or equal to a preset time threshold if the image classification result is a key target;

[0166] The data validity determination unit is configured to determine that the initial eye movement data is valid if the gaze time is greater than or equal to a time threshold.

[0167] Optionally, the driving behavior prediction module 403 includes:

[0168] The driving behavior output model processing submodule is configured to input the sensor data and the target eye movement data into a pre-generated driving behavior output model to obtain a predicted driving behavior.

[0169] Optionally, the driving behavior output model includes an operator, wherein the operator is used to process the target eye movement data;

[0170] The driving behavior correction module 405 includes:

[0171] A pre-data acquisition submodule, configured to obtain a loss value and a target threshold;

[0172] a data comparison submodule, configured to compare the loss value with a target threshold;

[0173] A first weight adjustment submodule is configured to reduce the weight of the operator if the loss value is greater than a target threshold;

[0174] The second weight adjustment submodule is configured to increase the weight of the operator if the loss value is less than or equal to the target threshold.

[0175] Optionally, the pre-acquisition data submodule includes:

[0176] A high-precision map data acquisition unit, configured to acquire high-precision map data;

[0177] a standard data determination unit configured to determine a standard driving behavior based on the high-precision map data and the sensor data;

[0178] The loss value determination unit is configured to determine a loss value according to the predicted driving behavior and the standard driving behavior.

[0179] Optionally, the pre-acquisition data submodule includes:

[0180] A data deviation value determination unit, which determines a data deviation value according to high-precision map data and sensor data;

[0181] a data deviation value comparison unit, for comparing the data deviation value with a preset second deviation threshold;

[0182] The target threshold determination unit uses the data deviation value as the target threshold value if the data deviation value is less than the second deviation threshold value.

[0183] Figure 5 This is a structural diagram of an electronic device M00 provided in an embodiment of the present application. In the figure, the electronic device M00 includes a processor M01 and a memory M02. The memory M02 stores programs or instructions that can be run on the processor M01. When the program or instruction is executed by the processor M01, the various steps of the above-mentioned driving behavior correction method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0184] In an embodiment of the present application, the memory M02 may be used to store software programs and various data. The memory M02 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory M02 may include a volatile memory or a non-volatile memory, or the memory M02 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory M02 in the embodiment of the present application includes but is not limited to these and any other suitable types of memories.

[0185] The processor M01 may include one or more processing units; optionally, the processor M01 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor M01.

[0186] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned driving behavior correction method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0187] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0188] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned driving behavior correction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0190] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A driving behavior correction method, characterized in that: Applied to a vehicle, the vehicle comprising an eye tracker, the method comprising: Acquiring sensor data of the vehicle and acquiring initial eye movement data through the eye tracker; Filtering target eye movement data from the initial eye movement data; Obtaining a predicted driving behavior based on the sensor data and the target eye movement data; Determining a deviation value of the driving behavior according to the predicted driving behavior and the actual driving behavior of the driver acquired in advance; If the deviation value of the driving behavior is greater than a preset first deviation threshold, the predicted driving behavior is corrected.

2. The method according to claim 1, characterized in that The initial eye movement data includes a fixation point and a fixation time of the fixation point; The step of filtering out target eye movement data from the initial eye movement data comprises: Acquire an image in front of the vehicle; Mapping the gaze point to the image in front of the vehicle to obtain an image of the gaze point; Classifying the image of the gaze point to obtain an image classification result; Judging whether the initial eye movement data is valid according to the judgment result of the image classification result and the judgment result of the fixation time; If the initial eye movement data is valid, the initial eye movement data is used as the target eye movement data.

3. The method according to claim 2, characterized in that The determining whether the initial eye movement data is valid according to the determination result of the image classification result and the determination result of the fixation time includes: Determining whether the image classification result is a preset key target; If the image classification result is the key target, determining whether the fixation time is greater than or equal to a preset time threshold; If the gaze time is greater than or equal to the time threshold, it is determined that the initial eye movement data is valid.

4. The method according to claim 1, characterized in that: The predicted driving behavior obtained according to the sensor data and the target eye movement data includes: The sensor data and the target eye movement data are input into a pre-generated driving behavior output model to obtain a predicted driving behavior.

5. The method according to claim 4, characterized in that The driving behavior output model includes an operator, wherein the operator is used to process the target eye movement data; The modifying of the predicted driving behavior comprises: Get the loss value and target threshold; comparing the loss value to the target threshold; If the loss value is greater than the target threshold, the weight of the operator is reduced; If the loss value is less than or equal to the target threshold, the weight of the operator is increased.

6. The method according to claim 5, characterized in that The obtaining of the loss value comprises: Obtain high-precision map data; Determining a standard driving behavior based on the high-precision map data and the sensor data; A loss value is determined based on the predicted driving behavior and the standard driving behavior.

7. The method according to claim 6, characterized in that The acquisition target threshold comprises: Determining a data deviation value according to the high-precision map data and the sensor data; Comparing the data deviation value with a preset second deviation threshold; If the data deviation value is smaller than the second deviation threshold, the data deviation value is used as the target threshold.

8. A driving behavior correction device, characterized in that: Applied to a vehicle, the vehicle includes an eye tracker, and the device includes: a data acquisition module, configured to acquire sensor data of the vehicle and to acquire initial eye movement data through the eye tracker; A data screening module, configured to screen target eye movement data from the initial eye movement data; a driving behavior prediction module, configured to obtain a predicted driving behavior based on the sensor data and the target eye movement data; a driving behavior deviation determination module, configured to determine a driving behavior deviation value according to the predicted driving behavior and the pre-acquired actual driving behavior of the driver; The driving behavior correction module is configured to correct the predicted driving behavior if the deviation value of the driving behavior is greater than a preset first deviation threshold.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the driving behavior correction method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the driving behavior correction method according to any one of claims 1 to 7.

Images