A method and system for autonomous driving data collection and closed-loop management

Abstract

A method and system for autonomous driving data collection and closed-loop management. The method includes: obtaining vehicle driving data (S102); preprocessing the vehicle driving data (S104); screening the preprocessed vehicle driving data according to a preset screening rule through a pre-trained neural network to obtain incremental data (S106); storing the incremental data, or sending the incremental data to the cloud (S108). This method can effectively screen out high-value incremental data, thereby reducing the system data storage volume and / or the requirement for data transmission bandwidth. The above system can be installed on the vehicle in a retrofit or pre-installed manner, does not depend on a specific vehicle model, and does not require the vehicle to be equipped with a costly truth system such as a lidar, greatly improving the convenience of system use and facilitating rapid and large-scale application.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and in particular, to a method and system for autonomous driving data collection and closed-loop management. Background Art

[0002] In recent years, the autonomous driving technology has developed rapidly. However, the large-scale deployment of autonomous vehicles is still restricted by the sensor cost and the ability of perception and decision-making algorithms. Among them, the ability of perception and decision-making algorithms depends on the performance of the neural network adopted, such as the performance of the deep neural network (DNN). The improvement of the neural network performance depends on the collection and accumulation of autonomous driving data.

[0003] Currently, there are generally the following two methods for collecting autonomous driving data: One is to use professional data collection vehicles and road test vehicles to collect the perceived data; the other is to deploy a data collection system on mass-produced vehicles, and upload the vehicle perception, status and control data to the cloud after a certain processing. However, when using collection vehicles and road test vehicles to collect data through a large number of road tests, on the one hand, there are problems such as a small collection scenario range, a large amount of single-vehicle data collection, recording a large amount of redundant data, and the need to be equipped with a ground truth system such as a lidar; on the other hand, the cost of exploring valuable data is gradually increasing, making the average value density of the data sparser and the collection cost higher. Collecting data using mass-produced vehicles enriches the quantity and diversity of the data, and also greatly saves the data collection cost. However, currently, the data uploaded to the cloud by most automobile enterprises through T-box and other means is still mainly numerical data such as the dynamic parameters of the vehicle and the power system, and does not support the collection and transmission of data such as images and videos, which is not sufficient to support the training of the perception and decision-making system DNN for autonomous driving, and there is still a problem of screening high-value data. Tesla has initially explored deploying a data collection system in mass-produced vehicles, and screening high-value data through methods such as the "shadow mode" and data scoring mechanism. However, its high-value data screening mechanism is not clear, which may have relatively high requirements for the system transmission bandwidth, and the system design is relatively complex. Therefore, how to combine the autonomous driving scenario and the vehicle advantages of vehicle manufacturers, and design an effective and large-scale applicable incremental high-value data collection and closed-loop solution for the need to improve the performance of the autonomous driving perception and decision-making system DNN has become a technical problem to be solved urgently in this field. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a method and system for autonomous driving data collection and closed-loop management that can overcome the above problems or at least partially solve the above problems.

[0005] An object of the present invention is to provide a method and system for autonomous driving data collection and closed-loop management that can effectively screen out high-value incremental data.

[0006] A further object of the present invention is to train the neural network of the perception decision system for autonomous driving by using the screened incremental data, and deploy the trained neural network to update the pre-trained neural network for data collection, so as to form an incremental data closed-loop management aiming at improving the perception ability of the neural network.

[0007] In particular, according to an aspect of an embodiment of the present invention, there is provided an autonomous driving data collection and closed-loop management method, which is applied to a vehicle end and includes:

[0008] Obtain vehicle driving data;

[0009] Preprocess the vehicle driving data;

[0010] Screen the preprocessed vehicle driving data according to a preset screening rule through a pre-trained neural network to obtain incremental data;

[0011] Store the incremental data, or send the incremental data to the cloud.

[0012] Optionally, after storing the incremental data, the method further includes:

[0013] Receive and deploy the trained neural network sent from the cloud to update the pre-trained neural network, where the trained neural network is a neural network trained with the incremental data.

[0014] Optionally, the vehicle driving data includes vehicle perception data, and the neural network is a deep neural network.

[0015] Optionally, the vehicle perception data includes on-vehicle visual perception data;

[0016] The preprocessing of the vehicle driving data includes:

[0017] Perform at least one of the following processes on the on-vehicle visual perception data: deformation recovery, noise reduction, deblurring, white balance, image compensation and restoration under strong light or weak light conditions.

[0018] Optionally, the vehicle perception data further includes vehicle navigation data and vehicle status data;

[0019] The preprocessing of the vehicle driving data further includes:

[0020] Perform time synchronization on the in-vehicle visual perception data, the vehicle navigation data, and the vehicle state data to form structured vehicle perception data, where each piece of the structured vehicle perception data includes a timestamp, a perception image at the current moment, the vehicle position at the current moment, and the vehicle dynamics parameters at the current moment.

[0021] Optionally, the preset screening rules include at least one of the following:

[0022] High-precision map and perception data comparison screening rule, prediction data and perception data comparison screening rule, cloud instruction screening rule, adjacent frame image difference screening rule.

[0023] Optionally, the high-precision map and perception data comparison screening rule includes:

[0024] Run the pre-trained neural network to identify the structured vehicle perception data to obtain identification data;

[0025] When the forward distance of the vehicle from the marked target road infrastructure in the high-precision map is less than the first threshold, determine whether the identified data at the current moment includes the marked target road infrastructure;

[0026] If not, use the structured vehicle perception data of a specified number of frames adjacent before and after the structured vehicle perception data at the current moment as the incremental data.

[0027] Optionally, the prediction data and perception data comparison screening rule includes:

[0028] Predict the first position of the obstacle vehicle in the first number of frames in the future starting from the current moment;

[0029] Based on the structured vehicle perception data of the second number of frames in the future starting from the current moment, obtain the second position of the obstacle vehicle in the second number of frames in the future actually perceived by the pre-trained neural network, where the second number is greater than or equal to the first number;

[0030] Judge whether the prediction data and the perception data are consistent according to the first position and the second position in each future frame;

[0031] If not, use the structured vehicle perception data of a specified number of frames adjacent before and after the structured vehicle perception data at the current moment as the incremental data.

[0032] Optionally, the judging whether the prediction data and the perception data are consistent according to the first position and the second position in each future frame includes:

[0033] Calculate the distance difference d between the predicted data and the perceived data according to the following formula (1):

[0034]

[0035] In formula (1), represents the second position of the obstacle vehicle in the i-th future frame actually perceived, represents the first position of the obstacle vehicle in the predicted i-th future frame, and n is equal to the first quantity;

[0036] Judge whether the distance difference d is greater than the second threshold;

[0037] If so, determine that the predicted data is inconsistent with the perceived data.

[0038] Optionally, the cloud instruction screening rule includes:

[0039] Screen the structured vehicle perception data according to the data screening instruction issued by the cloud to obtain the incremental data.

[0040] Optionally, the adjacent frame image difference screening rule includes:

[0041] Calculate the difference index between the perceived images in the structured vehicle perception data of the adjacent previous frame and the next frame;

[0042] Judge whether the difference index is greater than the third threshold;

[0043] If so, use the structured vehicle perception data of the specified number of frames adjacent before and after with the structured vehicle perception data of the next frame as the center as the incremental data.

[0044] Optionally, the difference index is the mean square error;

[0045] The calculation of the difference index between the perceived images in the structured vehicle perception data of the adjacent previous frame and the next frame includes:

[0046] Calculate the mean square error MSE between the perceived images in the structured vehicle perception data of the previous frame and the next frame according to the following formula (2):

[0047]

[0048] In formula (2), I t (i,j), I t+1 (i,j) respectively represent the pixel values of two adjacent frames of perceived images at time t and t+1, i and j respectively represent the pixel point coordinates in the perceived image, and m and n respectively represent the length and width of the size of the perceived image; or

[0049] The difference index is structural similarity;

[0050] Calculating the difference index between the perception images in the structured vehicle perception data of adjacent previous and subsequent frames includes:

[0051] Calculating the structural similarity SSIM between the perception images in the structured vehicle perception data of the previous and subsequent frames according to the following formula (3):

[0052]

[0053] In formula (3), x and y respectively represent the pixel values of two adjacent frames of perception images, and u x is the mean value of x, and u y is the mean value of y, σ x is the variance of x, and σ y is the variance of y, and σ xy is the covariance of x and y; c1 and c2 are constants to maintain stability, c1 = (k1 * L) 2 , c2 = (k2 * L) 2 , where k1 and k2 are constants, and L is the dynamic range of pixel values.

[0054] Optionally, the vehicle driving data includes vehicle control data, and the neural network is a deep reinforcement learning neural network.

[0055] Optionally, the vehicle control data includes the current vehicle state data S t and the current control action a human of the driver of the vehicle;

[0056] The screening of the pre - processed vehicle driving data by the pre - trained neural network according to the preset screening rules to obtain incremental data includes:

[0057] Collecting the current control action a t output by the pre - trained deep reinforcement learning neural network based on the current vehicle state data S nn ;

[0058] Calculating the difference between the current control action a nn output by the deep reinforcement learning neural network and the current control action a human of the driver;

[0059] Judging whether the difference is greater than the fourth threshold;

[0060] If so, determining the current vehicle state data S t and the current control action a nnThe reward function value r(t) of the deep reinforcement learning neural network described below;

[0061] Using the current vehicle state data S t , the current control action a output by the deep reinforcement learning neural network nn , the current control action a of the driver human The reward function value r(t) and the vehicle state data S at the next moment t+1 Construct a data packet as the incremental data.

[0062] Optionally, determining the current vehicle state data S t and the current control action a output by the deep reinforcement learning neural network nn The reward function value r(t) of the deep reinforcement learning neural network includes:

[0063] Calculate the reward function r(t) of the deep reinforcement learning neural network according to the following formula (4):

[0064] r(t) = R ca (t) + b1R ma (t) + b2R lk (t) + b3R speed (t) + b4R steer (t) (4)

[0065] Wherein, R ca (t) represents the collision avoidance index, R ma (t) represents the task completion degree index, R lk (t) represents the lane keeping ability index, R speed (t) represents the speed rationality index, R steer (t) represents the steering rationality index, R ca (t), R ma (t), R lk (t), R speed (t) and R steer (t) are calculated under the current vehicle state data S t assuming that the current control action a output by the deep reinforcement learning neural network is applied to the control system of the vehicle nn afterwards, and b1, b2, b3, b4 represent weight coefficients.

[0066] Optionally, the current vehicle state data S t includes at least one of the following: current environment information, current vehicle speed, current vehicle steering angle;

[0067] The current control action a of the driver humanincluding at least one of the following: a first decision instruction, a steering angle, an accelerator pedal / brake pedal opening;

[0068] the current control action a output by the deep reinforcement learning neural network nn including at least one of the following: a second decision instruction, a steering instruction, a speed instruction.

[0069] Optionally, before screening the preprocessed vehicle driving data by a pre-trained neural network according to a preset screening rule to obtain incremental data, it further includes:

[0070] caching the preprocessed vehicle driving data.

[0071] According to another aspect of the embodiments of the present invention, there is also provided an autonomous driving data acquisition and closed-loop management system, which is applied to the vehicle side and includes:

[0072] a data acquirer configured to acquire vehicle driving data;

[0073] a data processor including: a data preprocessing module configured to preprocess the vehicle driving data; and a data screening module on which a pre-trained neural network is deployed, configured to screen the preprocessed vehicle driving data by the pre-trained neural network according to a preset screening rule to obtain incremental data; and

[0074] a memory configured to store the incremental data; and / or

[0075] a sending module configured to send the incremental data to the cloud.

[0076] Optionally, the data screening module is further configured to:

[0077] receive and deploy the trained neural network sent from the cloud to update the pre-trained neural network, where the trained neural network is the neural network trained by the incremental data.

[0078] Optionally, the data acquirer includes one or more on-vehicle cameras, and the one or more on-vehicle cameras are configured to acquire on-vehicle visual perception data;

[0079] The data preprocessing module is further configured to:

[0080] perform at least one of the following on the on-vehicle visual perception data: distortion recovery, noise reduction, deblurring, white balance, image compensation and restoration under strong light or weak light conditions.

[0081] Optionally, the data acquirer further includes:

[0082] a navigation device configured to acquire vehicle navigation data; and

[0083] A vehicle network device, configured to obtain vehicle status data; and

[0084] The data processor further includes a data synchronization module, configured to perform time synchronization on the in-vehicle visual perception data, the vehicle navigation data, and the vehicle status data to form structured vehicle perception data, wherein each piece of the structured vehicle perception data includes a timestamp, a perception image at the current moment, a vehicle position at the current moment, and vehicle dynamics parameters at the current moment.

[0085] Optionally, the data processor further includes:

[0086] A data caching module, configured to cache the preprocessed vehicle driving data.

[0087] Optionally, the memory is an in-vehicle mobile hard disk.

[0088] In the autonomous driving data acquisition and closed-loop management method and system provided by the embodiments of the present invention, after obtaining incremental data by screening the preprocessed vehicle driving data according to a preset screening rule through a pre-trained neural network, the incremental data is stored or sent to the cloud for training of the neural network. The solution of the present invention can effectively screen out high-value incremental data, thereby reducing the system data storage amount and / or the requirement for data transmission bandwidth. The system of the present invention can be installed on a vehicle in a retrofit or pre-installed manner, does not depend on a specific vehicle model, and does not require the vehicle to be equipped with a high-cost lidar and other ground truth systems, greatly improving the convenience of system use and facilitating rapid and large-scale application.

[0089] Further, the solution of the present invention uses at least one of a high-precision map and perception data comparison screening rule, a prediction data and perception data comparison screening rule, a cloud instruction screening rule, an adjacent frame image difference screening rule, etc. to screen the collected vehicle driving data, and more accurate and valuable incremental data can be obtained, thereby greatly reducing the storage amount or transmission amount of the perception data.

[0090] Further, the solution of the present invention trains a neural network of an autonomous driving perception decision system by using the screened incremental data, and deploys the trained neural network to update the pre-trained neural network for data acquisition and screening, thereby forming an incremental data closed-loop management aiming at improving the perception ability of the neural network.

[0091] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other objects, features, and advantages of the present invention more obvious and understandable, the following specifically describes the embodiments of the present invention.

[0092] Those skilled in the art will become more clearly aware of the above and other objects, advantages and features of the present invention from the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings. Description of the Drawings

[0093] Some specific embodiments of the present invention will be described in detail hereinafter with reference to the accompanying drawings in an illustrative rather than restrictive manner. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0094] Figure 1 is a schematic flowchart of a method for autonomous driving data collection and closed-loop management according to an embodiment of the present invention;

[0095] Figure 2 is a schematic flowchart of a method for autonomous driving data collection and closed-loop management according to another embodiment of the present invention;

[0096] Figure 3 is a schematic flowchart of a method for autonomous driving data collection and closed-loop management according to yet another embodiment of the present invention;

[0097] Figure 4 is a schematic flowchart of a method for autonomous driving data collection and closed-loop management according to still another embodiment of the present invention;

[0098] Figure 5 is a schematic structural diagram of a system for autonomous driving data collection and closed-loop management according to an embodiment of the present invention;

[0099] Figure 6 is a schematic structural diagram of a system for autonomous driving data collection and closed-loop management according to another embodiment of the present invention. Detailed Embodiments

[0100] Exemplary embodiments of the present disclosure will be described in more detail hereinafter with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0101] Most of the perception modules of current autonomous driving systems use DNNs for object detection, recognition, classification, and prediction. Continuously optimizing and iterating the perception and decision-making system DNN with actual data, improving the performance metrics of the DNN, and enhancing the robustness and stability of the autonomous driving system in dealing with various scenarios are the keys to the large-scale deployment of current autonomous driving vehicles. Currently, most enterprises and research institutions usually adopt the method of using dedicated data collection vehicles or test vehicles for centralized data collection. These data collection vehicles generally need to be equipped with ground truth systems such as lidar, which have high requirements for equipment, and there is no screening mechanism or the screening mechanism is unclear during the data collection process, resulting in a large requirement for data storage space or a large requirement for data transmission bandwidth, which not only places high requirements on the hardware system but also increases the difficulty of subsequent data mining and finding valuable data.

[0102] In view of the above technical problems, an embodiment of the present invention proposes an autonomous driving data collection and closed-loop management method and system, which can be applied to the vehicle side.

[0103] Figure 1 The flowchart of the autonomous driving data collection and closed-loop management method according to an embodiment of the present invention is shown. Refer to Figure 1 As shown, the autonomous driving data collection and closed-loop management method can at least include the following steps S102 to S108.

[0104] Step S102, obtain vehicle driving data.

[0105] Step S104, preprocess the vehicle driving data.

[0106] Step S106, screen the preprocessed vehicle driving data according to a preset screening rule through a pre-trained neural network to obtain incremental data.

[0107] Step S108, store the incremental data, or send the incremental data to the cloud.

[0108] Specifically, in step S108, the incremental data can be stored locally (i.e., the vehicle-side memory), and then the incremental data stored locally is periodically copied and transferred to the cloud (such as a cloud computing center) for subsequent data processing and neural network training. Or, the incremental data can also be directly sent to the cloud through a network communication module (such as T-box) for subsequent data processing and neural network training. Of course, it is also possible to store locally and upload to the cloud at the same time, so as to achieve multiple backups and ensure that the data will not be lost.

[0109] The automatic driving data acquisition and closed-loop management method provided by the embodiment of the present invention adopts a distributed incremental data acquisition method, which can effectively screen out high-value incremental data, thereby reducing the system data storage volume and / or the requirement for data transmission bandwidth.

[0110] In one embodiment, as shown in Figure 2 After step S108, step S110 may further be executed: receiving and deploying the trained neural network sent from the cloud to update the pre-trained neural network, where the trained neural network is the neural network trained with incremental data. Specifically, the trained neural network sent from the cloud may be received through the T-box configured in the vehicle by means of OTA (Over-the-Air). In this embodiment, the neural network of the perception and decision-making system for automatic driving is trained with the screened incremental data, and the trained neural network is deployed to update the pre-trained neural network. The updated neural network is used again for data acquisition and screening, thereby forming an incremental data closed-loop management aiming at improving the perception ability of the neural network.

[0111] In one implementation manner, the incremental data stored locally may be regularly copied to the cloud computing center. After analyzing and annotating the incremental data, the neural network is incrementally trained with the annotated data to obtain the trained neural network. In another implementation manner, after the incremental data is sent to the cloud, the incremental data may be directly analyzed and annotated, and the neural network is incrementally trained with the annotated data to obtain the trained neural network. By analyzing and annotating the screened incremental data, the data processing efficiency can be effectively improved; on this basis, the neural network is incrementally trained with the annotated data, which can effectively improve the performance index of the neural network and improve the training and learning efficiency of the neural network.

[0112] In another embodiment, still referring to Figure 2 Before executing step S106, the following step S105 may be further executed: caching the preprocessed vehicle driving data. Specifically, the preprocessed vehicle driving data obtained within a preset time (such as 5 min) may be cached to leave enough time for the data screening operation in step S106.

[0113] In the field of automatic driving, the vehicle driving data may include vehicle perception data or vehicle control data. The acquisition and closed-loop management methods for these two types of data are introduced separately below.

[0114] For vehicle perception data, a deep neural network DNN is usually used to make perception decisions based on the vehicle perception data. In one embodiment, as shown in Figure 3 Step S102 above may be specifically implemented as:

[0115] Step S302: Obtain vehicle driving data, where the vehicle driving data includes vehicle perception data. Specifically, the vehicle perception data may include on-vehicle visual perception data, that is, perception data such as images and videos obtained by a visual system (specifically, one or more cameras) configured on the vehicle.

[0116] In this case, step S104 above may specifically include the following step S304: Perform at least one of the following processes on the on-vehicle visual perception data: deformation recovery, noise reduction, deblurring, white balance, image compensation and recovery under strong light or weak light conditions, etc., to enhance the clarity and accuracy of the on-vehicle visual perception data.

[0117] Furthermore, the vehicle perception data may also include vehicle navigation data and vehicle status data. Specifically, the vehicle navigation data can be obtained through a navigation device configured on the vehicle, and the vehicle status data can be obtained from various sensors (such as a vehicle speed sensor, an acceleration sensor, etc.) and / or instruments of the vehicle through the vehicle's vehicle network device. In this case, step S104 above may also specifically include the following step S306: Perform time synchronization on the on-vehicle visual perception data, vehicle navigation data, and vehicle status data to form structured vehicle perception data, where each structured vehicle perception data includes a timestamp, the perception image at the current moment, the vehicle position at the current moment, and the vehicle dynamics parameters at the current moment. The vehicle dynamics parameters may include vehicle speed, steering wheel angle, acceleration / deceleration, etc. When performing time synchronization on the on-vehicle visual perception data, vehicle navigation data, and vehicle status data, the system clock of the vehicle's navigation system or the system clock of the data acquisition and closed-loop management system used to implement this method can be used as the time reference.

[0118] Furthermore, for the preprocessed vehicle perception data, perform step S308: Screen the preprocessed vehicle perception data through a pre-trained deep neural network according to a preset screening rule to obtain incremental data, where the preset screening rule includes at least one of a comparison and screening rule between a high-precision map and perception data, a comparison and screening rule between prediction data and perception data, a cloud instruction screening rule, and an adjacent frame image difference screening rule. The above four preset screening rules are introduced separately below.

[0119] The first type: The comparison and screening rules for high-precision maps and perception data. In the comparison and screening rules for high-precision maps and perception data, a pre-trained neural network (specifically, a deep neural network) is run to identify each frame of structured vehicle perception data to obtain identification data. When the forward distance of the vehicle from the marked target road infrastructure in the high-precision map is less than the first threshold, it is determined whether the identified data at the current moment includes the marked target road infrastructure. If not, the structured vehicle perception data of a specified number of frames adjacent before and after the current moment's structured vehicle perception data centered on the current moment is used as incremental data. Specifically, the high-precision map (HD-Map) can be imported by the navigation device configured in the vehicle, and the road infrastructure in the high-precision map (such as traffic signal poles, road static barriers, etc.) is marked. The first threshold can be set according to actual application requirements. For example, it can be set to 50m. The specified number can be set according to the accuracy requirements of actual applications. For example, it can be set to 20. At this time, the structured vehicle perception data of 20 frames adjacent before and after including the structured vehicle perception data at the current moment is used as incremental data. This method is applicable to the comparison and screening of static data, and can screen out the vehicle perception data that the deep neural network fails to accurately identify, which is more effective for training the deep neural network and improving the perception ability of the deep neural network.

[0120] The second type: The comparison and screening rules for prediction data and perception data. In the comparison and screening rules for prediction data and perception data, the first position of the obstacle vehicle in the first number of frames in the future starting from the current moment is predicted, and the second position of the obstacle vehicle in the second number of frames in the future starting from the current moment is obtained through a pre-trained neural network based on the structured vehicle perception data in the second number of frames in the future starting from the current moment, where the second number is greater than or equal to the first number. Then, according to the first position and the second position in each frame in the future, it is determined whether the prediction data and the perception data are consistent. If not, the structured vehicle perception data of a specified number of frames adjacent before and after the current moment's structured vehicle perception data centered on the current moment is used as incremental data.

[0121] Specifically, methods such as vehicle kinematics and LSTM (Long Short-Term Memory) can be used to predict the future trajectory and position of the obstacle vehicle. The following method is used to determine whether the prediction data and the perception data are consistent: First, the distance difference d between the prediction data and the perception data is calculated according to the following formula (1):

[0122]

[0123] In formula (1), represents the second position of the obstacle vehicle in the actual perceived future i-th frame, Represents the first position of the obstacle vehicle in the predicted future i-th frame, and n is equal to the first quantity. Then, it is determined whether the calculated distance difference d is greater than the second threshold dmin. If so, it can be determined that the prediction data is inconsistent with the perception data.

[0124] The first quantity and the second quantity can be set according to the actual application requirements. For example, the first quantity can generally be set to 8 frames, and the second quantity can be set to 20 frames. If it is predicted that the obstacle vehicle will exceed the perception field of view of the vehicle's vision system within the next 8 frames, the first quantity can be set to the number of frames within the perception field of view of the vision system. In addition, if a missing frame of the obstacle vehicle is detected within the perception field of view, the values of this frame can be set to a large value (such as 1000) to ensure that this perception data can be filtered out. The second threshold dmin can be set according to the actual application, and the present invention does not limit this.

[0125] This method is applicable to the filtering of perception data without ground truth, and can filter out vehicle perception data that is inconsistent with the prediction data, which is more effective for training a deep neural network and improving the perception ability of the deep neural network.

[0126] The third type: Cloud instruction filtering rule. In the cloud instruction filtering rule, incremental data is obtained by filtering the structured vehicle perception data according to the data filtering instructions issued by the cloud. Specifically, according to the training requirements of the deep neural network, data filtering instructions can be set in the cloud (such as collecting vehicle perception data when the braking deceleration is greater than the threshold, collecting vehicle perception data at traffic light intersections, etc.), and the data filtering instructions are sent to the vehicle end through the T-box, and the vehicle end executes the data filtering instructions received from the cloud. This method can purposefully and specifically filter out the incremental data required for neural network training, improving the data processing efficiency and the neural network training efficiency.

[0127] The fourth type: Adjacent frame image difference filtering rule. In the adjacent frame image difference filtering rule, first, the difference index between the perception images in the structured vehicle perception data of the adjacent previous frame and the subsequent frame is calculated. Then, it is determined whether the difference index is greater than the third threshold. If so, the structured vehicle perception data of the specified number of frames adjacent before and after with the structured vehicle perception data of the subsequent frame as the center is used as the incremental data. The third threshold can be set according to the actual application requirements, and the present invention does not limit this. The limitation of the specified number is as described above. For example, when the specified number is set to 20, when the difference index of the perception images of the front and rear frames is greater than the third threshold, the structured vehicle perception data of 20 frames adjacent before and after including the structured vehicle perception data of this subsequent frame is used as the incremental data.

[0128] Specifically, the above difference index can be the Mean Square Error (MSE) or the Structural Similarity (SSIM) between two frames of perceptual images.

[0129] For the mean square error, the mean square error MSE between the perceptual images in the structured vehicle perception data of the previous frame and the next frame can be calculated according to the following formula (2):

[0130]

[0131] In formula (2), I t (i, j) and I t+1 (i, j) respectively represent the pixel values of two adjacent frames of perceptual images at time t and time t + 1, i and j respectively represent the pixel point coordinates in the perceptual image, and m and n respectively represent the length and width of the size of the perceptual image.

[0132] For the structural similarity, the structural similarity SSIM between the perceptual images in the structured vehicle perception data of the previous frame and the next frame can be calculated according to the following formula (3):

[0133]

[0134] In formula (3), x and y respectively represent the pixel values of two adjacent frames of perceptual images, u x is the mean of x, u y is the mean of y, σ x is the variance of x, σ y is the variance of y, σ xy is the covariance of x and y; c1 and c2 are constants to maintain stability, c1 = (k1 * L) 2 , c2 = (k2 * L) 2 , where k1 and k2 are constants, and L is the dynamic range of pixel values. Generally, k1 = 0.01 and k2 = 0.03.

[0135] When there are obvious differences between the front and rear frame images, it indicates that the vehicle enters a new scene or environment. At this time, the vehicle perception data has high value. This method can more effectively improve the perception ability of the deep neural network by screening out the vehicle perception data with the difference between adjacent frame images exceeding the threshold for neural network training.

[0136] The operation of step S310 is the same as that of step S108 in the previous text and will not be repeated here.

[0137] This embodiment filters the collected vehicle driving data by adopting at least one of the high-precision map and perception data comparison screening rules, prediction data and perception data comparison screening rules, cloud instruction screening rules, adjacent frame image difference screening rules, etc., to obtain more accurate and valuable incremental data, thereby greatly reducing the storage or transmission volume of perception data.

[0138] For vehicle control data, a deep reinforcement learning (DRL) neural network can be used for perception decision making. In one embodiment, see Figure 4 As shown, the autonomous driving data collection and closed-loop management method may include the following steps S402 to S416.

[0139] Step S402, obtaining vehicle control data, wherein the vehicle control data includes current vehicle state data S and the current control action a of the vehicle driver. human .

[0140] Specifically, the current vehicle status data S t It may include current environment information (such as image information, grid map, etc.), current vehicle speed, current vehicle steering angle, etc. The driver's current control action a human It may include the first decision instruction, steering angle, accelerator pedal / brake pedal opening, etc. The vehicle control data may be obtained through the visual system (such as a camera), navigation equipment, vehicle network equipment, etc. configured in the vehicle.

[0141] Step S404: pre-processing the vehicle control data.

[0142] In this step, the vehicle control data can be preprocessed using an existing common processing method, which can be selected according to the actual application and will not be described in detail in this article.

[0143] When the vehicle control data is screened, the above step S106 can be specifically implemented as the following steps S406 to S414.

[0144] Step S406, collecting the current control action a output by the pre-trained deep reinforcement learning neural network based on the current vehicle state data S nn .

[0145] The current control action a output by the deep reinforcement learning neural network nn It may include second decision instructions, steering instructions, speed instructions, etc.

[0146] Step S408, calculating the current control action a output by the deep reinforcement learning neural network nn The current control action of the driver human The difference between.

[0147] The difference calculation in this step should be known to those skilled in the art and will not be elaborated here.

[0148] Step S410: Determine whether the difference is greater than the fourth threshold. If so, execute step S412.

[0149] Step S412: Determine the current vehicle state data S t and the current control action a output by the deep reinforcement learning neural network nn and the reward function value r(t) of the deep reinforcement learning neural network at the next time.

[0150] Specifically, calculate the reward function r(t) of the deep reinforcement learning neural network according to the following formula (4):

[0151] r(t) = R ca (t) + b1R ma (t) + b2R lk (t) + b3R speed (t) + b4R steer (t) (4)

[0152] Wherein, R ca (t) represents the collision avoidance index, R ma (t) represents the task completion degree index, R lk (t) represents the lane keeping ability index, R speed (t) represents the speed rationality index, R steer (t) represents the steering rationality index, R ca (t), R ma (t), R lk (t), R speed (t) and R steer (t) are calculated under the assumption that the current control action a output by the deep reinforcement learning neural network is applied to the control system of the vehicle under the current vehicle state data S t , and b1, b2, b3, b4 represent weight coefficients. Each weight coefficient represents the degree of attention to the corresponding index and can be set according to experience. nn t

[0153] Step S414: Construct a data packet as incremental data with the current vehicle state data S t , the current control action a output by the deep reinforcement learning neural network nn , the current control action a of the driver human , the reward function value r(t) and the vehicle state data S at the next moment t+1

[0154] The vehicle state data S at the next moment​t+1 Refers to the vehicle status data S t The status data after the vehicle is subjected to the current control action a of the driver human . The data packet form constructed in this step can be [S t , a nn , a human , r(t), S t+1 .

[0155] Step S416, store the incremental data, or send the incremental data to the cloud

[0156] The operation of step S416 is the same as that of step S108 described above, and will not be elaborated

[0157] Further, after step S416, it is also possible to receive and deploy the trained deep reinforcement learning neural network sent from the cloud to update the pre-trained deep reinforcement learning neural network, where the trained deep reinforcement learning neural network is the deep reinforcement learning neural network trained with the incremental data of the selected vehicle control data

[0158] This embodiment performs incremental data acquisition and closed-loop management on vehicle control data, which can greatly reduce the system data storage volume and / or the requirement for data transmission bandwidth, improve data processing efficiency, and enhance the perception and decision-making ability of the deep reinforcement learning neural network

[0159] Based on the same technical concept, an embodiment of the present invention also provides an autonomous driving data acquisition and closed-loop management system, which is applied to the vehicle side Figure 5 Shows a schematic structural diagram of an autonomous driving data acquisition and closed-loop management system 10 according to an embodiment of the present invention. Refer to Figure 5 As shown, the autonomous driving data acquisition and closed-loop management system 10 may at least include a data acquirer 100 and a data processor 200. The data acquirer 100 is configured to acquire vehicle driving data. The data processor 200 may include a data preprocessing module 201 and a data screening module 202. The data preprocessing module 201 is configured to preprocess the vehicle driving data. A pre-trained neural network is deployed in the data screening module 202, and is configured to screen the preprocessed vehicle driving data according to a preset screening rule through the pre-trained neural network to obtain incremental data. The data processor 200 may be a GPU (Graphics Processing Unit) that can run a neural network (such as DNN). The autonomous driving data acquisition and closed-loop management system 10 may further include: a memory 300, configured to store incremental data; and / or, a sending module 400, configured to send the incremental data to the cloud. Specifically, the sending module 400 sends the incremental data to the cloud through a network communication module 500 (which may be a T-box, etc.)

[0160] In one embodiment, the data screening module 202 may also be configured to receive and deploy the trained neural network sent from the cloud to update the pre-trained neural network, where the trained neural network is the neural network trained with incremental data.

[0161] In one embodiment, referring to Figure 6 As shown, the data acquirer 100 may include a vision system 101 (specifically, one or more vehicle-mounted cameras), configured to acquire vehicle vision perception data. Correspondingly, the data preprocessing module 201 may also be configured to perform at least one of the following processes on the vehicle vision perception data: distortion recovery, noise reduction, deblurring, white balance, image compensation and recovery under strong light or weak light conditions.

[0162] In one embodiment, continuing to refer to Figure 6 As shown, the data acquirer 100 may also include a navigation device 102 and a vehicle network device 103. The navigation device 102 may be, for example, an integrated navigation system, configured to acquire vehicle navigation data. The vehicle network device 103 may be, for example, an in-vehicle CAN network bus, an in-vehicle Ethernet bus, etc. Vehicle state data is acquired from various sensors (such as a vehicle speed sensor, an acceleration sensor, etc.) and / or meters of the vehicle through the vehicle network device 103. The data processor 200 may also include a data synchronization module 203, configured to perform time synchronization on the vehicle vision perception data, the vehicle navigation data, and the vehicle state data to form structured vehicle perception data, where each structured vehicle perception data includes a timestamp, a perception image at the current moment, the vehicle position at the current moment, and the vehicle dynamics parameters at the current moment.

[0163] In one embodiment, continuing to refer to Figure 6 As shown, the data processor 200 may also include a data caching module 204, configured to cache the preprocessed vehicle driving data.

[0164] In one embodiment, the memory 300 may be an in-vehicle mobile hard disk. Since the in-vehicle mobile hard disk has a large capacity, a large amount of screened incremental data can be stored, and then copied and transferred to the cloud computing center in batches, so that the image or video data collected by this system can be transmitted without relying on the bandwidth of the wireless network.

[0165] In specific implementation, the data screening module 202 may be configured to perform the data screening operations described in any of the foregoing embodiments or combinations of embodiments, which will not be repeated herein.

[0166] The autonomous driving data collection and closed-loop management system 10 provided by the embodiments of the present invention can be installed on a vehicle in a pre-installed manner (i.e., installed during vehicle production) or a post-installed manner (i.e., installed after the vehicle leaves the factory), does not depend on a specific vehicle model, and does not require the vehicle to be equipped with an expensive truth system such as a lidar. The transmission of image or video data does not depend on the bandwidth of a wireless network, thereby greatly improving the convenience of system use and facilitating the rapid and large-scale application of the autonomous driving data collection and closed-loop management system 10.

[0167] The foregoing has introduced in detail various implementation manners of the autonomous driving data collection and closed-loop management method and system of the present invention. The following describes the implementation manners of the present invention through specific specific embodiments.

[0168] Embodiment 1

[0169] The autonomous driving data collection and closed-loop management system 10 in this embodiment is a pre-installed system. Compared with Figure 6 the sending module 400 is reduced. The vision system 101 uses one or more cameras, and the memory 300 is an in-vehicle mobile hard disk. In an application, the visual perception data obtained by the camera, the vehicle navigation data obtained by the navigation device 102, and the vehicle status data obtained by the vehicle network device 103 are inputs to the data processor 200. A pre-trained DNN is deployed in the data screening module 202 of the data processor 200, and the DNN can be upgraded by means of OTA. After the input data is pre-processed and synchronized by the data pre-processing module 201 and the data synchronization module 203, the data screening module 202 uses rules such as a high-precision map and perception data comparison and screening rules, prediction data and perception data comparison and screening rules, cloud instruction screening rules, and adjacent frame image difference screening rules for data screening, and saves the screened structured perception data to the in-vehicle mobile hard disk. In a pre-installed system, the DNN result output in the data processor 200 can also be used as one of the input sources of the vehicle's intelligent driving system to improve the reuse rate of the system.

[0170] After the autonomous driving data collection and closed-loop management system 10 is deployed on a large scale, the vehicle collects and screens incremental data in different scenarios. The incremental data collected and screened by the vehicle is first processed and classified at the computing center, and can be classified according to information such as location, vehicle speed, and acceleration. Of course, data under specific conditions can also be collected according to the screening instruction conditions set by the cloud. Then, the incremental data is labeled, and the labeled incremental data is used for algorithm incremental training of the DNN and performance improvement of the DNN. Finally, the iteratively updated DNN network is regularly deployed to the data processor 200 of the vehicle by means of OTA, and then a new round of incremental data collection is performed, thereby forming a data collection and closed-loop management system.

[0171] Embodiment 2

[0172] The autonomous driving data acquisition and closed-loop management method and system in this embodiment are used for acquiring incremental data of autonomous driving control for training the DRL neural network.

[0173] In this embodiment, the autonomous driving data acquisition and closed-loop management system 10 is installed on an autonomous driving test vehicle, and a pre-trained DRL neural network is deployed in the data processor 200 installed on the autonomous driving test vehicle, and the deployed DRL neural network runs in the background, that is, the actions output by the DRL neural network do not directly act on the real vehicle control.

[0174] In application, the autonomous driving data acquisition and closed-loop management system 10 detects the action difference between the output action of the DRL neural network and the action of the human driver in real time, obtains incremental control data whose difference exceeds the fourth threshold, and the incremental control data is stored in a data packet [S t ,a nn ,a human ,r(t),S t+1 . Among them, the calculation, judgment of the action difference between the output action of the DRL neural network and the action of the human driver, and the construction of the data packet of the incremental control data are as described above and will not be repeated here.

[0175] After obtaining the incremental control data, use the incremental control data to train the DRL neural network. Periodically deploy the trained DRL neural network into the test vehicle through OTA to form an incremental data closed-loop management system aiming to improve the KPI of the DRL neural network control ability.

[0176] According to any one of the above optional embodiments or a combination of multiple optional embodiments, the embodiments of the present invention can achieve the following

[0177] Beneficial effects:

[0178] In the autonomous driving data acquisition and closed-loop management method and system provided by the embodiments of the present invention, after screening the preprocessed vehicle driving data according to the preset screening rules through a pre-trained neural network to obtain incremental data, the incremental data is stored or sent to the cloud for neural network training. The solution of the present invention can effectively screen out high-value incremental data, thereby reducing the system data storage capacity and / or the requirement for data transmission bandwidth. The system of the present invention can be installed on the vehicle in a retrofittable or pre-installable manner, does not depend on a specific vehicle model, and does not require the vehicle to be equipped with a high-cost lidar and other ground truth systems, greatly improving the convenience of system use and facilitating rapid and large-scale application.

[0179] Furthermore, the solution of the present invention uses at least one of the comparison and screening rules for high-precision maps and perception data, the comparison and screening rules for prediction data and perception data, the cloud instruction screening rules, the adjacent frame image difference screening rules, etc. to screen the collected vehicle driving data, and more accurate and valuable incremental data can be obtained, thereby greatly reducing the storage amount or transmission amount of perception data.

[0180] Furthermore, the solution of the present invention trains the neural network of the perception decision system for autonomous driving by using the screened incremental data, and deploys the trained neural network to update the pre-trained neural network for data collection and screening, so as to form an incremental data closed-loop management aiming at improving the perception ability of the neural network.

[0181] At this point, those skilled in the art should recognize that although the exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived from the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and determined to cover all these other variations or modifications.

Claims

1. An autonomous driving data acquisition and closed-loop management method, applied to the vehicle side, comprising: Obtaining vehicle driving data; Preprocessing the vehicle driving data; Screening the preprocessed vehicle driving data according to a preset screening rule through a pre-trained neural network to obtain incremental data; Storing the incremental data or sending the incremental data to the cloud; Wherein, after storing the incremental data, it further includes: Receiving and deploying the trained neural network sent by the cloud to update the pre-trained neural network, wherein the trained neural network is the neural network trained with the incremental data; Wherein, the vehicle driving data includes vehicle perception data, and the neural network is a deep neural network; Wherein, the vehicle perception data includes in-vehicle visual perception data; The preprocessing of the vehicle driving data includes: Performing at least one of the following processes on the in-vehicle visual perception data: distortion recovery, noise reduction, deblurring, white balance, image compensation and restoration under strong light or weak light conditions; Wherein, the vehicle perception data further includes vehicle navigation data and vehicle state data; The preprocessing of the vehicle driving data further includes: Performing time synchronization on the in-vehicle visual perception data, the vehicle navigation data and the vehicle state data to form structured vehicle perception data, wherein each piece of structured vehicle perception data includes a timestamp, a perception image at the current moment, the vehicle position at the current moment, and the vehicle dynamics parameters at the current moment; The preset screening rule includes: a comparison and screening rule between a high-precision map and perception data; Wherein, the comparison and screening rule between the high-precision map and perception data includes: running the pre-trained neural network to identify the structured vehicle perception data to obtain identification data; when the forward distance of the vehicle from the target road infrastructure marked in the high-precision map is less than a first threshold, determining whether the identified target road infrastructure is included in the identification data at the current moment; if not, taking the structured vehicle perception data of a specified number of frames adjacent before and after the structured vehicle perception data at the current moment as the incremental data.

2. The method according to claim 1, wherein the preset screening rule further includes at least one of the following: A comparison and screening rule between predicted data and perception data, a cloud instruction screening rule, and a difference screening rule for adjacent frame images.

3. The method according to claim 2, wherein, The comparison and screening rule between predicted data and perception data includes: predicting the first position of an obstacle vehicle in the first number of frames in the future starting from the current moment; obtaining the second position of the obstacle vehicle in the second number of frames in the future starting from the current moment based on the structured vehicle perception data of the second number of frames in the future starting from the current moment through the pre-trained neural network, where the second number is greater than or equal to the first number; judging whether the predicted data and the perception data are consistent according to the first position and the second position in each frame in the future; if not, taking the structured vehicle perception data of a specified number of frames adjacent before and after the structured vehicle perception data at the current moment as the incremental data.

4. The method according to claim 3, wherein, Determining whether the predicted data is consistent with the sensed data according to the first position and the second position in each future frame includes: Calculating the distance difference d between the predicted data and the sensed data according to the following formula (1): In formula (1), represents the second position of the obstacle vehicle in the actually perceived future i-th frame, represents the first position of the obstacle vehicle in the predicted future i-th frame, and n is equal to the first quantity; Judging whether the distance difference d is greater than a second threshold; If so, determining that the predicted data is inconsistent with the sensed data.

5. The method according to claim 2, wherein, The cloud instruction screening rule includes: Screening the structured vehicle sensed data according to the data screening instruction sent from the cloud to obtain the incremental data.

6. The method according to claim 2, wherein The adjacent frame image difference screening rule includes: Calculating a difference index between the sensed images in the structured vehicle sensed data of the adjacent previous frame and the subsequent frame; Judging whether the difference index is greater than a third threshold; If so, using the structured vehicle sensed data of a specified number of frames adjacent before and after with the structured vehicle sensed data of the subsequent frame as the center as the incremental data.

7. The method according to claim 6, wherein The difference index is the mean square error; Calculating the difference index between the sensed images in the structured vehicle sensed data of the adjacent previous frame and the subsequent frame includes: Calculating the mean square error MSE between the sensed images in the structured vehicle sensed data of the previous frame and the subsequent frame according to the following formula (2): In formula (2), I t (i, j), I t+1 (i, j) respectively represent the pixel values of two adjacent frames of sensed images at time t and time t + 1, i and j respectively represent the pixel point coordinates in the sensed image, and m and n respectively represent the length and width of the size of the sensed image; or The difference index is the structural similarity; Calculating the difference index between the sensed images in the structured vehicle sensed data of the adjacent previous frame and the subsequent frame includes: Calculating the structural similarity SSIM between the sensed images in the structured vehicle sensed data of the previous frame and the subsequent frame according to the following formula (3): In Equation (3), x and y respectively represent the pixel values of two adjacent frames of perceptual images, and u x is the mean of x, and u y is the mean of y, σ x is the variance of x, and σ y is the variance of y, and σ xy is the covariance of x and y; c1 and c2 are constants for maintaining stability, c1 = (k1 * L) 2 , c2 = (k2 * L) 2 , where k1 and k2 are constants, and L is the dynamic range of the pixel values.

8. The method according to claim 1, wherein, The vehicle driving data includes vehicle control data, and the neural network is a deep reinforcement learning neural network.

9. The method according to claim 8, wherein, The vehicle control data includes the current vehicle state data S t and the current control action a of the driver of the vehicle human ; Screening the preprocessed vehicle driving data by the pre-trained neural network according to a preset screening rule to obtain incremental data includes: Collect the current control action a output by the pre-trained deep reinforcement learning neural network based on the current vehicle state data S t ; nn ; Calculate the current control action a output by the deep reinforcement learning neural network nn and the current control action a of the driver human to obtain the difference therebetween; Judging whether the difference is greater than a fourth threshold; If so, determine the current vehicle state data S t and the current control action a output by the deep reinforcement learning neural network nn and the reward function value r(t) of the deep reinforcement learning neural network With the current vehicle state data S t and the current control action a output by the deep reinforcement learning neural network nn and the current control action a of the driver human and the reward and punishment function value r(t) and the vehicle state data S at the next moment t+1 construct a data packet as the incremental data.

10. The method according to claim 9, wherein, Determining the current vehicle state data S t and the current control action a output by the deep reinforcement learning neural network nn The reward function value r(t) of the deep reinforcement learning neural network under the above conditions includes: Calculating the reward and punishment function r(t) of the deep reinforcement learning neural network according to the following formula (4): r(t) = R ca (t) + b1R ma (t) + b2R lk (t) + b3R speed (t) + b4R steer (t)(4) Among them, R ca (t) represents the collision avoidance index, R ma (t) represents the task completion index, R lk (t) represents the lane keeping ability index, R speed (t) represents the speed rationality index, R steer (t) represents the steering rationality index, R ca (t), R ma (t), R lk (t), R speed (t) and R steer (t) are calculated under the assumption that the current control action a output by the deep reinforcement learning neural network is applied to the control system of the vehicle under the current vehicle state data S t , where b1, b2, b3, and b4 represent weight coefficients. nn ​ 11. The method according to claim 9, wherein The current vehicle state data S t includes at least one of the following: current environmental information, current vehicle speed, current vehicle steering angle; The current control action a of the driver human includes at least one of the following: a first decision-making instruction, a steering angle, an accelerator pedal / brake pedal opening; The current control action a output by the deep reinforcement learning neural network nn includes at least one of the following: a second decision instruction, a steering instruction, and a speed instruction.

12. The method according to claim 1, wherein, Before screening the preprocessed vehicle driving data by the pre-trained neural network according to a preset screening rule to obtain incremental data, it further includes: Caching the preprocessed vehicle driving data.

13. An autonomous driving data acquisition and closed-loop management system, applied to a vehicle terminal, includes: A data acquirer configured to acquire vehicle driving data; A data processor, including: a data preprocessing module configured to preprocess the vehicle driving data; and a data screening module on which a pre-trained neural network is deployed, configured to screen the preprocessed vehicle driving data by the pre-trained neural network according to a preset screening rule to obtain incremental data; and A memory configured to store the incremental data; and / or A sending module configured to send the incremental data to the cloud; Wherein, the data screening module is further configured to: Receive and deploy the trained neural network sent from the cloud to update the pre-trained neural network, wherein the trained neural network is the neural network trained by the incremental data; Among them, the data acquirer includes one or more on-vehicle cameras, and the one or more on-vehicle cameras are configured to acquire on-vehicle visual perception data; The data preprocessing module is further configured to: perform at least one of the following processes on the on-vehicle visual perception data: deformation recovery, noise reduction, deblurring, white balance, image compensation and recovery under strong light or weak light conditions; Among them, the data acquirer further includes: a navigation device configured to acquire vehicle navigation data; and a vehicle network device configured to acquire vehicle status data; and The data processor further includes a data synchronization module configured to perform time synchronization on the on-vehicle visual perception data, the vehicle navigation data and the vehicle status data to form structured vehicle perception data, wherein each piece of the structured vehicle perception data includes a timestamp, a perception image at the current moment, the vehicle position at the current moment, and the vehicle dynamics parameters at the current moment; The preset screening rules include: a comparison and screening rule between a high-precision map and perception data; Among them, the comparison and screening rule between the high-precision map and perception data includes: running the pre-trained neural network to identify the structured vehicle perception data to obtain identification data; when the forward distance of the vehicle from the pre-marked target road infrastructure in the high-precision map is less than a first threshold, determining whether the pre-marked target road infrastructure is included in the identification data at the current moment; if not, using the structured vehicle perception data of a specified number of frames adjacent before and after the structured vehicle perception data centered on the current moment as the incremental data.

14. The system according to claim 13, wherein, The data processor further includes: a data caching module configured to cache the preprocessed vehicle driving data.

15. The system according to claim 13, wherein, The memory is an on-vehicle mobile hard disk.

Images