Data processing method, device, equipment, storage medium and product

Abstract

The present application discloses a data processing method, device, equipment, storage medium and product, which relate to the field of intelligent driving technology. The method comprises: acquiring intelligent driving data, and configuring parameters of the intelligent driving data to obtain configured intelligent driving data, so as to reduce data analysis anomalies caused by configuration errors and improve data analysis quality and efficiency; processing the configured intelligent driving data to obtain intelligent driving model training data, so as to ensure that a high-quality intelligent driving model training data set is finally obtained, so that a good intelligent driving model can be effectively trained to improve the safety and reliability of the automatic driving system; labeling the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label, and the attributes and categories of the data can be intuitively characterized through the labels, so as to provide effective support for data balance while ensuring the diversity of intelligent driving data.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to data processing methods, devices, equipment, storage media and products. Background Art

[0002] The importance of intelligent driving functions in the entire automotive industry is gradually increasing. The deployment of intelligent driving model training is also becoming more and more important in the development of intelligent driving functions. In order to train powerful intelligent driving models, a large amount of intelligent driving model data is needed to support it, and at the same time, the diversity of intelligent driving model data must be ensured. In the existing technology, there is no solution that can ensure the diversity of intelligent driving model data. Summary of the invention

[0003] The main purpose of this application is to provide a data processing method, device, equipment, storage medium and product, aiming to solve the technical problem of how to ensure the diversity of intelligent driving model data.

[0004] To achieve the above objectives, the present application proposes a data processing method, which includes:

[0005] Acquire intelligent driving data, and perform parameter configuration on the intelligent driving data to obtain configured intelligent driving data;

[0006] Processing and analyzing the configured intelligent driving data to obtain intelligent driving model training data;

[0007] The intelligent driving model training data is labeled to obtain the intelligent driving model training data corresponding to each label.

[0008] In one embodiment, the step of performing parameter configuration on the intelligent driving data to obtain the configured intelligent driving data includes:

[0009] Obtain basic parameters and parameters of each functional sub-module in intelligent driving data;

[0010] Correcting the error information in the basic parameters and optimizing the performance parameters in the basic parameters to obtain the configured basic parameters;

[0011] Identify the type of the parameters of each functional submodule, and based on the type of the parameters of each functional submodule, adopt a corresponding configuration strategy to configure the parameters of each functional submodule to obtain the configured parameters of each functional submodule;

[0012] Based on the configured basic parameters and the configured parameters of each functional sub-module, the configured intelligent driving data is obtained.

[0013] In one embodiment, the step of processing and analyzing the configured intelligent driving data to obtain intelligent driving model training data includes:

[0014] Processing the configured intelligent driving data by using a data parsing format version parameter and in combination with a preset parsing strategy to obtain parsed intelligent driving data, and converting the parsed intelligent driving data into intelligent driving data with a unified format;

[0015] Performing valid frame check on the intelligent driving data in the unified format;

[0016] Perform map clipping and map topology construction on the map data in the intelligent driving data after the valid frame check to obtain local topology map data;

[0017] Performing scene complexity analysis and scene recognition based on the local topology map data to obtain scene analysis results;

[0018] Identify the interactive object based on the intelligent driving data after the valid frame inspection;

[0019] Establishing the surrounding information of traffic participants, lane centerline and lane boundary line based on the intelligent driving data after the valid frame inspection;

[0020] Acquiring vehicle lane change information according to the surrounding information of the traffic participant, the lane center line and the lane boundary line;

[0021] Combining the scene analysis result, the interactive object, and the vehicle lane change information, performing scene scoring on the intelligent driving data after the valid frame check to obtain a scene complexity evaluation value corresponding to each intelligent driving data;

[0022] According to the scene complexity evaluation values ​​corresponding to the respective intelligent driving data, the intelligent driving data after the valid frame inspection is converted into intelligent driving model training data.

[0023] In one embodiment, the step of labeling the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label includes:

[0024] Get labels in different dimensions;

[0025] Through the labels under the different dimensions, the attributes and categories of the intelligent driving model training data are mined to obtain the corresponding intelligent driving model training data under each label.

[0026] In one embodiment, after labeling the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label, the following steps are further included:

[0027] The preprocessed intelligent driving model training data corresponding to each label are stored in a pre-established data structure.

[0028] In one embodiment, before the step of storing the pre-processed intelligent driving model training data corresponding to each label in a pre-established data structure, the method further includes:

[0029] Information statistics and data slicing processing are performed on the intelligent driving model training data corresponding to each label to complete the preprocessing operation of the intelligent driving model training data corresponding to each label.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a data processing device, the data processing device comprising:

[0031] A configuration module, used to obtain intelligent driving data and perform parameter configuration on the intelligent driving data to obtain configured intelligent driving data;

[0032] A processing module, used to process the configured intelligent driving data to obtain intelligent driving model training data;

[0033] The labeling module is used to label the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label.

[0034] In addition, to achieve the above objectives, the present application also proposes a data processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data processing method described above.

[0035] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data processing method described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the data processing method described above are implemented.

[0037] The embodiments of the present application provide a data processing method, apparatus, device, storage medium and product, which specifically include: acquiring intelligent driving data, and configuring parameters of the intelligent driving data to obtain configured intelligent driving data, so as to reduce data analysis anomalies caused by configuration errors and improve data analysis quality and efficiency; processing the configured intelligent driving data to obtain intelligent driving model training data, so as to ensure that a high-quality intelligent driving model training data set is finally obtained, so as to effectively train an intelligent driving model with good performance and improve the safety and reliability of the automatic driving system; labeling the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label, and the attributes and categories of the data can be intuitively characterized through the labels, so as to provide effective support for data balance while ensuring the diversity of intelligent driving data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 A flowchart of the first embodiment of the data processing method of the present application is provided;

[0041] Figure 2 A flowchart of the second embodiment of the data processing method of this application is provided;

[0042] Figure 3 A flowchart of the third embodiment of the data processing method of this application is provided;

[0043] Figure 4 A schematic diagram of a data analysis and labeling framework provided in Example 1 of the present application;

[0044] Figure 5 This is a schematic diagram of the module structure of the data processing device according to an embodiment of the present application;

[0045] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the data processing method in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] The main solution of the embodiment of the present application is: obtaining intelligent driving data, and configuring parameters of the intelligent driving data to obtain configured intelligent driving data; processing the configured intelligent driving data to obtain intelligent driving model training data; labeling the intelligent driving model training data to obtain intelligent driving model training data corresponding to each label.

[0050] The importance of intelligent driving functions in the entire automotive industry is gradually increasing. The deployment of intelligent driving model training is also becoming more and more important in the development of intelligent driving functions. In order to train powerful intelligent driving models, a large amount of intelligent driving model data is needed to support it, and at the same time, the diversity of intelligent driving model data must be ensured. In the existing technology, there is no solution that can ensure the diversity of intelligent driving model data.

[0051] In order to ensure the diversity of intelligent driving model data, it is necessary to have a comprehensive and detailed understanding of the intelligent driving model data, and conduct a detailed analysis of the intelligent driving model data to characterize the differences and importance of the intelligent driving model data from different dimensions.

[0052] The present application provides a solution, specifically to obtain intelligent driving data, and configure parameters of the intelligent driving data to obtain configured intelligent driving data. By configuring and processing the intelligent driving data, it is possible to reduce and avoid abnormal data analysis results caused by excessive parameter configuration errors, and effectively improve the quality and efficiency of intelligent driving data analysis; the configured intelligent driving data is processed to obtain intelligent driving model training data, ensuring that the final intelligent driving model training data set is of high quality, and can effectively train a good-performance intelligent driving model, thereby improving the safety and reliability of the automatic driving system; the intelligent driving model training data is labeled to obtain the intelligent driving model training data corresponding to each label. The attributes and categories of the data can be intuitively characterized through the labels, while ensuring the diversity of the intelligent driving data, providing effective support for data balance.

[0053] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a vehicle-mounted system, etc. The following takes a mobile phone as an example to illustrate this embodiment and the following embodiments.

[0054] Based on this, the present application embodiment provides a data processing method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the data processing method of the present application.

[0055] In this embodiment, the data processing method includes steps S10 to S30:

[0056] Step S10, acquiring intelligent driving data, and performing parameter configuration on the intelligent driving data to obtain configured intelligent driving data;

[0057] It is understandable that in order to ensure the diversity of intelligent driving model data, it is necessary to have a comprehensive and detailed understanding of the intelligent driving model data and conduct a detailed analysis of the intelligent driving model data to characterize the differences and importance of the intelligent driving model data from different dimensions.

[0058] Before analyzing the intelligent driving data, configuring the parameters of the intelligent driving data can reduce data analysis anomalies caused by configuration errors and improve the quality and efficiency of data analysis.

[0059] In a feasible implementation, step S10 may include steps S11 to S14:

[0060] Step S11, obtaining basic parameters and parameters of each functional submodule in the intelligent driving data;

[0061] It should be noted that, in this embodiment, parameter configuration mainly involves the basic parameter configuration of data processing and the configuration of each functional sub-module, including but not limited to the configuration of data path, configuration of function switch, configuration of parameter threshold, configuration of control data processing flow, etc.

[0062] In addition, it should be noted that the data analysis framework provided in this embodiment centrally manages different types of parameters in the entire data analysis process, thereby reducing and avoiding abnormal data analysis results caused by excessive parameter configuration errors.

[0063] Step S12, performing error correction processing on the error information in the basic parameters, and optimizing processing on the performance parameters in the basic parameters, to obtain the configured basic parameters;

[0064] Optionally, you can configure basic parameters, such as obtaining the source of smart driving data, obtaining extended information of smart driving data, whether to process the debug (Boolean value) mode of smart driving data, whether to filter out Jira (Boolean value) issues in smart driving data, whether to notify through WeChat if data anomalies are detected, whether to perform frame anomaly checks, whether to perform adaptive concurrent scheduling, and specify a working directory.

[0065] In addition, since a large amount of coordinate transformation is used in the intelligent driving data analysis process, especially some features need to be converted to the vehicle coordinate system for calculation and analysis, which places high requirements on the vehicle parameters. Therefore, in the process of configuring the basic parameters, the vehicle of the intelligent driving data will also be identified, so as to correct the basic information of the vehicle.

[0066] Since the sources of intelligent driving data are wide, the data collected by different vehicles and different models have different basic information about the vehicle, such as the length, width, wheelbase, and the relationship between the center point of the vehicle and the center point of the rear axle. In the intelligent driving data extraction process, some basic information about the vehicle is not easy to obtain directly. At this time, the vehicle type and other data will be obtained based on the original data information, and the basic information of the vehicle will be adaptively loaded and corrected according to the vehicle type.

[0067] Step S13, identifying the type of each functional submodule parameter, and based on the type of each functional submodule parameter, adopting a corresponding configuration strategy to configure each functional submodule parameter, to obtain configured parameters of each functional submodule;

[0068] Since intelligent driving data comes from a wide range of sources, different types of intelligent driving data contain sub-modules that perform different functions. Therefore, it is necessary to identify the type of parameters of each functional sub-module, adopt corresponding configuration strategies, and configure parameters for each type of functional sub-module based on the corresponding configuration strategies.

[0069] The following is a further example of important parameter configuration in each functional submodule:

[0070] Folder parameter configuration: all input and output data path configurations, such as input data path, output data path, video storage path, label storage path, etc.

[0071] Data storage configuration: for example, whether to store map topology, whether to save intermediate output results, whether to save lane change information, etc.

[0072] Visualization parameter configuration: such as video switch, whether to draw historical tracks, whether lane-changing objects are highlighted, video zoom ratio, whether to convert MP4 videos to TS format, etc.

[0073] Abnormal inspection parameter configuration: such as object heading angle abnormal threshold, object speed abnormal threshold, trajectory longitudinal jump abnormal threshold, trajectory smoothness abnormal threshold, etc.

[0074] Slice parameter configuration: such as the minimum frame length of the slice, visualization of data slices, etc.

[0075] Frame complexity parameter configuration: mainly involves the enabling switch and threshold setting of frame level complexity.

[0076] Lane binding parameter configuration: involves lane binding mode selection, whether to bind the lane centerline and boundary, etc.

[0077] Label parameter configuration: for example, whether to perform label correction, whether to perform target-level object labeling, etc.

[0078] Intersection-related parameter configuration: such as whether to enable intersection recognition, intersection recognition stability threshold, intersection recognition frame length parameter configuration, etc.

[0079] Step S15, obtaining configured intelligent driving data based on the configured basic parameters and the configured parameters of each functional sub-module.

[0080] By configuring the completed basic parameters and parameters of each functional sub-module, the configured intelligent driving data is obtained, which can effectively improve the quality and efficiency of intelligent driving data processing and ensure that the final intelligent driving model training data set is of high quality, thereby training an intelligent driving model with good performance and improving the safety and reliability of the autonomous driving system.

[0081] Step S20, processing the configured intelligent driving data to obtain intelligent driving model training data;

[0082] It should be noted that for large-scale offline data processing, since each intelligent driving data is quite different, in order to make full use of machine resources, this embodiment designs load balancing to ensure the reasonable adjustment of machine resources.

[0083] In addition, it should be noted that the data file format of the intelligent driving data analysis link is protobuf format. The data mainly contains object information (including detailed information of the vehicle itself and other vehicles), map information and vehicle driving status information, all of which are extracted from the original data.

[0084] It is understandable that in order to ensure that a high-quality intelligent driving model training data set is finally obtained, a good-performance intelligent driving model can be effectively trained to improve the safety and reliability of the autonomous driving system, therefore, step S20 is executed.

[0085] In a feasible implementation, step S20 may include steps S21 to S29:

[0086] Step S21, processing the configured intelligent driving data by combining the data parsing format version parameter with a preset parsing strategy to obtain parsed intelligent driving data, and converting the parsed intelligent driving data into intelligent driving data with a unified format;

[0087] It should be noted that in the intelligent driving system, the data parsing format version parameter is used to identify the data format version obtained from different sensors (such as cameras, lidars, etc.).

[0088] Alternatively, since smart driving data sets are usually quite large, the entire data collection time span will be relatively large. Although the protobuf format has upward and downward compatibility, it is difficult to ensure forward and backward compatibility for some major upgraded data. In this case, several versions of the data parsing format will be retained to facilitate the processing of smart driving data from different periods.

[0089] By configuring the data parsing format version parameters, the format required for the data parsing will be adaptively scheduled. Specifically, the data parsing version parameters are first defined to identify the format version required for data parsing. Then, different data parsing strategies and scheduling schemes are configured according to different version parameters. In addition, the configuration and scheduling logic of data parsing are dynamically adjusted according to the data parsing version parameters to obtain data with a unified format.

[0090] According to the above steps, the adaptive scheduling of data parsing logic is realized by configuring the data parsing format version parameters, which effectively solves the compatibility problem of data parsing in different historical periods and brings great convenience to processing different periods and different versions of data.

[0091] Step S22, performing a valid frame check on the intelligent driving data in a unified format;

[0092] Optionally, since smart driving data comes from a wide range of sources and has many types of data anomalies, substituting outliers into smart driving model training will lead to unexpected serious consequences. Therefore, before performing valid frame checks on smart driving data in a unified format, it is also necessary to perform comprehensive outlier detection on the smart driving data in a unified format and filter out the abnormal data.

[0093] After the abnormal data is filtered, the intelligent driving data with a unified format is checked for valid frames. The checking process includes: identifying abnormalities in the map information and the trajectory information of traffic participants in each frame of intelligent driving data with a unified format, such as abnormal trajectory jitter, abnormal speed fluctuation, abnormal angle fluctuation, etc. At the same time, special filtering is performed on intersection information. Then, the abnormal frame data is identified, and different processing rules are formulated to determine the processing strategy of the abnormal frame data according to the application scenario and data processing requirements. Based on different processing strategies, it is decided to retain the abnormal frame data and try to repair it, or to directly discard the abnormal frame to avoid negative impact on subsequent processing. In addition, the intelligent driving data fragments with a unified format after the outlier processing are merged to form a complete data sequence, and then specific parts or fragments are cut out from the complete data sequence for more detailed data analysis.

[0094] Step S23, performing map clipping and constructing map topology on the map data in the intelligent driving data after the valid frame check, and obtaining local topology map data;

[0095] Since the map range used in the entire intelligent driving dataset is relatively large, especially in scenarios with high-precision maps, we usually only care about the map situation around the vehicle, and the importance of the map distance decreases as the distance from the vehicle increases. The map is cropped using the corresponding map cropping scheme to remove the map information far away from the vehicle. Different map cropping schemes can be selected according to parameter configuration.

[0096] The purpose of constructing map topology is to associate the relationship between lanes, the relationship between lanes and vehicles, the relationship between vehicles and lanes, and the relationship between vehicles and vehicles. In intelligent driving data analysis, many functional submodules will greatly reduce the amount of calculation with the help of map topology and avoid unnecessary repeated and redundant calculation overhead.

[0097] Finally, the above-mentioned map clipping and map topology building functions are combined to process the intelligent driving data in a unified format to generate local topology map data.

[0098] Step S24, performing scene recognition and scene complexity analysis based on the local topology map data to obtain scene analysis results;

[0099] Optionally, the scene complexity analysis mainly determines the map complexity, obstacle complexity, and lane change complexity based on the local topological map data. Among them, the map complexity mainly describes the diversity and complexity of the scene from the map dimension; the obstacle complexity mainly describes the complexity of the scene from the vehicle type and vehicle behavior; the lane change complexity incorporates key information such as vehicle lane change and cutin (the behavior of the vehicle cutting into the main lane from the side lane) into the scene complexity.

[0100] Scene recognition mainly identifies vehicle congestion scenes, human driving scenes, NZP (Navigational Zero Pilot, intelligent driving state based on high-precision maps) driving mode, LCC (Lane Centering Control, intelligent driving state based on perception maps) driving mode, ACC (Adaptive Cruise Control, driving state based on adaptive cruise system) driving mode, etc. Identifying these scenes is mainly to distinguish the uses of intelligent driving data. For example, for the model of training the future trajectory planning of the vehicle, human driving data is needed, which requires identification and distinction of human driving data. In addition, congestion scene recognition can filter abnormal parking situations and reduce data noise input.

[0101] Based on the above steps, that is, by analyzing scene complexity and identifying specific scenes, the intelligent driving system can better understand the characteristics of the current driving environment and make more accurate and timely decisions.

[0102] Step S25, identifying an interactive object based on the intelligent driving data after the valid frame check;

[0103] It should be noted that the interaction object refers to the entity that may interact with the vehicle in the intelligent driving scenario.

[0104] Optionally, the identification of interactive objects mainly focuses on obstacles that have a greater impact on the ego-vehicle (i.e., the intelligent driving vehicle itself), and these obstacles are specially processed, such as increasing the impact weight of the obstacle.

[0105] Specifically, first, obstacles in the video frame are detected by using object detection algorithms (such as YOLO, Faster R-CNN, etc.). For each obstacle detected, its type (such as vehicle, pedestrian, bicycle, etc.) and location are determined. Then, the detected obstacles are classified, focusing on obstacles that will have a greater impact on the ego vehicle. Next, for the selected key obstacles, their future trajectories are predicted using a pre-trained motion model, where the prediction time range is usually the next few seconds, assuming the trajectory within the next 8 seconds. And determine whether these obstacles will interact with the ego vehicle within the future time range, including but not limited to collision risks, lane change risks, etc. Finally, obstacles with interactive behaviors are marked to reduce safety risks.

[0106] Step S26, establishing the surrounding information of the traffic participants, the lane centerline and the lane boundary line based on the intelligent driving data after the valid frame check;

[0107] Optionally, the surrounding information of the traffic participant is established, mainly including the traffic participants or obstacles adjacent to the traffic participant in front, behind, left, and right, and the distance from the traffic participant to the lane boundary, etc. The surrounding information of the traffic participant can be used as an important feature on the one hand, and on the other hand, the position of the traffic participant and the surrounding interaction can be quickly located based on this information.

[0108] The lane centerline, lane boundary line, and road boundary line association information are established to facilitate the rapid acquisition of soft and hard boundary conditions during vehicle traffic. Among them, traffic participants are bound to lanes, and the association between traffic participants and lanes is achieved through the lane binding algorithm.

[0109] Step S27, acquiring vehicle lane change information according to the surrounding information of the traffic participant, the lane center line and the lane boundary line;

[0110] Optionally, the vehicle lane change recognition is implemented based on the vehicle intention and the prediction of the vehicle's future driving trajectory. The vehicle intention and the prediction of the vehicle's future driving trajectory are two core issues in the field of intelligent driving prediction, and the vehicle intention is the focus of attention when the vehicle changes lanes. On the one hand, the lane change of other vehicles at a close distance from the vehicle will have an important impact on the vehicle; on the other hand, the risk of obstacles in the vehicle's driving trajectory planning may require lane change. Therefore, the vehicle's lane change behavior will be specially handled in the training of the intelligent driving model.

[0111] Optionally, this step combines the surrounding information of traffic participants and lane information to evaluate the safety and feasibility of lane changes, such as ensuring a sufficient safe distance from the front and rear vehicles. At the same time, based on the lane centerline and lane boundary line, a path planning algorithm is used to determine the optimal lane change path. And lane change recognition is performed for different types of objects in different scenarios (NZP, LCC, etc.). Combining the above three methods, the vehicle lane change information is obtained, and the vehicle lane change information is stored at the target level, and the vehicle number that changes lanes in any frame is stored, so that the vehicle can be specially processed according to the frame number and vehicle number when training the intelligent driving model.

[0112] Step S28, combining the scene analysis result, the interactive object and the vehicle lane change information, performing scene scoring on the intelligent driving data after the valid frame check to obtain a scene complexity evaluation value corresponding to each intelligent driving data;

[0113] Optionally, this step is mainly for data balancing. The scene score is obtained by combining the scene complexity, interactive objects, vehicle lane change behavior, and key obstacle analysis results obtained through the above steps. The scene score also represents the importance of intelligent driving data. The scene scoring process can be expressed as:

[0114] scores=scene_score*scene_weight+lane_change_scores* lane_change_weight + close_interaction_scores* close_interaction_scores_weight + focal_agent_scores * focal_agent _weight;

[0115] scores = minimum(max_value, score_sums);

[0116] Among them, scene_score is the scene complexity, scene_weight is the scene complexity weight, lane_change_scores is the lane change complexity, lane_change_weight is the lane change complexity weight, close_interaction_scores is the interaction scene complexity, close_interaction_scores_weight is the interaction scene weight, focal_agent_scores is the key obstacle complexity, focal_agent _weight is the key obstacle weight. max_value is the maximum score threshold, the total score is clipped, and the smaller value of max_value and score_sums is taken. scores is the scene complexity evaluation value.

[0117] Step S29: converting the intelligent driving data after the valid frame check into intelligent driving model training data according to the scene complexity evaluation value corresponding to each intelligent driving data.

[0118] Optionally, after obtaining the scene complexity evaluation value corresponding to each intelligent driving data, the intelligent driving data are divided into different complexity levels, such as low complexity, medium complexity and high complexity, according to the complexity score. Then, according to the result of intelligent driving data classification, the intelligent driving data of different complexity levels are converted into intelligent driving model training data suitable for different intelligent driving models, and they are reasonably allocated to the training set, the validation set and the test set to optimize the intelligent driving model.

[0119] Through the above steps, the configured intelligent driving data is processed to obtain intelligent driving model training data, ensuring that the final intelligent driving model training data set is of high quality, can effectively train an intelligent driving model with good performance, and improve the safety and reliability of the autonomous driving system.

[0120] Step S30: label the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label.

[0121] It is understandable that in order to train a powerful intelligent driving model, while ensuring a large amount of data support, it is also necessary to ensure the diversity of the data. Therefore, executing step S30 can characterize the differences and importance of the data from different dimensions to ensure the diversity of the data.

[0122] In a feasible implementation, step S30 may include steps S31-S32:

[0123] Step S31, obtaining labels in different dimensions;

[0124] It should be noted that labels under different dimensions refer to the process of labeling intelligent driving data from multiple angles according to different analysis needs and objectives when processing intelligent driving data. The labels include scene-level labels and target-level labels.

[0125] Among them, the scene-level label refers to the descriptive label of the entire scene; the object-level label is the label for a single object or a group of related objects in the scene.

[0126] Step S32: mining the attributes and categories of the intelligent driving model training data through the labels under the different dimensions to obtain the intelligent driving model training data corresponding to each label.

[0127] Optionally, after obtaining labels under different dimensions, each data record can be manually labeled by experts or relevant personnel. Subsequently, the labeled dataset is used to train the intelligent driving model to ensure that the model can learn the characteristics of the data from different perspectives.

[0128] Through the above steps, that is, labeling the intelligent driving model training data with labels under different dimensions, the attributes and categories of the data can be intuitively characterized, while ensuring the diversity of intelligent driving training data and providing effective support for data balance.

[0129] The above-mentioned embodiment method specifically includes acquiring intelligent driving data, and configuring parameters of the intelligent driving data to obtain configured intelligent driving data. By configuring and processing the intelligent driving data, it is possible to reduce and avoid abnormal data analysis results caused by excessive parameter configuration errors, and effectively improve the quality and efficiency of intelligent driving data analysis; processing the configured intelligent driving data to obtain intelligent driving model training data, ensuring that the final intelligent driving model training data set is of high quality, and can effectively train a good-performance intelligent driving model, thereby improving the safety and reliability of the automatic driving system; labeling the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label. The labels can intuitively characterize the attributes and categories of the data, while ensuring the diversity of the intelligent driving data, providing effective support for data balance.

[0130] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 After step S30, the data processing method further includes step S31:

[0131] Step S31, storing the preprocessed intelligent driving model training data corresponding to each label in a pre-established data structure.

[0132] Optionally, in this embodiment, an intermediate result is designed to save a data structure, which is divided into scene level, frame level and target level, and is used to save the more important process quantity data in the data analysis process at the corresponding position of the data structure. After the data analysis work is completed, the intermediate result saves the data structure together with the output data.

[0133] If the data needs to be further analyzed or mined later, the data structure can be directly loaded to directly extract relevant information for data processing, which can greatly reduce the overhead of data processing calculations and improve data processing efficiency.

[0134] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned second embodiment can refer to the above introduction, and will not be repeated in the following. Figure 3 , before step S31, the data processing method further includes step S301:

[0135] Step S301, performing information statistics and data slicing processing on the intelligent driving model training data corresponding to each label to complete the preprocessing operation of the intelligent driving model training data corresponding to each label.

[0136] Optionally, after completing the scene labeling and obtaining the corresponding intelligent driving model training data under each label, the output intelligent driving model training data is statistically analyzed, including outlier statistics, number of valid fragments, effective frame ratio statistics, abnormal frame classification statistics, analysis of the time consumption statistics of each module, data distribution statistics (vehicle type statistics, vehicle quantity distribution, map lane distribution, etc.), so as to further understand the intelligent driving model training data and provide data support for optimizing the intelligent driving model.

[0137] Then, the training data of the intelligent driving model corresponding to each label is sliced ​​according to the data segmentation algorithm. In order to ensure that the sliced ​​data fully meets the training requirements, unreasonable or abnormal frame data is removed.

[0138] Finally, the training data of the intelligent driving model corresponding to each label that has completed information statistics and data slicing processing are normalized in format for training different intelligent driving models.

[0139] In addition, since the intelligent driving model training data contains many types of data, among which the map information and vehicle information are very complex, in order to accurately and intuitively understand the data, this implementation also designed a data visualization module. The data visualization module is divided into two modes: video visualization and image visualization. Through the video visualization module, you can understand the change information of the time series in detail; through image visualization, you can understand the interaction of traffic participants at the frame level in detail.

[0140] Through the above-mentioned embodiment method, the number and distribution of labels in each dimension of the data set can be statistically analyzed through information statistics, which helps to understand the characteristics of the data set and discover potential problems; through the process of dividing the data set into different subsets based on labels, special analysis and training can be performed on different subsets to ensure that the model can perform well in all important scenarios.

[0141] For example, to help understand the implementation process of the data processing method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 4 , Figure 4 A data analysis and labeling framework diagram of a data processing method is provided, specifically:

[0142] First, obtain intelligent driving data information, including object information, map information, and status labels, etc. And perform parameter configuration on the acquired intelligent driving data information to obtain optimized intelligent driving data. The parameter configuration process in the embodiment of the present application mainly involves the basic parameter configuration of data processing, the configuration of each functional submodule, including the configuration of data path, the configuration of function switch, the configuration of parameter threshold, the configuration of control data processing flow, etc.

[0143] Then, for large-scale offline data processing, the data differences are large. In order to make full use of machine resources, this embodiment designs load balancing to ensure the reasonable allocation of machine resources. Through load balancing, the intelligent driving data information file is parsed, compatible with different versions of intelligent driving data, and a unified version of intelligent driving data is obtained. After the intelligent driving data in a unified format is processed for outliers, a validity frame check is performed, and abnormal frames are marked. The retention of abnormal frames and the integration and interception of data fragments can be determined based on different processing strategies.

[0144] Next, the map information in the intelligent driving data after the valid frame check is used for cropping to remove the map information far away from the vehicle, and by constructing the map topology, the relationship between lanes, the relationship between lanes and vehicles, the relationship between vehicles and lanes, and the relationship between vehicles.

[0145] At the same time, the scene complexity in the intelligent driving data after the valid frame inspection is analyzed, including the map information complexity, obstacle complexity, and vehicle lane change complexity, etc., to obtain the effective scene complexity of the vehicle. And scene recognition is performed, mainly identifying congestion scenes, human driving scenes, NZP driving mode, LCC driving mode, ACC driving mode, etc., mainly to distinguish the purpose of the data.

[0146] At the same time, the interactive objects in the intelligent driving data after the valid frame inspection are identified, and obstacles that will interact with the vehicle in the next n seconds (the value can be 8 seconds) are marked. Based on the intelligent driving data after the valid frame inspection, the surrounding information of traffic participants, lane center lines and lane boundaries are established. Combined with the surrounding information of traffic participants, lane center lines and lane boundaries, vehicle lane change recognition is performed for different types of objects in different scenarios (NZP, LCC, etc.), and the vehicle lane change information is stored at the target level, storing the vehicle number that changes lanes in any frame.

[0147] The comprehensive scores of intelligent driving data in each scenario are obtained by combining the analysis results of scene complexity, interactive objects, lane change behavior, key obstacles, etc. According to the comprehensive scores in each scenario, the intelligent driving data is converted into training set data for training intelligent driving models.

[0148] Subsequently, the training set data used to train the intelligent driving model is labeled accordingly through labels in different dimensions, including scene-level labels and target-level labels, to distinguish which intelligent driving model each intelligent driving model training data is suitable for.

[0149] Finally, the intelligent driving model training data corresponding to each label is pre-processed by data visualization, information statistics, data slicing, and filtering. The intelligent driving model training data for different scenarios is analyzed and trained to ensure that the intelligent driving model can perform well in all important scenarios.

[0150] Through the above-mentioned embodiment method, the embodiment of the present application provides a data processing and analysis implementation framework, which includes multiple sub-function modules to implement different functional analyses. The coupling between each sub-function module is low, and each sub-function can be turned on or off by configuring parameters, which brings a higher degree of freedom to the processing of the entire data.

[0151] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data processing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0152] This application also provides a data processing device, please refer to Figure 5 , the data processing device comprises:

[0153] A configuration module 10 is used to obtain intelligent driving data and perform parameter configuration on the intelligent driving data to obtain configured intelligent driving data;

[0154] A processing module 20 is used to process the configured intelligent driving data to obtain intelligent driving model training data;

[0155] The labeling module 30 is used to label the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label.

[0156] The data processing device provided by the present application adopts the data processing method in the above embodiment, which can solve the technical problem of how to ensure the diversity of intelligent driving model data. Compared with the prior art, the beneficial effects of the data processing device provided by the present application are the same as the beneficial effects of the data processing method provided by the above embodiment, and the other technical features in the data processing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0157] The present application provides a data processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data processing method in the above-mentioned embodiment 1.

[0158] Reference below Figure 6 , which shows a schematic diagram of the structure of a data processing device suitable for implementing an embodiment of the present application. The data processing device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The data processing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0159] like Figure 6As shown, the data processing device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data processing device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a data processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0160] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0161] The data processing device provided by this application adopts the data processing method in the above embodiment to solve the technical problem of how to ensure the diversity of intelligent driving model data. Compared with the prior art, the beneficial effects of the data processing device provided by this application are the same as the beneficial effects of the data processing method provided by the above embodiment, and the other technical features in the data processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0162] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0163] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0164] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data processing method in the above-mentioned embodiment.

[0165] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0166] The computer-readable storage medium may be included in a data processing device, or may exist independently without being incorporated into a data processing device.

[0167] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a data processing device, the data processing device:

[0168] Acquire intelligent driving data, and perform parameter configuration on the intelligent driving data to obtain configured intelligent driving data;

[0169] Processing and analyzing the configured intelligent driving data to obtain intelligent driving model training data;

[0170] The intelligent driving model training data is labeled to obtain the intelligent driving model training data corresponding to each label.

[0171] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0172] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0173] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0174] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data processing method, and can solve the technical problem of how to ensure the diversity of intelligent driving model data. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the data processing method provided in the above-mentioned embodiment, and will not be repeated here.

[0175] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0176] The computer program product provided by this application can solve the technical problem of how to ensure the diversity of intelligent driving model data. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the data processing method provided by the above embodiment, which will not be repeated here.

[0177] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Acquiring intelligent driving data, and performing parameter configuration on the intelligent driving data to obtain configured intelligent driving data; Processing the configured intelligent driving data to obtain intelligent driving model training data includes: parsing and converting the configured intelligent driving data to obtain intelligent driving data in a unified format; performing scene recognition and scene complexity analysis on the intelligent driving data in a unified format to obtain scene analysis results; combining the scene analysis results, interactive objects, and vehicle lane change information, performing scene scoring on the intelligent driving data in a unified format to obtain a scene complexity evaluation value; converting intelligent driving data of different complexity levels into intelligent driving model training data of different intelligent driving models according to the scene complexity evaluation value; The intelligent driving model training data is labeled to obtain the intelligent driving model training data corresponding to each label.

2. The method according to claim 1, characterized in that The step of performing parameter configuration on the intelligent driving data to obtain the configured intelligent driving data comprises: Obtain basic parameters and parameters of each functional sub-module in intelligent driving data; Correcting the error information in the basic parameters and optimizing the performance parameters in the basic parameters to obtain the configured basic parameters; Identify the type of the parameters of each functional submodule, and based on the type of the parameters of each functional submodule, adopt a corresponding configuration strategy to configure the parameters of each functional submodule to obtain the configured parameters of each functional submodule; Based on the configured basic parameters and the configured parameters of each functional sub-module, the configured intelligent driving data is obtained.

3. The method according to claim 1, characterized in that The step of processing and analyzing the configured intelligent driving data to obtain intelligent driving model training data includes: Processing the configured intelligent driving data by using a data parsing format version parameter and in combination with a preset data parsing strategy to obtain parsed intelligent driving data, and converting the parsed intelligent driving data into intelligent driving data with a unified format; Performing valid frame check on the intelligent driving data in the unified format; Performing map clipping and constructing map topology on the map data in the intelligent driving data after the valid frame check to obtain local topology map data; Performing scene recognition and scene complexity analysis based on the local topology map data to obtain scene analysis results; Identify the interactive object based on the intelligent driving data after the valid frame inspection; Establishing the surrounding information of traffic participants, lane centerline and lane boundary line based on the intelligent driving data after the valid frame inspection; Acquiring vehicle lane change information according to the surrounding information of the traffic participant, the lane center line and the lane boundary line; Combining the scene analysis result, the interactive object, and the vehicle lane change information, performing scene scoring on the intelligent driving data after the valid frame check to obtain a scene complexity evaluation value corresponding to each intelligent driving data; According to the scene complexity evaluation values ​​corresponding to the respective intelligent driving data, the intelligent driving data after the valid frame inspection is converted into intelligent driving model training data.

4. The method according to claim 1, characterized in that The step of labeling the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label includes: Get labels in different dimensions; Through the labels under the different dimensions, the attributes and categories of the intelligent driving model training data are mined to obtain the corresponding intelligent driving model training data under each label.

5. The method according to claim 1, characterized in that After the step of labeling the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label, the method further includes: The preprocessed intelligent driving model training data corresponding to each label are stored in a pre-established data structure.

6. The method according to claim 5, characterized in that Before the step of storing the pre-processed intelligent driving model training data corresponding to each label in a pre-established data structure, the method further includes: Information statistics and data slicing processing are performed on the intelligent driving model training data corresponding to each label to complete the preprocessing operation of the intelligent driving model training data corresponding to each label.

7. A data processing device, characterized in that: The device comprises: A configuration module, used to obtain intelligent driving data and perform parameter configuration on the intelligent driving data to obtain configured intelligent driving data; A processing module, used to process the configured intelligent driving data to obtain intelligent driving model training data, including: parsing and converting the configured intelligent driving data to obtain intelligent driving data in a unified format; performing scene recognition and scene complexity analysis on the intelligent driving data in a unified format to obtain scene analysis results; combining the scene analysis results, interactive objects, and vehicle lane change information, performing scene scoring on the intelligent driving data in a unified format to obtain a scene complexity evaluation value; and converting intelligent driving data of different complexity levels into intelligent driving model training data of different intelligent driving models according to the scene complexity evaluation value; The labeling module is used to label the intelligent driving model training data to obtain the intelligent driving model training data corresponding to each label.

8. A data processing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data processing method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

Images