Data processing method and device, equipment and storage medium

Abstract

The invention discloses a data processing method and device, equipment and a storage medium, and relates to the technical field of data processing.The method comprises the steps that in response to a data processing instruction, to-be-processed driving data and preset model data requirements are obtained, and a data processing task is created; acquiring associated cloud data from a cloud database according to the data processing task; based on a pre-configured server cluster, performing scene analysis on the associated cloud data according to the data processing task to obtain effective scene fragments; and carrying out training preprocessing on the effective scene segments to obtain model training data. According to the method, large-scale and diversified model training data can be constructed in real time.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0002] In the current automotive industry, the importance of intelligent driving functions in the entire car is gradually increasing. Intelligent driving refers to the technology that machines help people drive, and completely replace human driving in special circumstances. In order to support the development of intelligent driving functions, the relevant model training and deployment are particularly important.

[0003] In order to build a powerful and intelligent driving model, large-scale and diverse model training data is needed as a solid support. At the same time, it is also necessary to ensure that the model training data can cover a wide range of driving scenarios and support rapid iteration and real-time updates to ensure that the model can continue to learn and optimize, so as to accurately respond to complex and changing road environments. However, existing data processing methods cannot achieve real-time construction of large-scale and diverse model training data.

[0004] Therefore, how to build large-scale and diverse model training data in real time is a technical problem that needs to be solved urgently.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a data processing method, device, equipment and storage medium, aiming to solve the technical problem of how to build large-scale and diversified model training data in real time.

[0007] To achieve the above objectives, the present application proposes a data processing method, which includes:

[0008] In response to the data processing instruction, obtaining the driving data to be processed and the preset model data requirements, and creating a data processing task;

[0009] According to the data processing task, obtaining related cloud data from a cloud database;

[0010] Based on the preconfigured server cluster, according to the data processing task, the associated cloud data is subjected to scene analysis to obtain valid scene fragments;

[0011] The effective scene segments are subjected to training preprocessing to obtain model training data.

[0012] In one embodiment, before the step of performing scene analysis on the associated cloud data based on the preconfigured server cluster and according to the data processing task to obtain valid scene fragments, the step further includes:

[0013] Determining a corresponding data parsing format based on the associated cloud data;

[0014] The data parsing format is configured into the server cluster to obtain a preconfigured server cluster.

[0015] In one embodiment, the associated cloud data includes first cloud data, and before the step of acquiring the associated cloud data from the cloud database according to the data processing task, the step further includes:

[0016] According to the data processing task, obtaining driving data to be processed;

[0017] Based on the preconfigured server cluster and according to the data processing task, extracting and merging the required subject data in the to-be-processed driving data to obtain first cloud data;

[0018] The first cloud data is saved in a cloud database.

[0019] In one embodiment, the associated cloud data includes second cloud data, and the step of extracting and merging the required subject data in the to-be-processed driving data according to the data processing task based on the preconfigured server cluster to obtain the first cloud data further includes:

[0020] Based on a preconfigured server cluster, performing effective data analysis on the first cloud data to obtain effective data of several topics;

[0021] Time-aligning the valid data of the several subjects, merging the valid data of the several subjects according to the data frames to obtain merged data;

[0022] Slicing the merged data to obtain second cloud data;

[0023] The second cloud data is saved in a cloud database.

[0024] In one embodiment, the step of performing scene analysis on the associated cloud data based on the preconfigured server cluster and according to the data processing task to obtain valid scene fragments includes:

[0025] Based on a preconfigured server cluster and a multi-concurrency mechanism, a scenario analysis is performed on the associated cloud data to obtain a scenario analysis result;

[0026] Based on the scene analysis results, scene data mining and scene label mining are performed, and effective scene fragments are extracted according to the scene label classification.

[0027] In one embodiment, the preset model data requirement includes a model training data format, and the step of performing training preprocessing on the valid scene fragment to obtain the model training data includes:

[0028] Confirming whether the valid scene segment meets a preset data checking condition;

[0029] After a preset data checking condition is met, extracting relevant scene data from the valid scene fragment;

[0030] The scene data is stored in the model training data format.

[0031] In one embodiment, the scene data includes map data, and the step of extracting relevant scene data from the valid scene fragment after satisfying a preset data checking condition includes:

[0032] After the preset data checking condition is met, the map data in the valid scene segment is subjected to thinning processing;

[0033] The step of storing the scene data in the model training data format includes:

[0034] The map data after the thinning process is stored at the frame level in the model training data format.

[0035] In one embodiment, the method further comprises:

[0036] Performing data production and abnormal statistics on the driving data to be processed to obtain data statistical results;

[0037] Analyze the relevant logs of the data processing task and the data statistical results to generate a data analysis report;

[0038] Based on the model training data and the data statistical results, data visualization is performed.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a data processing device, the data processing device comprising:

[0040] A task creation module, used to respond to data processing instructions, obtain driving data to be processed and preset model data requirements, and create a data processing task;

[0041] A data acquisition module, used to acquire relevant cloud data from a cloud database according to the data processing task;

[0042] A scene analysis module, configured to perform scene analysis on the associated cloud data based on a preconfigured server cluster and according to the data processing task to obtain valid scene fragments;

[0043] The training preprocessing module is used to perform training preprocessing on the effective scene fragments to obtain model training data.

[0044] In addition, to achieve the above objectives, the present application also proposes a data processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data processing method described above.

[0045] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data processing method described above are implemented.

[0046] One or more technical solutions proposed in this application have at least the following technical effects:

[0047] The data processing method, apparatus, device and storage medium proposed in the embodiments of the present application specifically obtain the driving data to be processed and the preset model data requirements in response to the data processing instructions, and create a data processing task; according to the data processing task, obtain the related cloud data from the cloud database; based on the preconfigured server cluster, according to the data processing task, perform scene analysis on the related cloud data to obtain valid scene fragments; perform training preprocessing on the valid scene fragments to obtain model training data.

[0048] This application creates a data processing task through the original driving data and the model data requirements related to intelligent driving, which is used to indicate how to build model training data; then, according to the data processing task, the associated cloud data is obtained from the cloud database, and then combined with the pre-configured server cluster, the associated cloud data is analyzed for scenes to extract scene fragments that are effective for model training; finally, the effective scene fragments are pre-processed for training to ensure that the obtained model training data is reliable and effective. Through the above scheme, this application realizes real-time scheduling and processing of large-scale driving data sets, and constructs large-scale and diverse model training data by performing scene analysis on large-scale driving data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0051] Figure 1 A flowchart of the first embodiment of the data processing method of the present application is provided;

[0052] Figure 2 A flowchart of the second embodiment of the data processing method of this application is provided;

[0053] Figure 3 A flowchart of the third embodiment of the data processing method of this application is provided;

[0054] Figure 4 A flowchart of the fourth embodiment of the data processing method of the present application is provided;

[0055] Figure 5 A flowchart of the fifth embodiment of the data processing method of the present application is provided;

[0056] Figure 6 This is an example flow chart of a data processing method involved in Embodiment 5 of the present application;

[0057] Figure 7 This is a schematic diagram of the module structure of the data processing device according to an embodiment of the present application;

[0058] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the data processing method in the embodiment of the present application.

[0059] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0061] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0062] The main solution of the embodiment of the present application is: in response to data processing instructions, obtaining the driving data to be processed and the preset model data requirements, and creating a data processing task; according to the data processing task, obtaining related cloud data from the cloud database; based on the preconfigured server cluster, according to the data processing task, performing scene analysis on the related cloud data to obtain valid scene fragments; performing training preprocessing on the valid scene fragments to obtain model training data.

[0063] Technical terms involved in the embodiments of this application:

[0064] Server cluster: A server cluster is a system that brings together multiple computer servers and connects them through a high-speed network to jointly complete specific tasks or provide services. By working together, server clusters can significantly improve the reliability, scalability and performance of the system. Server clusters can provide enterprises and users with more stable, efficient and scalable services through load balancing, high availability, resource sharing and data consistency mechanisms.

[0065] ROS (Robot Operating System): ROS is an open source, flexible framework for writing robot software. ROS provides functions similar to traditional operating systems (such as Windows, Linux), but focuses on robotic applications, which can greatly simplify the process of creating complex and robust robotic behaviors across robotic platforms. ROS packets are the main form of data flow in the ROS system. ROS packets mainly use mechanisms such as messages, services or topics, and actions to achieve communication and data exchange between nodes. In the fields of self-driving cars, drones, robots, etc., ROS packets are used to transmit information such as environmental maps, target locations, path planning results, and can also be used to transmit sensor data. At the same time, ROS packets support data recording and playback functions.

[0066] Protobuf: Protobuf (Protocol Buffers) is a lightweight and efficient structured data storage format that can be used for structured data serialization and is particularly suitable for network communication and data storage. Protobuf stores data in binary format, which is more compact than text formats (such as JSON, XML), thereby reducing the amount of data during network transmission or storage, saving bandwidth and storage space. Protobuf supports multiple programming languages, such as C++, Java, Python, etc., making data exchange between different platforms simple. Protobuf's design allows new fields to be added without affecting old data.

[0067] Pickle: Pickle is a Python-specific object serialization module, mainly used for persistent storage of Python objects and object transmission between programs, and can realize binary serialization and deserialization of Python object structures. Through pickle, almost all Python objects can be serialized into byte streams, and these byte streams can be deserialized back to the original Python objects.

[0068] However, existing data processing methods cannot support the real-time production of large-scale and diverse model training data when processing large-scale data sets. Therefore, how to build large-scale and diverse model training data in real time is a technical problem that needs to be solved urgently.

[0069] The present application provides a solution. The present application creates a data processing task through the original driving data and the model data requirements related to intelligent driving, which is used to indicate how to build the model training data; then, according to the data processing task, the associated cloud data is obtained from the cloud database, and then combined with the pre-configured server cluster, the associated cloud data is analyzed in the scene to extract the scene fragments that are effective for model training; finally, the effective scene fragments are pre-processed for training to ensure that the obtained model training data is reliable and effective. Through the above scheme, the present application realizes real-time scheduling and processing of large-scale driving data sets, and constructs large-scale and diverse model training data by performing scene analysis on large-scale driving data sets.

[0070] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a data processing device capable of realizing the above functions, etc. The following takes a data processing device as an example to illustrate this embodiment and the following embodiments.

[0071] Based on this, the present application embodiment provides a data processing method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the data processing method of the present application.

[0072] In this embodiment, the data processing method includes steps S110 to S140:

[0073] Step S110, in response to the data processing instruction, obtaining the driving data to be processed and the preset model data requirements, and creating a data processing task;

[0074] Specifically, the data processing device can first monitor whether the driving data in the cloud is updated. When the driving data is updated and the preliminary analysis is completed, the data processing instruction is automatically generated. Then, in response to the data processing instruction, the driving data to be processed is obtained, and the corresponding data processing task is automatically created in combination with the pre-set production requirements of the model training data. Among them, the driving data includes vehicle road test data and mass-produced vehicle data at the user end, which often include vehicle basic data (such as license plate number, engine number, vehicle model, body length and width, etc.), vehicle driving dynamic data (such as driving speed, driving time, driving trajectory, acceleration, braking force, steering angle, etc.), vehicle working condition data (such as the working status of each component of the vehicle, fault information, etc.), environmental perception data (vehicle external monitoring data, laser point cloud perception data, other vehicle data, etc.), map information, vehicle positioning information, user driving behavior data, user interaction data, in-vehicle video and voice monitoring data, vehicle safety event information, etc.; the preset model data requirements refer to the requirements pre-set by relevant personnel based on the relevant production requirements of the model training data, including but not limited to the data format, data type, definition of valid data, and the ratio of potential data to valid data of the model training data. Among them, effective data refers to data that has a training effect on model training; potential data refers to data that has a fine-tuning effect on model training.

[0075] The creation of data processing tasks can also be manually triggered by users, and corresponding data processing tasks are created by receiving and responding to the user's data processing instructions. The data processing instructions may include the user filtering the driving data to be processed according to the time or type of the data, as well as the pre-set production requirements of the model training data.

[0076] It should be understood that when there are multiple preset model data requirements, multiple corresponding data processing tasks can be created.

[0077] Step S120, acquiring related cloud data from a cloud database according to the data processing task;

[0078] Step S130, based on the preconfigured server cluster and according to the data processing task, performing scene analysis on the associated cloud data to obtain valid scene fragments;

[0079] Specifically, according to the created data processing task, the data processing device can download the associated cloud data to be processed from the cloud database. It should be noted that the associated cloud data refers to the associated data obtained and stored in the cloud database after the driving data to be processed is processed at different stages according to the preset model data requirements, or the driving data to be processed that has not been processed.

[0080] Then, in order to ensure the load balancing of the data processing equipment and improve the efficiency of data processing, the data processing equipment can pre-configure the environment variables of the server cluster according to the relevant information of the associated cloud data to meet the different requirements of the machines in different data processing links and efficiently execute data processing tasks.

[0081] Specifically, through the pre-configured server cluster, combined with the data processing flow or steps included in the data processing task, the data associated with the cloud data is first cleaned, and the scenes associated with the cloud data are automatically identified and scored. Then, data analysis is performed on the identified scenes to confirm the target categories and dynamic changes of the targets in the scenes, and the scenes are classified based on the scene analysis results. According to the preset model training requirements, data segments that are effective for model training are extracted, thereby obtaining effective scene segments.

[0082] In a feasible implementation manner, the step S130 may include steps A01 to A02:

[0083] Step A01, based on the preconfigured server cluster and multi-concurrency mechanism, according to the data processing task, performing scenario analysis on the associated cloud data to obtain a scenario analysis result;

[0084] Step A02: Based on the scene analysis results, scene data mining and scene label mining are performed, and effective scene segments are extracted according to the scene label classification.

[0085] In this implementation, because there may be multiple preset model data requirements and multiple corresponding associated cloud data, multi-threaded parallel data processing operations can be performed based on a multi-concurrency mechanism, effectively improving the efficiency of data processing.

[0086] In this embodiment, the scene refers to a series of data sets that can describe and simulate real or virtual driving environments, which not only include static geographic environment information, but also cover dynamic information such as traffic participants, weather conditions, and vehicle driving status.

[0087] Specifically, through the pre-configured server cluster, combined with the multi-concurrency mechanism, combined with the data processing flow or steps included in the data processing task, multiple related cloud data are cleaned, and the image features in the video data and map data in the related cloud data after data cleaning are extracted using machine learning algorithms such as deep learning, and the target objects (such as vehicles, pedestrians, traffic signs, lane lines, other obstacles, etc.) are detected and identified to determine the key elements and potential risks in the scene. Then, the clustering algorithm is used to cluster scenes with similar features, and the dynamic changes of the scenes in the video data and map data are continuously tracked and analyzed in combination with the time series analysis technology, so as to obtain the scene analysis results of several data segments in the video data and map data.

[0088] Then, through the scene analysis results of the data fragments, the scene analysis results are statistically analyzed using machine learning algorithms such as deep learning. For example, the scene analysis results of different data fragments contain statistical indicators such as the number of different types of targets, speed, direction, number of different types of traffic signs, and the distance between other vehicles and the vehicle on the lane line. At the same time, the association between different target objects in the scene analysis results of different data fragments is further confirmed, such as the interaction between vehicles and pedestrians, the impact of traffic signs on vehicle driving, the position relationship between other vehicles and the vehicle, etc., and abnormal scenes that do not conform to normal driving modes are identified, such as obstacles that suddenly appear, other vehicles that drive illegally, etc. In combination with the pre-defined scene label system, scene labels are annotated for different data fragments, and abnormal frame data that does not conform to the scene labels in the data fragments are eliminated to obtain mining data based on scene label classification.

[0089] Among them, the scene labeling system is constructed by relevant personnel based on the scenarios of training data of different intelligent driving related models. The scene labeling system can be designed in a hierarchical manner, for example, from the perspective of traffic scenes, traffic roads, scene complexity, and the dynamic behavior of targets in the scenes.

[0090] Finally, according to the preset model training requirements, the scene labels corresponding to the scene fragments that are effective for model training are determined, and the data fragments corresponding to the scene labels are extracted from the mining data classified based on the scene labels, so as to obtain the effective scene fragments.

[0091] It should be understood that because the preset model data requirements require the presence of potential data in the model training data, some data segments that do not match the scene labels can be appropriately selected as potential data to be added to the valid scene segments for subsequent fine-tuning of the model. At the same time, in order to facilitate the compatibility of scene data in the data processing process, the valid scene segment data in this embodiment can be stored in the protobuf data format.

[0092] Step S140: performing training preprocessing on the valid scene segments to obtain model training data.

[0093] Finally, the data processing equipment performs training preprocessing operations on the extracted valid scene fragments, such as data format conversion, virtual scene generation and deduction, data screening and division, etc., and finally obtains the model training data corresponding to the preset model training requirements, so as to provide effective data support for related models of intelligent driving.

[0094] It should be understood that after the server cluster processes large amounts of pending driving data according to the data processing task, in order to improve the utilization of the server cluster, the intermediate output data occupying the server cluster resources can be deleted after storing the model training data.

[0095] Specifically, the model training data, the driving data information corresponding to the model training data and other information are bucketed and stored in the cloud according to the generation time, the model to which the data belongs and other information, and the intermediate output data on the server cluster is deleted.

[0096] It should be understood that because the model training data is composed of valid scene segments corresponding to scene labels, the model training data uploaded to the cloud can support retrieval by scene labels.

[0097] The present embodiment provides a data processing method, which obtains driving data to be processed and preset model data requirements in response to data processing instructions, and creates a data processing task; obtains related cloud data from a cloud database according to the data processing task; based on a preconfigured server cluster, performs scene analysis on the related cloud data according to the data processing task to obtain valid scene fragments; and performs training preprocessing on the valid scene fragments to obtain model training data.

[0098] Through the above scheme, this embodiment creates a data processing task through the original driving data and the model data requirements related to intelligent driving, which is used to indicate how to build model training data; then, according to the data processing task, the associated cloud data is obtained from the cloud database, and then combined with the pre-configured server cluster, the associated cloud data is analyzed for scenes to extract scene fragments that are effective for model training; finally, the effective scene fragments are pre-processed for training to ensure that the obtained model training data is reliable and effective. Through the above scheme, this application realizes real-time scheduling and processing of large-scale driving data sets, and constructs large-scale and diverse model training data by performing scene analysis on large-scale driving data sets.

[0099] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 , before the above step S130, it also includes steps S210 to S220:

[0100] Step S210, determining a corresponding data parsing format based on the associated cloud data;

[0101] Step S220: configuring the data parsing format into the server cluster to obtain a preconfigured server cluster.

[0102] Understandably, since intelligent driving data is usually stored in the format of ros packages, and the data in ros packages can support users to customize and expand specific message types, it is strongly dependent on the format version of the data. Different ROS versions or ROS versions in different periods may define different versions of message types, making their data formats different, which brings difficulties to the subsequent data processing of intelligent driving data. The current common practice is to upgrade the data of the ROS package to the same version and then process the data uniformly, but this approach has high computational and time costs for data processing.

[0103] In this embodiment, in order to solve the problem of high computational and time costs of data processing in the above method, the data parsing format of the corresponding ROS version is determined based on the ROS version of the associated cloud data obtained.

[0104] Then, the corresponding data parsing format is downloaded from the cloud database and dynamically configured in the server cluster, so that the server cluster can subsequently directly use the configured data parsing format to quickly parse the associated cloud data, thereby effectively improving the data processing efficiency of large-scale associated cloud data.

[0105] The present embodiment provides a data processing method, which obtains driving data to be processed and preset model data requirements in response to a data processing instruction, and creates a data processing task; obtains related cloud data from a cloud database according to the data processing task; determines a corresponding data parsing format based on the related cloud data; configures the data parsing format to the server cluster to obtain a preconfigured server cluster; based on the preconfigured server cluster and according to the data processing task, performs scene analysis on the related cloud data to obtain valid scene fragments; performs training preprocessing on the valid scene fragments to obtain model training data.

[0106] Through the above scheme, this embodiment obtains the data parsing format corresponding to the ROS version of the associated cloud data, and pre-configures the data parsing format in the server cluster, so that the server cluster can subsequently quickly parse the associated cloud data, reduce the error rate of real-time scheduling processing of large-scale driving data sets, and further improve the data processing efficiency of large-scale driving data sets.

[0107] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned second embodiment can refer to the above introduction, and no further description will be given later. On this basis, the associated cloud data includes the first cloud data, please refer to Figure 3 , before the above step S120, it also includes steps S310 to S330:

[0108] Step S310, obtaining driving data to be processed according to the data processing task;

[0109] Step S320, based on the preconfigured server cluster and according to the data processing task, extracting and merging the required subject data in the to-be-processed driving data to obtain first cloud data;

[0110] Step S330: save the first cloud data to a cloud database.

[0111] Specifically, according to the created data processing task, the driving data to be processed, that is, the unprocessed original driving data, is downloaded from the cloud.

[0112] Since the original driving data usually includes multiple data (such as environmental perception data, own vehicle data, other vehicle data, map data, etc.), the data volume is often too large. If the original driving data is directly analyzed and processed, it will have a negative impact on the data processing efficiency.

[0113] Because for a certain intelligent driving model, the original driving data often contains some highly correlated subject data, so in this embodiment, according to the pre-configured server cluster, the subject data with high correlation for model training is extracted and merged from the driving data to be processed to obtain the first cloud data, so that the first cloud data is used as the associated cloud data for this data processing task. In actual data, the data volume of the first cloud data obtained by the above method usually only accounts for 5% of the original driving data, which can effectively improve the efficiency of subsequent data processing.

[0114] It can be understood that the obtained first cloud data is saved in the cloud database. If it is necessary to repeat similar data processing on the original driving data later, the subsequent data processing operation can be directly performed through the first cloud data in the cloud database.

[0115] It can be understood that after extracting data from the driving data to be processed to improve data processing efficiency, valid data in the first cloud data can be further extracted. Therefore, the associated cloud data includes the second cloud data. After step S320, steps B01 to B04 are also included:

[0116] Step B01, based on a preconfigured server cluster, performing effective data analysis on the first cloud data to obtain effective data of several topics;

[0117] Specifically, through a pre-configured server cluster, valid data in the first cloud data is parsed to obtain valid data of several topics, such as other vehicle perception data and laser point cloud data in environmental perception data, self-vehicle driving status data and vehicle positioning data in self-vehicle data, etc.

[0118] Step B02, time-aligning the valid data of the plurality of subjects, merging the valid data of the plurality of subjects according to data frames to obtain merged data;

[0119] Step B03, slicing the merged data to obtain second cloud data;

[0120] Then, in order to facilitate subsequent scene analysis based on time series analysis technology, the data processing device can align the timestamps of the valid data of several subjects in the time series, and then merge the valid data of several subjects according to the data frame, that is, put the valid data of all subjects with the same timestamp into the same frame data, thereby obtaining merged data.

[0121] Then, since the length of the merged data is still relatively long (for example, more than 300 seconds), it puts great pressure on subsequent data processing. Therefore, the data processing device in this embodiment can also slice the merged data to obtain data segments of appropriate length (for example, 100 seconds).

[0122] Step B04: Save the second cloud data to a cloud database.

[0123] It can be understood that the obtained second cloud data is saved in the cloud database. If it is necessary to repeat similar data processing on the first cloud data later, the subsequent data processing operation can be directly performed through the second cloud data in the cloud database.

[0124] The present embodiment provides a data processing method, which obtains driving data to be processed and preset model data requirements in response to a data processing instruction, and creates a data processing task; obtains the driving data to be processed according to the data processing task; based on a preconfigured server cluster, extracts and merges the required subject data in the driving data to be processed according to the data processing task to obtain first cloud data; saves the first cloud data to a cloud database; obtains the first cloud data from the cloud database according to the data processing task; determines a corresponding data parsing format based on the associated cloud data; configures the data parsing format to the server cluster to obtain a preconfigured server cluster; based on the preconfigured server cluster, according to the data processing task, performs scene analysis on the associated cloud data to obtain valid scene fragments; performs training preprocessing on the valid scene fragments to obtain model training data.

[0125] Through the above scheme, this embodiment extracts related subject data from the driving data to be processed in advance, thereby reducing the data volume of the data set to be processed as much as possible. At the same time, it further performs effective data extraction, data merging according to timestamps, data segmentation and other operations to improve data processing efficiency, realize real-time scheduling and processing of large-scale driving data sets, and construct large-scale and diverse model training data in real time, providing reliable and effective data support for intelligent driving model training.

[0126] Based on the first embodiment and / or the second embodiment and / or the third embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first embodiment, the second embodiment and / or the third embodiment can refer to the above introduction and will not be repeated later. On this basis, the preset model data requirements include the model training data format, please refer to Figure 4 The step S140 includes steps S410 to S430:

[0127] Step S410, confirming whether the valid scene segment meets a preset data checking condition;

[0128] It is understandable that in order to ensure that the data sent to the model training is reliable and valid, it is necessary to confirm whether the valid scene segments meet the preset data inspection conditions. Among them, the preset data inspection conditions are the data reliability requirements pre-set by relevant personnel based on the model training data requirements, including data validity, integrity, data quality and other requirements.

[0129] Specifically, through operations such as abnormal data check (checking duplicate data, file format and size, etc.), data integrity check (checking file corruption or missing), data validity check (for example, data type, value range, logical consistency, etc.), and data fragment timestamp consistency check, it is confirmed whether the valid scene fragment meets the preset data check conditions.

[0130] Step S420, extracting relevant scene data from the valid scene fragment after the preset data checking condition is met;

[0131] Step S430: storing the scene data in the model training data format.

[0132] Then, since the training input data of different intelligent driving models are different, after the valid scene fragments meet the preset data inspection conditions, only the scene data related to the model input is extracted from the valid scene fragments. For example, the training input of the intelligent driving path planning model requires information related to the vehicle itself, other vehicles, map information, etc.

[0133] Finally, in order to ensure that the relevant models of intelligent driving are compatible with data from all sources, it is necessary to store the relevant scene data in the model training data format according to the preset model data requirements. In this embodiment, the model training data format can be a pickle format.

[0134] It should be understood that when the valid scene segment does not meet the preset data checking condition, it is necessary to re-execute step S130 to regenerate the valid scene segment.

[0135] In a feasible implementation manner, the scene data includes map data, and the step S420 may include step C01:

[0136] Step C01, after the preset data checking condition is met, the map data in the valid scene segment is subjected to thinning processing;

[0137] The step 430 may include step C02:

[0138] Step C02, storing the map data after the thinning process at the frame level in the model training data format.

[0139] In this embodiment, since the map data is usually high-precision map data with a large amount of data, and the map data information of vehicle driving in a short period of time does not differ much, the map data in the effective scene segment can be thinned out so that adjacent frames of data in the effective scene segment share one map data.

[0140] At the same time, the thinned map data is stored separately at the frame level, that is, one frame of map data is saved separately, and an index relationship is established between the single frame of map data and the timestamps of other scene data, so that the intelligent driving related models can quickly learn the scene data.

[0141] It can be understood that in order to improve data compatibility, the map data after thinning processing is also stored in the model training data format according to the preset model data requirements, preferably in pickle format.

[0142] The present embodiment provides a data processing method, which obtains driving data to be processed and preset model data requirements in response to a data processing instruction, and creates a data processing task; obtains related cloud data from a cloud database according to the data processing task; based on a preconfigured server cluster, performs scene analysis on the related cloud data according to the data processing task to obtain valid scene fragments; confirms whether the valid scene fragments meet preset data check conditions; after the preset data check conditions are met, extracts relevant scene data from the valid scene fragments; and stores the scene data in accordance with the model training data format.

[0143] This embodiment uses the above scheme to perform data checks on the extracted valid scene data fragments. When the valid scene data fragments meet the preset data check conditions, the scene data related to the model input is extracted according to the preset model training requirements, and the data storage format is unified to ensure that the data sent to the model training is reliable, effective and compatible, reduce the error rate of real-time scheduling processing of large-scale driving data sets, and further improve the data processing efficiency of large-scale driving data sets.

[0144] Based on the first embodiment and / or the second embodiment and / or the third embodiment and / or the fourth embodiment of the present application, in the fifth embodiment of the present application, the same or similar contents as those in the first embodiment, the second embodiment, the third embodiment and / or the fourth embodiment can refer to the above description and will not be described in detail later. Figure 5 , after the above step S140, including steps S510 to S530:

[0145] Step S510, performing data generation and abnormal statistics on the driving data to be processed to obtain data statistical results;

[0146] Step S520, analyzing the relevant logs of the data processing task and the data statistical results to generate a data analysis report;

[0147] It can be understood that in order to understand the quality of the generated model training data and the utilization rate of the original driving data (i.e., the driving data to be processed), after processing large-scale driving data, data analysis can be performed using the generated model training data and the corresponding original driving data.

[0148] Specifically, in the process of processing the full amount of large-scale driving data, there are many abnormal data that are filtered out. Therefore, after the data processing is completed, the data processing device can perform statistical analysis on the ROS package data corresponding to the original driving data by number, such as counting the number and duration of model training data segments produced by a ROS package, the number and duration of abnormal data segments, the number of abnormal data frames in different categories of scene data, and other statistical information of different dimensions to obtain data statistical results. Then upload and store the data statistical results to the cloud for subsequent data analysis.

[0149] Then, the data processing equipment combines the relevant execution logs and data statistical results of the data processing tasks for further analysis to understand the occurrence rate of data anomalies and the efficiency of data processing during the data processing process, and then generates a data analysis report to facilitate subsequent optimization of the data processing process.

[0150] It should be understood that when a data anomaly occurs during data processing, the data processing device can notify relevant developers of the data anomaly information through messages, such as notifying specific information of the data anomaly through enterprise system messages or associated enterprise WeChat accounts.

[0151] Step S530: Perform data visualization based on the model training data and the data statistical results.

[0152] Specifically, in order to enable relevant personnel to more intuitively understand the production status of model training data, the corresponding time periods of the original driving data can be sliced ​​and visualized through the time information of the model training data, and the visual image information of the model training data and the corresponding original driving data can be compared to intuitively understand the difference between the model training data and the corresponding original driving data. At the same time, the two-dimensional view of the data can be visualized according to the statistical information of different dimensions in the data statistical results to further understand the quality effect of data production.

[0153] The present embodiment provides a data processing method, which obtains driving data to be processed and preset model data requirements in response to a data processing instruction, and creates a data processing task; obtains related cloud data from a cloud database according to the data processing task; based on a preconfigured server cluster, according to the data processing task, performs scene analysis on the related cloud data to obtain valid scene fragments; performs training preprocessing on the valid scene fragments to obtain model training data; performs data production and abnormality statistics on the driving data to be processed to obtain data statistical results; analyzes relevant logs of the data processing task and the data statistical results to generate a data analysis report; and performs data visualization based on the model training data and the data statistical results.

[0154] Through the above scheme, this embodiment supports data anomaly inspection, data analysis and data visualization on the model training data obtained after data processing, further ensures the reliability of the model training data, realizes real-time scheduling and processing of large-scale driving data sets, and constructs large-scale and diverse model training data in real time, providing reliable and effective data support for intelligent driving model training.

[0155] For example, to help understand the implementation process of the data processing method obtained by combining this embodiment with the above-mentioned embodiment 5, please refer to Figure 6 , Figure 6 An example flow chart of a data processing method is provided, specifically:

[0156] First, a data processing task is created based on the drive test data and user data. The triggering of the data processing task can be automatically triggered by monitoring whether the data is updated, or it can be manually triggered.

[0157] Then the created data processing tasks are scheduled for execution. When there are multiple data processing tasks, multiple scheduling data information can be processed concurrently using the multi-concurrency mechanism.

[0158] The processing flow for a single data processing task is as follows:

[0159] First, according to the scheduling data information, the original data is downloaded; then the data parsing format of the original data is obtained, and the data parsing format is configured in the server cluster; then the relevant topic data is extracted from the original data and the data is merged to obtain the first cloud data, and the first cloud data is uploaded to the cloud database; then the first cloud data is parsed and the data format is converted to obtain the second cloud data, and the second cloud data is uploaded to the cloud database; then the second cloud data is analyzed for the scene to obtain valid data, that is, valid scene fragments, and stored in protobuf format; then the valid scene fragments are trained and preprocessed and stored in pickle format; after the training preprocessing, the local intermediate data results are cleared, and the valid data information is uploaded to the cloud database, so that the cloud database supports tags to retrieve valid data; at the same time, the relevant logs of the data processing process can be analyzed, and the data after the scene analysis can be statistically analyzed to obtain a statistical analysis report; at the same time, when data abnormalities occur during the data processing process, it is preferred to notify the abnormal message in the form of WeChat notification; finally, data visualization operations can be performed based on the valid data retrieved from the cloud database.

[0160] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data processing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0161] This application also provides a data processing device, please refer to Figure 7 , the data processing device comprises:

[0162] The task creation module 10 is used to obtain the driving data to be processed and the preset model data requirements in response to the data processing instruction, and create a data processing task;

[0163] A data acquisition module 20, used to acquire relevant cloud data from a cloud database according to the data processing task;

[0164] A scene analysis module 30, configured to perform scene analysis on the associated cloud data based on a preconfigured server cluster and according to the data processing task to obtain valid scene segments;

[0165] The training preprocessing module 40 is used to perform training preprocessing on the valid scene segments to obtain model training data.

[0166] The data processing device provided by the present application adopts the data processing method in the above embodiment, which can solve the technical problem of how to build large-scale and diversified model training data in real time. Compared with the prior art, the beneficial effects of the data processing device provided by the present application are the same as the beneficial effects of the data processing method provided by the above embodiment, and other technical features in the data processing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0167] The present application provides a data processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data processing method in the above-mentioned embodiment.

[0168] Reference below Figure 8, which shows a schematic diagram of the structure of a data processing device suitable for implementing an embodiment of the present application. The data processing device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The data processing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0169] like Figure 8 As shown, the data processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a data processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0170] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0171] The data processing device provided by the present application adopts the data processing method in the above embodiment, which can solve the technical problem of how to build large-scale and diversified model training data in real time. Compared with the prior art, the beneficial effects of the data processing device provided by the present application are the same as the beneficial effects of the data processing method provided by the above embodiment, and the other technical features in the data processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0172] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0174] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data processing method in the above-mentioned embodiment.

[0175] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0176] The computer-readable storage medium may be included in a data processing device, or may exist independently without being incorporated into a data processing device.

[0177] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the data processing device, the data processing device: responds to the data processing instructions, obtains the driving data to be processed and the preset model data requirements, and creates a data processing task; obtains related cloud data from the cloud database according to the data processing task; based on the preconfigured server cluster, according to the data processing task, performs scene analysis on the related cloud data to obtain valid scene fragments; performs training preprocessing on the valid scene fragments to obtain model training data.

[0178] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0179] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0181] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data processing method, and can solve the technical problem of how to build large-scale and diversified model training data in real time. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the data processing method provided in the above-mentioned embodiment, and will not be repeated here.

[0182] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0183] The computer program product provided in this application can solve the technical problem of how to build large-scale and diverse model training data in real time. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the data processing method provided in the above embodiment, which will not be repeated here.

[0184] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: In response to the data processing instruction, obtaining the driving data to be processed and the preset model data requirements, and creating a data processing task; According to the data processing task, obtaining related cloud data from a cloud database; Based on the preconfigured server cluster, according to the data processing task, the associated cloud data is subjected to scene analysis to obtain valid scene fragments; The effective scene segments are subjected to training preprocessing to obtain model training data.

2. The method according to claim 1, characterized in that Before the step of performing scene analysis on the associated cloud data based on the preconfigured server cluster and according to the data processing task to obtain valid scene fragments, the step further includes: Determining a corresponding data parsing format based on the associated cloud data; The data parsing format is configured into the server cluster to obtain a preconfigured server cluster.

3. The method according to claim 2, characterized in that The associated cloud data includes first cloud data. Before the step of acquiring the associated cloud data from the cloud database according to the data processing task, the step further includes: According to the data processing task, obtaining driving data to be processed; Based on the preconfigured server cluster, according to the data processing task, extract and merge the required subject data in the to-be-processed driving data to obtain first cloud data; The first cloud data is saved in a cloud database.

4. The method according to claim 3, characterized in that The associated cloud data includes second cloud data, and the step of extracting and merging the required subject data in the to-be-processed driving data according to the data processing task based on the preconfigured server cluster to obtain the first cloud data further includes: Based on the preconfigured server cluster, the first cloud data is analyzed for valid data to obtain valid data of several topics; Time-aligning the valid data of the plurality of subjects, merging the valid data of the plurality of subjects according to data frames to obtain merged data; Slicing the merged data to obtain second cloud data; The second cloud data is saved in a cloud database.

5. The method according to claim 1, characterized in that The step of performing scene analysis on the associated cloud data based on the preconfigured server cluster and according to the data processing task to obtain valid scene fragments includes: Based on a preconfigured server cluster and a multi-concurrency mechanism, a scenario analysis is performed on the associated cloud data to obtain a scenario analysis result; Based on the scene analysis results, scene data mining and scene label mining are performed, and effective scene segments are extracted according to the scene label classification.

6. The method according to claim 1, characterized in that The preset model data requirement includes a model training data format, and the step of performing training preprocessing on the valid scene fragments to obtain model training data includes: Confirming whether the valid scene segment meets a preset data checking condition; After a preset data checking condition is met, extracting relevant scene data from the valid scene fragment; The scene data is stored in the model training data format.

7. The method according to claim 6, characterized in that The scene data includes map data, and the step of extracting relevant scene data from the valid scene fragment after satisfying a preset data checking condition includes: After the preset data checking conditions are met, the map data in the valid scene segment is subjected to thinning processing; The step of storing the scene data in the model training data format includes: The map data after the thinning process is stored at the frame level in the model training data format.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Performing data production and abnormal statistics on the driving data to be processed to obtain data statistical results; Analyze the relevant logs of the data processing task and the data statistical results to generate a data analysis report; Based on the model training data and the data statistical results, data visualization is performed.

9. A data processing device, characterized in that: The data processing device comprises: A task creation module, used to respond to data processing instructions, obtain driving data to be processed and preset model data requirements, and create a data processing task; A data acquisition module, used to acquire relevant cloud data from a cloud database according to the data processing task; A scene analysis module, configured to perform scene analysis on the associated cloud data based on a preconfigured server cluster and according to the data processing task to obtain valid scene fragments; The training preprocessing module is used to perform training preprocessing on the effective scene fragments to obtain model training data.

10. A data processing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data processing method according to any one of claims 1 to 8.

11. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 8 are implemented.

Images