Data closed-loop system, data closed-loop method, controller and readable storage medium

Abstract

The invention relates to the technical field of intelligent driving, in particular to a data closed-loop system, a data closed-loop method, a controller and a readable storage medium, and aims to solve the technical problem of how to carry out effective data closed-loop to realize high-efficiency model iteration. In order to achieve the purpose, the data closed-loop system comprises a data mining module, a data processing module and a model iteration training module. And the data mining module is used for performing data mining based on the mining model according to a preset data mining requirement to obtain mining data. And the data processing model is configured to perform data processing according to the mining data to obtain first training data. The model iteration training module is configured to perform model iteration training on the to-be-iterated model according to the first training data to obtain a model iteration training result, so that the model performance of the to-be-iterated model can be effectively improved, the iteration efficiency of the model is improved, better expansibility is achieved, and the iteration requirements of multiple to-be-iterated models with different functions can be met.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly to a data closed-loop system, a data closed-loop method, a controller, and a readable storage medium. Background Art

[0002] Intelligent driving functions involve multiple vision-based classification models. The training and deployment of these models rely on high-quality data collection, annotation, and corresponding model training. To cope with complex and changing environmental information and interaction scenarios, the models need to be updated and iterated periodically in a targeted manner. However, in the previous process of model update, iteration, and improvement, it is impossible to avoid long cross-departmental cooperation, multiple link platforms, and manual processes, resulting in slow model iteration.

[0003] Correspondingly, there is a need in the art for a new model iteration solution to solve the above problems. Summary of the Invention

[0004] In order to overcome the above defects, this application is proposed to solve or at least partially solve the technical problem of how to perform effective data closed-loop to achieve high-efficiency model iteration.

[0005] In a first aspect, a data closed-loop system is provided. The system includes:

[0006] A data mining module configured to perform data mining based on a mining model according to a preset data mining requirement to obtain mined data; wherein the data mining method includes at least one of cloud data mining and vehicle-side data mining; the data mining requirement is set according to the model to be iterated;

[0007] A data processing module configured to process the mined data to obtain first training data;

[0008] A model iteration training module configured to perform model iteration training on the model to be iterated according to the first training data to obtain a model iteration training result.

[0009] In a technical solution of the above data closed-loop system, the data mining module includes a mining model iteration unit and a data mining unit;

[0010] The mining model iteration unit is configured to obtain second training data of the mining model corresponding to the target category information from a cloud database according to the target category information corresponding to the data mining requirement; and perform iteration training on the mining model based on the second training data to obtain a trained mining model;

[0011] The data mining unit is configured to perform data mining according to the trained mining model to obtain the mined data.

[0012] In a technical solution of the above data closed-loop system, the mining model iteration unit is further configured to:

[0013] According to the data mining requirement, perform vectorized data search based on the cloud database to obtain cold start data of the mining model;

[0014] According to the cold start data, perform initial training on the mining model to obtain an initial mining model;

[0015] According to the target category information corresponding to the data mining requirement, obtain the second training data from the cloud database;

[0016] According to the second training data, perform iterative training on the initial mining model to obtain the trained mining model.

[0017] In a technical solution of the above data closed-loop system, the data mining unit is further configured to:

[0018] Determine the data mining method according to the target category information and data volume information corresponding to the data mining requirement;

[0019] If the data mining method is cloud data mining, perform data mining on the cloud historical return data in the cloud data database according to the trained mining model to obtain the mining data;

[0020] If the data mining method is vehicle-side data mining, send the trained mining model to the vehicle side to perform data mining on the vehicle-side data to obtain the mining data.

[0021] In a technical solution of the above data closed-loop system, the data processing module includes a training data production unit and a data cleaning unit;

[0022] The training data production unit is configured to perform training data format conversion according to the mining data to obtain format conversion data;

[0023] The data cleaning unit is configured to perform data cleaning on the format conversion data to obtain the first training data.

[0024] In a technical solution of the above data closed-loop system, the training data production unit is further configured to:

[0025] Perform data screening on the mining data to obtain screened data;

[0026] Perform target tracking detection on the screened data to obtain target tracking detection data;

[0027] Perform format conversion on the target tracking detection data to obtain the format-converted data.

[0028] In a technical solution of the above data closed-loop system, the model iterative training module includes a model training unit, a model forward inference tracking unit, a model quantization unit, and a model packaging unit;

[0029] The model training unit is configured to perform model iterative training on the model to be iterated according to the first training data to obtain a model with completed iterative training;

[0030] The model forward inference tracking unit is configured to track the forward inference process of the model according to the model with completed iterative training in accordance with the inference execution process of the model to obtain the forward inference workflow of the model;

[0031] The model quantization unit is configured to perform model quantization on the forward inference workflow according to the model usage environment and inference speed requirements to obtain a quantized model;

[0032] The model packaging unit is configured to perform packaging according to the quantized model to obtain the model iterative training result.

[0033] In a technical solution of the above data closed-loop system, the model training unit includes a first evaluation subunit; the model forward inference tracking unit includes a second evaluation subunit; the model quantization unit includes a third evaluation subunit;

[0034] The first evaluation subunit is configured to evaluate the model with completed iterative training according to the evaluation dataset to obtain a first evaluation result;

[0035] The second evaluation subunit is configured to evaluate the forward inference workflow according to the evaluation dataset to obtain a second evaluation result;

[0036] The third evaluation unit is configured to evaluate the quantized model according to the evaluation dataset to obtain a third evaluation result.

[0037] In a technical solution of the above data closed-loop system, the model packaging unit includes a quality evaluation subunit;

[0038] The quality evaluation subunit is configured to evaluate the model iterative training result according to the quality evaluation dataset to obtain an evaluation result of the output model.

[0039] In a technical solution of the above data closed-loop system, the mined data is autonomous driving data;

[0040] Among them, the autonomous driving data includes at least one of environmental perception data, vehicle state data, and planning control data of an autonomous driving vehicle.

[0041] In a second aspect, a data closed-loop method is provided, and the method includes:

[0042] According to a preset data mining requirement, based on a mining model, perform data mining to obtain mined data; wherein, the data mining method includes at least one of cloud data mining and vehicle-side data mining; the data mining requirement is set according to a model to be iterated;

[0043] Perform data processing according to the mined data to obtain first training data;

[0044] Perform model iterative training on the model to be iterated according to the first training data to obtain a model iterative training result.

[0045] In a technical solution of the above data closed-loop method, the step of performing data mining according to a preset data mining requirement, based on a mining model, to obtain mined data includes:

[0046] According to the target category information corresponding to the data mining requirement, obtain second training data of the mining model corresponding to the target category information from a cloud database; and based on the second training data, perform iterative training on the mining model to obtain a trained mining model;

[0047] Perform data mining according to the trained mining model to obtain the mined data.

[0048] In a technical solution of the above data closed-loop method, the step of obtaining second training data of the mining model corresponding to the target category information from a cloud database according to the data mining requirement includes:

[0049] Based on the cloud database, perform vectorized data search according to the data mining requirement to obtain cold start data of the mining model;

[0050] Perform initial training on the mining model according to the cold start data to obtain an initial mining model;

[0051] Obtain the second training data from the cloud database according to the target category information corresponding to the data mining requirement.

[0052] In a technical solution of the above data closed-loop method, the step of performing data mining according to the trained mining model to obtain the mined data includes:

[0053] Determine the data mining method according to the target category information and data volume information corresponding to the data mining requirements;

[0054] If the data mining method is cloud data mining, perform data mining on the cloud historical return data in the cloud data database according to the trained mining model to obtain the mined data;

[0055] If the data mining method is vehicle - end data mining, send the trained mining model to the vehicle - end to perform data mining on the vehicle - end data to obtain the mined data.

[0056] In a technical solution of the above - mentioned data closed - loop method, the obtaining of the first training data by performing data processing on the mined data includes:

[0057] Perform training data format conversion on the mined data to obtain format - converted data;

[0058] Perform data cleaning on the format - converted data to obtain the first training data.

[0059] In a technical solution of the above - mentioned data closed - loop method, the performing of training data format conversion on the mined data to obtain format - converted data includes:

[0060] Perform data screening on the mined data to obtain screened data;

[0061] Perform target tracking detection on the screened data to obtain target tracking detection data;

[0062] Perform format conversion on the target tracking detection data to obtain the format - converted data.

[0063] In a technical solution of the above - mentioned data closed - loop method, the obtaining of the model iteration training result by performing model iteration training on the model to be iterated according to the first training data includes:

[0064] Perform model iteration training on the model to be iterated according to the first training data to obtain a model with completed iteration training;

[0065] According to the model with completed iteration training, track the forward inference process of the model according to the inference execution process of the model to obtain the forward inference workflow of the model;

[0066] Perform model quantization on the forward inference workflow according to the model usage environment and inference speed requirements to obtain a quantized model;

[0067] Package according to the quantized model to obtain the model iteration training result.

[0068] In one technical solution of the above data closed-loop method, the method further includes:

[0069] Evaluating the iteratively trained model according to the evaluation data set to obtain a first evaluation result;

[0070] Evaluating the forward inference workflow according to the evaluation data set to obtain a second evaluation result;

[0071] Evaluating the quantized model according to the evaluation data set to obtain a third evaluation result.

[0072] In one technical solution of the above data closed-loop method, the method further includes:

[0073] Evaluating the model iterative training result according to the quality evaluation data set to obtain a quasi-output model evaluation result.

[0074] In one technical solution of the above data closed-loop method, the mined data is autonomous driving data;

[0075] Wherein, the autonomous driving data includes at least one of environmental perception data, vehicle state data, and planning control data of an autonomous driving vehicle.

[0076] In a third aspect, a controller is provided, which includes at least one processor; and a memory communicatively connected to the at least one processor; wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above data closed-loop method is implemented.

[0077] In a fourth aspect, a computer-readable storage medium is provided, which stores multiple program codes, and the program codes are adapted to be loaded and run by a processor to execute the method described in any one of the technical solutions of the above data closed-loop method.

[0078] Solution 1. A data closed-loop system, characterized in that the system includes:

[0079] A data mining module, configured to perform data mining based on a mining model according to a preset data mining requirement to obtain mined data; wherein, the data mining method includes at least one of cloud data mining and vehicle-side data mining; the data mining requirement is set according to the model to be iterated;

[0080] A data processing module, configured to process the mined data to obtain first training data;

[0081] The model iterative training module is configured to perform model iterative training on the model to be iterated according to the first training data, and obtain the model iterative training result.

[0082] Solution 2. The data closed-loop system according to Solution 1, wherein the data mining module includes a mining model iterative unit and a data mining unit;

[0083] The mining model iterative unit is configured to obtain the second training data of the mining model corresponding to the target category information according to the data mining requirement from the cloud database; and perform iterative training on the mining model based on the second training data to obtain the trained mining model;

[0084] The data mining unit is configured to perform data mining according to the trained mining model to obtain the mining data.

[0085] Solution 3. The data closed-loop system according to Solution 2, wherein the mining model iterative unit is further configured to:

[0086] Based on the data mining requirement, perform vectorized data search on the cloud database to obtain the cold start data of the mining model;

[0087] Perform initial training on the mining model according to the cold start data to obtain an initial mining model;

[0088] Obtain the second training data from the cloud database according to the target category information corresponding to the data mining requirement;

[0089] Perform iterative training on the initial mining model according to the second training data to obtain the trained mining model.

[0090] Solution 4. The data closed-loop system according to Solution 2, wherein the data mining unit is further configured to:

[0091] Determine the data mining method according to the target category information and data volume information corresponding to the data mining requirement;

[0092] If the data mining method is cloud data mining, perform data mining on the cloud historical return data in the cloud data database according to the trained mining model to obtain the mining data;

[0093] If the data mining method is vehicle-side data mining, send the trained mining model to the vehicle side to perform data mining on the vehicle-side data to obtain the mining data.

[0094] Solution 5. The data closed-loop system according to Solution 1, characterized in that the data processing module includes a training data production unit and a data cleaning unit;

[0095] The training data production unit is configured to perform training data format conversion according to the mined data to obtain format conversion data;

[0096] The data cleaning unit is configured to perform data cleaning on the format conversion data to obtain the first training data.

[0097] Solution 6. The data closed-loop system according to Solution 5, characterized in that the training data production unit is further configured to:

[0098] Perform data screening on the mined data to obtain screened data;

[0099] Perform target tracking detection on the screened data to obtain target tracking detection data;

[0100] Perform format conversion on the target tracking detection data to obtain the format conversion data.

[0101] Solution 7. The data closed-loop system according to Solution 1, characterized in that the model iterative training module includes a model training unit, a model forward inference tracking unit, a model quantization unit, and a model packaging unit;

[0102] The model training unit is configured to perform model iterative training on the model to be iterated according to the first training data to obtain a model with iterative training completed;

[0103] The model forward inference tracking unit is configured to track the forward inference process of the model according to the model with iterative training completed in accordance with the inference execution process of the model to obtain the forward inference workflow of the model;

[0104] The model quantization unit is configured to perform model quantization on the forward inference workflow according to the model usage environment and inference speed requirements to obtain a quantized model;

[0105] The model packaging unit is configured to perform packaging according to the quantized model to obtain the model iterative training result.

[0106] Solution 8. The data closed-loop system according to Solution 7, characterized in that

[0107] The model training unit includes a first evaluation subunit; the model forward inference tracking unit includes a second evaluation subunit; the model quantization unit includes a third evaluation subunit;

[0108] The first evaluation subunit is configured to evaluate the iteratively trained model according to an evaluation dataset to obtain a first evaluation result;

[0109] The second evaluation subunit is configured to evaluate the forward inference workflow according to the evaluation dataset to obtain a second evaluation result;

[0110] The third evaluation unit is configured to evaluate the quantized model according to the evaluation dataset to obtain a third evaluation result.

[0111] Solution 9. The data closed-loop system according to Solution 7, characterized in that

[0112] The model packaging unit includes a quality evaluation subunit;

[0113] The quality evaluation subunit is configured to evaluate the model iterative training result according to a quality evaluation dataset to obtain a quasi-output model evaluation result.

[0114] Solution 10. The data closed-loop system according to any one of Solutions 1 to 9, characterized in that the mined data is autonomous driving data;

[0115] Wherein, the autonomous driving data includes at least one of environmental perception data, vehicle state data, and planning control data of an autonomous driving vehicle.

[0116] Solution 11. A data closed-loop method, characterized in that the method includes:

[0117] Performing data mining based on a mining model according to a preset data mining requirement to obtain mined data; wherein, the data mining method includes at least one of cloud data mining and vehicle-side data mining; the data mining requirement is set according to the model to be iterated;

[0118] Performing data processing on the mined data to obtain first training data;

[0119] Performing model iterative training on the model to be iterated according to the first training data to obtain a model iterative training result.

[0120] Solution 12. The model iteration method according to Solution 11, characterized in that

[0121] The performing data mining based on a mining model according to a preset data mining requirement to obtain mined data includes:

[0122] Obtain the second training data of the mining model corresponding to the target category information according to the target category information corresponding to the data mining requirement from the cloud database; and based on the second training data, perform iterative training on the mining model to obtain a trained mining model.

[0123] Perform data mining according to the trained mining model to obtain the mining data.

[0124] Solution 13. The data closed-loop method according to Solution 12, characterized in that

[0125] The obtaining the second training data of the mining model corresponding to the target category information according to the target category information corresponding to the data mining requirement includes:

[0126] According to the data mining requirement, perform vectorized data search based on the cloud database to obtain the cold start data of the mining model.

[0127] Perform initial training on the mining model according to the cold start data to obtain an initial mining model.

[0128] Obtain the second training data from the cloud database according to the target category information corresponding to the data mining requirement.

[0129] Solution 14. The data closed-loop method according to Solution 12, characterized in that

[0130] The performing data mining according to the trained mining model to obtain the mining data includes:

[0131] Determine the data mining method according to the target category information and data volume information corresponding to the data mining requirement.

[0132] If the data mining method is cloud data mining, perform data mining on the cloud historical return data in the cloud data database according to the trained mining model to obtain the mining data.

[0133] If the data mining method is vehicle-side data mining, send the trained mining model to the vehicle side to perform data mining on the vehicle-side data to obtain the mining data.

[0134] Solution 15. The data closed-loop method according to Solution 11, characterized in that

[0135] The obtaining first training data by performing data processing on the mining data includes:

[0136] Perform training data format conversion according to the mining data to obtain format conversion data.

[0137] Perform data cleaning on the format conversion data to obtain the first training data.

[0138] Solution 16. The data closed-loop method according to Solution 15, characterized in that

[0139] Performing training data format conversion according to the mined data to obtain format conversion data, including:

[0140] Perform data screening on the mined data to obtain the screened data;

[0141] Perform target tracking detection on the screened data to obtain target tracking detection data;

[0142] Perform format conversion on the target tracking detection data to obtain the format conversion data.

[0143] Solution 17. The data closed-loop method according to Solution 11, characterized in that

[0144] Performing model iterative training on the model to be iterated according to the first training data to obtain a model iterative training result, including:

[0145] Perform model iterative training on the model to be iterated according to the first training data to obtain a model with iterative training completed;

[0146] According to the model with iterative training completed, track the forward inference process of the model according to the inference execution process of the model to obtain the forward inference workflow of the model;

[0147] Quantify the forward inference workflow according to the model usage environment and inference speed requirements to obtain a quantified model;

[0148] Package according to the quantified model to obtain the model iterative training result.

[0149] Solution 18. The data closed-loop method according to Solution 17, characterized in that the method further includes:

[0150] Evaluate the model with iterative training completed according to the evaluation dataset to obtain a first evaluation result;

[0151] Evaluate the forward inference workflow according to the evaluation dataset to obtain a second evaluation result;

[0152] Evaluate the quantified model according to the evaluation dataset to obtain a third evaluation result.

[0153] Solution 19. The data closed-loop method according to Solution 17, wherein the method further comprises:

[0154] Evaluating the iterative training result of the model according to the quality evaluation data set to obtain the evaluation result of the output model.

[0155] Solution 20. The data closed-loop method according to any one of Solutions 11 to 19, wherein the mined data is autonomous driving data;

[0156] wherein the autonomous driving data includes at least one of environmental perception data, vehicle state data, and planning control data of an autonomous driving vehicle.

[0157] Solution 21. A controller, comprising:

[0158] At least one processor;

[0159] And a memory communicatively connected to the at least one processor;

[0160] wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the data closed-loop method according to any one of Solutions 11 to 20 is implemented.

[0161] Solution 22. A computer-readable storage medium storing multiple program codes, wherein the program codes are adapted to be loaded and run by a processor to execute the data closed-loop method according to any one of Solutions 11 to 20.

[0162] One or more of the above technical solutions of the present application have at least one or more of the following beneficial effects:

[0163] In implementing the technical solution of the data closed-loop system provided by the present application, the data closed-loop system of the present application includes a data mining module, a data processing module, and a model iterative training module. The data mining module performs data mining based on a mining model according to preset data mining requirements to obtain mined data. The data processing module processes the mined data to obtain first training data; wherein, the data mining method includes at least one of cloud data mining and vehicle-side data mining. The model iterative training module is configured to perform model iterative training on the model to be iterated according to the first training data to obtain a model iterative training result. Through the above setting method, the data closed-loop system of the present application can meet the closed-loop requirements of data mining and model iterative training for the model to be iterated, and can effectively improve the model performance of the model to be iterated. At the same time, since there is no need to perform separate data mining and model iteration for each model to be iterated, the iteration efficiency of the model is also effectively improved. When there is a need for an iterative model, the data closed-loop system of the present application can be directly applied to implement the data mining of the first training data, and the model iterative training of the model to be iterated can be realized according to the mined first training data, which can effectively save labor input and computing resources and accelerate the model iterative training process. In addition, the data closed-loop system of the present application can be applied to multiple models to be iterated with similar functions, has better scalability, and can meet the iteration requirements of multiple models to be iterated with different functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0164] With reference to the accompanying drawings, the disclosure of the present application will become more understandable. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the protection scope of the present application. Among them:

[0165] Figure 1 is a schematic diagram of the main component structure of a data closed-loop system according to an embodiment of the present application;

[0166] Figure 2 is a schematic diagram of the main step flow of a data closed-loop method according to an embodiment of the present application;

[0167] Figure 3 is a schematic diagram of the main component structure of a training data acquisition module according to an embodiment of an implementation manner of the present application;

[0168] Figure 4 is a schematic diagram of the main component structure of a model iterative training model according to an embodiment of an implementation manner of an example of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0169] Some embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principle of the present application and are not intended to limit the protection scope of the present application.

[0170] In the description of the present application, "module" and "processor" may include hardware, software, or a combination of both. A module may include a hardware circuit, various appropriate sensors, communication ports, memory, and may also include a software part, such as program code, or may be a combination of software and hardware. The term "A and / or B" represents all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and / or B", and may include only A, only B, or A and B. The singular terms "a" and "the" may also include the plural form.

[0171] For the relevant personal information of users that may be involved in the embodiments of the present application, it is all processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, and for reasonable purposes based on business scenarios, for the personal information actively provided by users during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the authorization of users.

[0172] The personal information of users processed by the present application will vary depending on the specific product / service scenario, and it is subject to the specific scenario of the user's use of the product / service. It may involve the user's account information, device information, driving information, vehicle information, or other relevant information. The present application will treat the personal information of users and its processing with a high degree of diligence.

[0173] The present application attaches great importance to the security of the personal information of users and has taken security protection measures that meet industry standards and are reasonable and feasible to protect the information of users and prevent personal information from being accessed, publicly disclosed, used, modified, damaged, or lost without authorization.

[0174] Refer to the appendix Figure 1 , Figure 1 is a schematic diagram of the main composition structure of the data closed-loop system according to an embodiment of the present application. As Figure 1 shown, the data closed-loop system in the embodiment of the present application mainly includes a data mining module, a data processing model, and a model iterative training model.

[0175] In this embodiment, the data mining module can be configured to perform data mining based on a mining model according to a preset data mining requirement to obtain mined data; wherein, the data mining method includes at least one of cloud data mining and vehicle-side data mining; the data mining requirement is set according to the model to be iterated. The data processing module can be configured to process the mined data to obtain first training data. The model iterative training module can be configured to perform model iterative training on the model to be iterated according to the first training data to obtain a model iterative training result. That is, the data mining requirement can be determined according to the type of training data required by the model to be iterated, and data mining can be performed based on the data mining requirement. The data mining method can include vehicle-side data mining and cloud data mining.

[0176] In one embodiment, the model to be iterated can be various vision classification models for implementing intelligent driving functions, including but not limited to a vision classification model for detecting road signs, a vision classification model for detecting traffic lights, a vision classification model for detecting static obstacles, a vision classification model for detecting dynamic obstacles, etc. Cloud data mining refers to a method of performing data mining on the historical return data of vehicles flowing back to the cloud server by applying a mining model. Vehicle-side data mining refers to a method of performing data mining on the real-time vehicle-side data of a vehicle by applying a mining model.

[0177] In one embodiment, the mining model can be an object detection model.

[0178] In one embodiment, the mining model can be an object classification model.

[0179] In one embodiment, the mining model can include an object detection model and an object classification model. That is, during the data mining process, the object detection model detects the target box of the data to be mined, and the object classification model detects the category of the detected target box to determine the target type. The mining model includes an object detection model and an object classification model, which can effectively improve the accuracy of the data mining process to obtain more accurate first training data.

[0180] In one embodiment, the mined data can be autonomous driving data; wherein, the autonomous driving data can include at least one of environmental perception data, vehicle state data, and planning control data of an autonomous driving vehicle. Autonomous driving data refers to various information for the operation and decision-making of an autonomous driving system. Environmental perception data is information such as pedestrians, vehicles, traffic signs, and traffic lights around the vehicle captured in real time by sensors such as cameras and lidar. Vehicle state information is information reflecting the real-time operation of the vehicle, such as speed, acceleration, steering angle, etc. Planning control data is driving decision-making information made according to environmental perception data, such as a planned path, etc. Data mining can be performed on the autonomous driving data to obtain the mined data.

[0181] In one embodiment of the example of the present application, the data mining module may include a mining model iteration unit and a data mining unit. The mining model iteration unit may be configured to obtain second training data of the mining model corresponding to the target category information according to the target category information corresponding to the data mining requirement from the cloud database; and iteratively train the mining model based on the second training data to obtain a trained mining model. The data mining unit may be configured to perform data mining according to the trained mining model to obtain mined data.

[0182] In this embodiment, the mining model iteration unit may perform vectorized data search based on the cloud database according to the data mining requirement to obtain cold start data of the mining model; perform initial training on the mining model according to the cold start data to obtain an initial mining model; obtain second training data from the cloud database according to the target category information corresponding to the data mining requirement; and perform iterative training on the initial mining model according to the second training data to obtain a trained mining model. Among them, vectorized data search is a process of finding related objects with similar characteristics to the target information corresponding to the data mining requirement in the cloud database, and the related objects are stored in the database in a vectorized form, and the vector is a data list representing the element size and direction. Cold start data is small-scale and high-quality data used in the initial stage of model training to quickly guide the model to understand the basic pattern of the task.

[0183] The data mining unit may determine the data mining method according to the target category information and data volume information corresponding to the data mining requirement; if the data mining method is cloud data mining, perform data mining on the cloud historical return data in the cloud data database according to the trained mining model to obtain mined data; if the data mining method is vehicle-side data mining, send the trained mining model to the vehicle side to perform data mining on the vehicle-side data to obtain mined data.

[0184] In one implementation, for the initial mining model obtained through initial training based on cold start data, second training data can be obtained from the historical image-level base library in the cloud database. The process of obtaining the second training data can incorporate a hard example mining strategy to obtain the second training data, thereby further iteratively training the mining model and enhancing the precision and recall capabilities of the mining model. Among them, the Online Hard Example Mining (OHEM) is a commonly used technique in object detection tasks, aiming to improve the model's ability to identify hard examples. Hard examples include hard negative examples and hard positive examples. Hard negative examples refer to those negative examples that are easily misjudged, while hard positive examples refer to those positive examples that are misclassified during the training process. The core idea of OHEM is to focus on those samples misjudged by the model during the training process and improve the model's ability to identify these samples by increasing the weights of these samples.

[0185] In one implementation, the target category information and data volume information corresponding to the data mining requirements can be determined. And based on the data volume information, the accuracy rate and recall rate of the mining model can be determined, thereby guiding the mining model to conduct data mining. For example, different accuracy rates and recall rates can be set for different data mining requirements.

[0186] In one implementation, for specific target categories or data mining requirements with a small data volume, data mining can be performed on the cloud historical return data in the cloud database based on the mining model to obtain mining data. For other categories except the above specific target categories and data mining requirements with a large data volume, the mining model can be sent to one or more vehicle terminals to perform real-time data mining on the vehicle terminal data to meet the data mining requirements of large data volumes.

[0187] In one implementation, the data closed-loop system of this application can be deployed on the cloud server. After data mining on the vehicle terminal data to obtain mining data, the mining data can be returned to the cloud server.

[0188] In one implementation, for data mining, corresponding data mining rules can be set, and the mining model can obtain mining data based on the data mining rules. Among them, data mining rules refer to the rules that help discover hidden patterns and knowledge in data during the data mining process. Data mining rules can include, but are not limited to, association rules, classification rules, clustering rules, sequence rules, regression rules, and anomaly detection, etc.

[0189] In an implementation manner of the embodiment of the present application, the data processing module may include a training data production unit and a data cleaning unit. The training data production unit may be configured to perform training data format conversion based on the mined data to obtain format conversion data. The data cleaning unit may be configured to perform data cleaning on the format conversion data to obtain the first training data.

[0190] In this implementation manner, the training data production unit may perform data screening on the mined data to obtain screened data; perform target tracking detection on the screened data to obtain target tracking detection data; and perform format conversion on the target tracking detection data to obtain format conversion data.

[0191] Specifically, for the obtained mined data, data quality detection may be first performed on the mined data to ensure the availability of the mined data. Among them, data quality detection refers to a comprehensive, accurate, and reliable evaluation of the mined data to ensure that the detection results of the mined data meet the preset rules and standards. For example, data quality detection may be performed based on machine learning algorithms or automated data quality detection tools.

[0192] For the mined data that passes the data quality detection, data screening may be performed to obtain screened data. Among them, the mined data that passes the above quality detection may be screened according to the video level to further screen out the data corresponding to the target category information that meets the data mining requirements as the screened data, so as to improve the accuracy of the data and reduce the cost loss in the subsequent process.

[0193] For the screened data, an OD (Object Detection) annotation model may be applied to perform target tracking detection, and frame-level data annotation may be performed on the target to achieve target-level tracking annotation of the screened data, so as to obtain target tracking detection data. The target tracking detection data may include the ID of the target category and the positioning information of the bbox (bounding box).

[0194] For the target tracking detection data, the target may be cropped according to the ID of the target category and the bbox of the target tracking detection data, and positive and negative samples may be produced. Finally, the cropped target is subjected to format conversion to be converted into the training data format, thereby obtaining format conversion data.

[0195] Since there are often new target categories added as negative sample data during the data mining process and issues related to annotation quality are involved, it is necessary to clean the format-converted data to improve the data quality of the first training data. For the newly added target categories, a binary classification model can be applied to clean the format-converted data. And a binary classification model without overfitting noise can be used to perform full-scale inference on the format-converted data to achieve dirty data screening at the target level for all data. For the screened dirty data, the target data annotation can be redone to update the labels for all data, so as to obtain higher-quality first training data.

[0196] In an implementation manner of the embodiment of the present application, the model iterative training module may include a model training unit, a model forward inference tracking unit, a model quantization unit, and a model packaging unit. The model training unit may be configured to perform model iterative training on the model to be iterated according to the first training data to obtain a model with iterative training completed. The model forward inference tracking unit may be configured to track the forward inference process of the model according to the model with iterative training completed in accordance with the inference execution process of the model to obtain the forward inference workflow of the model. The model quantization unit may be configured to perform model quantization on the forward inference workflow according to the model usage environment and inference speed requirements to obtain a quantized model. The model packaging unit may be configured to package according to the quantized model to obtain the model iterative training result.

[0197] In this implementation manner, after obtaining high-quality first training data, iterative training can be performed on the model to be iterated according to the first training data to improve the performance of the model to be iterated, thereby obtaining a model with iterative training completed. Among them, the model with iterative training completed can be a pth model. The pth model is a model file format mainly used to save the weight parameters of a trained neural network model.

[0198] To unify the model quantization process of the model iterative training module, the forward inference workflow of the model after iterative training can be traced according to the model inference execution process, removing the performance overhead of tools such as the Python parser, thereby reducing the overhead during the model execution process and obtaining the forward inference workflow of the model. Among them, the forward inference workflow of the model refers to the process of applying the model after iterative training to new input data for prediction or classification. The specific steps of the forward inference workflow can include but are not limited to data preprocessing, model loading, forward propagation, and result postprocessing. The forward inference workflow of the model can be saved as a.pt model file. The.pt model file is usually a model file stored in the form of a weight file, and these weight files contain model parameters and learned features. Since the.pt model only traces the forward inference workflow during the model inference execution process and saves the model parameters and learned features corresponding to the forward inference workflow tracing, it is more suitable for the model quantization process.

[0199] To improve the inference speed of the model and make the model more adaptable to the usage environment (such as the vehicle terminal environment), the forward inference workflow of the model can be quantized to obtain a quantized model. Among them, model quantization refers to the process of converting the weights and activation values of the model from floating-point storage to integer storage, thereby achieving model compression and acceleration to better adapt to running on devices with limited computing resources. The forward inference workflow of the model can be compiled to obtain a compiled file as the quantized model.

[0200] The quantized model can be packaged, that is, multiple model files and related resources corresponding to the quantized model are integrated into a compressed package file for easy storage, transmission, and management, thereby obtaining the model iterative training result.

[0201] In one example, the model to be iterated is a visual classification model for the vehicle terminal. After iterative training of the model to be iterated according to the data closed-loop system of the present application, the obtained model iterative training result can be sent to the vehicle terminal to replace the original visual classification model on the vehicle terminal (such as replacing it in the daily bundle package on the vehicle terminal), thereby realizing the replacement and performance update of the expandable visual classification model.

[0202] In one embodiment, the model training unit includes a first evaluation subunit; the model forward inference tracking unit includes a second evaluation subunit; the model quantization unit includes a third evaluation subunit. The first evaluation subunit can be configured to evaluate the model after iterative training according to the evaluation dataset to obtain a first evaluation result. The second evaluation subunit can be configured to evaluate the forward inference workflow according to the evaluation dataset to obtain a second evaluation result. The third evaluation unit can be configured to evaluate the quantized model according to the evaluation dataset to obtain a third evaluation result.

[0203] That is, evaluation sub-units are set in the model training unit, the model forward inference tracking unit, and the model quantization unit. These evaluation sub-units all use the same evaluation data set for performance evaluation, so as to achieve performance alignment. For the second evaluation result, if there is a case where its value is less than the first evaluation result, it indicates that the model performance has declined, and it is necessary to confirm whether there are errors or omissions in the model forward inference tracking link. For the third evaluation result, if there is a case where its value is less than the first evaluation result or less than the second evaluation result, it also indicates that the model performance has declined, and it is necessary to confirm whether there are errors or omissions in the model quantization link.

[0204] In one implementation, the model packaging unit may include a quality evaluation sub-unit. The quality evaluation sub-unit may be configured to evaluate the model iterative training result according to the quality evaluation data set to obtain the evaluation result of the output model. That is, in the model packaging link, based on the quality evaluation data set, the output evaluation of the model iterative training result is performed to obtain the output evaluation result. If the output evaluation result is qualified, the model iterative training result can be output.

[0205] In one implementation, reference may be made to the appendix Figure 3 , Figure 3 is a schematic diagram of the main composition structure of the training data acquisition module according to an implementation manner of an embodiment of the present application. Figure 3 In the figure, reference numeral 1 is a data mining module, reference numeral 2 is a data processing module, reference numeral 11 is a mining model iteration unit, reference numeral 12 is a data mining unit, reference numeral 21 is a data production unit, and reference numeral 22 is a data cleaning unit.

[0206] As Figure 3 shown, the mining model iteration unit searches for vectorized data from the cloud database based on the data mining requirements to obtain cold start data, and cold starts (i.e., initial training) the mining model according to the cold start data. And based on the target category information corresponding to the data mining requirements, the second training data is obtained from the cloud database, and the mined model after the initial training is iteratively trained according to the second training data to obtain the mined model after the training is completed.

[0207] The data mining unit can realize vehicle-cloud integrated data mining based on the mined model after the training is completed. That is, the data mining method is determined to be cloud data mining or vehicle-side data mining according to different target category information and data volume information. For different data mining methods, data mining can be performed based on the mining model and mining rules to obtain mined data.

[0208] The data production unit obtains format conversion data after performing data screening, target model detection, target category annotation, target category cropping, and format conversion on the mined data.

[0209] The data cleaning unit trains a binary classification model for the target category, performs dirty data cleaning based on the trained binary classification model, and determines whether to stop cleaning; if so, it re-labels the dirty data to update the labels of all data, thereby obtaining the first training data; if not, it continues to perform dirty data cleaning.

[0210] In one embodiment, reference can be made to the attached Figure 4 , Figure 4 which is a schematic diagram of the main component structure of the model iterative training model according to an embodiment of the present application. Figure 4 In the figure, reference numeral 3 is the model iterative training module, reference numeral 31 is the model training unit, reference numeral 32 is the model forward inference tracking unit, reference numeral 33 is the model quantization unit, and reference numeral 34 is the model packaging unit.

[0211] As Figure 4 shown, the model training unit performs iterative training on the model to be iterated according to the first training data to obtain a pth model. The model forward inference tracking unit tracks the forward inference process of the model according to the pth model to obtain a pt model file. The model quantization unit quantizes the pt model file to obtain a quantized model. The model packaging unit packages the quantized model to obtain an output model (i.e., the result of model iterative training). Among them, the model training unit, the model forward inference tracking unit, and the model quantization unit are all set with self-test data evaluation. The self-test data evaluation evaluates the generation results of each unit according to the same evaluation data set to align the model performance. The model packaging unit sets the QA output evaluation. The QA output evaluation evaluates the result of model iterative training according to the quality evaluation data set to obtain the output model evaluation result. If the output model evaluation result meets the requirements, the output model can be output.

[0212] Furthermore, the present application also provides a data closed-loop method.

[0213] Reference can be made to the attached Figure 2 , Figure 2 which is a schematic diagram of the main step flow of the data closed-loop method according to an embodiment of the present application. As Figure 2 shown, the model iterative method of the embodiment of the present application mainly includes the following steps S101 and step S103.

[0214] Step S101: According to the preset data mining requirements, based on the mining model, perform data mining to obtain mining data; wherein, the data mining method includes at least one of cloud data mining and vehicle-side data mining; the data mining requirements are set according to the model to be iterated.

[0215] In this embodiment, step S101 may further include the following steps S1011 and step S1012:

[0216] Step S1011: According to the target category information corresponding to the data mining requirement, obtain the second training data of the mining model corresponding to the target category information from the cloud database; and based on the second training data, perform iterative training on the mining model to obtain a trained mining model.

[0217] Specifically, according to the data mining requirement, vectorized data search can be performed based on the cloud database to obtain the cold start data of the mining model; according to the cold start data, initial training is performed on the mining model to obtain an initial mining model; according to the target category information corresponding to the data mining requirement, the second training data is obtained from the cloud database.

[0218] Step S1012: Perform data mining according to the trained mining model to obtain mined data.

[0219] Specifically, the data mining method can be determined according to the target category information and data volume information corresponding to the data mining requirement; if the data mining method is cloud data mining, then according to the trained mining model, data mining is performed on the cloud historical return data in the cloud data database to obtain mined data; if the data mining method is vehicle-side data mining, then the trained mining model is sent to the vehicle side to perform data mining on the vehicle-side data to obtain mined data.

[0220] Step S102: Process the mined data to obtain the first training data.

[0221] In this embodiment, step S102 may further include the following steps S1021 and S1022:

[0222] Step S1021: According to the mined data, perform training data format conversion to obtain format conversion data.

[0223] Specifically, the mined data can be screened to obtain screened data; target tracking detection is performed on the screened data to obtain target tracking detection data; format conversion is performed on the target tracking detection data to obtain format conversion data; data cleaning is performed on the format conversion data to obtain the first training data.

[0224] Step S1022: Perform data cleaning on the format conversion data to obtain the first training data.

[0225] Step S103: According to the first training data, perform model iterative training on the model to be iterated to obtain the model iterative training result.

[0226] In this embodiment, step S103 may further include the following steps S1031 to S1034:

[0227] Step S1031: Iteratively train the model to be iterated based on the first training data to obtain a model with completed iterative training.

[0228] Step S1032: According to the model with completed iterative training, track the forward inference process of the model according to the inference execution process of the model to obtain a forward inference workflow.

[0229] Step S1033: Quantize the forward inference workflow according to the model usage environment and inference speed requirements to obtain a quantized model.

[0230] Step S1034: Package according to the quantized model to obtain the model iterative training result.

[0231] In one embodiment, the data closed-loop method of the present application may further include the following steps S104 to S106:

[0232] Step S104: Evaluate the model with completed iterative training according to the evaluation dataset to obtain a first evaluation result.

[0233] Step S105: Evaluate the forward inference workflow according to the evaluation dataset to obtain a second evaluation result.

[0234] Step S106: Evaluate the quantized model according to the evaluation dataset to obtain a third evaluation result.

[0235] In one embodiment, the model iteration method of the present application may further include the following step S107:

[0236] Step S107: Evaluate the model iterative training result according to the quality evaluation dataset to obtain an evaluation result of the output-ready model.

[0237] In one embodiment, the mined data may be autonomous driving data; among them, the autonomous driving data may include at least one of environmental perception data, vehicle state data, and planning control data of an autonomous driving vehicle.

[0238] It should be noted that although the above steps are described in a specific order in the above embodiments, those skilled in the art can understand that in order to achieve the effects of the present application, different steps do not necessarily have to be executed in such an order, and they can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent technical solutions to the technical solutions described in the present application, and therefore will also fall within the protection scope of the present application.

[0239] Those skilled in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments of the present application can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be realized. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium, etc., that can carry the computer program code.

[0240] Another aspect of the present application also provides a computer-readable storage medium.

[0241] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium can be configured to store a program for executing the data closed-loop method of the above-mentioned method embodiment. The program can be loaded and run by a processor to implement the above-mentioned data closed-loop method. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The computer-readable storage medium can be a storage device formed by various electronic devices, such as magnetic disks, hard disks, optical discs, flash memories, read-only memories, random access memories, etc. Optionally, the computer-readable storage medium in the embodiments of the present application is a non-transitory computer-readable storage medium.

[0242] Another aspect of the present application also provides a controller.

[0243] In an embodiment of a controller according to the present application, the controller can include at least one processor; and a memory communicatively connected to the at least one processor; wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above-mentioned embodiments is realized.

[0244] The controller described in the present application can be, but is not limited to, mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, vehicle-mounted devices, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, etc. The embodiments of the present application do not make limitations thereto.

[0245] So far, the technical solution of the present application has been described in conjunction with one embodiment shown in the accompanying drawings. However, it is easily understood by those skilled in the art that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principle of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A data closed-loop system, characterized in that, The system includes: A data mining module configured to perform data mining based on a mining model according to a preset data mining requirement to obtain mined data; wherein, the data mining method includes at least one of cloud data mining and vehicle - end data mining; the data mining requirement is set according to the model to be iterated; A data processing module configured to process the mined data to obtain first training data; A model iterative training module configured to perform model iterative training on the model to be iterated according to the first training data to obtain a model iterative training result.

2. The data closed-loop system according to claim 1, wherein The data mining module includes a mining model iteration unit and a data mining unit; The mining model iteration unit is configured to obtain second training data of the mining model corresponding to the target category information from a cloud database according to the target category information corresponding to the data mining requirement; and perform iterative training on the mining model based on the second training data to obtain a trained mining model; The data mining unit is configured to perform data mining according to the trained mining model to obtain the mined data.

3. The data closed-loop system according to claim 2, wherein The mining model iteration unit is further configured to: Perform vectorized data search based on the cloud database according to the data mining requirement to obtain cold - start data of the mining model; Perform initial training on the mining model according to the cold - start data to obtain an initial mining model; Obtain the second training data from the cloud database according to the target category information corresponding to the data mining requirement; Perform iterative training on the initial mining model according to the second training data to obtain the trained mining model.

4. The data closed-loop system according to claim 2, wherein The data mining unit is further configured to: Determine the data mining method according to the target category information and data volume information corresponding to the data mining requirement; If the data mining method is cloud data mining, perform data mining on the cloud historical return data in the cloud data database according to the trained mining model to obtain the mined data; If the data mining method is vehicle - end data mining, send the trained mining model to the vehicle - end to perform data mining on the vehicle - end data to obtain the mined data.

5. The data closed-loop system according to claim 1, wherein The data processing module includes a training data production unit and a data cleaning unit; The training data production unit is configured to perform training data format conversion according to the mined data to obtain format - converted data; The data cleaning unit is configured to clean the format - converted data to obtain the first training data.

6. The data closed-loop system according to claim 5, wherein, The training data production unit is further configured to: Perform data screening on the mined data to obtain screened data; Perform target tracking detection on the screened data to obtain target tracking detection data; Perform format conversion on the target tracking detection data to obtain the format - converted data.

7. The data closed-loop system according to claim 1, characterized in that The model iterative training module includes a model training unit, a model forward inference tracking unit, a model quantization unit, and a model packaging unit; The model training unit is configured to perform iterative model training on the model to be iterated according to the first training data, and obtain a model with completed iterative training; The model forward inference tracking unit is configured to track the forward inference process of the model according to the model with completed iterative training in accordance with the inference execution process of the model, and obtain the forward inference workflow of the model; The model quantization unit is configured to perform model quantization on the forward inference workflow according to the model usage environment and inference speed requirements, and obtain a quantized model; The model packaging unit is configured to package according to the quantized model, and obtain the model iterative training result.

8. The data closed-loop system according to claim 7, wherein The model training unit includes a first evaluation subunit; the model forward inference tracking unit includes a second evaluation subunit; the model quantization unit includes a third evaluation subunit; The first evaluation subunit is configured to evaluate the model with completed iterative training according to the evaluation data set, and obtain a first evaluation result; The second evaluation subunit is configured to evaluate the forward inference workflow according to the evaluation data set, and obtain a second evaluation result; The third evaluation unit is configured to evaluate the quantized model according to the evaluation data set, and obtain a third evaluation result.

9. The data closed-loop system according to claim 7, wherein The model packaging unit includes a quality evaluation subunit; The quality evaluation subunit is configured to evaluate the model iterative training result according to the quality evaluation data set, and obtain an evaluation result of the model ready for release.

10. The data closed-loop system according to any one of claims 1 to 9, characterized in that, The mined data is autonomous driving data; Wherein, the autonomous driving data includes at least one of environmental perception data, vehicle state data, and planning control data of an autonomous driving vehicle.

Images