Data acquisition and processing method, electronic equipment, vehicle and storage medium
By deploying a deep learning model on the vehicle-side device to identify data scenarios and filtering the specified scenario data, the problem of waste of storage resources and low analysis efficiency in data acquisition is solved, and the accurate filtering of corner case, bad case and hard case data is achieved. Storage improves data effectiveness and analysis efficiency.
Patent Information
- Application Number
- CN202411175946.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-17
AI Technical Summary
The prior art can easily cause waste of storage resources and low data mining and analysis efficiency in data collection, especially in the operation data of equipment and software programs. The collected data scenes are numerous and the amount is large, resulting in data that does not need attention to be collected. Influence analysis efficiency.
By deploying an inference model on the vehicle end device, using the deep learning model to identify the scene to which the data belongs, only the data in the preset scene library is stored and uploaded locally, and the data of specified scenarios such as corner case, bad case and hard case are filtered out. Reduce the storage and transmission of invalid data.
It effectively reduces the overhead of data storage resources, improves the effectiveness of data and the efficiency of data mining and analysis, and improves the accuracy and automatic screening capabilities of corner case, bad case and hard case scenario data.
Smart Images
Figure CN119150019A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data acquisition and processing method, electronic equipment, a vehicle and a storage medium. Background Art
[0002] With the development of big data, the effectiveness of collected data will directly affect the efficiency and accuracy of data mining and analysis. For example, for the operation data of equipment and software programs, currently, all data is usually collected on a large scale and then put into the cloud for data mining and analysis. Since the collected data involves many scenarios and a large amount of data, it is easy for the collected data to contain data of scenarios that do not need to be paid attention to, which easily leads to a waste of storage resources and low efficiency of data mining and analysis. Summary of the invention
[0003] In view of this, the purpose of the embodiments of the present application is to provide a data collection and processing method, an electronic device, a vehicle and a storage medium, which can improve the problem that the collected data easily causes waste of storage resources and the low efficiency of data mining and analysis.
[0004] In order to achieve the above technical objectives, the technical solutions adopted in this application are as follows:
[0005] In a first aspect, an embodiment of the present application provides a data collection and processing method, the method comprising:
[0006] Obtain data collected by vehicle-side equipment;
[0007] Inputting the data into an inference model deployed on the vehicle side, and obtaining an inference result output by the inference model that represents the scenario to which the data belongs, wherein the inference model has a function of identifying a specified scenario, and the specified scenario includes at least one of a bad case scenario, a corner case scenario, and a hard case scenario;
[0008] Based on the inference result, the data is processed, wherein when the inference result indicates that the scene in which the data is located belongs to a scene in a preset scene library, local storage and / or uploading operations are performed on the data.
[0009] In combination with the first aspect, in some optional implementations, before acquiring the data collected by the vehicle-side device, the method further includes:
[0010] Acquire a sample data set, wherein the sample data set includes sample data labeled with scene types, and the scene types include types corresponding to bad case scenes, corner case scenes, and hard case scenes, respectively;
[0011] The sample data set is used to train and test the deep learning model to obtain an inference model that has the function of identifying the scene to which the data belongs.
[0012] In combination with the first aspect, in some optional implementations, the sample data set is used to train and test the deep learning model to obtain an inference model with the function of identifying the scene to which the data belongs, including:
[0013] Dividing the sample data set into a training set and a test set;
[0014] Using the training set to train the deep learning model, and using the test set to test the trained deep learning model to obtain an evaluation index of the deep learning model;
[0015] When the accuracy represented by the evaluation index is greater than or equal to a first preset threshold, the deep learning model that has been trained and tested is used as the inference model;
[0016] When the accuracy represented by the evaluation index is less than the first preset threshold, the deep learning model that has been trained and tested is optimized by an optimization algorithm until a preset stop condition is met, and the optimized deep learning model is used as the inference model.
[0017] In combination with the first aspect, in some optional implementations, the deep learning model that has been trained and tested is optimized by an optimization algorithm until a preset stop condition is met, and the optimized deep learning model is used as the inference model, including:
[0018] Repeatedly adjusting the model parameters of the deep learning model, and using the sample data set to train and test the deep learning model after adjusting the model parameters, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or the number of repetitions reaches a preset number, and stopping, and using the finally obtained deep learning model as the inference model, wherein the model parameters include at least one of a learning rate, a batch size, a parameter weight, and a dropout rate;
[0019] Alternatively, repeatedly using different new sample data sets to train and test the deep learning model that has been trained and tested, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or the number of repetitions reaches a preset number, and the deep learning model finally obtained is used as the inference model;
[0020] Alternatively, repeatedly adjust the model parameters of the deep learning model, and use a new sample data set to train and test the deep learning model after the model parameters are adjusted, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or stop when the number of repetitions reaches a preset number, and use the final deep learning model as the inference model.
[0021] In combination with the first aspect, in some optional implementations, the method further includes:
[0022] When an operation instruction for model update is received, the latest model corresponding to the inference model is downloaded from the server to replace the inference model; or, based on local historical data, the inference model is updated to obtain an updated inference model, wherein the latest model is a model obtained by updating the inference model based on cloud historical data, and both the local historical data and the cloud historical data include data belonging to the scenes in the preset scene library.
[0023] In combination with the first aspect, in some optional implementations, the inference result includes at least one scene label and a confidence level corresponding to the scene label;
[0024] Based on the inference result, the data is processed, including:
[0025] When there is a scene label belonging to the scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is greater than a second preset threshold, performing a local storage and / or upload operation on the data;
[0026] When the inference result contains a scene label belonging to a scene in the preset scene library and the confidence level is less than or equal to a second preset threshold, the data is discarded;
[0027] When there is no scene label belonging to the scene in the preset scene library in the inference result, the data is discarded.
[0028] In combination with the first aspect, in some optional implementations, the method further includes:
[0029] When there is a scene label belonging to a scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is greater than a second preset threshold, the scene label corresponding to the scene in the preset scene library and with the maximum confidence is used as the target scene label of the data, and the target scene label is added to the data.
[0030] In a second aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory coupled to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device executes the above method.
[0031] In a third aspect, an embodiment of the present application further provides a vehicle, comprising a vehicle body and the above-mentioned electronic device.
[0032] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed on a computer, the computer executes the above method.
[0033] The invention adopting the above technical solution has the following advantages:
[0034] In the technical solution provided in the present application, the data collected by the vehicle-side device is input into the inference model to obtain the inference result of the scene to which the data belongs, which is output by the inference model; when the inference result indicates that the scene in which the data is located belongs to the scene in the preset scene library, the data is locally stored and / or uploaded. Since the inference model can identify the data of specified scenes such as corner cases, bad cases and hard cases, and the preset scene library includes the specified scenes, it is beneficial to automatically screen and extract the data of corner cases, bad cases and hard case scenes accurately. In addition, for scene data that does not need to be paid attention to, it does not need to be stored and uploaded, which is beneficial to reduce the overhead of data storage resources, improve the effectiveness of stored and uploaded data, and further help to improve the efficiency of subsequent data mining and analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The present application may be further described by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings may be obtained based on these drawings without creative effort.
[0036] Figure 1 One of the flow charts of the data acquisition and processing method provided in the embodiment of the present application.
[0037] Figure 2 The second flowchart of the data acquisition and processing method provided in the embodiment of the present application.
[0038] Figure 3 for Figure 2 Schematic diagram of the flow chart of the sub-steps of step 020.
[0039] Figure 4 A schematic diagram of the circuit structure of an electronic device provided in an embodiment of the present application.
[0040] Icon: 200 - electronic device; 210 - processor; 220 - memory. DETAILED DESCRIPTION
[0041] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that in the drawings or descriptions, similar or identical parts use the same figure numbers, and the implementation methods not shown or described in the drawings are forms known to ordinary technicians in the relevant technical field. In the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0042] The existing technology mainly uses two strategies to identify data from corner case (extreme) scenarios, bad case (abnormal) scenarios, and hard case (difficult case) scenarios, but the recognition effect is far from expected. Strategy 1: Collect all data and put it in the cloud, and then use the model in the cloud to mine corner case / bad case / hard case data. Strategy 2: Use true value data and rules on the vehicle side, rather than using models for condition matching, to filter out cornercase / bad case / hard case data and upload it to the cloud. Both strategies have defects. Strategy 1 will collect a large amount of invalid data, which is time-consuming and labor-intensive; Strategy 2 only uploads valid data. Since it relies solely on rules, the effectiveness and accuracy of the identified data are low.
[0043] In view of the above-mentioned defects of the prior art, the inventors, after long-term research and experiments, propose the following embodiments to solve the above-mentioned problems.
[0044] First embodiment
[0045] Please refer to Figure 1 The present application provides a data collection and processing method, which can be applied to electronic devices, so that the electronic devices can automatically screen the collected data to reduce the amount of collected data and improve the effectiveness of the collected data. The electronic device can be, but is not limited to, a personal computer, a smart phone, a vehicle-side device deployed on a vehicle, an electronic system, etc. The data collection and processing method can include the following steps:
[0046] Step 110, obtaining data collected by the vehicle-side device;
[0047] Step 120, inputting the data into an inference model deployed on the vehicle side, and obtaining an inference result output by the inference model that represents the scenario to which the data belongs, wherein the inference model has a function of identifying a specified scenario, and the specified scenario includes at least one of a bad case scenario, a corner case scenario, and a hard case scenario;
[0048] Step 130, based on the inference result, the data is processed, wherein when the inference result indicates that the scene in which the data is located belongs to a scene in a preset scene library, local storage and / or uploading operations are performed on the data, and the preset scene library includes the specified scene.
[0049] The following will describe the various steps of the data collection and processing method in detail as follows:
[0050] Please refer to Figure 2 Before step 110, the method may further include the steps of model training and testing. For example, before step 110, the method may further include:
[0051] Step 010, obtaining a sample data set, wherein the sample data set includes sample data labeled with scene types, and the scene types include types corresponding to bad case scenes, corner case scenes, and hard case scenes respectively;
[0052] Step 020, using the sample data set, train and test the deep learning model to obtain an inference model that has the function of identifying the scene to which the data belongs.
[0053] In step 010, the sample data set can be understood as a data set collected in advance by engineers for model training and testing, which is collected by the same type of equipment as the vehicle-side equipment. The scenarios to which the data in the sample data set belongs may include, but are not limited to, bad case scenarios, corner case scenarios, and hard case scenarios.
[0054] Corner case scenarios refer to extreme scenarios. For example, situations and scenarios that rarely occur during autonomous driving but may lead to danger. These scenarios are often not within the normal operating range of intelligent driving and account for a small proportion of the data used in perception model training. They are a type of long-tail data problem. For example, pedestrians suddenly rush into the middle of the road, "ghostly peeking out" and other scenarios and the corresponding data collected.
[0055] Bad case scenarios refer to abnormal scenarios. For example, in this application scenario, there are unexpected responses. These cases are usually manifested as the model's prediction results for certain input data being inconsistent with the actual results, resulting in incorrect or misleading predictions.
[0056] Hard case scenarios refer to difficult scenarios, such as complex, difficult and challenging scenarios, such as driving scenarios in complex environments such as bad weather, backlighting, and dimness.
[0057] Understandably, in the prior art, the recognition of data for corner case, bad case and hard case scenarios is far less than expected. If the training data is collected according to the strategy of full data, it will not only cause a great waste of data, but also will not help improve the shortcomings of the model. In this application, during the model training and testing phase, the data of specific scenarios such as corner case, bad case and hard case are used to train and test the deep learning model. In this way, the accuracy of the model detection data can be improved to determine whether it belongs to specific scenarios such as corner case, bad case and hard case. In addition, using the strategy of model plus rules on the vehicle side, data belonging to scenes such as corner case, bad case, hard case, etc. can be directly mined on the vehicle side, and then these valid data can be uploaded to the cloud instead of full collection and uploading, which is conducive to reducing the amount of data collected / transmitted. Among them, the rule is to store the data that meets the "scenes in the preset scene library" locally or perform an upload operation.
[0058] In this embodiment, engineers can use the labeling tool to label each frame / data with the label of the scene to which the data belongs. If the minimum granularity of the data in the sample data set is one frame of data, each frame of data is labeled when performing sample labeling. Each frame of data can be understood as the data collected by the sensor of the corresponding device per frame.
[0059] It should be noted that the amount of data in the sample data set and the scenarios to which it belongs can be flexibly determined according to actual conditions and are not specifically limited here.
[0060] Please refer to Figure 3 In this embodiment, step 020 may include:
[0061] Step 021, dividing the sample data set into a training set and a test set;
[0062] Step 022, using the training set to train the deep learning model, and using the test set to test the trained deep learning model to obtain an evaluation index of the deep learning model;
[0063] Step 023, when the accuracy represented by the evaluation index is greater than or equal to a first preset threshold, the deep learning model that has been trained and tested is used as the inference model;
[0064] Step 024, when the accuracy represented by the evaluation index is less than the first preset threshold, the deep learning model that has been trained and tested is optimized by an optimization algorithm until the preset stop condition is met, and the optimized deep learning model is used as the inference model.
[0065] In step 021, the sample data set can be divided into a training set and a test set according to a corresponding ratio. The ratio can be flexibly determined according to actual conditions, such as 4:1, which is not specifically limited here.
[0066] In step 022, the deep learning model can be used for classification or semantic recognition models, such as deep neural network models, convolutional neural network models, etc. The types of deep learning models are not specifically limited here, as long as they can meet the scene recognition of data. In addition, the evaluation indicators of deep learning models may include, but are not limited to, accuracy, precision, recall, F1 score, etc., which are not specifically limited here. Generally speaking, users are more concerned about the accuracy of the model. Therefore, during the model testing process, the accuracy of the model needs to be tested so that the accuracy of the model meets the design requirements.
[0067] For example, if the accuracy of the evaluation index is greater than or equal to the first preset threshold, it means that the design requirements are met. At this time, the deep learning model for training and testing can be used as an inference model. The first preset threshold can be flexibly set according to actual conditions and is not specifically limited here.
[0068] If the accuracy of the evaluation index is less than the first preset threshold, it means that the design requirements are not met. At this time, the model needs to be optimized to improve the accuracy. For example, in step 024, the deep learning model that has been trained and tested is optimized by an optimization algorithm until the preset stop condition is met, and the optimized deep learning model is used as the inference model, which may include:
[0069] Repeatedly adjust the model parameters of the deep learning model, and use the sample data set to train and test the deep learning model after adjusting the model parameters, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or stop when the number of repetitions reaches a preset number, and use the finally obtained deep learning model as the inference model, wherein adjusting the model parameters of the deep learning model includes: adjusting the batch size, learning rate, parameter weight and dropout rate in the model parameters by traversing, and the preset number of times can be flexibly determined according to actual conditions, such as 3 times, 4 times, etc., which is not specifically limited here;
[0070] Alternatively, repeatedly using different new sample data sets to train and test the deep learning model that has been trained and tested, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or the number of repetitions reaches a preset number, and the deep learning model finally obtained is used as the inference model;
[0071] Alternatively, repeatedly adjust the model parameters of the deep learning model, and use a new sample data set to train and test the deep learning model after the model parameters are adjusted, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or stop when the number of repetitions reaches a preset number, and use the final deep learning model as the inference model.
[0072] Understandably, the way to optimize the deep learning model can be to adjust the model parameters and then train and test the model; or, use a new sample data set to train and test the model again; or combine the previous two methods to train and test the model until the accuracy of the optimized deep learning model meets the design requirements, or the number of repeated training and testing reaches a preset number, so that an inference model with higher accuracy can be obtained.
[0073] When adjusting the model parameters, one parameter in the model can be adjusted each time based on the control variable method, that is, by traversing and adjusting the batch size, learning rate, parameter weight, and dropout rate in the model parameters, and recording the evaluation indicators of the model after adjusting the parameters, so as to select the model parameters corresponding to the optimal evaluation indicators, and then obtain an inference model with higher accuracy.
[0074] In step 110, the vehicle-side device may be an electronic system deployed on the vehicle, and the vehicle may collect data in real time based on various scenarios (such as bad case / corner case / hard case scenarios) through its own sensors. The data may be the operating data of various electronic devices on the vehicle and the operating data of software, and the type of data is not specifically limited here.
[0075] In step 120, the vehicle-side device is pre-deployed with a trained and tested inference model. As an example, the vehicle can input each frame of collected data into the inference model so that the inference model can identify the scene to which each frame of data belongs. Among them, the inference result output by the inference model may include at least one scene label and the confidence corresponding to the scene label. The higher the confidence value of a scene label, the greater the probability that the frame data belongs to the scene label.
[0076] In step 130, the vehicle-side device can filter the collected data, store the data of the scene that needs attention locally, upload it to the cloud for storage, or store it locally and in the cloud at the same time. The data of the scene that does not need attention can be directly discarded. The uploading operation can refer to uploading to the cloud for storage or uploading to other devices for storage, which is not specifically limited here.
[0077] For example, step 130 may include:
[0078] When there is a scene label belonging to the scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is greater than a second preset threshold, performing local storage and / or uploading operations on the data, wherein the second preset threshold can be flexibly determined according to actual conditions and is not specifically limited here;
[0079] When the inference result contains a scene label belonging to a scene in the preset scene library and the confidence level is less than or equal to a second preset threshold, the data is discarded;
[0080] When there is no scene label belonging to the scene in the preset scene library in the inference result, the data is discarded.
[0081] The preset scene library can be a scene list, which usually only records scenes that the user needs to pay attention to, and scenes that do not need to be paid attention to are not recorded in the scene list. For example, the preset scene library can record bad case scenes, corner case scenes, hard case scenes and other scenes.
[0082] When there is a scene label belonging to a scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is greater than the second preset threshold, it means that the data belongs to the scene in the preset scene library and is data that needs attention and needs to be stored locally or in the cloud.
[0083] When there is a scene label belonging to a scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is less than or equal to the second preset threshold, it means that the probability that the data belongs to the scene in the preset scene library is small, and the data does not need to be paid attention to, so there is no need to store it locally or in the cloud to reduce the usage of storage resources.
[0084] When there is no scene label belonging to the scene in the preset scene library in the inference result, it means that the data is not data that the user needs to pay attention to and does not need to be stored locally or in the cloud.
[0085] As an optional implementation, after step 130, the method may further include:
[0086] When an operation instruction for model update is received, the latest model corresponding to the inference model is downloaded from the server to replace the inference model; or, based on local historical data, the inference model is updated to obtain an updated inference model, wherein the latest model is a model obtained by updating the inference model based on cloud historical data, and both the local historical data and the cloud historical data include data belonging to the scenes in the preset scene library.
[0087] In this embodiment, when the method is applied to a vehicle, the operation instruction of the model update can be triggered by the user through a virtual button on the touch screen of the vehicle, or automatically generated based on a set update cycle. The model update can be to download the latest model from the cloud, or directly update the model locally.
[0088] For example, the server may store a recommendation model. In addition, the server may periodically or under the control of an administrator use the latest cloud historical data to train and test the recommendation model to update the model. The vehicle-side device may obtain the latest recommendation model from the server and perform local updates.
[0089] In another implementation of model update, the vehicle-side device can use the latest local historical data to train and test the recommendation model, thereby updating the recommendation model locally. In this way, using the latest historical data to update the model is conducive to improving the accuracy of the model.
[0090] In this embodiment, after step 130, the method may further include:
[0091] When there is a scene label belonging to a scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is greater than a second preset threshold, the scene label corresponding to the scene in the preset scene library and with the maximum confidence is used as the target scene label of the data, and the target scene label is added to the data.
[0092] Understandably, if the data belongs to a scene in a preset scene library, the scene label with the highest confidence in the preset scene library will be used as the target scene label to which the frame data belongs. This is beneficial for subsequent mining and analysis of the frame data.
[0093] The following example illustrates the implementation process of the method:
[0094] First, based on the performance of the deep learning model, it is necessary to select the inference model and applicable scenarios that can be placed on the vehicle side. The initial deep learning model on the vehicle side can come from the deep learning model that performs poorly on the hard case / bad case / cornercase scene recognition deployed on the road test vehicle. This model performs well for most scenes, but performs below expectations for the hard case / bad case / corner case scenes. Use the initial model on the vehicle side of the data collection vehicle to automatically filter out the scene data of the hard case / bad case / corner case and upload it to the cloud for manual labeling. When the above scene data reaches a certain scale, model training and testing can be performed to obtain a deep learning model. If the model improves the scene recognition accuracy in the scene library and performs better than the model deployed on the vehicle side, the deep learning model is deployed on the vehicle side. Otherwise, it is necessary to use an optimization algorithm or increase the data in the scene library to retrain the model to meet the requirements for deployment on the vehicle side.
[0095] Secondly, the sensors on the vehicle acquire single-frame data and input it into the vehicle-side inference model. The vehicle-side inference model infers the single-frame data, obtains inference results such as the scene label and confidence of the single-frame data, and labels the data with a scene label.
[0096] Finally, the automatic screening logic on the vehicle side is executed. The vehicle compares the model inference results with the hard case / corner case / bad case in the scenario library and other operations. If the inference result meets the requirements, the frame data is retained and uploaded to the cloud or saved on the local disk. Otherwise, the data is discarded.
[0097] As an example, the implementation and design process of the method can be as follows:
[0098] Engineers need to prepare a deep learning model that can be deployed to the vehicle and a list of bad case / corner case / hardcase scenarios. The preparation work can be as follows:
[0099] 1) When the deep learning model is trained to a certain level, it will have relatively good performance for most scenarios, but there will still be bad cases / corner cases / hard cases, and these scenarios will be made into a scenario list / scenario library.
[0100] 2) The deep learning model has a certain effect on the scenario defined in 1), but the confidence of the reasoning is relatively low, not completely unrecognizable. In this case, the model can be deployed on the vehicle side as a vehicle-side model.
[0101] 3) Deploy the model to the vehicle side as the inference module of the data collection vehicle side model. Deploy the above scenario list to the vehicle side as the data collection vehicle side scenario list module. Deploy the data screening algorithm to the vehicle side as the data collection vehicle side data screening module.
[0102] The overall design of the vehicle-side scene data model reasoning and automatic screening algorithm can be as follows:
[0103] A single frame of data is obtained through the sensor and sent to the vehicle-side reasoning model. After reasoning by the vehicle-side reasoning model, the single frame of data will obtain a scene label and a confidence score. Then the data screening module automatically determines whether the frame of data is in the scene list based on the scene label and confidence score of the single frame of data and the scene list in the vehicle-side scene list module. If the data is in the scene list, the frame of data is valid data, and the frame of data is marked with a scene label. The data and label are stored in the local hard disk or uploaded to the cloud. Otherwise, the frame of data is invalid data and needs to be discarded or destroyed.
[0104] The design of vehicle-side reasoning model update iteration can be as follows:
[0105] Use the training sets in the manually annotated cloud scene library to continuously iterate the deep learning algorithm and model, and use the hardcase / bad case / cornercase scene data to evaluate the performance of the model. If the model's indicators are improved compared to before training, the model can be deployed to the data collection vehicle and replace the original vehicle-side model; if the model's indicators are not improved compared to before training, it is necessary to adjust the algorithm or use the original model of the data collection vehicle to continue collecting scene data and upload it to the cloud, and then retrain the model until the model's recognition effect in the hard case / bad case / cornercase scene data is improved, and then the deep learning model can be deployed to the vehicle.
[0106] The design of the vehicle-side automatic screening module can be as follows:
[0107] According to the result of the vehicle-side model inference, it is compared with the list of bad case / hard case / corner case scenarios in the scenario library. If the scene label of the model inference result is in the scenario library, the frame data is valid data, and it is necessary to add the scene label to the data and retain the frame data. The frame data is saved to the vehicle side or directly uploaded to the cloud; otherwise, the frame data is invalid data and needs to be discarded.
[0108] In summary, this embodiment can obtain single-frame data through the sensor on the vehicle side, and then send the single-frame data to the reasoning model on the vehicle side; after the reasoning of the vehicle-side reasoning model, the single-frame data will obtain the scene label label and confidence score; then, according to the scene label label and confidence score of the single-frame data and the scene list stored on the vehicle side, it is automatically determined whether the frame data is in the scene list, that is, the scene label is in the scene list, and the confidence exceeds the corresponding threshold, which means that the frame data is in the scene list. If the data is in the scene list, the frame data is valid data, and the frame data is marked with a scene label, and the data and the label are stored in the local hard disk or uploaded to the cloud; otherwise, the frame data is invalid data and needs to be discarded or destroyed. In this way, invalid data can be directly filtered out, valid data can be retained, and scene labels can be marked on valid data, which can not only improve the collection efficiency of valid data, but also improve the utilization rate of data, so as to facilitate more efficient use of data. In the process of collecting and screening data, the present application can reduce manual participation, which is conducive to improving the accuracy and effectiveness of stored data and saving human resources. Compared with full data collection, this application collects valid data and filters out invalid data, which can save bandwidth resources for data transmission and save data storage resources.
[0109] It should be noted that, in this embodiment, model training, testing and optimization can be performed locally on the electronic device or by a server in the cloud, and there is no specific limitation on the execution entity.
[0110] Second embodiment
[0111] Please refer to Figure 4 , an embodiment of the present application provides an electronic device 200, which may include a processor 210 and a memory 220. The memory 220 stores a computer program, and when the computer program is executed by the processor 210, the electronic device 200 can perform the corresponding steps in the following data acquisition and processing method.
[0112] In this embodiment, the processor 210 and the memory 220 may be integrated into one body to serve as a controller on the vehicle. In other embodiments, the processor 210 and the memory 220 may exist independently.
[0113] Understandably, Figure 4 The structure of the electronic device 200 shown in FIG. 1 is only a schematic diagram of the structure. The electronic device 200 may also include Figure 4 More components shown, for example, may also include a communication module for establishing a communication connection with a server or other devices. Figure 4 Each component shown in the figure can be implemented by hardware, software or a combination thereof.
[0114] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic device 200 described above can refer to the corresponding process of each step in the aforementioned method, and will not be elaborated herein.
[0115] Third embodiment
[0116] The embodiment of the present application also provides a vehicle, which may include a vehicle body and the above-mentioned electronic device. In this way, it is beneficial for the vehicle end to collect valid data, store the valid data locally or in the cloud, filter out invalid data, save bandwidth resources for data transmission, save data storage resources, reduce the amount of stored data, and help improve the subsequent mining efficiency of valid data.
[0117] Fourth embodiment
[0118] The embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the computer executes the data collection and processing method described in the above embodiment.
[0119] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by hardware, and can also be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.
[0120] In the embodiments provided in the present application, it should be understood that the disclosed equipment and methods can also be implemented in other ways. The equipment and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of the module, program segment or code includes one or more executable instructions for implementing the specified logical functions. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.
[0121] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A data collection and processing method, characterized in that: The method comprises: Obtain data collected by vehicle-side equipment; Inputting the data into an inference model deployed on the vehicle side, and obtaining an inference result output by the inference model that represents the scenario to which the data belongs, wherein the inference model has a function of identifying a specified scenario, and the specified scenario includes at least one of a bad case scenario, a corner case scenario, and a hard case scenario; Based on the inference result, the data is processed, wherein when the inference result indicates that the scene in which the data is located belongs to a scene in a preset scene library, local storage and / or upload operations are performed on the data, and the preset scene library includes the specified scene.
2. The method according to claim 1, characterized in that Before acquiring the data collected by the vehicle-side device, the method further includes: Acquire a sample data set, wherein the sample data set includes sample data labeled with scene types, and the scene types include types corresponding to bad case scenes, corner case scenes, and hard case scenes, respectively; The sample data set is used to train and test the deep learning model to obtain an inference model that has the function of identifying the scene to which the data belongs.
3. The method according to claim 2, characterized in that Using the sample data set, the deep learning model is trained and tested to obtain an inference model with the function of identifying the scene to which the data belongs, including: Dividing the sample data set into a training set and a test set; Using the training set to train the deep learning model, and using the test set to test the trained deep learning model to obtain an evaluation index of the deep learning model; When the accuracy represented by the evaluation index is greater than or equal to a first preset threshold, the deep learning model that has been trained and tested is used as the inference model; When the accuracy represented by the evaluation index is less than the first preset threshold, the deep learning model that has been trained and tested is optimized by an optimization algorithm until a preset stop condition is met, and the optimized deep learning model is used as the inference model.
4. The method according to claim 3, characterized in that The deep learning model that has been trained and tested is optimized by an optimization algorithm until a preset stop condition is met, and the optimized deep learning model is used as the inference model, including: Repeatedly adjusting the model parameters of the deep learning model, and using the sample data set to train and test the deep learning model after adjusting the model parameters, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or the number of repetitions reaches a preset number, and the deep learning model finally obtained is used as the inference model, wherein adjusting the model parameters of the deep learning model includes: adjusting the batch size, learning rate, parameter weight and dropout rate in the model parameters by traversing; Alternatively, repeatedly using different new sample data sets to train and test the deep learning model that has been trained and tested, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or the number of repetitions reaches a preset number, and the deep learning model finally obtained is used as the inference model; Alternatively, repeatedly adjust the model parameters of the deep learning model, and use a new sample data set to train and test the deep learning model after the model parameters are adjusted, until the accuracy of the optimized deep learning model is greater than or equal to the first preset threshold, or stop when the number of repetitions reaches a preset number, and use the final deep learning model as the inference model.
5. The method according to claim 1, characterized in that The method further comprises: When an operation instruction for model update is received, the latest model corresponding to the inference model is downloaded from the server to replace the inference model; or, based on local historical data, the inference model is updated to obtain an updated inference model, wherein the latest model is a model obtained by updating the inference model based on cloud historical data, and both the local historical data and the cloud historical data include data belonging to the scenes in the preset scene library.
6. The method according to claim 1, characterized in that The reasoning result includes at least one scene label and a confidence level corresponding to the scene label; Based on the inference result, the data is processed, including: When there is a scene label belonging to the scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is greater than a second preset threshold, performing a local storage and / or upload operation on the data; When the inference result contains a scene label belonging to a scene in the preset scene library and the confidence level is less than or equal to a second preset threshold, the data is discarded; When there is no scene label belonging to the scene in the preset scene library in the inference result, the data is discarded.
7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: When there is a scene label belonging to a scene in the preset scene library in the inference result, and the confidence corresponding to the scene label is greater than a second preset threshold, the scene label corresponding to the scene in the preset scene library and with the maximum confidence is used as the target scene label of the data, and the target scene label is added to the data.
8. An electronic device, characterized in that: The electronic device comprises a processor and a memory coupled to each other, wherein the memory stores a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 7.
9. A vehicle, characterized in that: The vehicle comprises a vehicle body and the electronic device as claimed in claim 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.