Model processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a model processing method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a target running state data set of a target vehicle; labeling the target operation state data set to obtain a target data set; matching a corresponding reference model from a preset model library according to the target task type, and selecting to-be-trained data from a target data set according to the target task type and the reference model; training the reference model through the to-be-trained data, comparing an output result with an evaluation result of a truth value system, and correcting the reference model according to a comparison result to obtain a target model. According to the method, the reference model is trained according to the multi-source data set of the target vehicle, the reference model is corrected through the truth value system, the target model is obtained, the problem that the weather condition and the driving road condition in the driving process of the target vehicle cannot be comprehensively considered in the prior art is solved, and the detection capacity of the target model is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a model processing method, device, electronic equipment and storage medium. Background Art

[0002] The core of autonomous driving is to obtain raw data through sensors, and through model reasoning, obtain information about various obstacles around the vehicle (cars, trucks, cones, cyclists, etc.), or add some post-processing strategies to make the perception results more in line with the needs of autonomous driving, thereby providing a decision-making basis for the vehicle's driving route and driving strategy.

[0003] However, the existing data cleaning methods are only for certain sensor data and cannot meet the needs of cleaning the multi-source acquisition device data generated by the vehicle. In addition, the existing model training methods for vehicles are usually used to deal with a single vehicle operation problem and most of the data used are related to the vehicle operation problem. This leads to the inability to comprehensively consider the various operating states of the vehicle, the various weather conditions when the vehicle is driving, and the conditions of the road. This leads to inaccurate obstacle information, which affects vehicle driving. Summary of the invention

[0004] The present invention provides a model processing method, device, electronic device and storage medium to solve the problem that various operating states of a vehicle, various weather conditions when the vehicle is driving and the conditions of the driving road cannot be comprehensively considered, which leads to inaccurate obstacle information and thus affects vehicle driving.

[0005] According to one aspect of the present invention, there is provided a model processing method, comprising:

[0006] Determine a target operating state data set of the target vehicle, wherein the target operating state data set is data obtained by extracting a reference operating state data set according to a preset data amount and segmenting it according to a preset ratio, and the reference operating state data set is a data set obtained by processing original operating state data collected by a multi-source acquisition device;

[0007] Annotating the target operating status data set to obtain a target data set;

[0008] Matching a corresponding reference model from a preset model library according to the target task type, and selecting data to be trained from a target data set according to the target task type and the reference model, wherein the target task type is used to characterize the type of action to be completed by the target vehicle during operation, and the reference model is used to provide auxiliary information for the completion of the target task;

[0009] The reference model is trained using the data to be trained, and the output result is compared with the evaluation result of the true value system, and the reference model is corrected according to the comparison result to obtain the target model.

[0010] According to another aspect of the present invention, there is provided a model processing device, comprising:

[0011] A target operating state data set determination module is used to determine a target operating state data set of a target vehicle, wherein the target operating state data set is data extracted from a reference operating state data set according to a preset data volume and segmented according to a preset ratio, and the reference operating state data set is a data set obtained by processing original operating state data collected by a multi-source acquisition device;

[0012] A target data set determination module, used to label the target operating status data set to obtain a target data set;

[0013] A reference model determination module is used to match the corresponding reference model from a preset model library according to the target task type, and select the data to be trained from the target data set according to the target task type and the reference model, wherein the target task type is used to characterize the type of action to be completed by the target vehicle during operation, and the reference model is used to provide auxiliary information for the completion of the target task;

[0014] The correction module is used to train the reference model with the data to be trained, compare the output result with the evaluation result of the true value system, and correct the reference model according to the comparison result to obtain the target model.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the model processing method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the model processing method described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention determines the target operating status data set of the target vehicle, which can ensure that the obtained multi-source original operating status data is converted into a unified format and the sensitive information therein is eliminated, providing a basis for subsequent processing; the target operating status data set is labeled to obtain the target data set, and the labeling of the data set can improve the generalization ability and operation speed of the target model while facilitating the classification and retrieval of the data set; the corresponding reference model is matched from the preset model library according to the target task type, and the target task type can provide a basis for the selection of the reference model, and a more suitable reference model is selected; the data to be trained is selected from the target data set according to the target task type and the reference model, which can make the data to be trained more suitable for the target task and also include a wider range of data types; the reference model is trained with the data to be trained, and the output result is compared with the evaluation result of the true value system, and the reference model is corrected according to the comparison result to obtain the target model, which can improve the detection ability of the target model. The method trains a reference model based on a multi-source data set of the target vehicle, and corrects the reference model through a truth system to obtain a target model. This solves the problem that the existing technology cannot comprehensively consider the weather conditions and road conditions during the driving of the target vehicle, and improves the detection capability of the target model.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flow chart of a model processing method provided by an embodiment of the present invention;

[0024] Figure 2 A block diagram of a cloud data management platform provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of a data desensitization department provided in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a cloud platform provided by an embodiment of the present invention;

[0027] Figure 5A block diagram of a cloud platform provided in an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the structure of a model processing device provided by an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of the structure of an electronic device for implementing the model processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Figure 1 This is a flow chart of a model processing method provided by an embodiment of the present invention. This embodiment can be applied to train the model according to the driving environment of the vehicle to ensure that the model fits the actual scene. The method can be executed by a model processing device, which can be implemented in the form of hardware and / or software. The model processing device can be configured in any electronic device with network communication function. Figure 1 As shown, the method includes:

[0033] S110 , determining a target operating state data set of a target vehicle.

[0034] Among them, the target operating status data set is the data obtained by extracting the reference operating status data set according to a preset data volume and dividing it according to a preset ratio, and the reference operating status data set is the data set obtained by processing the original operating status data collected by multi-source acquisition equipment.

[0035] Among them, the multi-source acquisition equipment includes: millimeter wave radar, 4D millimeter wave radar, ultrasonic radar, positioning sensor, radar and camera, etc. The original operation status data set includes: millimeter wave data set, 4D millimeter wave data set, ultrasonic data set, positioning data set, image data set, etc.

[0036] The preset data volume can be determined based on the reference collection volume. The reference collection volume is the collection volume corresponding to the data set whose collection volume in the original running status data set is less than the collection volume of other data sets. The collection volume is the amount of data collected by the collection device per second.

[0037] For example, assuming that the sampling frequency of the millimeter wave radar is 60 Hz, that is, 60 data are collected per second; the sampling frequency of the positioning radar is 100 Hz, that is, 100 data are collected per second, and the sampling frequencies of the 4D millimeter wave radar, ultrasonic radar, radar and camera are all greater than 60 Hz, that is, more than 60 data are collected per second, then the preset data volume can be 60. That is, when extracting the 4D millimeter wave data set, ultrasonic data set and image data set, 60 data are extracted from the data per second.

[0038] In the above steps, the preset data volume is set to avoid different amounts of data collected by different sensors in the same time period, which may cause mismatches when the data is subsequently matched according to the original timestamps, thereby ensuring the accuracy and consistency of the data.

[0039] The operating status data is used to characterize the driving road data and weather data of the target vehicle when it is driving.

[0040] Specifically, the original running state data set of the target vehicle during driving is collected through multi-source acquisition equipment, the original running state data set is desensitized and data parsed, and the parsed data set is used as the reference running state data set. The reference running state data set is extracted according to the preset data volume, and the extracted data set is coordinate transformed, and the data set after coordinate transformation is used as the target running state data set.

[0041] Furthermore, the extraction process is: according to the timestamp of the data set corresponding to the preset data volume, data corresponding to the timestamp closest to the timestamp in the millimeter wave data set are selected from the data sets other than the millimeter wave data set.

[0042] For example, assuming that in the millimeter wave data set, the collection time of one of the data is time A; the times corresponding to the data adjacent to time A in the ultrasonic data set are time B and time C, and the difference between time B and time A is less than the difference between time C and time A, then the data corresponding to time B is selected.

[0043] S120: Label the target operating status data set to obtain a target data set.

[0044] Specifically, determine the preset label type, classify the target operating status data set according to the preset label type, and obtain the data set to be labeled. Label the data set to be labeled using the labeling model to obtain the labeled data set. Verify the accuracy of the labeled data set. If the labeling is correct, export the labeled data set and the data set to be labeled to obtain the target data set.

[0045] Among them, the preset labels can be determined according to the actual operating requirements of the target vehicle, such as vehicles, pedestrians, lane lines and roadblocks.

[0046] For example, assuming that the preset tag type is vehicle, the running status data of the vehicle contained in the target running status data set is extracted, and the vehicles contained in the extracted running status data are labeled by the labeling model to obtain a vehicle labeling data set. The labeling of the vehicle labeling data set is verified, and if the verification is successful, the vehicle to-be-labeled data set and the vehicle labeling data set are exported.

[0047] Furthermore, the preset tags contain more tag information, so the target running status data needs to be repeatedly labeled. For example, if some target running status data has been labeled with vehicle information, it still needs to be labeled when labeling pedestrians.

[0048] The above steps of labeling the target operation data set are to distinguish the information contained in the target operation status data set that can affect the vehicle operation, to prevent the confusion of multiple information and affect the accuracy of target model detection.

[0049] S130: Match the corresponding reference model from the preset model library according to the target task type, and select the data to be trained from the target data set according to the target task type and the reference model.

[0050] The target task type is used to characterize the type of action that the target vehicle will complete when it is running. The target task is the action that the target vehicle needs to complete in the next step according to the current state when it is running. For example, assuming that the target task of the target vehicle is to enter an intersection, the corresponding target task type can be detecting vehicles at the intersection or detecting pedestrians at the intersection.

[0051] The reference model is used to provide auxiliary information for the completion of the target task. For example, assuming that the target task is to enter an intersection and the target task type is to detect vehicles at the intersection, the reference model can be an image recognition model. The reference model determines the vehicle at the intersection based on the current operating status data of the target vehicle and marks the location of the vehicle, thereby providing assistance for the target vehicle to pass through the intersection.

[0052] Specifically, the target task type is determined according to the action that the target vehicle is going to complete during operation, the historical processing information is determined according to the target task type, and the corresponding reference model is matched from the preset model library according to the historical processing information. The data set related to the processing target task is determined from the target data set according to the target task type, and the matched data set is used as the reference training data set. The reference model selects the training data set according to the proportion of environmental information in the reference training data set.

[0053] Among them, the preset model library is composed of existing models and algorithms. For example, the models may include: convolutional neural network model, recurrent neural network model, generative adversarial network model, graph neural network model, etc.; the algorithms may include: edge detection algorithm and Hough transform algorithm, etc.

[0054] The historical processing information is the result of processing the target task at the historical moment and the model information adopted, etc. The environmental information may include: obstacle type information, weather scene information, road scene information, etc.

[0055] Exemplarily, assuming that the target task type is backing out into a garage, the historical processing information is determined according to the task of backing into a garage, and the corresponding model for processing backing into a garage is matched from the preset model library according to the model information used in the historical processing information, such as a lane line detection model, and the obtained lane line detection model is used as a reference model, wherein the lane line detection model is used to detect the distance between the target vehicle and the garage line. According to the target task type, data with lane lines marked are selected from the target data set to obtain reference training data. The reference training data is further screened by the lane line detection model to obtain a data set to be trained.

[0056] In the above steps, the selection of the training data set can ensure the balance of various types of data in the data set and the weather and road condition information represented by the data.

[0057] S140, training the reference model with the data to be trained, and comparing the output result with the evaluation result of the true value system, and correcting the reference model according to the comparison result to obtain the target model.

[0058] Among them, the truth system is used to generate software and evaluation software to ensure that it can provide intelligent packaging and visualization functions of sensor data based on the collected data, provide offline display and playback of truth system data (including truth results), generate truth data based on perception and perception fusion, true value manual labeling functions and perception evaluation and other related functions.

[0059] For example, the perception evaluation can be: lane line evaluation, target object evaluation and roadside evaluation, etc. Among them, the lane line evaluation includes: Precision, Recall, Accuracy evaluation, lateral distance error and curvature radius error evaluation, and color and type, which can be manually corrected.

[0060] Among them, the target object evaluation includes: Precision, Recall, Accuracy evaluation; lateral distance error evaluation, longitudinal distance error evaluation, heading angle error evaluation, lateral velocity error evaluation, longitudinal velocity error evaluation, lateral acceleration error evaluation, longitudinal acceleration error evaluation, target length error evaluation, target width error evaluation, target height error evaluation of vehicles / pedestrians / two-wheeled vehicles; Mean Average Precision (mAP) evaluation; MOTA (Multiple Object Tracking Accuracy) and MOTP (Multiple Object Tracking Precision) evaluation; target type recognition accuracy, CIPV (closest vehicle on the path) recognition accuracy evaluation; FPPI (False Positive per Image) evaluation.

[0061] Among them, the curb assessment includes: lateral distance error assessment and slope stability error assessment.

[0062] Specifically, the data to be trained is input into the reference model, the reference model is trained, the output result is obtained, and the output result is compared with the evaluation result of the true value system. If the error between the output result and the evaluation result meets the preset error, the reference model is saved; if the error between the output result and the evaluation result does not meet the preset error, the reference model is corrected according to the evaluation result until the error between the output result and the evaluation result meets the preset error, and the reference model that meets the preset error is used as the target model.

[0063] Furthermore, after obtaining the target model, the target model is stored.

[0064] Optionally, determining a target operating state data set of a target vehicle includes steps A1-A4:

[0065] Step A1: Obtain the original running status data set of the target vehicle.

[0066] Specifically, the original operating status data collected by the multi-source collection equipment is obtained through the data management platform, and the original operating status data is summarized to obtain the original operating status data set of the target vehicle.

[0067] Furthermore, data management platforms such as Figure 2 As shown, the functions of the data management platform include: data collection management, data upload, source data management, data set management, workflow engine and unified portal.

[0068] Among them, the data collection management function includes: collection requirements, management of collection tasks, collection monitoring, driver management, vehicle management and collection location management, etc. The data collection management function mainly realizes the collection of the target vehicle's operating status data by controlling multi-source collection equipment such as data collection host, sensors, and collection software, and summarizes the road information, vehicle movement information, driver information, weather information, location information, traffic sign information, and surrounding vehicle information of the target vehicle during driving collected by the multi-source collection equipment.

[0069] The data upload function includes: uploading the original state data set and statistics on the number of uploads. The data upload function is mainly used to upload the original operation state data set to the cloud platform, and can also be used to count the upload status of the original operation state data.

[0070] Among them, the source data management function is mainly used to search for existing original operating status data sets. It can locate a certain vehicle, a batch of data, or data for a certain period of time for viewing, downloading, processing, etc., and supports data playback and vehicle trajectory display.

[0071] The data set management function includes: data set screening and data discovery. The data set management function mainly implements data segmentation management based on project dimensions. Users can create their own data sets, provide data annotation functions, filter annotated data, and mark important data scenarios. Users can retrieve and filter the entire internal data set to find data sets that are suitable for their business. Users can initiate "authorization applications" for data sets that meet their usage requirements.

[0072] The workflow engine functions include workflow management, workflow jobs, project management, and data processing operators. The workflow engine function is mainly used to create, configure, view, associate jobs, delete jobs, and configure permissions for workflows, and to display information such as resource usage, platform operators, and project operators. Furthermore, users can complete operations such as adding, deleting, or connecting each step by dragging tools or forming resources, dragging in resources, initiating workflow tasks, and executing jobs.

[0073] Furthermore, the workflow engine function can also realize operations such as project creation, permission configuration and project editing, so as to process and manage project resources through the workflow engine. Among them, projects include: platform operators and project operators, which can be managed through operations such as uploading, disabling, deleting, starting and stopping, and can display operator details and help documents on the display interface, which is convenient for pipeline processing and automatic archiving of multi-source data and multi-project data.

[0074] Among them, the platform operator is the task that the data management platform needs to complete, such as the platform operator can collect the original operation status data every 10 minutes and classify the original operation status data. The project operator is the task corresponding to the action that the target vehicle needs to complete, such as the target vehicle reversing. The project operator is used to issue a reversing task to the cloud platform and analyze and save the information processed by the cloud platform.

[0075] Among them, the unified portal functions mainly include: application management, single sign-on, and user authority management. The unified portal module supports application module-level management of various parts of the cloud platform, and users with different permissions view different functional modules, supports single sign-on, and supports user authority management.

[0076] Step A2: desensitize and analyze the original running status data set to obtain a reference running status data set.

[0077] Among them, desensitization is used to remove sensitive data from the original operating status data set. Sensitive data refers to data that can expose driver information and geographic entity information.

[0078] Among them, data analysis is used to convert the data format in the original running data set into a format that can be recognized by the reference model.

[0079] Specifically, the data that can expose driver information and geographic entity information in the original operating status data set is removed, and after the removal is completed, it is exported into rosbag format data. The data desensitization department sends the exported rosbag format data to the cloud platform. The original data export function of the cloud platform converts the rosbag format data into pcap format data, and matches the corresponding parsing program according to the data type corresponding to the different original operating status data in the original operating status data set. The data is parsed by the parsing program to obtain the reference operating status data set.

[0080] Furthermore, the desensitization of the original operating status data set is completed by the data desensitization department, such as Figure 3 As shown, the data desensitization department includes: data desensitization (ground information) and deflection module and rosbag format file module.

[0081] Among them, the data desensitization (geo-information) and deflection modules include: data desensitization (geo-information) service and data desensitization (personal information) service. Among them, the data desensitization (geo-information) service is used to complete the desensitization processing related to geographic information, and provide subsequent operation and maintenance service capabilities to ensure the normal operation of the data desensitization platform capabilities; the data desensitization (personal information) service is used to complete the anonymization processing of personal information (face, license plate), and provide subsequent operation and maintenance service capabilities to ensure the normal operation of the data desensitization platform capabilities.

[0082] Among them, the rosbag format file module is used to export the desensitized original running status data set into rosbag format data and save it in the rosbag file.

[0083] Furthermore, the data desensitization department can meet the hierarchical storage of the original operating status data set before data annotation and the original operating status data set after desensitization, as well as the classified storage and management of point cloud, video, image and other data to form a data list for subsequent processing.

[0084] The above steps are to desensitize the original operation status data set because most of the original operation status data set is the most original geographic information data, and the original geographic information data contains a lot of sensitive and confidential geographic entity information. If used directly, it will cause a lot of important information leakage.

[0085] Furthermore, a corresponding parsing program is matched according to the data type corresponding to different original running status data in the original running status data set, and the data is parsed by the parsing program, such as Figure 4 As shown, the cloud platform includes a raw data export function, which can convert rosbag format data into pcap format data and match the corresponding parsing program according to the data type.

[0086] Exemplarily, millimeter wave data and 4D millimeter wave data are matched with the millimeter wave analysis program, and the millimeter wave data are analyzed by the millimeter wave analysis program to obtain the millimeter wave point cloud; ultrasonic data are matched with the ultrasonic analysis program, and the ultrasonic data are analyzed by the ultrasonic analysis program to obtain the ultrasonic radar data; the positioning data pcap package is matched with the positioning analysis program, and the positioning data pcap package is analyzed by the positioning analysis program to obtain the positioning text data; the radar data pcap package is matched with the radar analysis program, and the radar data pcap package and the positioning data after motion compensation are analyzed by the radar analysis program to obtain the pcd point cloud; the video data is matched with the video to picture program, and the video is converted into a picture by the video to picture program.

[0087] The above step of parsing the original running status data set is to parse the original running status data set into a data set that is easy to add annotations and easy to be recognized by the reference model, which can reduce the computing power of the model.

[0088] Step A3: clean the reference running status data set to obtain the data set to be sub-packaged.

[0089] Specifically, the reference operating status data set is filtered according to the timestamp in the scene label. After the filtering is completed, the filtered reference operating status data set is extracted according to the preset data volume, and the coordinate transformation is performed, and the data set after the coordinate transformation is used as the data set to be subpackaged.

[0090] Step A4: segment the data set to be subpackaged according to a preset ratio, and perform alignment processing to obtain a target operating status data set of the target vehicle.

[0091] The preset ratio is the size of the cutting data determined in advance according to the requirements.

[0092] Specifically, the data set to be subpackaged is segmented according to a preset ratio to obtain multiple small data sets, and various types of data in the small data sets are time-aligned according to the original timestamps to obtain the target running state data set of the target vehicle.

[0093] The original timestamp is the recording time corresponding to when the original operating status data set is collected.

[0094] The above steps determine the target running status data set, which can eliminate invalid data in the original running status data set and correspond the real-line data in time to ensure that the running status of the target vehicle at a certain moment is described from different angles.

[0095] Optionally, cleaning and classifying the reference running status data set to obtain the data set to be packaged includes steps B1-B2:

[0096] Step B1: Filter the reference running status data to obtain a valid running status data set.

[0097] Specifically, the reference running status data set is filtered according to the timestamp in the scene label to obtain the valid running status data set.

[0098] Step B2: extract the effective operation status data set according to the preset data volume, and perform coordinate system conversion after extraction to obtain the data set to be subpackaged.

[0099] Specifically, the effective operation status data set is extracted according to the preset data volume, the extracted data set is coordinate transformed, and the data set after the coordinate transformation is used as the data set to be packaged.

[0100] Exemplarily, according to the timestamp in the scene label file, the reference running status data set is cleaned to obtain the effective running status data set. The effective running status data set is subjected to frame extraction (extraction) processing according to the preset data volume, and the extracted data is subjected to coordinate transformation, and the coordinates are transformed into a unified coordinate to obtain the data set to be sub-packaged. The data set to be sub-packaged is cut into multiple small data sets according to the preset ratio, and the data in the small data sets are time-correlated according to the original timestamp to obtain the target running status data set.

[0101] For example, Figure 4 As shown, the cloud platform includes a data preprocessing function, which can clean the millimeter wave point cloud, ultrasonic radar data and PCD point cloud according to the timestamp in the scene label file to obtain a valid operation status data set. The effective operation status data set is framed (extracted) according to the preset data volume, and the coordinates of the extracted data are transformed to a unified coordinate to obtain the data set to be sub-packaged. The data set to be sub-packaged is cut into multiple small data sets according to the preset ratio, and the image data and positioning data are matched with the cut small data sets according to the original timestamp to obtain the target operation status data set.

[0102] Optionally, labeling the target running status data set to obtain the target data set includes steps C1-C3:

[0103] Step C1: classify the target operating status data set according to the preset label type to obtain the data set to be labeled.

[0104] Among them, the preset label types can be: vehicles, pedestrians, lane lines and obstacles, etc.

[0105] Specifically, a preset label type is determined, and the target operating status data set is classified according to the preset label type to obtain a data set to be labeled.

[0106] Exemplarily, assuming that the preset label type is pedestrian, the running status data containing pedestrians in the target running status data set is separated through the classification model to obtain the data set to be labeled.

[0107] Step C2: label the dataset to be labeled to obtain a labeled dataset.

[0108] Specifically, the dataset to be labeled is labeled according to the labeling model to obtain a labeled dataset.

[0109] Furthermore, if Figure 4 As shown, after obtaining the labeled data set, the labeled data set is converted into a point cloud data set (PCD), and the point cloud data set is archived and stored in a database.

[0110] Step C3: verify the annotations of the annotated data set. If the verification passes, export the annotated data set to obtain the target data set.

[0111] Specifically, the annotations in the annotated dataset are verified. If the verification passes, the annotated dataset is exported, and the annotated dataset and the dataset to be annotated before annotation are aggregated to obtain the target dataset. If the verification fails, the failed annotations are marked for manual correction, and the corrected annotations are fed back to the annotation model to correct the annotation model.

[0112] Furthermore, if Figure 4 As shown, before the annotation of the annotated dataset is verified, the annotated dataset needs to be converted into json format (JavaScript Object Notation, a lightweight data exchange format).

[0113] For example, Figure 4 As shown, the labeled data set is converted into json format, and it is determined whether the true value export is correct. If the true value is correct, the true value file is archived and saved in the database, where the true value is the annotation information of the labeled data set; judging whether the true value export is correct is to verify whether the annotations in the labeled data set are correct; the true value file is archived and the labeled data set is archived.

[0114] Furthermore, the true value file archive is organized according to a determined folder organization method. For example, the naming format of the folder organization method can be: vehicle-date-weather-first-level scene-second-level scene-third-level scene-multi-source acquisition device data, that is, vehicle A-XX year XX month XX day-rainy day-B area-C street-D supermarket entrance-ultrasonic data.

[0115] In the above step, the target operation data set is labeled in order to distinguish the information contained in the target operation status data set.

[0116] Optionally, matching a corresponding reference model from a preset model library according to the target task type includes steps D1-D2:

[0117] Step D1: Match the corresponding reference model type from the historical processing information according to the target task type.

[0118] Exemplarily, assuming that the target task type is moving vehicle detection, and assuming that the CenterNet network is used for moving vehicle detection task processing in historical processing information, the matched reference model type is the CenterNet network.

[0119] Exemplarily, assuming that the target task type is pedestrian detection, and assuming that the Yolo algorithm and the HyperLearner algorithm are used in the historical processing information for processing the moving vehicle detection task, the recognition accuracy of the Yolo algorithm and the HyperLearner algorithm are obtained from the historical processing information. Assuming that the HyperLearner algorithm (super learning algorithm) has the highest recognition accuracy, the reference model type is the HyperLearner algorithm.

[0120] Step D2: Match the reference model corresponding to the target task from the preset model library according to the reference model type.

[0121] Exemplarily, assuming that the reference model type is a CenterNet network, a CenterNet network model is matched from a preset model library, and the obtained CenterNet network model is used as a reference model.

[0122] The above steps, based on the target task type, process the information from the reference model of the history, which can ensure that the selected model is suitable for the target task.

[0123] Optionally, selecting data to be trained from the target data set according to the target task type and the reference model includes steps E1-E2:

[0124] Step E1: Determine the reference data type according to the target task type, and select a reference training data set from the target data set according to the reference data type.

[0125] Specifically, the reference data type required to achieve the target task is determined according to the target task type, a data set is selected from the target data set according to the reference data type, and the obtained data set is used as a reference training data set.

[0126] For example, assuming that the target task type is to detect pedestrians in area A in time period XX t1-t2, then the data set corresponding to area A in time period XX t1-t2 is searched from the target data set, and the obtained data set is used as a reference training data set.

[0127] Step E2: intercept and filter the reference training data set through the reference model to obtain the data set to be trained.

[0128] Specifically, the reference model selects the reference training data set according to the proportion of obstacle types, the proportion of each weather scene, the proportion of each road scene, the level of model detection indicators, etc. contained in the reference training data set to obtain the data set to be trained.

[0129] The above steps can ensure the balance of the data to be trained while taking more factors into consideration, thereby improving the generalization ability of the reference model.

[0130] Optionally, the output result is compared with the evaluation result of the true value system, and the reference model is modified according to the comparison result to obtain the target model, including steps F1-F3:

[0131] Step F1: Determine the evaluation result through the truth value system according to the target task type, compare the evaluation result with the output result, and obtain a comparison result.

[0132] The evaluation result may be a result predicted by a general model; or it may be a labeled data set.

[0133] The general model is a pre-built model that can adapt to all scenarios during the driving process of the target vehicle, and the model type is the same as the reference model.

[0134] In the above steps, the construction of the general model is to provide a more reasonable evaluation basis for the reference model. That is, assuming that the target vehicle is active in area A and / or area B most of the time; the general model can ensure that the target vehicle operates in most areas (including area A and area B). However, when the general model processes the operating status of the target vehicle, it cannot fully adapt to area A and / or area B, but the result of the general model can meet the driving needs of the target vehicle, so it can be used as the basis for correcting the reference model.

[0135] Specifically, if the evaluation result is a labeled data set, the truth value system retrieves the corresponding labeled data set from the database according to the target task type, and compares the output result generated by the reference model according to the data set to be trained with the labeled data set to obtain a comparison result. If the evaluation result is a general model result, the truth value system retrieves the general model result corresponding to the target task type and stored in the truth value system according to the target task type, and compares the output result generated by the reference model according to the data set to be trained with the general model result to obtain a comparison result.

[0136] Exemplarily, the evaluation result is a general model result. Assuming that the target task type is to detect pedestrians in area A between time periods t1 and t2, the general model is matched from the true value system to detect the pedestrians in area A between time periods t1 and t2, and the obtained detection results are compared with the output results of the reference model to obtain a comparison result.

[0137] Step F2: If the comparison result meets the preset conditions, the reference model is saved.

[0138] Specifically, if the comparison result meets the preset conditions, it is considered that the reference model can meet the requirements of the target task, and the reference model is stored.

[0139] Step F3: If the comparison result does not meet the preset conditions, the reference model is corrected.

[0140] Specifically, if the comparison result does not meet the preset conditions, it is considered that the reference model cannot meet the requirements of the target task, and the reference model is modified according to the comparison result.

[0141] For example, Figure 4 As shown in the figure, the cloud platform includes a database export training data function, which can search the corresponding target data set from the database according to the target task type, separate the labeled data set and the data set to be labeled in the target data set, and import them into Octopus respectively. Octopus calls the reference model to train the labeled data set and the data set to be labeled, summarizes the training results, obtains the output results, and sends the output results to the truth value system for evaluation. Among them, Octopus is used to provide a training platform for the training of the reference model.

[0142] Furthermore, if Figure 5 As shown, the cloud platform also includes the following functions: algorithm management, algorithm framework support, data set management, model management, model format conversion, automatic definition of training tasks, model evaluation, model compression, Bad case evaluation and cluster management, etc.

[0143] The technical solution of this embodiment determines the target running state data set of the target vehicle, which can ensure that the obtained multi-source original running state data is converted into a unified format, and the sensitive information therein is removed, providing a basis for subsequent processing; the target running state data set is annotated to obtain the target data set, and the data set annotation can improve the generalization ability and operation speed of the target model, and it is also convenient to classify and retrieve the data set; the corresponding reference model is matched from the preset model library according to the target task type, and the target task type can provide a basis for the selection of the reference model, and a more suitable reference model is selected; the data to be trained is selected from the target data set according to the target task type and the reference model, which can make the data to be trained more suitable for the target task and also include a wider range of data types; the reference model is trained by the data to be trained, and the output result is compared with the evaluation result of the truth value system, and the reference model is corrected according to the comparison result to obtain the target model, which can improve the detection ability of the target model. The method trains the reference model according to the multi-source data set of the target vehicle, corrects the reference model by the truth value system, and obtains the target model, which solves the problem that the existing technology cannot comprehensively consider the weather conditions and the road conditions during the driving process of the target vehicle, and improves the detection ability of the target model.

[0144] Figure 6This is a schematic diagram of the structure of a model processing device provided by an embodiment of the present invention. This embodiment can be used to train the model according to the driving environment of the vehicle to ensure that the model fits the actual scene. The model processing device can be implemented in the form of hardware and / or software, and the model processing device can be configured in any electronic device with network communication function. Figure 6 As shown, the device includes: a target operating state data set determination module 210, a target data set determination module 220, a reference model determination module 230 and a correction module 240, wherein:

[0145] The target running state data set determination module 210 is used to determine the target running state data set of the target vehicle. The target running state data set is the data obtained by extracting the reference running state data set according to a preset data amount and dividing it according to a preset ratio. The reference running state data set is the data set obtained by processing the original running state data collected by the multi-source collection device.

[0146] The target data set determination module 220 is used to label the target running status data set to obtain the target data set;

[0147] Reference model determination module 230: used to match the corresponding reference model from the preset model library according to the target task type, and select the data to be trained from the target data set according to the target task type and the reference model. The target task type is used to characterize the type of action that the target vehicle will complete during operation, and the reference model is used to provide auxiliary information for the completion of the target task;

[0148] Correction module 240: used to train the reference model with the training data, compare the output result with the evaluation result of the true value system, and correct the reference model according to the comparison result to obtain the target model.

[0149] Optionally, the target operating status data set determination module 210 includes:

[0150] Original data acquisition unit: used to acquire the original running status data set of the target vehicle;

[0151] Reference running state data set determination unit: used to perform desensitization processing and data analysis on the original running state data set to obtain the reference running state data set. The desensitization processing is used to remove sensitive data in the original running state data set. The sensitive data is data that can expose driver information and geographic entity information. The data analysis is used to convert the original running data into a format that can be recognized by the model.

[0152] The unit for determining the data set to be sub-packaged is used to clean the reference running status data set to obtain the data set to be sub-packaged;

[0153] The target running state data set determination unit is used to segment the data set to be subpackaged according to a preset ratio and perform alignment processing to obtain the target running state data set of the target vehicle.

[0154] Optionally, the unit for determining the data set to be subcontracted includes:

[0155] The effective operation status data set determining unit is used to filter the reference operation status data to obtain the effective operation status data set;

[0156] The unit for determining the data to be sub-packaged is used to extract the effective operation status data set according to the preset data volume, and perform coordinate system conversion after extraction to obtain the data set to be sub-packaged.

[0157] Optionally, the target data set determination module 220 includes:

[0158] The to-be-labeled data set determination unit is used to classify the target operating status data set according to the preset label type to obtain the to-be-labeled data set;

[0159] A labeled data set determining unit: used for labeling the data set to be labeled to obtain a labeled data set;

[0160] Target data set determination unit: used to verify the annotation of the annotated data set. If the verification passes, the annotated data set is exported to obtain the target data set.

[0161] Optionally, the reference model determination module 230 includes:

[0162] A reference model type determination unit: used to match the corresponding reference model type from the historical processing information according to the target task type;

[0163] Reference model determination unit: used to match the reference model corresponding to the processing target task from the preset model library according to the reference model type.

[0164] Optionally, the reference model determination module 230 includes:

[0165] A reference training data determination unit is used to determine a reference data type according to a target task type, and select a reference training data set from a target data set according to the reference data type;

[0166] The unit for determining the data to be trained is used to intercept and filter the reference training data set through the reference model to obtain the data set to be trained.

[0167] Optionally, the correction module 240 includes:

[0168] Comparison result determination unit: used to determine the evaluation result through the truth value system according to the target task type, compare the evaluation result with the output result, and obtain the comparison result;

[0169] Model saving unit: used to save the reference model if the comparison result meets the preset conditions;

[0170] Correction determination unit: used to correct the reference model if the comparison result does not meet the preset conditions.

[0171] The model processing device provided in the embodiment of the present invention can execute the model processing method provided in any embodiment of the present invention mentioned above, and has the corresponding functions and beneficial effects of executing the model processing method. For detailed process, please refer to the relevant operations of the model processing method in the above embodiment.

[0172] Figure 7 A schematic diagram of the structure of an electronic device for implementing the model processing method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0173] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0174] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0175] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the model processing method.

[0176] In some embodiments, the model processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the model processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the model processing method in any other appropriate manner (e.g., by means of firmware).

[0177] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0178] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0179] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0181] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0182] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0183] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0184] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A model processing method, characterized in that: include: Determine a target operating state data set of the target vehicle, wherein the target operating state data set is data obtained by extracting a reference operating state data set according to a preset data amount and segmenting it according to a preset ratio, and the reference operating state data set is a data set obtained by processing original operating state data collected by a multi-source acquisition device; Annotating the target operating status data set to obtain a target data set; Matching a corresponding reference model from a preset model library according to the target task type, and selecting data to be trained from a target data set according to the target task type and the reference model, wherein the target task type is used to characterize the type of action to be completed by the target vehicle during operation, and the reference model is used to provide auxiliary information for the completion of the target task; The reference model is trained using the data to be trained, and the output result is compared with the evaluation result of the true value system, and the reference model is corrected according to the comparison result to obtain the target model.

2. The method according to claim 1, characterized in that The step of determining a target operating state data set of a target vehicle includes: Obtain the original running status data set of the target vehicle; Performing desensitization processing and data analysis on the original operating status data set to obtain a reference operating status data set, wherein the desensitization processing is used to remove sensitive data in the original operating status data set, wherein the sensitive data is data that can expose driver information and geographic entity information, and the data analysis is used to convert the original operating data into a format that can be recognized by the model; Cleaning the reference running status data set to obtain a data set to be subpackaged; The data set to be packaged is segmented according to a preset ratio and aligned to obtain a target operating state data set of the target vehicle.

3. The method according to claim 2, characterized in that The step of cleaning and classifying the reference running status data set to obtain the data set to be packaged includes: Filtering the reference operating status data to obtain a valid operating status data set; The effective operation status data set is extracted according to a preset data volume, and a coordinate system conversion is performed after the extraction to obtain the data set to be packaged.

4. The method according to claim 1, characterized in that The step of labeling the target operating status data set to obtain the target data set includes: Classify the target operating status data set according to preset label types to obtain a data set to be labeled; Annotating the data set to be annotated to obtain an annotated data set; The annotations of the annotated dataset are verified. If the verification passes, the annotated dataset is exported to obtain the target dataset.

5. The method according to claim 1, characterized in that: The matching of the corresponding reference model from the preset model library according to the target task type includes: Match the corresponding reference model type from the historical processing information according to the target task type; A reference model corresponding to the processing target task is matched from a preset model library according to the reference model type.

6. The method according to claim 1, characterized in that Selecting data to be trained from the target data set according to the target task type and the reference model includes: Determine a reference data type according to the target task type, and select a reference training data set from the target data set according to the reference data type; The reference training data is intercepted and screened through a reference model to obtain data to be trained.

7. The method according to claim 1, characterized in that The output result is compared with the evaluation result of the true value system, and the reference model is modified according to the comparison result to obtain the target model, including: Determine an evaluation result through a truth value system according to the target task type, and compare the evaluation result with the output result to obtain a comparison result; If the comparison result meets the preset conditions, the reference model is saved; If the comparison result does not meet the preset condition, the reference model is corrected.

8. A model processing device, characterized in that: include: A target operating state data set determination module is used to determine a target operating state data set of a target vehicle, wherein the target operating state data set is data extracted from a reference operating state data set according to a preset data volume and segmented according to a preset ratio, and the reference operating state data set is a data set obtained by processing original operating state data collected by a multi-source acquisition device; A target data set determination module, used to label the target operating status data set to obtain a target data set; A reference model determination module is used to match the corresponding reference model from a preset model library according to the target task type, and select the data to be trained from the target data set according to the target task type and the reference model, wherein the target task type is used to characterize the type of action to be completed by the target vehicle during operation, and the reference model is used to provide auxiliary information for the completion of the target task; The correction module is used to train the reference model with the data to be trained, compare the output result with the evaluation result of the true value system, and correct the reference model according to the comparison result to obtain the target model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the model processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the model processing method according to any one of claims 1 to 7 when executed.

Images