An on-vehicle data cleaning method and device, electronic equipment and storage medium
By generating an autonomous driving status filtering rule table and filtering vehicle status log files, the problem of poor data cleaning quality on the vehicle was solved, achieving efficient and refined data filtering and cost reduction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WHITE RHINO ZHIDA (BEIJING) TECH CO LTD
- Filing Date
- 2022-09-06
- Publication Date
- 2026-05-19
AI Technical Summary
Existing onboard data cleaning methods are of poor quality, resulting in low-quality data being filtered out, with a large amount of uninterested data or important data being lost.
By reading vehicle status log files, an autonomous driving status filtering rule table is generated, and this table is used to filter bag files, thereby refining the data filtering and reducing human intervention.
It enables fine-grained filtering of bag files, improves data cleaning quality, and reduces labor costs.
Smart Images
Figure CN115422178B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data cleaning technology, and in particular to a method, apparatus, electronic device, and storage medium for cleaning data on a vehicle. Background Technology
[0002] Currently, the combination of autonomous driving and data closed loop has become a major solution because autonomous driving engineering has been recognized as a task to solve the "long tail problem" of data distribution. The occasional corner cases (extreme cases) are one of the sources for upgrading data-driven algorithm models.
[0003] What are the core technologies and modules that constitute this autonomous driving data closed loop? First, the autonomous driving algorithms and modules are data-driven; second, the continuous flow of data needs to be utilized in a reasonable and effective way.
[0004] On the data side, sample collection focuses on two key elements: massive volume and high quality. Ideally, data collection should include both real vehicle data and simulation data, with the amount of real vehicle data equal to the number of real vehicles multiplied by the collection time.
[0005] Real-world data for autonomous driving is typically stored in BAT files, which collect topic data from various modules, such as data frames from LiDAR and cameras, point cloud data frames, and the vehicle's chassis status. BAT files are very large; based on real-world testing, they typically contain several GB per minute. For example, with a data rate of 2 GB per minute, and considering unmanned delivery vehicles operating for approximately 10 hours per day (including parking, loading, and unloading time), the total data volume would be around 1.2 TB.
[0006] In existing technologies, all data is usually saved first, and then data from a certain period or certain topics is trimmed to reduce the amount of data. However, this method is relatively mechanical, and the quality of the filtered data is not high. In addition to the possibility of a large amount of data that is not of interest, important data may also be lost. Therefore, human intervention is needed to improve the quality of the cleaned data. Summary of the Invention
[0007] This invention provides a method, apparatus, electronic device, and storage medium for cleaning up in-vehicle data, which addresses the technical problem of poor quality in existing in-vehicle data cleaning methods.
[0008] This invention provides a method for cleaning up in-vehicle data, applied to an autonomous driving system; the autonomous driving system stores an autonomous driving state filtering rule table; the method includes:
[0009] Read the vehicle status log file;
[0010] A filter table is generated using the aforementioned autonomous driving state filtering rule table and the vehicle state log file;
[0011] Get the bag file;
[0012] The bag file is filtered using the filter table to obtain the target bag file, and then the bag file is deleted.
[0013] Optionally, the step of generating a filter table using the autonomous driving state filtering rules and the vehicle state log file includes:
[0014] Read unprocessed vehicle status logs from the vehicle status log file;
[0015] Match the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table;
[0016] If a match is successful, a filter table record is generated using the target autonomous driving state filtering rule and the unprocessed vehicle state log.
[0017] Determine whether there are any unprocessed vehicle status logs in the vehicle status log file;
[0018] If so, return to the step of reading unprocessed vehicle status logs from the vehicle status log file;
[0019] If not, generate a filter table using all the aforementioned filter table records.
[0020] Optionally, the autonomous driving state filtering rule table records multiple autonomous driving state filtering rules; the step of matching the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table includes:
[0021] Obtain the first initial state before the state change and the first target state after the state change from the unprocessed vehicle state log;
[0022] Obtain the second initial state and the second target state from the autonomous driving state filtering rules;
[0023] The autonomous driving state filtering rule that is the same as the first initial state and the same as the second target state is determined as the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log.
[0024] Optionally, it also includes:
[0025] If no target autonomous driving state filtering rule is found in the autonomous driving state filtering rule table for the unprocessed vehicle state log, the unprocessed vehicle state log is determined as a processed vehicle state log, and the step of reading the unprocessed vehicle state log from the vehicle state log file is returned.
[0026] Optionally, the step of generating a filter table record using the target autonomous driving state filtering rule and the unprocessed vehicle state log includes:
[0027] Obtain the switching time from the unprocessed vehicle status log;
[0028] Obtain the collection start time, collection end time, and collection list from the target autonomous driving state filtering rules;
[0029] The start time is calculated using the switching time and the acquisition start time;
[0030] The end time is calculated using the switching time and the acquisition end time;
[0031] A filter table record is generated using the start time, the end time, and the collection list.
[0032] The present invention also provides an in-vehicle data cleaning device for use in an autonomous driving system; the autonomous driving system stores an autonomous driving state filtering rule table; the device includes:
[0033] The vehicle status log file reading module is used to read the vehicle status log file;
[0034] The filter table generation module is used to generate a filter table using the autonomous driving state filter rule table and the vehicle state log file.
[0035] The bag file retrieval module is used to retrieve bag files;
[0036] The filtering module is used to filter the bag file using the filtering table to obtain the target bag file and delete the bag file.
[0037] Optionally, the filter table generation module includes:
[0038] An unprocessed vehicle status log reading submodule is used to read unprocessed vehicle status logs from the vehicle status log file.
[0039] The matching submodule is used to match the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table.
[0040] The filter table record generation submodule is used to generate a filter table record if a match is successful, using the target autonomous driving state filtering rules and the unprocessed vehicle state log.
[0041] The judgment submodule is used to determine whether there are any unprocessed vehicle status logs in the vehicle status log file;
[0042] The first return submodule is used to return the step of reading unprocessed vehicle status logs from the vehicle status log file if the condition is met.
[0043] The filter table generation submodule is used to generate a filter table using all the filter table records, if not.
[0044] Optionally, the autonomous driving state filtering rule table records multiple autonomous driving state filtering rules; the matching submodule includes:
[0045] The first initial state and first target state acquisition unit is used to acquire the first initial state before the state change and the first target state after the state change from the unprocessed vehicle state log.
[0046] The second initial state and the second target state acquisition unit is used to acquire the second initial state and the second target state from the autonomous driving state filtering rules;
[0047] The target autonomous driving state filtering rule determination unit is used to determine the autonomous driving state filtering rule that has the same second initial state as the first initial state and the same second target state as the second target state as the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log.
[0048] The present invention also provides an electronic device, the device comprising a processor and a memory:
[0049] The memory is used to store program code and transmit the program code to the processor;
[0050] The processor is configured to execute the vehicle data cleanup method as described above, according to the instructions in the program code.
[0051] The present invention also provides a computer-readable storage medium for storing program code for performing the on-vehicle data cleanup method as described in any of the preceding claims.
[0052] As can be seen from the above technical solutions, the present invention has the following advantages: The present invention discloses a method for cleaning up vehicle data, including: reading a vehicle status log file; generating a filter table using an autonomous driving status filter rule table and the vehicle status log file; obtaining a bag file; filtering the bag file using the filter table to obtain a target bag file, and deleting the bag file.
[0053] This invention generates a filter table using vehicle status log files and a set autonomous driving status filtering rule table. The filter table is then used to filter bag files, thereby achieving fine-grained filtering of bag files, reducing manual intervention, and lowering labor costs. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0055] Figure 1 A flowchart of an existing vendor data closed loop;
[0056] Figure 2 A flowchart for another existing vendor data closed loop;
[0057] Figure 3 A flowchart illustrating the steps of an on-vehicle data cleaning method provided in an embodiment of the present invention;
[0058] Figure 4 This is a state transition diagram of an autonomous vehicle during operation.
[0059] Figure 5 A flowchart illustrating the steps of a vehicle data cleaning method according to another embodiment of the present invention;
[0060] Figure 6 This is a structural block diagram of an on-vehicle data cleaning device provided in an embodiment of the present invention. Detailed Implementation
[0061] Currently, the combination of autonomous driving and data closed loop has become a major solution. To construct this autonomous driving data closed loop, one first needs data-driven autonomous driving algorithms and modules, and secondly, one needs reasonable and effective methods to utilize the continuous flow of data.
[0062] Taking the manufacturer's data closed loop as an example:
[0063] like Figure 1 As shown, Figure 1This is a flowchart of a vendor's data closed loop. After obtaining the data source, the vendor confirms the model error through unit testing, obtains a large amount of effective data through data augmentation and amplification technology, and performs data labeling. After multiple rounds of data cleaning, the model is trained and finally deployed.
[0064] like Figure 2 As shown, Figure 2 This is a flowchart of another type of manufacturer data closed loop. After obtaining data from real vehicles, manufacturers use it to complete intelligent data filtering, data annotation, model search, training, evaluation, debugging and deployment.
[0065] However, the above methods all require saving all the data collected from the actual vehicle first, and then cutting off the data for a certain period of time or certain topics to reduce the amount of data. However, this method is relatively mechanical, and the quality of the filtered data is not good. Either there is a lot of data that is not relevant, or important data is lost.
[0066] In view of this, embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for cleaning up in-vehicle data, which solves the technical problem of poor quality in existing in-vehicle data cleaning methods.
[0067] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0068] Please see Figure 3 , Figure 3 A flowchart illustrating the steps of an on-vehicle data cleaning method provided in an embodiment of the present invention.
[0069] This invention provides a method for cleaning up in-vehicle data, applied to an autonomous driving system; the autonomous driving system stores an autonomous driving state filtering rule table; specifically, it may include the following steps:
[0070] Step 301: Read the vehicle status log file;
[0071] A vehicle status log file is a log file that records the vehicle status at different times. Different vehicle statuses can be set according to different driving needs. During the driving process, the autonomous driving system will automatically collect the vehicle status to form a vehicle status log.
[0072] To facilitate understanding, the following examples illustrate vehicle status:
[0073] Please see Figure 4 , Figure 4 This is a state transition diagram for an autonomous vehicle during operation. For example... Figure 4 As shown, the driverless car experienced different vehicle states during operation, including:
[0074] LOCATE_VEHICLE: The initial state of the autonomous vehicle, and its position on the map before it departs.
[0075] FORWARD_DRIVE: This state includes execution, lane keeping, and obstacle avoidance. When the autonomous vehicle is not in a parking lot (i.e., in an area without open roads), the autonomous vehicle state machine will choose this state first.
[0076] STOP_SIGN_WAIT: This state is entered when an autonomous vehicle is waiting at a stop sign.
[0077] CROSS_INTERSECTION: In this state, the autonomous vehicle will wait until it is confirmed that it is safe to cross the intersection.
[0078] UTURN_DRIVE: This state is invoked when making a U-turn.
[0079] UTURN_STOP: The parking status before making a U-turn.
[0080] CROSS_DIVIDER: The state of driving across the yellow line.
[0081] PARKING_NAVIGATE: Normal driving mode in the parking lot.
[0082] TRAFFIC_JAM and ESCAPE: Two states for handling traffic congestion.
[0083] BAD_RNDF: This state is entered when the current road is different from the pre-made road network map. In this state, the autonomous vehicle will use the hybrid A* algorithm to complete the vehicle's path planning.
[0084] MISSION_COMPLETE: When the DARPA challenge ends, the autonomous vehicle enters this state, which is the final state of the entire state machine.
[0085] In one example, the format of the vehicle status log file can be shown in Table 1 below:
[0086]
[0087] Table 1
[0088] This means that each time an autonomous vehicle undergoes a state transition, a log entry can be recorded. Each log entry includes: a timestamp of the vehicle state transition (format: YYYY-MM-DD HH:MM:SS), the original state (from state), and the new state (to state). Taking the data in Table 2 as an example, at 10:23:14 on January 1, 2022, the vehicle transitioned from the LOCATE_VEHICLE state to the FORWARD_DRIVE state, remained in the FORWARD_DRIVE state until 10:32:14 on January 1, 2022, then transitioned to the CROSS_DIVIDER state, and finally transitioned back to the FORWARD_DRIVE state at 10:32:24 on January 1, 2022.
[0089] Step 302: Generate a filter table using the autonomous driving state filter rule table and the vehicle state log file;
[0090] In this embodiment of the invention, the autonomous driving state filtering rules can be set by technicians according to actual needs. In one example, the autonomous driving state filtering rules can be shown in Table 2 below:
[0091]
[0092]
[0093] Table 2
[0094] Each autonomous driving state filtering rule contains five fields: the original state of the vehicle (from state), the new state of the vehicle (to state), the sample start time, the sample end time, and the list of topics to be monitored. As shown in Table 2, the explanations of each autonomous driving state filtering rule are as follows:
[0095] The first autonomous driving state filtering rule: Collect / camera_image data for 5 seconds (out of a total of 10 seconds) before and after the LOCATE_VEHICLE state switches to FORWARD_DRIVE state.
[0096] The second autonomous driving state filtering rule: Collect / camera_image, / ivhdmap, and / path data for 10 seconds (out of a total of 20 seconds) before and after switching from any state (ANY_STATE) to CROSS_DIVIDER state.
[0097] In this embodiment of the invention, after obtaining the vehicle status log file, a filter table can be generated by combining it with a preset autonomous driving status filter rule table.
[0098] Step 303, obtain the bag file;
[0099] Step 304: Filter the bag files using a filter table to obtain the target bag files, and then delete the bag files.
[0100] bag files: bag files are a common data storage format in ROS (Robot Operating System), used to store message data of topics sent by other ROS nodes in the ROS system.
[0101] The topic is the name of the information being transmitted. Whether a node is a publisher or a subscriber, they are unaware of each other's activities. When they need information, they all look for the topic name. Multiple nodes can publish information to the same topic. Similarly, multiple nodes can subscribe to the same topic.
[0102] A message is a data format for information transmitted between nodes. ROS supports standard data types, as well as nested data types and custom message types. ROS itself also defines a complete set of data formats for representing robots, including velocity, pose, trajectory, point cloud, and so on.
[0103] After generating the filter table, each bag file can be read sequentially using the bag file filter, and the bag files can be filtered using the filter table to obtain the target bag file, while the original bag files are cleaned up.
[0104] This invention generates a filter table using vehicle status log files and a set autonomous driving status filtering rule table. The filter table is then used to filter bag files, thereby achieving fine-grained filtering of bag files, reducing manual intervention, and lowering labor costs.
[0105] Please see Figure 5 , Figure 5 A flowchart illustrating the steps of a vehicle data cleanup method according to another embodiment of the present invention. Specifically, it may include the following steps:
[0106] Step 501: Read the vehicle status log file;
[0107] Step 501 is the same as step 301. For details, please refer to the description of step 301. It will not be repeated here.
[0108] Step 502: Read unprocessed vehicle status logs from the vehicle status log file;
[0109] Step 503: Match the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table;
[0110] In this embodiment of the invention, unprocessed vehicle status logs in the vehicle status log file can be obtained sequentially, and then the target autonomous driving status filtering rule corresponding to the unprocessed vehicle status log can be matched in the autonomous driving status filtering rule table.
[0111] In one example, the autonomous driving state filtering rule table records multiple autonomous driving state filtering rules; step 503 may include the following sub-steps:
[0112] S31, obtain the first initial state before the state change and the first target state after the state change from the unprocessed vehicle state log;
[0113] S32, Obtain the second initial state and the second target state from the autonomous driving state filtering rules;
[0114] S33, the autonomous driving state filtering rule that is the same as the first initial state and the second target state is the same as the second target state is determined as the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log.
[0115] In this embodiment of the invention, a first initial state before the state change (i.e., the original state before the vehicle state switch recorded in the vehicle state log) and a first target state after the change (i.e., the new state after the vehicle state switch recorded in the vehicle state log) can be obtained from the unprocessed vehicle state log. Then, the first initial state and the first target state of the unprocessed vehicle state log are sequentially matched with the second initial state (i.e., the original state after the vehicle state switch) and the second target state (the new state after the vehicle state switch) in each autonomous driving state filtering rule in the autonomous driving state filtering rule table. When an autonomous driving state filtering rule is matched where the second initial state is the same as the first initial state and the second target state is the same as the first target state, this autonomous driving state filtering rule is taken as the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log.
[0116] Step 504: If the match is successful, a filter table record is generated using the target autonomous driving state filtering rules and unprocessed vehicle state logs.
[0117] In this embodiment of the invention, when a corresponding target autonomous driving state filtering rule is matched, a filtering table record can be generated using the target autonomous driving state filtering rule and unprocessed vehicle state logs.
[0118] In one example, step 504 may include the following sub-steps:
[0119] S41, obtain the switchover time from the unprocessed vehicle status log;
[0120] S42, obtain the collection start time, collection end time and collection list from the target autonomous driving state filtering rules;
[0121] S43, calculate the start time using the switching time and the acquisition start time;
[0122] S44, calculate the end time using the switching time and the data acquisition end time;
[0123] S45 uses start time, end time, and collection list to generate filter table records.
[0124] In this embodiment of the invention, the unprocessed vehicle status log records the switching time (i.e., the timestamps of the vehicle status switching). After obtaining the sample start time (samplebegin) and sample end time (sample end) from the autonomous driving status filtering rules, the start time of the filter table record can be calculated based on the switching time and the sample start time, and the end time of the filter table record can be calculated based on the switching time and the sample end time. Finally, the start time, end time, and the sample list in the target autonomous driving status filtering rule (the topic list in the target autonomous driving status filtering rule) are combined to form a complete filter table record.
[0125] Furthermore, in this embodiment of the invention, if no target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log is found in the autonomous driving state filtering rule table, the unprocessed vehicle state log is determined as a processed vehicle state log, and the step of reading the unprocessed vehicle state log from the vehicle state log file is returned.
[0126] Step 505: Determine whether there are any unprocessed vehicle status logs in the vehicle status log file;
[0127] Step 506: If yes, return to the step of reading unprocessed vehicle status logs from the vehicle status log file;
[0128] Step 507: If not, generate a filter table using all filter table records;
[0129] After a new filter table record is generated, it can be determined whether there are any unprocessed vehicle status logs in the vehicle status log file. If so, return to step 502, reread an unprocessed vehicle status log, and generate a new filter table record. If all vehicle status logs in the vehicle status log file have been processed, then all filter table records are used to generate the filter table.
[0130] In one example, the filter table generated by combining Table 1 and Table 2 is shown in Table 3 below:
[0131]
[0132] Table 3
[0133] Each row in the filter table represents a filter table record, and each filter table record includes three fields: start time (begin timestamps), end time (end timestamps), and topic list. As shown in Table 3, the meaning of each filter table record is as follows:
[0134] The first filter record: filters the / camera_image data between 10:23:09 on 2022-01-01 and 10:23:19 on 2022-01-01.
[0135] The second filter table record: Filter out the data for / camera_image, / ivhdmap, and / path between 10:32:04 on 2022-01-01 and 10:32:24 on 2022-01-01.
[0136] To facilitate understanding, the embodiments of the present invention will be described below through specific examples:
[0137] First, the first vehicle status log entry is read from the vehicle status log file in Table 1, indicating that the vehicle switched from the LOCATE_VEHICLE state to the FORWARD_DRIVE state at 10:23:14 on 2022-01-01. Then, the autonomous driving state filtering rule table in Table 2 is searched to find the matching autonomous driving state filtering rule, which is: collect / camera_image data for 5 seconds (a total of 10 seconds) before and after the switch from LOCATE_VEHICLE state to FORWARD_DRIVE state. The matching condition is that the from state and to state records in the two tables match (it is worth noting that ANY_STATE is a virtual state that can match any vehicle). Therefore, a filter table record can be generated, that is, to filter out the / camera_image data between 10:23:09 on 2022-01-01 and 10:23:19 on 2022-01-01; while begin timestamps and end timestamps are obtained by adding the timestamps recorded in the vehicle status log to sample begin and sample end in the autonomous driving status filtering rules.
[0138] Step 508, obtain the bag file;
[0139] Step 509: Filter the bag files using a filter table to obtain the target bag files, and then delete the bag files.
[0140] After generating the filter table, each bag file can be read sequentially using the bag file filter, and the bag files can be filtered using the filter table to obtain the target bag file, while the original bag files are cleaned up.
[0141] This invention generates a filter table using vehicle status log files and a set autonomous driving status filtering rule table. The filter table is then used to filter bag files, thereby achieving fine-grained filtering of bag files, reducing manual intervention, and lowering labor costs.
[0142] Please see Figure 6 , Figure 6 This is a structural block diagram of an on-vehicle data cleaning device provided in an embodiment of the present invention.
[0143] This invention provides an on-vehicle data cleaning device for use in an autonomous driving system; the autonomous driving system stores an autonomous driving state filtering rule table; the device includes:
[0144] The vehicle status log file reading module 601 is used to read the vehicle status log file;
[0145] The filter table generation module 602 is used to generate a filter table using the autonomous driving state filter rule table and the vehicle state log file.
[0146] The bag file acquisition module 603 is used to acquire bag files;
[0147] The filtering module 604 is used to filter bag files using a filter table, obtain the target bag file, and delete the bag file.
[0148] In this embodiment of the invention, the filter table generation module 602 includes:
[0149] The unprocessed vehicle status log reading submodule is used to read unprocessed vehicle status logs from the vehicle status log file;
[0150] The matching submodule is used to match the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table.
[0151] The filter table record generation submodule is used to generate filter table records based on the target autonomous driving state filtering rules and unprocessed vehicle state logs if a match is successful.
[0152] The judgment submodule is used to determine whether there are any unprocessed vehicle status logs in the vehicle status log file;
[0153] The first return submodule is used to return the steps of reading unprocessed vehicle status logs from the vehicle status log file if the condition is met.
[0154] The filter table generation submodule is used to generate a filter table from all filter table records if no.
[0155] In this embodiment of the invention, the autonomous driving state filtering rule table records multiple autonomous driving state filtering rules; the matching submodule includes:
[0156] The first initial state and first target state acquisition unit is used to acquire the first initial state before the state change and the first target state after the state change from the unprocessed vehicle state log.
[0157] The second initial state and the second target state acquisition unit is used to acquire the second initial state and the second target state from the autonomous driving state filtering rules;
[0158] The target autonomous driving state filtering rule determination unit is used to determine the autonomous driving state filtering rule that is the same as the first initial state and the second target state is the same as the second target state as the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log.
[0159] In this embodiment of the invention, the filter table generation module 602 further includes:
[0160] If no target autonomous driving state filtering rule is found in the autonomous driving state filtering rule table for the unprocessed vehicle state log, the unprocessed vehicle state log is determined as a processed vehicle state log, and the step of reading the unprocessed vehicle state log from the vehicle state log file is returned.
[0161] In this embodiment of the invention, the filter table record generation submodule includes:
[0162] The switching time acquisition unit is used to obtain the switching time from the unprocessed vehicle status log;
[0163] The data collection start time, data collection end time, and data collection list acquisition unit is used to obtain the data collection start time, data collection end time, and data collection list from the target autonomous driving state filtering rules;
[0164] The start time calculation unit is used to calculate the start time using the switching time and the acquisition start time.
[0165] The end time calculation unit is used to calculate the end time using the switching time and the acquisition end time.
[0166] The filter table record generation unit is used to generate filter table records using start time, end time, and collection list.
[0167] This invention also provides an electronic device, which includes a processor and a memory:
[0168] The memory is used to store program code and transfer the program code to the processor;
[0169] The processor is used to execute the vehicle data cleaning method of this invention according to the instructions in the program code.
[0170] This invention also provides a computer-readable storage medium for storing program code for executing the vehicle data cleanup method of this invention.
[0171] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0173] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0174] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0175] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0177] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.
[0178] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0179] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for cleaning up data on a vehicle, characterized in that, The method is applied to an autonomous driving system; the autonomous driving system stores an autonomous driving state filtering rule table; the method includes: Read the vehicle status log file; A filter table is generated using the aforementioned autonomous driving state filtering rule table and the vehicle state log file; Get the bag file; The bag file is filtered using the filter table to obtain the target bag file, and then the bag file is deleted. The step of generating a filter table using the autonomous driving state filtering rules and the vehicle state log file includes: Read unprocessed vehicle status logs from the vehicle status log file; Match the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table; If a match is successful, a filter table record is generated using the target autonomous driving state filtering rule and the unprocessed vehicle state log. Determine whether there are any unprocessed vehicle status logs in the vehicle status log file; If so, return to the step of reading unprocessed vehicle status logs from the vehicle status log file; If not, generate a filter table using all the aforementioned filter table records.
2. The method according to claim 1, characterized in that, The autonomous driving state filtering rule table records multiple autonomous driving state filtering rules; the step of matching the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table includes: Obtain the first initial state before the state change and the first target state after the state change from the unprocessed vehicle state log; Obtain the second initial state and the second target state from the autonomous driving state filtering rules; The autonomous driving state filtering rule that is the same as the first initial state and the same as the second target state is determined as the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log.
3. The method according to claim 1, characterized in that, Also includes: If no target autonomous driving state filtering rule is found in the autonomous driving state filtering rule table for the unprocessed vehicle state log, the unprocessed vehicle state log is determined as a processed vehicle state log, and the step of reading the unprocessed vehicle state log from the vehicle state log file is returned.
4. The method according to claim 1, characterized in that, The step of generating a filter table record using the target autonomous driving state filtering rules and the unprocessed vehicle state log includes: Obtain the switching time from the unprocessed vehicle status log; Obtain the collection start time, collection end time, and collection list from the target autonomous driving state filtering rules; The start time is calculated using the switching time and the acquisition start time; The end time is calculated using the switching time and the acquisition end time; A filter table record is generated using the start time, the end time, and the collection list.
5. A vehicle data cleaning device, characterized in that, The device is applied to an autonomous driving system; the autonomous driving system stores an autonomous driving state filtering rule table; the device includes: The vehicle status log file reading module is used to read the vehicle status log file; The filter table generation module is used to generate a filter table using the autonomous driving state filter rule table and the vehicle state log file. The bag file retrieval module is used to retrieve bag files; The filtering module is used to filter the bag file using the filtering table to obtain the target bag file and delete the bag file. The filter table generation module includes: An unprocessed vehicle status log reading submodule is used to read unprocessed vehicle status logs from the vehicle status log file. The matching submodule is used to match the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log in the autonomous driving state filtering rule table. The filter table record generation submodule is used to generate a filter table record if a match is successful, using the target autonomous driving state filtering rules and the unprocessed vehicle state log. The judgment submodule is used to determine whether there are any unprocessed vehicle status logs in the vehicle status log file; The first return submodule is used to return the step of reading unprocessed vehicle status logs from the vehicle status log file if the condition is met. The filter table generation submodule is used to generate a filter table using all the filter table records, if not.
6. The apparatus according to claim 5, characterized in that, The autonomous driving state filtering rule table records multiple autonomous driving state filtering rules; the matching submodule includes: The first initial state and first target state acquisition unit is used to acquire the first initial state before the state change and the first target state after the state change from the unprocessed vehicle state log. The second initial state and the second target state acquisition unit is used to acquire the second initial state and the second target state from the autonomous driving state filtering rules; The target autonomous driving state filtering rule determination unit is used to determine the autonomous driving state filtering rule that has the same second initial state as the first initial state and the same second target state as the second target state as the target autonomous driving state filtering rule corresponding to the unprocessed vehicle state log.
7. An electronic device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the vehicle data cleaning method according to any one of claims 1-4 according to the instructions in the program code.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for performing the on-vehicle data cleanup method according to any one of claims 1-4.