A Plog file reading method and system

Abstract

The present invention discloses a Plog file reading method and system. When a user needs to obtain a Plog file at a time point of interest, the system automatically obtains vehicle ID and start time meta information according to a Plog file list provided by the user, stores the meta information in a tree structure, and splices different Plog files in the tree storage structure according to vehicle ID and time sequence. Then, according to the vehicle ID and time frame information to be searched input by the user, the Plog file stored in the frame is found from the tree structure. The Plog file is loaded in a Plog file loading mode according to whether the number of files loaded in the memory exceeds an upper limit. The frame is found in the Plog file and returned to the user, thereby realizing the reading of the Plog file with as little memory as possible, so that the technical problem that the existing offline reading of Plog files requires a relatively large memory space is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle PNC offline data processing, and in particular to a Plog file reading method and system. Background Art

[0002] Autonomous vehicles are equipped with a variety of monitoring devices, such as millimeter-wave radar, lidar, ultrasonic radar, and cameras, which can acquire real-time monitoring information. The perception module integrates and processes the monitoring information obtained by these various monitoring devices to obtain environmental information surrounding the autonomous vehicle. The PNC (Plan and Control) module generates PNC information based on the environmental information output by the perception module. When the vehicle is online, the onboard PNC automatically records its status and writes it to a Plog file (a serialized file format for the PNC's internal status). When the vehicle is offline, administrators often need to read the Plog file for a long time to process the data. Because Plog files are quite large, reading them requires a large amount of memory space, which can lead to slow responses and affect system stability. Therefore, how to read Plog files using the minimum amount of memory is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0003] The embodiments of the present invention provide a Plog file reading method and system, which use as little memory as possible to read the Plog file, thereby solving the technical problem that the existing offline reading of the Plog file requires a relatively large memory space.

[0004] In view of this, a first aspect of the present invention provides a method for reading a Plog file, the method comprising:

[0005] Get the target vehicle ID input by the user and the frame time corresponding to the target reading frame;

[0006] According to the target vehicle ID and the frame time corresponding to the target read frame, the corresponding meta information is searched in the tree structure storing the Plog file to determine the target Plog file where the target read frame of the target vehicle ID is located, wherein the meta information includes the vehicle ID and the start time of the Plog file;

[0007] Check whether the target Plog file has been loaded. If not, load the target Plog file based on whether the number of files loaded in the memory exceeds the upper limit, search for the target read frame from the target Plog file and return it to the user.

[0008] Optionally, based on whether the number of files loaded in the memory exceeds the upper limit, a target Plog file is loaded, and a target read frame is searched from the target Plog file and returned to the user, including:

[0009] When the number of files loaded into the memory does not exceed the upper limit, the target Plog file is loaded directly, and the target read frame is searched from the target Plog file and returned to the user;

[0010] When the number of files loaded in the memory exceeds the upper limit, all loaded files are scanned, unused files are deleted from the memory, and then the target Plog file is loaded, and the target read frame is searched from the target Plog file and returned to the user.

[0011] Optionally, before obtaining the target vehicle ID input by the user and the frame time corresponding to the target reading frame, the method further includes:

[0012] Get the Plog list to be read. The Plog file name in the Plog list contains the vehicle ID and the start time of the Plog file;

[0013] Scan the Plog list and obtain meta information from the Plog file name in the Plog list;

[0014] The metadata is stored in a two-layer tree structure in an orderly manner. The first layer of the tree structure stores the vehicle ID, and the second layer stores the start time of the Plog file.

[0015] Optionally, an LRU algorithm is used to delete unused files from memory.

[0016] Optionally, before detecting whether the target Plog file has been loaded, the following steps are further included:

[0017] Enables a read-write lock on each frame in the Plog files in the Plog list.

[0018] Optionally, it also includes:

[0019] In offline state, simulate the internal state of the PNC of the target vehicle ID and obtain the Plog file corresponding to the simulation;

[0020] Compare the Plog file corresponding to the target vehicle ID simulation with the Plog file written by the online PNC of the target vehicle ID, and output the comparison result.

[0021] A second aspect of the present invention provides a Plog file reading system, the system comprising:

[0022] A target information acquisition module is used to obtain the target vehicle ID input by the user and the frame time corresponding to the target reading frame;

[0023] a target Plog file search module, configured to search for corresponding meta information in a tree structure storing Plog files based on a target vehicle ID and a frame time corresponding to a target read frame, and determine a target Plog file where a target read frame corresponding to the target vehicle ID is located, wherein the meta information includes the vehicle ID and the start time of the Plog file;

[0024] The loading detection module is used to detect whether the target Plog file has been loaded. If not, the target Plog file is loaded according to whether the number of files loaded in the memory exceeds the upper limit, and the target reading frame is searched from the target Plog file and returned to the user.

[0025] Optionally, the loading detection module is specifically used to:

[0026] When the number of files loaded into the memory does not exceed the upper limit, the target Plog file is loaded directly, and the target read frame is searched from the target Plog file and returned to the user;

[0027] When the number of files loaded in the memory exceeds the upper limit, all loaded files are scanned, unused files are deleted from the memory, and then the target Plog file is loaded, and the target read frame is searched from the target Plog file and returned to the user.

[0028] Optionally, it also includes:

[0029] The Plog list acquisition module is used to obtain the Plog list to be read. The Plog file name in the Plog list contains the vehicle ID and the start time of the Plog file;

[0030] Scanning module, used to scan the Plog list and obtain meta information from the Plog file name in the Plog list;

[0031] The meta information storage module is used to store the meta information in a two-layer tree structure in an orderly manner. The first layer of the tree structure stores the vehicle ID, and the second layer stores the start time of the Plog file.

[0032] Optionally, a read-write lock module is also included;

[0033] The read-write lock module is used to enable a read-write lock for each frame in the Plog file in the Plog list before detecting whether the target Plog file has been loaded.

[0034] It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:

[0035] In the Plog file reading method provided in an embodiment of the present invention, when a vehicle is online, the vehicle's PNC automatically records its own status and writes it into a Plog file, which is stored in a database. When a user needs to obtain the Plog file of a time point of interest, the system automatically obtains the vehicle ID and start time metadata based on the Plog file list provided by the user, stores the metadata in a tree structure, and splices different Plog files in the tree storage structure based on the vehicle ID and time sequence. Then, based on the vehicle ID and time frame information to be searched, the Plog file stored in the frame is searched from the tree structure. The system detects whether the number of files loaded in the memory exceeds an upper limit, and selects a PLog file loading method based on whether the number of files loaded in the memory exceeds the upper limit to load the Plog file. The frame is found in the Plog file and returned to the user, thereby achieving the goal of reading the Plog file with as little memory as possible, thereby solving the technical problem that the existing offline reading of Plog files requires a relatively large memory space. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic flow chart of a Plog file reading method provided in an embodiment of the present invention;

[0037] Figure 2 Another schematic flow chart of a Plog file reading method provided in an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of the Plog file storage structure provided in an embodiment of the present invention;

[0039] Figure 4 The figure is a schematic structural diagram of a Plog file reading system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0041] For easier understanding, see Figure 1 , Figure 1 Schematic diagram of the flow of the Plog file reading method provided in an embodiment of the present invention. The Plog file reading method provided in an embodiment of the present invention includes:

[0042] Step 101: Obtain the target vehicle ID input by the user and the frame time corresponding to the target reading frame.

[0043] Each Plog file records several frames (pre-settable) of PNC internal status data. When the user needs to find the Plog file data corresponding to a certain frame of a certain vehicle, the user can directly enter the vehicle ID and the frame to be searched, and provide the system with the target vehicle ID and the frame time corresponding to the target reading frame.

[0044] Step 102: According to the target vehicle ID and the frame time corresponding to the target read frame, the corresponding meta information is searched in the tree structure storing the Plog file to determine the target Plog file where the target read frame of the target vehicle ID is located, wherein the meta information includes the vehicle ID and the start time of the Plog file.

[0045] According to the frame time and vehicle ID provided by the user, the system searches for the corresponding meta information in the tree structure, that is, it searches for the target vehicle ID and the target Plog file corresponding to the frame time under the target vehicle ID in the tree structure. For example, Figure 3 As shown, the vehicle ID that the user wants to find is MKZ_0001, and the frame time is 100002.1 seconds. Then, the target Plog file is Figure 3 The Plog file corresponding to ID: MKZ_0001 and time: 100002.

[0046] In one embodiment, a read-write lock is enabled for each frame in the Plog file in the Plog list. Then, when the user writes, the data will be copied in the memory and modified on the copied data. After the user completes the modification, if there is no read lock, the write lock needs to be locked and the data will be directly replaced with the original data. After the replacement is completed, the write lock is unlocked.

[0047] Step 103: Check whether the target Plog file has been loaded. If not, load the target Plog file based on whether the number of files loaded in the memory exceeds the upper limit, search the target read frame from the target Plog file and return it to the user.

[0048] To effectively control the memory usage of Plog file reading, the present invention uses lazy loading to load the target Plog file. After determining the target Plog file, a check is performed to see if the target Plog file has been loaded. If so, the target read frame is directly retrieved from the target Plog file and returned to the user. If the target Plog file has not been loaded, a method for loading the target Plog file is selected based on the number of files loaded in memory. After the target Plog file is loaded, the target read frame is retrieved from the target Plog file and returned to the user. The upper limit on the number of files loaded in memory can be set according to actual needs.

[0049] In one embodiment, if the target Plog file has not been loaded, it is necessary to determine whether the number of files loaded in the memory exceeds the upper limit before executing step 104A or step 104B.

[0050] Step 104A: When the number of files loaded into the memory does not exceed the upper limit, directly load the target Plog file, search the target read frame from the target Plog file, and return it to the user.

[0051] Step 104B: When the number of files loaded in the memory exceeds the upper limit, scan all loaded files, delete unused files from the memory, load the target Plog file, search the target read frame from the target Plog file, and return it to the user.

[0052] When loading a file, a check is performed to determine whether the number of Plog frames loaded into memory exceeds a preset upper limit. If it does not, there is sufficient memory to read the Plog file, and the target read frame can be directly searched from the target Plog file and returned to the user. If it exceeds the preset upper limit, indicating that memory is tight, unused files are deleted from memory, and the specific deletion method can be implemented using the LRU algorithm. It should be noted that when a file is read, it will be marked as in use and cannot be deleted from memory. Conversely, if a file is not marked as in use, it can be deleted from memory.

[0053] Before step 101, the Plog files can be pre-processed based on the user's points of interest, and the Plog files corresponding to the user's points of interest can be extracted from the database. The extracted Plog files are stored in a tree structure according to the meta information, so that the corresponding Plog files can be found according to the meta information. Figure 2 Specifically, the following steps may be included:

[0054] Step 001: Obtain the Plog list to be read. The Plog file name in the Plog list contains meta information: vehicle ID and start time of the Plog file.

[0055] When a vehicle is online, the onboard PNC automatically records its status and writes it to a Plog file. Plog files collected around each time point are stored in a database. Users provide a Plog list based on their interests. The Plog file names in the Plog list contain key information needed for indexing, such as the vehicle ID, the Plog file's start time, the number of frames in the Plog file, the region, and the map version. This information is encoded in the file name. Users simply decode this key information from the file name.

[0056] Step 002: Scan the Plog list and obtain meta information from the Plog file name in the Plog list.

[0057] The Plog list is scanned and meta information including vehicle ID and start time of the Plog file is extracted from the Plog file name in the Plog list.

[0058] Step 003: The meta information is stored in a two-layer tree structure in an orderly manner. The first layer of the tree structure stores the vehicle ID, and the second layer stores the start time of the Plog file.

[0059] All the extracted meta information is stored in a tree structure, which can be an LSM tree or other trees, preferably an LSM tree. The tree structure has two layers. The first layer stores vehicle information and the second layer stores the start time of the Plog file. The meta information is stored in the two-layer tree structure in an orderly manner, such as Figure 3 As shown, Figure 3 The ID:MKZ_0001 in the file indicates that the vehicle ID is MKZ_0001, and the time:100003 and time:100002 are the start times of the Plog file. Figure 3The time interval between 100003 and 100002 is 1 second. In actual use, the number of frames contained in each Plog file can be more than 1 second, and can be 10 seconds or other values. This is not limited in the present invention, and those skilled in the art can set it according to actual needs. In practical applications, the preferred time interval is between 10 seconds and 60 seconds, that is, the time frame data contained in each Plog file is between 10 seconds and 60 seconds, to avoid too many Plog files due to too little data and too much data causing excessive pressure on Plog file data storage. When the metadata is stored in an orderly manner in a two-layer tree structure, a quick sorting method can be used for sorting, first sorting according to the vehicle ID, and then sorting according to the start time of the Plog file. Thus, the splicing of adjacent Plog files is achieved.

[0060] It should also be noted that, according to the tree-structured metadata storage method in the Plog file reading method provided in the present invention, when a user requests the next frame of the current frame, the corresponding frame can be quickly found, and the requested data can be automatically read from different Plog files based on the metadata and returned to the user. For the user, it is equivalent to storing all Plogs of consecutive time in one file.

[0061] The Plog file reading method provided by the embodiment of the present invention can be used on a local reading platform, an AWS cloud computing platform, and a Spark cloud computing platform, and has uniformity. At the same time, read and write operations can be performed through multiple threads.

[0062] In a Plog file reading method provided by an embodiment of the present invention, when a vehicle is online, the vehicle's PNC automatically records its own status and writes it into a Plog file, which is stored in a database. When a user needs to obtain a Plog file at a time point of interest, the system automatically obtains the vehicle ID and start time metadata based on the Plog file list provided by the user, stores the metadata in a tree structure, and splices different Plog files in the tree storage structure based on the vehicle ID and time sequence. Then, based on the vehicle ID and time frame information to be searched for input by the user, the Plog file stored in the frame is searched from the tree structure. The system detects whether the number of files loaded in the memory exceeds an upper limit, and selects a PLog file loading method based on whether the number of files loaded in the memory exceeds the upper limit to load the Plog file. The frame is found in the Plog file and returned to the user, thereby achieving the goal of reading the Plog file with as little memory as possible, thereby solving the technical problem that existing offline Plog file reading requires a relatively large memory space.

[0063] In one embodiment, the internal state of the PNC of the target vehicle ID can also be simulated in an offline state to obtain a Plog file corresponding to the simulation. The Plog file corresponding to the simulation of the target vehicle ID is compared with the Plog file written by the online PNC of the target vehicle ID, and the comparison result is output. If the comparison result is basically consistent (within a preset deviation range), it indicates that the PNC algorithm that generates the Plog is relatively stable. If there is a large deviation, it indicates that the PNC algorithm that generates the Plog is unstable, thereby providing guidance for the improvement of the PNC algorithm.

[0064] For easier understanding, see Figure 4 , Figure 4 Schematic diagram of the structure of the Plog file reading system provided in an embodiment of the present invention. The Plog file reading system provided in an embodiment of the present invention includes:

[0065] The target information acquisition module 304 is used to obtain the target vehicle ID input by the user and the frame time corresponding to the target reading frame.

[0066] The target Plog file search module 305 is used to search for corresponding meta information in the tree structure storing the Plog file according to the target vehicle ID and the frame time corresponding to the target read frame, and determine the target Plog file where the target read frame of the target vehicle ID is located, wherein the meta information includes the vehicle ID and the start time of the Plog file.

[0067] The loading detection module 306 is used to detect whether the target Plog file has been loaded. If not, the target Plog file is loaded according to whether the number of files loaded in the memory exceeds the upper limit, and the target read frame is searched from the target Plog file and returned to the user.

[0068] The loading detection module 306 is specifically used for:

[0069] When the number of files loaded into the memory does not exceed the upper limit, the target Plog file is loaded directly, and the target read frame is searched from the target Plog file and returned to the user;

[0070] When the number of files loaded in the memory exceeds the upper limit, all loaded files are scanned, unused files are deleted from the memory, and then the target Plog file is loaded, and the target read frame is searched from the target Plog file and returned to the user.

[0071] Specifically, it also includes:

[0072] A Plog list acquisition module 301 is used to acquire a Plog list to be read, where the Plog file name in the Plog list includes the vehicle ID and the start time of the Plog file;

[0073] A scanning module 302 is used to scan the Plog list and obtain meta information from the Plog file name in the Plog list;

[0074] The meta information storage module 303 is used to store the meta information in a two-layer tree structure in an orderly manner. The first layer of the tree structure stores the vehicle ID, and the second layer stores the start time of the Plog file.

[0075] Specifically, an LRU algorithm is used to delete unused files from the memory.

[0076] Specifically, it also includes a read-write lock module 307;

[0077] The read-write lock module 307 is used to enable a read-write lock for each frame in the Plog file in the Plog list before detecting whether the target Plog file has been loaded.

[0078] Specifically, it also includes a simulation verification module 308 for:

[0079] In the offline state, the internal state of the PNC of the target vehicle ID is simulated to obtain the Plog file corresponding to the simulation, the Plog file corresponding to the simulation of the target vehicle ID is compared with the Plog file written by the online PNC of the target vehicle ID, and the comparison result is output.

[0080] In a Plog file reading system provided by an embodiment of the present invention, when a vehicle is online, the vehicle's PNC automatically records its own status and writes it into a Plog file, storing the Plog file in a database. When a user needs to obtain a Plog file at a time point of interest, the system automatically obtains the vehicle ID and start time metadata based on the Plog file list provided by the user, stores the metadata in a tree structure, and splices different Plog files in the tree storage structure based on the vehicle ID and time sequence. Then, based on the vehicle ID and time frame information to be searched for input by the user, the Plog file stored in the frame is searched from the tree structure. The system detects whether the number of files loaded in the memory exceeds an upper limit, selects a PLog file loading method based on whether the number of files loaded in the memory exceeds the upper limit, and finds the frame in the Plog file and returns it to the user, thereby achieving the goal of reading the Plog file with as little memory as possible, thereby solving the technical problem that the existing offline Plog file reading requires a relatively large memory space.

[0081] The Plog file reading system provided in the embodiment of the present invention is used to execute the Plog file reading method in the aforementioned embodiment of the Plog file reading method, and can achieve the same technical effect as the aforementioned embodiment of the Plog file reading method. Its principle is the same as that of the aforementioned embodiment of the Plog file reading method, and will not be repeated here.

[0082] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0083] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Plog file reading method, characterized in that: include: Get the target vehicle ID input by the user and the frame time corresponding to the target reading frame; Based on the target vehicle ID and the frame time corresponding to the target read frame, the corresponding meta information is searched in the tree structure storing the Plog file to determine the target Plog file where the target read frame of the target vehicle ID is located. The meta information includes the vehicle ID and the start time of the Plog file. The Plog file is a file generated by the planning and control module of the target vehicle after serializing the internal state of the monitoring data perceived during the autonomous driving process. Check whether the target Plog file has been loaded. If not, load the target Plog file based on whether the number of files loaded in the memory exceeds the upper limit, search for the target read frame from the target Plog file and return it to the user; The target Plog file is loaded based on whether the number of files loaded in the memory exceeds the upper limit. The target read frame is searched from the target Plog file and returned to the user, including: When the number of files loaded into the memory does not exceed the upper limit, the target Plog file is loaded directly, and the target read frame is searched from the target Plog file and returned to the user; When the number of files loaded in the memory exceeds the upper limit, all loaded files are scanned, unused files are deleted from the memory, and then the target Plog file is loaded, and the target read frame is searched from the target Plog file and returned to the user.

2. The Plog file reading method according to claim 1, wherein: Before obtaining the target vehicle ID input by the user and the frame time corresponding to the target reading frame, it also includes: Get the Plog list to be read. The Plog file name in the Plog list contains the vehicle ID and the start time of the Plog file; Scan the Plog list and obtain meta information from the Plog file name in the Plog list; The metadata is stored in a two-layer tree structure in an orderly manner. The first layer of the tree structure stores the vehicle ID, and the second layer stores the start time of the Plog file.

3. The Plog file reading method according to claim 1, wherein: Use the LRU algorithm to delete unused files from memory.

4. The Plog file reading method according to claim 1, wherein: Before checking whether the target Plog file has been loaded, it also includes: Enables a read-write lock on each frame in the Plog files in the Plog list.

5. The Plog file reading method according to any one of claims 1 to 4, characterized in that: Also includes: In offline state, simulate the internal state of the PNC of the target vehicle ID and obtain the Plog file corresponding to the simulation; Compare the Plog file corresponding to the target vehicle ID simulation with the Plog file written by the online PNC of the target vehicle ID, and output the comparison result.

6. A Plog file reading system, characterized in that: include: A target information acquisition module is used to obtain the target vehicle ID input by the user and the frame time corresponding to the target reading frame; A target Plog file search module is used to search for corresponding meta information in the tree structure storing Plog files based on the target vehicle ID and the frame time corresponding to the target read frame, and determine the target Plog file where the target read frame of the target vehicle ID is located. The meta information includes the vehicle ID and the start time of the Plog file. The Plog file is a file generated by the planning and control module of the target vehicle after serializing the internal state of the monitoring data perceived during the autonomous driving process; The loading detection module is used to detect whether the target Plog file has been loaded. If not, the target Plog file is loaded according to whether the number of files loaded in the memory exceeds the upper limit, and the target read frame is searched from the target Plog file and returned to the user; The loading detection module is specifically used to: When the number of files loaded into the memory does not exceed the upper limit, the target Plog file is loaded directly, and the target read frame is searched from the target Plog file and returned to the user; When the number of files loaded in the memory exceeds the upper limit, all loaded files are scanned, unused files are deleted from the memory, and then the target Plog file is loaded, and the target read frame is searched from the target Plog file and returned to the user.

7. The Plog file reading system according to claim 6, characterized in that: Also includes: The Plog list acquisition module is used to obtain the Plog list to be read. The Plog file name in the Plog list contains the vehicle ID and the start time of the Plog file; Scanning module, used to scan the Plog list and obtain meta information from the Plog file name in the Plog list; The meta information storage module is used to store the meta information in a two-layer tree structure in an orderly manner. The first layer of the tree structure stores the vehicle ID, and the second layer stores the start time of the Plog file.

8. The Plog file reading system according to claim 6, characterized in that: Also includes a read-write lock module; The read-write lock module is used to enable a read-write lock for each frame in the Plog file in the Plog list before detecting whether the target Plog file has been loaded.

Images