Data processing method and device for autonomous driving equipment, program product and equipment

CN121542219BActive Publication Date: 2026-09-18XIAN MAIN FUNCTION INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511654747.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-11-12
Publication Date
2026-09-18
Estimated Expiration
2045-11-12

AI Technical Summary

Technical Problem

然而,现有自动驾驶设备的数据处理方式存在数据生产混乱、回传至远程服务器的准确性及效率低,以及缺乏自动化数据后处理,从而一定程度上影响了对自动驾驶设备的一系列分析处理

Benefits of technology

[0020]On the one hand, by storing the collected autonomous driving equipment data into different subdirectories according to data type, effective data classification and organization are achieved at the source, avoiding the overhead of repeatedly sifting through mixed data in subsequent processing, and providing a foundation for quickly and accurately locating and acquiring target data. On the other hand, a global task queue is constructed based on time slices, decomposing the massive data processing task into discrete, time-window-based unit tasks, making the data processing process clear and controllable, facilitating system scheduling, monitoring, and fault tolerance, and improving the systematicness and efficiency of the overall processing flow. Furthermore, each task is bound to a specific time slice, ensuring that the system can accurately extract target data corresponding to the current task's time range from massive amounts of stored data, thereby avoiding blind and redundant data processing and ensuring the targeting and accuracy of data analysis. In addition, preprocessing is performed on the local autonomous driving equipment before synchronizing data to the remote server, providing a basis for saving bandwidth resources and reducing the storage and computing pressure on the remote server. Therefore, this disclosure can achieve accurate and efficient data transmission and automated post-processing of autonomous driving equipment data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121542219B_ABST
    Figure CN121542219B_ABST
Patent Text Reader

Abstract

The present disclosure relates to the technical field of computer, and relates to a data processing method and device of an automatic driving device, a computer program product and an electronic device. The method comprises: collecting automatic driving device data, and storing the automatic driving device data into corresponding storage sub-directories according to data types; creating a time slice based on a current system time and a time interval configuration, and constructing a global task queue in time sequence according to the time slice, different tasks in the global task queue having respective time slices; for a current task in the global task queue, obtaining target data corresponding to the time slice of the current task from the storage sub-directories, and pre-processing the target data to obtain processed data; and synchronizing the processed data to a remote server to generate a data analysis result of the automatic driving device according to the processed data. The present disclosure can realize accurate and efficient backhaul and automatic post-processing of the automatic driving device data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a data processing method for an autonomous driving device, a data processing apparatus for an autonomous driving device, a computer program product, and an electronic device. Background Technology

[0002] In the research and development and operation of autonomous driving equipment (such as driverless cars and intelligent connected vehicles), the massive amounts of critical data generated by the operation of autonomous driving equipment (including sensor data packets, system operation logs, crash dump files, etc.) are the core basis for fault diagnosis, behavior analysis, and algorithm model optimization. However, the existing data processing methods for autonomous driving equipment suffer from chaotic data production, low accuracy and efficiency in transmitting data back to remote servers, and a lack of automated data post-processing, which to some extent affects a series of analyses and processing of autonomous driving equipment.

[0003] It should be noted that the information in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide a data processing method, a data processing device, a computer program product, and an electronic device for autonomous driving equipment, thereby enabling accurate and efficient transmission and automated post-processing of data from autonomous driving equipment.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a data processing method for an autonomous driving device is provided, comprising: collecting data from the autonomous driving device and storing the data in corresponding storage subdirectories according to data type; creating time slices based on the current system time and time interval configuration, and constructing a global task queue by arranging the time slices in chronological order, wherein different tasks in the global task queue have their own time slices; for the current task in the global task queue, retrieving target data corresponding to the time slice of the current task from the storage subdirectory, and preprocessing the target data to obtain processed data; and synchronizing the processed data to a remote server to generate data analysis results for the autonomous driving device based on the processed data.

[0007] In one exemplary embodiment of this disclosure, time slices are created based on the current system time and time interval configuration, and a global task queue is constructed by arranging the time slices in chronological order. This includes: when the system starts, creating a date directory based on the current system time, and creating a slice directory, a temporary copy directory, and a directory to be uploaded under the date directory; creating a status log file under the date directory to record the current status of the time slices; creating time slices according to the time interval configuration, wherein each time slice corresponds to an independent status log file; and constructing a global task queue by arranging the time slices in chronological order based on the date directory and the status log file.

[0008] In one exemplary embodiment of this disclosure, the time slices include a 0 o'clock time slice, a power-on time slice, and a current time slice created based on a time interval configuration; based on a date directory and a status log file, a global task queue is constructed by arranging the time slices in chronological order, including: scanning all date directories and reading the status log file under each date directory to obtain consecutive time slices in a first state, where the first state indicates that the time slice directory has been created; and constructing a global task queue by arranging the time slices in the first state in ascending chronological order.

[0009] In one exemplary embodiment of this disclosure, for the current task in the global task queue, retrieving the target data corresponding to the time slice of the current task from the storage subdirectory includes: retrieving the current task and the next task adjacent to the current task from the global task queue; performing cross-day verification based on the time slices of the current task and the next task; if the current task and the next task are on different days and the next task is not at 0:00, then the current task is determined to be an abnormal task and the processing is skipped; otherwise, the temporary copy directory is cleared, and the target data corresponding to the time slice of the current task in the storage subdirectory is copied to the temporary copy directory; the current state of the status record file corresponding to the time slice of the current task is set to a second state, the second state being used to indicate that there is no data slice or that the data slice has been completed.

[0010] In one exemplary embodiment of this disclosure, preprocessing the target data to obtain processed data includes: compressing the uncompressed data in the target data and packaging the compressed data in the target data in a zero-compression manner to obtain processed data; copying the processed data from a temporary copy directory to a directory to be uploaded; setting the current state of the status record file corresponding to the time slice of the current task to a third state, the third state being used to indicate that data has been copied to the directory to be uploaded.

[0011] In one exemplary embodiment of this disclosure, synchronizing processed data to a remote server includes: scanning the directory to be uploaded to determine that the status record file corresponding to the time slice of the current task is in the third state; and calling the rsync command to upload the processed data in the directory to be uploaded to the remote server.

[0012] In one exemplary embodiment of this disclosure, calling the rsync command to upload processed data in the directory to be uploaded to a remote server includes: requesting recommended bandwidth from the remote server; determining a bandwidth limit based on the obtained recommended bandwidth, the bandwidth limit being used to limit the maximum network bandwidth that the rsync command can use when uploading data; determining a maximum rate limit based on the device type of the current system; and performing rate limiting control on the process of calling the rsync command to upload processed data in the directory to be uploaded to the remote server based on the bandwidth limit and the maximum rate limit.

[0013] In one exemplary embodiment of this disclosure, synchronizing the processed data to a remote server further includes: reading a status log file on the remote server, adding files not recorded in the processed data as new tasks to the status log file on the remote server, and setting the status of the new tasks to a first target status, which indicates that the task has been added; filtering tasks in the first target status, copying the files corresponding to the tasks in the first target status to a conversion directory, and updating the corresponding tasks to a second target status, which indicates that the files have been copied; obtaining files in the conversion directory and decompressing them to obtain decompressed data, and updating the decompressed tasks to a third target status, which indicates that the directory has been decompressed; converting the target decompressed data in the decompressed data to obtain data to be analyzed; and generating data analysis results for the autonomous driving device based on the processed data, including: extracting key information from the data to be analyzed based on configuration time information to obtain a key information set, and generating data analysis results based on the key information set according to a preset template.

[0014] In one exemplary embodiment of this disclosure, the method further includes: updating the data to be analyzed based on configuration time information according to a preset period, and extracting key information from the updated data to update the key information set; and generating data analysis results based on the updated key information set according to a preset template.

[0015] In one exemplary embodiment of this disclosure, collecting autonomous driving device data includes: subscribing to node messages of the autonomous driving device and storing the node messages of each node in a corresponding cache container; when a preset trigger condition is met, storing the data in the cache container in a corresponding storage subdirectory according to the data type, and recording real-time data and storing it in a corresponding storage subdirectory according to the data type, so as to obtain autonomous driving device data.

[0016] According to one aspect of this disclosure, a data processing apparatus for an autonomous driving device is provided, comprising: a data acquisition module for acquiring data from the autonomous driving device and storing the data in a corresponding storage subdirectory according to data type; a time slice creation module for creating time slices based on the current system time and time interval configuration, and constructing a global task queue according to the time order of the time slices, wherein different tasks in the global task queue have their own time slices; a target data acquisition module for acquiring target data corresponding to the time slice of the current task in the global task queue from the storage subdirectory, and preprocessing the target data to obtain processed data; and a data synchronization module for synchronizing the processed data to a remote server to generate data analysis results for the autonomous driving device based on the processed data.

[0017] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the above methods.

[0018] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any of the above methods by executing the executable instructions.

[0019] The data processing method for an autonomous driving device in an exemplary embodiment of this disclosure involves: collecting data from the autonomous driving device and storing the data in corresponding storage subdirectories according to data type; creating time slices based on the current system time and time interval configuration, and constructing a global task queue by arranging the time slices in chronological order, with different tasks in the global task queue having their own time slices; for the current task in the global task queue, retrieving target data corresponding to the time slice of the current task from the storage subdirectory, preprocessing the target data to obtain processed data; and synchronizing the processed data to a remote server to generate data analysis results for the autonomous driving device based on the processed data.

[0020] On the one hand, by storing the collected autonomous driving equipment data into different subdirectories according to data type, effective data classification and organization are achieved at the source, avoiding the overhead of repeatedly sifting through mixed data in subsequent processing, and providing a foundation for quickly and accurately locating and acquiring target data. On the other hand, a global task queue is constructed based on time slices, decomposing the massive data processing task into discrete, time-window-based unit tasks, making the data processing process clear and controllable, facilitating system scheduling, monitoring, and fault tolerance, and improving the systematicness and efficiency of the overall processing flow. Furthermore, each task is bound to a specific time slice, ensuring that the system can accurately extract target data corresponding to the current task's time range from massive amounts of stored data, thereby avoiding blind and redundant data processing and ensuring the targeting and accuracy of data analysis. In addition, preprocessing is performed on the local autonomous driving equipment before synchronizing data to the remote server, providing a basis for saving bandwidth resources and reducing the storage and computing pressure on the remote server. Therefore, this disclosure can achieve accurate and efficient data transmission and automated post-processing of autonomous driving equipment data.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0022] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation.

[0023] Figure 1 A flowchart illustrating a data processing method for an autonomous driving device according to an exemplary embodiment of the present disclosure is shown.

[0024] Figure 2 A flowchart illustrating an implementation of constructing a global task queue according to an exemplary embodiment of the present disclosure is shown.

[0025] Figure 3 A flowchart illustrating an implementation method for acquiring target data according to an exemplary embodiment of the present disclosure is shown.

[0026] Figure 4 A flowchart illustrating an implementation of preprocessing target data according to an exemplary embodiment of the present disclosure is shown.

[0027] Figure 5 A flowchart illustrating an implementation of controlling transmission speed according to an exemplary embodiment of the present disclosure is shown.

[0028] Figure 6A flowchart illustrating an implementation of automated task scheduling processing according to an exemplary embodiment of the present disclosure is shown.

[0029] Figure 7 A schematic diagram of the composition of a data processing apparatus for an autonomous driving device according to an exemplary embodiment of the present disclosure is shown.

[0030] Figure 8 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0031] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and therefore their detailed description will be omitted.

[0033] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details described, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known structures, methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more software-hardened modules, or in different network and / or processor devices and / or microcontroller devices.

[0035] In the research and development and operation of autonomous driving equipment (such as driverless cars and intelligent connected vehicles), the massive amounts of critical data generated during the operation of these devices (including sensor data packets, system operation logs, crash dump files, etc.) are the core basis for fault diagnosis, behavior analysis, and algorithm model optimization. However, existing data processing methods for autonomous driving equipment have the following problems: First, data production management is chaotic. Multiple data types generated by autonomous vehicles, such as logs, sensor data packets, and crash files (coredumps), are stored in a mixed manner, easily leading to accidental deletion or wasted storage of critical data. Second, remote data transmission is inefficient. Autonomous driving devices generate massive amounts of data, making it impossible to transmit data that meets actual user needs. Interruptions and data loss are also common during transmission. Current methods require manual intervention for overall data analysis, resulting in a cumbersome and inefficient process. This fails to automate the entire process from data transmission to analysis results output, making it difficult to support the efficient analysis needs of large-scale fleet data.

[0036] It is evident that current data processing for autonomous driving equipment suffers from chaotic data production, low accuracy and efficiency in transmitting data back to remote servers, and a lack of automated data post-processing, which to some extent affects a series of analyses and processing of autonomous driving equipment.

[0037] Based on one or more of the above-mentioned problems, the exemplary embodiments of this disclosure provide a data processing method for autonomous driving equipment, which has structured production management, efficient intelligent backhaul, automated post-processing and highly robust end-to-end data processing, solving problems such as chaotic data management, low transmission efficiency, cumbersome post-processing and insufficient robustness in the prior art.

[0038] It should be noted that the exemplary embodiments disclosed herein can be applied to the automated collection and intelligent storage management of key data of autonomous driving equipment, and can also be extended to fields such as industrial equipment operation data collection and edge computing node data synchronization.

[0039] like Figure 1 The diagram shown is a flowchart of a data processing method for an autonomous driving device according to an exemplary embodiment of this disclosure, with reference to... Figure 1 As shown, the method includes steps S110 to S140, specifically: Step S110: Collect data from the autonomous driving device and store the data in the corresponding storage subdirectory according to the data type.

[0040] Step S120: Create time slices based on the current system time and time interval configuration, and build a global task queue by arranging the time slices in chronological order. Different tasks in the global task queue have their own time slices.

[0041] Step S130: For the current task in the global task queue, retrieve the target data corresponding to the time slice of the current task from the storage subdirectory, and preprocess the target data to obtain the processed data.

[0042] Step S140: Synchronize the processed data to a remote server to generate data analysis results for the autonomous driving device based on the processed data.

[0043] The data processing method for autonomous driving devices in the exemplary embodiments of this disclosure, on the one hand, effectively classifies and organizes the collected autonomous driving device data at the source by storing the data according to data type into different subdirectories, avoiding the overhead of repeatedly filtering mixed data in subsequent processing, and providing a foundation for quickly and accurately locating and obtaining target data. On the other hand, a global task queue is constructed based on time slices, decomposing the massive data processing task into discrete, time-window-based unit tasks, making the data processing process clear and controllable, facilitating system scheduling, monitoring, and fault tolerance, and improving the systematicness and efficiency of the overall processing flow. Furthermore, each task is bound to a specific time slice, ensuring that the system can accurately extract target data corresponding to the current task's time range from massive amounts of stored data, thereby avoiding blind and redundant data processing and ensuring the targeting and accuracy of data analysis. In addition, preprocessing is performed on the local autonomous driving device before synchronizing the data to the remote server, providing a basis for saving bandwidth resources and reducing the storage and computing pressure on the remote server. Therefore, this disclosure can achieve accurate and efficient data transmission and automated post-processing of autonomous driving device data.

[0044] Steps S110 to S140 will be described in more detail below.

[0045] In step S110, data from the autonomous driving device is collected and stored in the corresponding storage subdirectory according to the data type.

[0046] Data from autonomous driving devices can include sensor data packets, system operation logs, crash dump files, etc. The collected data can be automatically written to the corresponding storage subdirectories according to data type. For example, Log data is written to ` / data / 2025-08-12 / log / `, and Bag data is written to ` / data / 2025-08-12 / bag / `. Specifically, a date directory (e.g., ` / data / 2025-08-12 / ` for the current day) can be automatically created upon system startup, and subdirectories `log / `, `bag / `, and `coreDump / ` can be created under this date directory to store Log, Bag, and CoreDump data types respectively.

[0047] In one exemplary embodiment, collecting data from an autonomous driving device may include: Subscribe to node messages from autonomous driving devices and store each node's message in its corresponding cache container.

[0048] When the preset trigger conditions are met, the data in the cache container is stored in the corresponding storage subdirectory according to the data type, and real-time data is recorded and stored in the corresponding storage subdirectory according to the data type to obtain the autonomous driving device data.

[0049] Specifically, when the cached data function (such as the cachedata function) is started, it can subscribe to all node messages of the autonomous driving device and initialize a storage container for each node message. The size of the cache container can be determined based on the message frequency and a preset cache duration. For example, if the message frequency is 10Hz and the cache duration is 30s, the container size is 300 messages. Furthermore, it can receive node messages from the autonomous driving device in real time and store each node's message in its corresponding cache container. When the cache container is full, the oldest message can be removed (i.e., first-in, first-out).

[0050] When preset trigger conditions are met, the system's packet recording interface can be called to first store the data in the cache container to the corresponding storage subdirectory (i.e., disk write) according to data type, and then start recording real-time data. After packet recording is complete, the Bag file can be automatically named (e.g., 20250812_210738_W028-R5.6.1-percep.bag), containing tags such as trigger time, vehicle number, software version, and trigger reason. Preset trigger conditions, such as characteristic events, manual takeover, and abnormal events, can be flexibly set according to actual scenario requirements.

[0051] Optionally, data deletion strategies can be implemented for the disks corresponding to the storage subdirectories. Predefined disk usage thresholds and check cycles can be set, for example, four levels of disk usage thresholds: low, medium, high, and very high (the threshold settings can be adjusted flexibly according to specific scenarios), with a check cycle of 30 seconds. Then, disk usage can be periodically retrieved, and corresponding deletion strategies (pre-set) can be triggered based on the usage rate. For example, if the usage rate is greater than the low threshold: delete Log, Bag, and CoreDump data older than 5 days. If the usage rate is greater than the medium threshold: delete Log, Bag, and CoreDump data older than 3 days. If the usage rate is greater than the high threshold: in cross-day scenarios, delete Log, Bag, and CoreDump data older than 1 day; in non-cross-day scenarios, delete Log data older than 1 day, some Bag data from the current day, and immediate CoreDump data. If the usage rate is greater than the very high threshold: delete Log data older than 1 day, most Bag data from the current day, and CoreDump data, and delete large files larger than 150MB; if the threshold is still exceeded, delete files larger than 30MB.

[0052] Optionally, logs recorded during the process can be deleted to ensure traceability of the operation.

[0053] It should be noted that the specific number of days or data file size involved in this example are exemplary. The exemplary embodiments of this disclosure can flexibly set the specific threshold value and file size according to actual needs.

[0054] It should be understood that the data production layer modules that collect data from autonomous driving devices, such as the cachedata caching module and the dynamic disk management module, are deployed in autonomous driving devices, for example, in in-vehicle embedded systems.

[0055] Autonomous driving devices generate massive amounts of node messages instantaneously during operation. By first writing the data to a cache container, the high-frequency, instantaneous data stream is effectively smoothed, avoiding peak impacts from write operations. Based on disk usage thresholds, data deletion strategies can be dynamically adjusted, prioritizing the deletion of expired, non-critical data, and different weights are set for different data types (e.g., CoreDump is retained with higher priority than ordinary logs), preventing disk overflow while ensuring that critical data is not lost.

[0056] In step S120, time slices are created based on the current system time and time interval configuration, and the time slices are arranged in chronological order to build a global task queue, with different tasks in the global task queue having their own time slices.

[0057] At system startup, the "MakeDateCutDir" function can be called to obtain the current system time, which can be used as the starting reference coordinate for constructing time slices. The time interval configuration is a preset, configurable duration parameter. For example, it can be 5 minutes, 30 minutes, 1 hour, etc., which determines the granularity of the continuous time stream to be discretized into processing units. A time slice is a logical, fixed-duration time window, such as "10:00-10:30". The global task queue is a list of pending tasks maintained internally by the system and arranged in chronological order; each task corresponds to a time slice.

[0058] In one exemplary embodiment, an implementation method for constructing a global task queue is provided. For example... Figure 2 As shown, creating time slices based on the current system time and time interval configuration, and constructing a global task queue by arranging the time slices in chronological order, may include: Step S210: When starting the system, create a date directory based on the current system time, and create a slice directory, a temporary copy directory, and a directory to be uploaded under the date directory.

[0059] When the system starts, the "MakeDateCutDir" function can be called to obtain the current system time, create a date directory for the current day (date directory), and create subdirectories such as slice directories (e.g. tmp_cut), temporary copy directories (e.g. tmp_cp), and directories to be uploaded (e.g. tmp_live) under the date directory.

[0060] Step S220: Create a status log file in the date directory to record the current status of the time slice.

[0061] Each time slice corresponds to an independent status log file (such as a data_info.json file). This step can first create the status log file in the date directory and initialize the status log. The data_info.json file records the lifecycle status of each time slice (initialized, sliced, packaged, uploaded, etc.), supporting task recovery after abnormal system restart.

[0062] Step S230: Create time slices according to the time interval configuration, where each time slice corresponds to an independent status record file.

[0063] In the exemplary embodiments of this disclosure, the time slices include a 0:00 time slice, a power-on time slice, and a current time slice created based on a time interval configuration. This can be understood as creating three types of time slices in this step: a 0:00 time slice (e.g., 2025-08-12-00-00-00), a power-on time slice (created only once to record the startup time), and a current time slice created based on a time interval configuration (e.g., one per hour). The 0:00 time slice is used to determine the cutoff time for the previous day.

[0064] Optionally, the time slice length can be dynamically adjusted through the "GetTimeSlot" interface to adapt to different data volume scenarios.

[0065] Step S240: Based on the date directory and status log file, construct a global task queue by arranging the time slices in chronological order.

[0066] Arranging them in chronological order can be done from front to back, such as "10:00-10:30", "10:30-11:00", "11:30-12:00", etc.

[0067] In one exemplary embodiment, constructing a global task queue by arranging time slices in chronological order based on a date directory and a status log file may include: First, all date directories are scanned, and the status log file under each date directory is read to obtain consecutive time slices in the first state, which indicates that the time slice directory has been created.

[0068] Then, the time slices in the first state are used to construct a global task queue in ascending order of time.

[0069] First, the "MakeCutTaskDeque" function is called to scan all date directories, sort them by time, and read the status log file (data_info.json file) under each date directory to obtain consecutive time slices in the first state (such as the inited state). Then, the filtered time slices can be added to the global task queue (such as CUT_TASK) in ascending order of time to ensure that tasks are processed in chronological order.

[0070] By scanning the status log file, ready time slices are added to the task queue, fundamentally avoiding the task scheduling system from processing empty or unfinished time slices. This prevents processing failures or incorrect results, and improves the reliability and success rate of the entire data processing flow.

[0071] In step S130, for the current task in the global task queue, the target data corresponding to the time slice of the current task is obtained from the storage subdirectory, and the target data is preprocessed to obtain the processed data.

[0072] The current task is the single task item being executed, retrieved from the head of the global task queue. Storage subdirectories are different disk directories created in step S110 according to data type classification. Target data refers to specific data files stored in all storage subdirectories, but whose timestamps fall within the time slice bound to the current task. This can be understood as data being correlated in the time dimension but dispersed in the type dimension. Preprocessing refers to a series of operations performed on the target data locally before sending it to the remote server, including but not limited to compression and packaging. Processed data is the higher-quality, smaller-sized data packet suitable for synchronous transmission obtained after preprocessing; it is the object ultimately synchronized to the remote server.

[0073] In one exemplary embodiment, a method for acquiring target data is also provided. For example... Figure 3 As shown, for the current task in the global task queue, retrieving the target data corresponding to the time slice of the current task from the storage subdirectory may include: Step S310: Retrieve the current task and the next task adjacent to the current task from the global task queue.

[0074] The current task (now_task) and the next task (next_task, which is the end time of the current task) can be retrieved from the global task queue through the "ConsumerCutTask" thread.

[0075] Step S320: Perform cross-day verification based on the time slices of the current task and the next task. If the current task and the next task are on different days and the next task is not at 0:00, then the current task is determined to be an abnormal task and the processing is skipped.

[0076] "Non-00:00" means the start time of the next task's time slice is not 00:00:00. Cross-day verification is a logical process used to check if any anomalies occur in the time slice near the date change point (i.e., 00:00), ensuring that the division of time slices is continuous and logical during date changes. An abnormal task refers to a task in a cross-day scenario whose end point of the time slice or the start point of the next time slice does not conform to expectations. "Skip processing" means the system will mark this task as failed or remove it from the queue instead of acquiring and preprocessing its data, thus avoiding processing potentially problematic data.

[0077] Step S330: If the time slices of the current task and the next task do not span days, clear the temporary copy directory and copy the target data corresponding to the time slice of the current task in the storage subdirectory to the temporary copy directory.

[0078] If the time slices of the current task and the next task do not span across days, the tmp_cp directory can be cleared, and the Log, Bag, and CoreDump data within the current time slice can be copied sequentially. Different methods can be used to obtain the target data for different data types. For example, for Log data, the "GrepFromLogPath" function can be called, combined with binary search to locate the time interval and / or chunked reading of log data to extract the content of a specified time period (corresponding to the current task's time slice). When crossing files, intermediate files are copied directly, and the first and last files are truncated by time. For Bag / CoreDump data, the corresponding files can be copied according to the time range. After obtaining the target data, it should be copied to a temporary copy directory.

[0079] Step S340: Set the current state of the status record file corresponding to the time slice of the current task to the second state.

[0080] The second state indicates whether there is no data slice or the data slice has been completed. After the copy is complete, the current task status in the status log file (data_info.json) corresponding to the current task's time slice can be updated to "cutted" (sliced) or "cutted_null" (no data slice).

[0081] By proactively identifying and intercepting problematic tasks that span multiple days and are not at midnight, the system avoids processing slices with incorrect time ranges or incomplete data, preventing contamination of the data and ensuring the accuracy of subsequent analysis results. Furthermore, it ensures that the data processed by each task precisely corresponds to its time slice, eliminating data residue or cross-contamination between different tasks. After data is successfully copied to the temporary directory, the status record is immediately updated to a second state (no data or completed), ensuring strict synchronization between the task status and the physical preparation of the data.

[0082] In one exemplary embodiment, a method for preprocessing the target data is also provided. For example... Figure 4 As shown, preprocessing the target data to obtain processed data may include: Step S410: Compress the uncompressed data in the target data and pack the compressed data in the target data using zero compression to obtain the processed data.

[0083] The “PreprocessSendTask” thread can be called to preprocess the slice data, such as compressing non-.gz format Log and CoreDump files, and calling the “PackPath” function to package the time slice directory into a .zip file in zero compression (zip-0) mode, thereby obtaining the processed data.

[0084] Step S420: Copy the processed data from the temporary copy directory to the directory to be uploaded.

[0085] Step S430: Set the current status of the status record file corresponding to the time slice of the current task to the third status. The third status indicates that data has been copied to the directory to be uploaded. The status can be updated to "copyed" (packaged and ready for upload).

[0086] By performing differentiated preprocessing on sliced ​​data (compressing uncompressed files, discarding incomplete data, and packaging with zero compression, etc.), the bandwidth usage can be reduced.

[0087] In step S140, the processed data is synchronized to a remote server to generate data analysis results for the autonomous driving device based on the processed data.

[0088] Post-processed data refers to data packets refined through local preprocessing. These packets are characterized by small size, high quality, and standardized format, and can be directly used for analysis. Synchronization to a remote server is the process of securely, reliably, and efficiently transmitting post-processed data from the vehicle (or edge) to a cloud data center. This refers to industrial-grade data synchronization services with features such as resume capability and flow control. A remote server refers to a server cluster located in the cloud with powerful computing and storage capabilities, serving as a platform for large-scale, centralized data analysis. Data analysis results are generated by the remote server applying various algorithms and models to the received post-processed data, producing instructive information, knowledge, or decision-making support. These results can take the form of structured reports, visualizations, or charts.

[0089] In one exemplary embodiment, synchronizing processed data to a remote server may include: First, the upload directory is scanned to determine if the status log file corresponding to the current task's time slice is in the third state. Then, the rsync command is called to upload the processed data in the upload directory to the remote server.

[0090] Specifically, the "ConsumerSendTask" thread can scan the .zip files in the tmp_live directory to check if the corresponding task status in data_info.json is "copyed," meaning it checks if the status record file corresponding to the current task's time slice is in the third state. Then, the rsync command is called to execute the upload. Configuration parameters include, but are not limited to, preventing accidental handling of some transferred files (partial-dir=.rsync-partial), deleting local files after successful upload (remove-source-files), and rate limiting (bwlimit).

[0091] Using the rsync command for file copying not only enables incremental synchronization but also supports resuming from the point of interruption, effectively addressing network instability. Even in weak network environments, it can reliably deliver data to the cloud through multiple retries and breakpoint resumption, ensuring no data loss.

[0092] In one exemplary embodiment, a method for controlling the transmission speed is also provided. For example... Figure 5 As shown, the process of calling the rsync command to upload the processed data in the directory to be uploaded to the remote server includes: Step S510: Request recommended bandwidth from the remote server. This can be achieved by calling the "GetRemoteRsyncSpeed" function to request recommended bandwidth from the remote server every preset time interval (e.g., 60 seconds) via a UDP (User Datagram Protocol Socket) socket, i.e., dynamically adjusting the bwlimit parameter.

[0093] Step S520: Determine the bandwidth limit based on the obtained recommended bandwidth. The bandwidth limit is used to limit the maximum network bandwidth that the rsync command can use when uploading data.

[0094] Step S530: Determine the maximum rate limit based on the current system's device type. This allows for setting maximum rate limits for different types of devices to avoid impacting autonomous driving operations.

[0095] Step S540: Based on bandwidth limitations and maximum rate limits, the process of uploading processed data from the directory to be uploaded to the remote server by calling the rsync command is rate-limited.

[0096] By requesting recommended bandwidth from a remote server, the vehicle and cloud work collaboratively, adhering to the cloud's global load balancing strategy. This prevents a single device or task from monopolizing the uplink, avoiding communication delays in other critical vehicle functions (such as real-time control and safety communication) that could occur due to network congestion, thus ensuring the overall real-time performance and stability of the autonomous driving system. A mechanism for determining the maximum speed limit based on device type is introduced, giving this speed-limiting strategy adaptive capabilities. Different types of devices (such as the onboard main control computer and roadside units) have different network hardware and processing capabilities. This mechanism prevents setting unrealistically high speeds on low-performance devices, avoiding hardware overload, and allowing this synchronization scheme to flexibly adapt to various hardware platforms, enhancing the technology's versatility and robustness.

[0097] After uploading the processed data in the directory to be uploaded to the remote server, the current status of the status record file corresponding to the time slice of the current task can be set to the fourth status. For example, after the upload is completed, the task status in data_info.json is updated to "rsynced" (uploaded back), realizing the status monitoring of the entire life cycle of the time slice.

[0098] In one exemplary embodiment, an implementation method for automated task scheduling is also provided. For example... Figure 6 The processed data will be synchronized to the remote server, including: Step S610: Read the status log file of the remote server, add the files not recorded in the processed data as new tasks to the status log file of the remote server, and set the status of the new tasks to the first target status, which is used to indicate that the task has been added.

[0099] On the server side, it is also possible to periodically scan the raw data directory of the remote server (e.g., via the "TaskProduct" thread), identify .zip files that conform to the date format, read the status log file data_info.json on the server side, add unrecorded .zip files as new tasks to the status log file, and set the status to the first target status (i.e., "added").

[0100] Step S620: Filter the tasks in the first target state, copy the files corresponding to the tasks in the first target state to the conversion directory, and update the corresponding tasks to the second target state. The second target state is used to indicate that the files have been copied.

[0101] Among them, tasks with a status of "added" (e.g., through the "ConsumerCpTask" thread) can be filtered to copy the .zip file corresponding to the task in the first target status to the conversion directory (e.g., CONV_PATH) and update the status to the second target status (i.e., "copyed").

[0102] Based on the server-side status log file (data_info.json), tasks can be automatically discovered, created, and scheduled, providing accurate status data for automated task scheduling.

[0103] Step S630: Obtain the files in the conversion directory, decompress them to obtain decompressed data, and update the decompression completed task to the third target status. The third target status is used to indicate that the directory has been decompressed.

[0104] It can process files in the conversion directory and decompress them to obtain decompressed data. Specifically, it can decompress .zip files in the conversion directory to the corresponding directory and update the status to a third-target status (e.g., "dirdecompressed" (directory decompressed)). More specifically, it can decompress .gz files in the Log and CoreDump subdirectories within the directory, extract CoreDump exception node information and Bag file event information (such as manual packet recording and obstacle parking), read the software version number, and update the decompression result to various categories such as "decompressed_hasbag" (Bag data included, decompressed) or "decompressed" (no Bag data, decompressed).

[0105] Step S640: Convert the format of the target decompressed data in the decompressed data to obtain the data to be analyzed.

[0106] Among the decompressed data, there may be target decompressed data that requires format conversion. Therefore, format conversion can be performed on such target decompressed data. Specifically, the "ConsumerTrans2rosTask" thread can handle tasks with a status of "decompressed_hasbag", calling the "TransBag2Ros" function to convert Bag to RosBag format. After the conversion is complete, the status is updated to the fourth target status (to indicate that it has been converted to the target format), such as "transed2rosbag" (to indicate that it has been converted to rosbag). Thus, the data to be analyzed is determined based on all converted and target decompressed data that does not require format conversion. Of course, the specific type of target format can be flexibly set according to the actual needs of the scenario.

[0107] It's important to note that the server-side status log file and the autonomous driving device-side status log file operate independently. That is, the server-side `data_info.json` and the vehicle-side `data_info.json` are different and function independently. The autonomous driving device-side status log file records the creation of time slices, data fragmentation, data compression, transfer, and data transmission. The server-side status log file records the data transfer status, whether decompression is complete, and whether format conversion has been performed after the autonomous driving device transmits the data back.

[0108] Based on the aforementioned exemplary embodiments, generating data analysis results for autonomous driving devices from processed data may include: extracting key information from the data to be analyzed based on configuration time information to obtain a key information set, and generating data analysis results based on the key information set according to a preset template.

[0109] The configured time information defines the required data time range, which can be set by the user. Analysis tasks can be executed periodically (e.g., by calling the AnaTask function), filtering the data directory to be analyzed according to the configured time information. Key information is extracted from the data, such as mine number, vehicle number, date, software version, data type (Log / Bag / CoreDump), event type (manual packet recording / obstacle parking / CoreDump exception, etc.), event quantity, and details. Then, data analysis results are generated according to a preset template, for example, sorted by the dimension of "mine number-vehicle number-date," and written to a CSV (plain text file) report (the header includes: mine number, vehicle number, date, autonomous driving version, data type, event type, quantity, and details). The preset template can be set according to scenario requirements and is not specifically limited.

[0110] By sequentially performing tasks such as decompressing the compressed package, decompressing the internal .gz file, converting the Bag file format, performing CoreDump anomaly analysis, and extracting Bag file events (such as mode switching, faults, and abnormal parking events), a structured CSV report is generated, providing an accurate data foundation for supporting rapid data retrieval and analysis.

[0111] In one exemplary embodiment, the data to be analyzed can be updated according to a preset period based on the configured time information, and key information can be extracted from the updated data to update the key information set; then, data analysis results can be generated based on the updated key information set according to a preset template.

[0112] A preset period is a pre-configured, periodic time interval used to trigger the entire update process, such as daily or weekly at 8:00 AM. The configured time information dynamically determines the data range to be covered in each periodic execution. Based on the updated data to be analyzed, the key information extraction and report generation processes are re-executed to produce analytical results reflecting the latest data status. For example, the AnaTask function can be used to execute every 30 minutes (preset time) to periodically retrieve the data to be analyzed, thus enabling periodic report updates.

[0113] It should be understood that time slice management, data preprocessing, and intelligent backhaul modules belong to the data backhaul layer and can be deployed in autonomous driving equipment, such as in an in-vehicle embedded system. The data post-processing layer may include task scheduling, data decompression and transformation, anomaly analysis, and report generation modules, and can be deployed on a remote server. The state management hub layer can achieve end-to-end state persistence and coordination through data_info.json (in-vehicle) and data_info.json (server).

[0114] The data processing method for autonomous driving devices in the exemplary embodiments of this disclosure, on the one hand, effectively classifies and organizes the collected autonomous driving device data at the source by storing the data according to data type into different subdirectories, avoiding the overhead of repeatedly filtering through mixed data in subsequent processing, and providing a foundation for quickly and accurately locating and acquiring target data. On the other hand, a global task queue is constructed based on time slices, decomposing the massive data processing task into discrete, time-window-based unit tasks, making the data processing process clear and controllable, facilitating system scheduling, monitoring, and fault tolerance, and improving the systematicness and efficiency of the overall processing flow. Furthermore, each task is bound to a specific time slice, ensuring that the system can accurately extract target data corresponding to the current task's time range from massive amounts of stored data, thereby avoiding blind and redundant data processing and ensuring the targeting and accuracy of data analysis. In addition, preprocessing is performed on the local autonomous driving device before synchronizing the data to the remote server, providing a basis for saving bandwidth resources and reducing the storage and computing pressure on the remote server. As can be seen, this disclosure constructs a three-in-one toolchain of "data production-data feedback-data post-processing", adopts a modular design and multi-threaded collaboration model, and combines technologies such as time slice management, dynamic storage control, precise data preprocessing and state persistence to achieve full-link automated processing of autonomous driving equipment data, enabling accurate and efficient feedback and automated post-processing of autonomous driving equipment data.

[0115] In addition, the entire link records the task lifecycle through a state persistence mechanism, supporting breakpoint resume after vehicle power failure, system restart, and network interruption, which greatly improves data processing efficiency and reliability, provides efficient data support for autonomous driving fault diagnosis and algorithm optimization, and can be extended to data processing scenarios such as industrial equipment and edge computing nodes.

[0116] In an exemplary embodiment of this disclosure, a data processing apparatus for an autonomous driving device is also provided. (See reference...) Figure 7 As shown, the device 700 may include a data acquisition module 710, a time slice creation module 720, a target data acquisition module 730, and a data synchronization module 740. Specifically: The data acquisition module 710 is used to collect data from the autonomous driving device and store the data in the corresponding storage subdirectory according to the data type. The time slice creation module 720 is used to create time slices based on the current system time and time interval configuration, and to build a global task queue according to the time order of the time slices. Different tasks in the global task queue have their own time slices. The target data acquisition module 730 is used to obtain the target data corresponding to the time slice of the current task in the global task queue from the storage subdirectory, and to preprocess the target data to obtain processed data. The data synchronization module 740 is used to synchronize the processed data to a remote server to generate data analysis results of the autonomous driving device based on the processed data.

[0117] Since the details of each functional module of the data processing apparatus of the autonomous driving device of the present disclosure have been described in the exemplary embodiments of the data processing method of the autonomous driving device described above, they will not be repeated here.

[0118] It should be noted that although several modules or units of the data processing apparatus of an autonomous driving device have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0119] Exemplary embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the methods described above.

[0120] In one implementation, the computer program product can be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0121] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.

[0122] Computer program code can be written in one or more programming languages. The program code can execute entirely on the user's computing device, or partially on the user's computing device, or as a standalone software package, or partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or it can be connected to an external computing device (e.g., through an internet connection provided by a mobile network operator).

[0123] Computer programs can be carried or transmitted via signals such as electricity, magnetism, light, electromagnetic radiation, and infrared radiation. Electronic devices can convert the signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, to execute by the processor of the electronic device) the method steps of various exemplary embodiments of this disclosure, such as the steps of the methods described above.

[0124] Furthermore, in exemplary embodiments of this disclosure, an electronic device capable of implementing the above-described methods is also provided. Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be specifically implemented as: entirely hardware embodiments, entirely software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, collectively referred to herein as "circuit," "module," or "system."

[0125] The following reference Figure 8 To describe an electronic device 800 according to such an embodiment of the present disclosure. Figure 8 The electronic device 800 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0126] like Figure 8 As shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, a bus 830 connecting different system components (including storage unit 820 and processing unit 810), and a display unit 840.

[0127] The storage unit stores program code that can be executed by the processing unit 810, causing the processing unit 810 to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0128] Storage unit 820 may include readable media in the form of volatile storage units, such as random access memory (RAM) 821 and / or cache memory 822, and may further include read-only memory (ROM) 823.

[0129] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0130] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0131] Electronic device 800 can also communicate with one or more external devices 900 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 800, and / or with any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0132] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0133] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0134] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A data processing method for an autonomous driving device, characterized in that, include: Collect data from autonomous driving devices and store the data in the corresponding storage subdirectories according to the data type; Time slices are created based on the current system time and time interval configuration, and the time slices are used to build a global task queue in chronological order. Different tasks in the global task queue have their own time slices. For the current task in the global task queue, retrieve the target data corresponding to the time slice of the current task from the storage subdirectory, and preprocess the target data to obtain the processed data; The processed data is synchronized to a remote server to generate data analysis results for the autonomous driving device based on the processed data; The step of creating time slices based on the current system time and time interval configuration, and constructing a global task queue from the time slices in chronological order, includes: When the system starts, a date directory is created based on the current system time, and a slice directory, a temporary copy directory, and a directory to be uploaded are created under the date directory; a status log file is created under the date directory to record the current status of the time slice; Based on the time interval configuration, time slices are created, each time slice corresponding to an independent status record file; based on the date directory and status record files, the time slices are constructed in chronological order to form the global task queue; The time slices include a 0:00 time slice, a boot time slice, and a current time slice created based on the time interval configuration; the step of constructing the global task queue based on the date directory and status log file, according to the time slices in chronological order, includes: Scan all date directories and read the status log file under each date directory to obtain consecutive time slices in the first state, which indicates that the time slice directory has been created; construct the global task queue by sorting the time slices in the first state in ascending order of time.

2. The method according to claim 1, characterized in that, The step of retrieving target data corresponding to the time slice of the current task from the storage subdirectory for the current task in the global task queue includes: Obtain the current task and the next task adjacent to the current task from the global task queue; Cross-day verification is performed based on the time slices of the current task and the next task. If the current task and the next task are on different days and the next task is not at 0:00, then the current task is determined to be an abnormal task and is skipped. Otherwise, clear the temporary copy directory and copy the target data corresponding to the time slice of the current task in the storage subdirectory to the temporary copy directory; Set the current state of the status record file corresponding to the time slice of the current task to the second state, which is used to indicate that there is no data slice or that the data slice has been completed.

3. The method according to claim 2, characterized in that, The preprocessing of the target data to obtain processed data includes: The uncompressed data in the target data is compressed, and the compressed data in the target data is packaged using zero compression to obtain the processed data. Copy the processed data from the temporary copy directory to the directory to be uploaded; Set the current state of the status record file corresponding to the time slice of the current task to the third state, which indicates that data has been copied to the directory to be uploaded.

4. The method according to claim 1, characterized in that, The step of synchronizing the processed data to a remote server includes: The directory to be uploaded is scanned to determine whether the status record file corresponding to the time slice of the current task is in the third state; The rsync command is invoked to upload the processed data in the directory to be uploaded to the remote server.

5. The method according to claim 4, characterized in that, The step of calling the rsync command to upload the processed data in the directory to be uploaded to the remote server includes: Request recommended bandwidth from the remote server; A bandwidth limit is determined based on the obtained recommended bandwidth, and the bandwidth limit is used to limit the maximum network bandwidth that the rsync command can use when uploading data. The maximum rate limit is determined based on the current system's device type; Based on the bandwidth limit and the maximum rate limit, the process of uploading processed data from the directory to be uploaded to the remote server by calling the rsync command is subject to rate limiting control.

6. The method according to claim 3, characterized in that, After synchronizing the processed data to the remote server, the process also includes: Read the status log file of the remote server, add the files not recorded in the processed data as new tasks to the status log file of the remote server, and set the status of the new tasks to a first target status, which indicates that the task has been added; Filter the tasks that are in the first target state, copy the files corresponding to the tasks in the first target state to the conversion directory, and update the corresponding tasks to the second target state, which is used to indicate that the files have been copied; The files in the conversion directory are obtained and decompressed to obtain decompressed data. The task of decompression completion is updated to the third target status, which indicates that the directory has been decompressed. The target decompressed data in the decompressed data is converted into a new format to obtain the data to be analyzed. The step of generating the data analysis results of the autonomous driving device based on the processed data includes: Based on the configured time information, key information is extracted from the data to be analyzed to obtain a key information set, and the data analysis results are generated based on the key information set according to a preset template.

7. The method according to claim 6, characterized in that, The method further includes: According to a preset cycle, the data to be analyzed is updated based on the configured time information, and key information is extracted from the updated data to update the key information set. The data analysis results are generated based on the updated key information set according to the preset template.

8. The method according to any one of claims 1 to 7, characterized in that, The data collected from the autonomous driving device includes: Subscribe to the node messages of the autonomous driving device and store the node messages of each node in the corresponding cache container; When the preset triggering conditions are met, the data in the cache container is stored in the corresponding storage subdirectory according to the data type, and real-time data is recorded and stored in the corresponding storage subdirectory according to the data type to obtain the autonomous driving device data.

Citation Information

Patent Citations

  • Task scheduling method for automatic driving vehicle, equipment and storage medium

    CN111694647A

  • Data acquisition method, device and equipment for autonomous vehicle in vehicle-road cooperation scene, and medium

    CN114980027A