Automatic driving log data returning method and device, and storage medium
By decoding video data from autonomous driving logs into image data and storing it along with non-video data, the problem of storage space occupation is solved, while ensuring the normal functioning of consumer nodes, thus achieving data volume reduction and functional transparency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU WERIDE TECH LTD CO
- Filing Date
- 2023-11-13
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies use image formats that increase storage space when storing autonomous driving log data, while changing the storage format can affect the processing capabilities of consumer nodes.
The camera data in video format is decoded into image data and written into a data queue along with non-video data. This data is then returned to the consumer node in a transparent manner, preserving the functionality of the consumer node.
This reduces the amount of data stored while ensuring that consumer nodes can perform their functions normally without requiring changes to the program code.
Smart Images
Figure CN117351590B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving technology, and in particular to an autonomous driving log data return method, apparatus, storage medium and computer device. Background Technology
[0002] During vehicle operation, autonomous vehicles record vehicle driving control data and environmental data collected by onboard data acquisition devices through autonomous driving log data, such as steering wheel data, camera data, and radar data. All autonomous driving log data are stored on the vehicle's hard drive.
[0003] Currently, existing technologies store camera data in image format. When downstream nodes and toolchain consumer nodes access camera data, the autonomous vehicle returns the camera data in image format, i.e., image data, allowing consumer nodes to perform subsequent processing based on the acquired image data. However, image format increases the amount of stored data, leading to excessive storage space occupied by autonomous driving log data. Changing the storage format of camera data, on the other hand, affects the subsequent processing by consumer nodes. Therefore, existing technologies struggle to reduce the amount of stored data while ensuring that consumer nodes can perform their functions correctly. Summary of the Invention
[0004] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the technical deficiency in the prior art that makes it difficult to ensure that consumer nodes can perform their functions normally while reducing the amount of stored data.
[0005] In a first aspect, embodiments of this application provide an autonomous driving log data return method, applied to a vehicle, wherein the vehicle stores various autonomous driving log data, each of the autonomous driving log data including camera data, the camera data being video data; the method includes:
[0006] If it is determined that autonomous driving log data needs to be returned, determine whether a data enqueue operation needs to be performed on the data queue according to the preset enqueue rules.
[0007] When a data queuing operation is required, log data to be processed is read from the vehicle, video data in the log data to be processed is decoded into image data, and the image data and non-video data in the log data to be processed are written into the data queue.
[0008] Retrieve and return the target queue log data from the data queue.
[0009] In one embodiment, the steps of reading log data to be processed from the vehicle, decoding video data in the log data to be processed into image data, and writing the image data and non-video data in the log data to be processed into the data queue include:
[0010] Read the log data to be processed corresponding to the current round from the vehicle;
[0011] If the log data to be processed in the current round is video data, then the video decoder is called based on the log data to be processed in the current round, and it is determined whether the log data to be processed corresponding to this data enqueue operation has not been completely read.
[0012] If the log data to be processed in the current round is non-video data, then the log data to be processed in the current round is written into the data queue, and it is determined whether the log data to be processed corresponding to this data enqueue operation has not been completely read.
[0013] If it is determined that the pending log data corresponding to this data enqueue operation has not been completely read, then proceed to the next round; otherwise, obtain the image data output by the video decoder and write the image data into the data queue.
[0014] In one embodiment, the step of invoking the video decoder based on the log data to be processed corresponding to the current round includes:
[0015] The video identifier of the log data to be processed in the current round is used as the target identifier, and the video decoder corresponding to the target identifier is called to decode the log data to be processed in the current round. The same video identifier corresponds to the same video decoder, and different video identifiers correspond to different video decoders.
[0016] In one embodiment, each piece of autonomous driving log data corresponds to a log recording timestamp; the step of reading the log data to be processed corresponding to the current round from the vehicle includes:
[0017] If the current round is the first round corresponding to this data enqueue operation, then the earliest log record timestamp corresponding to this data enqueue operation will be used as the data acquisition timestamp of the current round; otherwise, the log record timestamp that is later than the data acquisition timestamp of the previous round and adjacent to the data acquisition timestamp of the previous round will be used as the data acquisition timestamp of the current round.
[0018] Read the autonomous driving log data corresponding to the timestamp of the current round from the vehicle as the log data to be processed for the current round;
[0019] After the step of writing the image data into the data queue, the method further includes:
[0020] The log data in each queue of the data queue is sorted according to the log record timestamp from earliest to latest.
[0021] In one embodiment, the step of determining whether a data enqueue operation needs to be performed on the data queue according to a preset enqueue rule includes:
[0022] When the data queue includes at least one queue log data, the data time span of the data queue is determined; wherein, the data time span is the time interval between the latest log record timestamp and the earliest log record timestamp corresponding to each of the queue log data;
[0023] If the data time span is less than a first threshold, then it is determined that a data enqueue operation needs to be performed; wherein the first threshold is greater than 0.
[0024] In one embodiment, the steps of reading log data to be processed from the vehicle, decoding video data in the log data to be processed into image data, and writing the image data and non-video data in the log data to be processed into the data queue further include:
[0025] The time interval between the log record timestamp of the log data to be processed in the current round and the earliest log record timestamp of the data enqueue operation is taken as the read time span of the current round.
[0026] If the read time span corresponding to the current round is greater than or equal to the custom first threshold, it is determined that the log data to be processed has been read completely; otherwise, it is determined that the log data to be processed has not been read completely.
[0027] In one embodiment, the step of acquiring the image data output by the video decoder and writing the image data into the data queue includes:
[0028] Determine the total number of decoder calls corresponding to this data enqueue operation;
[0029] Based on the total number of decoder calls, each image data output by each video decoder is obtained and written into the data queue.
[0030] Secondly, embodiments of this application provide an autonomous driving log data return device, applied to a vehicle, wherein the vehicle stores various autonomous driving log data, each of the autonomous driving log data including camera data, the camera data being video data; the device includes:
[0031] The enqueue determination module is used to determine whether to perform a data enqueue operation on the data queue based on preset enqueue rules when it is determined that autonomous driving log data needs to be returned.
[0032] The data queuing module is used to read log data to be processed from the vehicle when a data queuing operation is required, decode the video data in the log data to be processed into image data, and write the image data and the non-video data in the log data to be processed into the data queue.
[0033] The data return module is used to retrieve and return target queue log data from the data queue.
[0034] Thirdly, embodiments of this application provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the autonomous driving log data return method described in any of the above embodiments.
[0035] Fourthly, embodiments of this application provide a computer device disposed in a vehicle, the computer device including: one or more processors, and a memory;
[0036] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the autonomous driving log data return method described in any of the above embodiments.
[0037] In the autonomous driving log data return method, apparatus, storage medium, and computer device provided in some embodiments of this application, the vehicle can store camera data in video format, thereby reducing the amount of autonomous driving log data stored. When storing camera data in video format, when autonomous driving log data needs to be returned, the vehicle can read the log data to be processed, decode the video data in the log data to be processed into image data and write it to a data queue, and write the non-video data in the log data to be processed into the data queue. Thus, when the consumer node calls the autonomous driving log data, the vehicle can return camera data in image format to the consumer node, thereby providing the camera data to the consumer node completely transparently in image format without modifying the consumer node's program code, and ensuring that the consumer node can perform its functions normally. Therefore, this application can reduce the amount of stored data while ensuring that the consumer node can perform its functions normally. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 This is an application environment diagram of an autonomous driving log data return method in one embodiment;
[0040] Figure 2 This is a flowchart illustrating an autonomous driving log data return method in one embodiment;
[0041] Figure 3 This is a flowchart illustrating the data queuing step in one embodiment;
[0042] Figure 4 This is a schematic structural block diagram of an autonomous driving log data return device in one embodiment;
[0043] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0045] In one embodiment, the autonomous driving log data return method provided in this application can be applied to Figure 1 In the application environment shown, vehicle 102 can have autonomous driving capabilities and is equipped with onboard data acquisition equipment. Vehicle 102 can generate autonomous driving log data based on vehicle driving control data and environmental data collected by the onboard data acquisition equipment, and store the autonomous driving log data in the local storage device of vehicle 102. The environmental data includes, but is not limited to, any one or any combination of data such as steering wheel data, camera data, and radar data.
[0046] In some situations, such as during algorithm iteration, simulation data collection, and road condition analysis, other devices need to acquire the autonomous driving data of vehicle 102 to facilitate subsequent analysis and processing based on the autonomous driving log data of vehicle 102. In this case, vehicle 102, which provides the autonomous driving log data, can act as a producer node, and other devices acquiring the autonomous driving data can act as consumer nodes 104. In one example, consumer node 104 can be, but is not limited to, a server, a remote control device, or other vehicles besides the vehicle itself.
[0047] In one embodiment, this application provides a method for returning autonomous driving log data. The following embodiments apply this method to... Figure 1 The vehicle shown is used as an example for illustration. This vehicle can store various autonomous driving log data, and each autonomous driving log data includes camera data stored in video format, that is, the camera data is stored in the vehicle as video data.
[0048] like Figure 2 As shown, the autonomous driving log data return method of this application may specifically include the following steps:
[0049] S202: If it is determined that autonomous driving log data needs to be returned, determine whether it is necessary to perform a data enqueue operation on the data queue according to the preset enqueue rules.
[0050] It is understandable that a vehicle can determine whether it needs to return its own autonomous driving log data based on any rules and in any way. For example, a vehicle can determine that it needs to return autonomous driving log data upon receiving a data retrieval request from a consumer node. Alternatively, a vehicle can determine that it needs to return autonomous driving log data if it detects that it contains at least one piece of autonomous driving log data.
[0051] When it is determined that autonomous driving log data needs to be returned, the vehicle can decide whether to perform a data enqueue operation on the data queue based on preset enqueue rules. The data queue can be a queue used to store log data to be returned; that is, all log data in the data queue is log data to be returned. Performing a data enqueue operation on the data queue means writing data into the data queue, thus making the data queue store that data.
[0052] It is understood that the specific content of the preset enqueue rules in this document can be determined based on the actual situation, and this application does not impose specific limitations on it. For example, the preset enqueue rules can determine that a data enqueue operation needs to be performed when the data queue is detected to be empty, and determine that a data enqueue operation does not need to be performed when the data queue is detected to be not empty. Alternatively, the preset enqueue rules can determine that a data enqueue operation needs to be performed when the amount of data in the data queue is detected to be less than a preset data amount threshold; otherwise, it determines that a data enqueue operation does not need to be performed.
[0053] S204: When a data enqueue operation is required, read the log data to be processed from the vehicle, decode the video data in the log data to be processed into image data, and write the image data and the non-video data in the log data to be processed into the data queue.
[0054] When it is determined that a data queuing operation needs to be performed, the vehicle can read at least one piece of autonomous driving log data from its local storage device as log data to be processed. Since camera data is video data, while other autonomous driving data (such as vehicle driving control data, steering wheel data, and radar data) is non-video data, the composition of the log data to be processed can be divided into three cases: the log data to be processed includes only video data, the log data to be processed includes only non-video data, and the log data to be processed includes both video data and non-video data.
[0055] When the log data to be processed includes video data, to ensure that consumer nodes can obtain processable autonomous driving log data without changing the original API (Application Programming Interface) and program code, the vehicle can decode the video data in the log data to be processed, converting the video data into image format data and obtaining the corresponding image data. The vehicle can then write the decoded image data into a data queue. Furthermore, in one example, the vehicle can obtain a custom target video format and target image format, and decode the video data in the target video format into image data in the target image format, thereby improving the flexibility of data return.
[0056] When the log data to be processed includes non-video data, since the consumer node can process non-video data, the vehicle can directly write the non-video data in the log data to be processed into the data queue.
[0057] Based on this, for the aforementioned three data composition scenarios, the vehicle can implement data entry in the following manner: If the log data to be processed only includes video data, the vehicle can decode the log data to be processed into image data and write the image data into the data queue; if the log data to be processed only includes non-video data, the vehicle can directly write the log data to be processed into the data queue; if the log data to be processed includes both video data and non-video data, the vehicle can decode the video data into image data and write both the image data and non-video data into the data queue. This ensures that the data format of the log data in each queue is one that can be directly processed by the consumer nodes.
[0058] S206: Retrieve and return the target queue log data from the data queue.
[0059] If it is determined that no data enqueue operation is needed, or if the current data enqueue operation has been completed, the vehicle can retrieve the target queue log data from the data queue and return it to the consumer. This means dequeuing the target queue log data from the data queue; after dequeueing, the vehicle will delete the target queue log data from the data queue.
[0060] Furthermore, in one example, before returning the target queue data, the vehicle can encapsulate the target queue data according to a preset format to obtain the target message. The vehicle can then return this target message to the consumer node.
[0061] In this embodiment, the vehicle can store camera data in video format, thereby reducing the amount of autonomous driving log data stored. When storing camera data in video format, when autonomous driving log data needs to be returned, the vehicle can read the log data to be processed, decode the video data in the log data to be processed into image data and write it into the data queue, and write the non-video data in the log data to be processed into the data queue. Thus, when the consumer node calls the autonomous driving log data, the vehicle can return camera data in image format to the consumer node, thereby providing the camera data to the consumer node completely transparently in image format without modifying the consumer node's program code, ensuring that the consumer node can perform its functions normally. Therefore, this application can reduce the amount of stored data while ensuring that the consumer node can perform its functions normally.
[0062] In one embodiment, such as Figure 3 As shown, the steps of reading log data to be processed from the vehicle, decoding video data in the log data to be processed into image data, and writing the image data and non-video data in the log data to be processed into the data queue include:
[0063] S302: Read the log data to be processed corresponding to the current round from the vehicle;
[0064] S304: If the log data to be processed in the current round is video data, then call the video decoder based on the log data to be processed in the current round, and determine whether the log data to be processed corresponding to this data enqueue operation has not been completely read.
[0065] S306: If the log data to be processed corresponding to the current round is non-video data, then write the log data to be processed corresponding to the current round into the data queue, and determine whether the log data to be processed corresponding to this data enqueue operation has not been completely read.
[0066] S308: If it is determined that the log data to be processed corresponding to this data enqueue operation has not been completely read, proceed to the next round; otherwise, obtain the image data output by the video decoder and write the image data into the data queue.
[0067] Since both video decoding and data queuing require a certain amount of time, in order to improve data return efficiency, in this embodiment, the vehicle can write multiple autonomous driving log data into the data queue in each data queuing operation, and simultaneously perform video decoding and non-video data queuing in each data queuing operation through asynchronous decoding.
[0068] Specifically, the vehicle can read multiple pending log data sets across multiple rounds, and read only one pending log data set within a single round. For the current round, the vehicle can read the pending log data set corresponding to the current round from the local storage device. Further, in one example, each autonomous driving log data set stored in the vehicle can correspond to a log recording timestamp, recording the generation time of each autonomous driving log data set, or the time when the log data was written to the storage device. In this example, the vehicle can determine the pending log data set corresponding to the current round based on the log recording timestamps from earliest to latest, thus achieving sequential reading.
[0069] The log data to be processed in the current round can be either video data or non-video data. When the log data to be processed in the current round is video data, the vehicle can call the video decoder to decode the log data. After calling the video decoder, regardless of whether the video decoder has finished decoding, if it is determined that the log data to be processed corresponding to this data enqueue operation has not been completely read, the vehicle can proceed to the next round to continue reading the next log data to be processed.
[0070] When the log data to be processed in the current round is non-video data, the vehicle can directly write the log data to be processed into the data queue, so that the data queue stores the log data to be processed. The vehicle can proceed to the next round to continue reading the next log data to be processed if it is determined that the log data to be processed corresponding to the current data enqueue operation has not been completely read.
[0071] After calling the video decoder or writing non-video data into the data queue, and after confirming that the log data to be processed corresponding to this data enqueue operation has been read, the vehicle can obtain the decoding result output by the video decoder, that is, obtain the image data, and write the image data into the data queue to enqueue all the log data corresponding to this data enqueue operation.
[0072] It should be noted that in this article, a single data enqueue operation can refer to the process of writing multiple autonomous driving log data into a data queue. In other words, writing N autonomous driving log data into a data queue is considered a single data enqueue operation. Here, N is a positive number greater than 1, and its value can be determined based on the actual situation. The N values for different data enqueue operations can be the same or different. In one example, if the amount of logs M read corresponding to this data enqueue operation is less than the N corresponding to this data enqueue operation, it can be determined that the log data to be processed for this data enqueue operation has not been completely read; otherwise, it can be determined that the log data to be processed for this data enqueue operation has been completely read.
[0073] In one embodiment, the step of invoking the video decoder based on the log data to be processed corresponding to the current round includes:
[0074] The video identifier of the log data to be processed in the current round is used as the target identifier, and the video decoder corresponding to the target identifier is called to decode the log data to be processed in the current round. The same video identifier corresponds to the same video decoder, and different video identifiers correspond to different video decoders.
[0075] Currently, video decoders introduce frame delay, meaning that video frames need to be decoded sequentially during the video decoding process. Due to various factors such as decoding complexity, dependencies between video frames, or system limitations, video decoders cannot immediately output the image data corresponding to each video frame after receiving it.
[0076] In this embodiment, if the log data to be processed corresponding to the current round is video data, the vehicle can determine the video identifier of the log data to be processed corresponding to the current round and use it as the target identifier. The video identifier can be used to establish decoding correlation between different video data. For example, video data collected by different vehicle-mounted cameras can correspond to different video identifiers, while video data collected by the same vehicle-mounted camera can correspond to the same video identifier.
[0077] In the current round, the vehicle can call the video decoder corresponding to the target identifier to perform video decoding. For any two video data points, if the video identifiers of the two video data points are the same, the vehicle can call the same video decoder to decode both video data points. If the video identifiers of the two video data points are different, the vehicle can call different video decoders to decode both video data points. Therefore, by calling the video decoder corresponding to the target identifier, the vehicle can assign video data points with the same video identifier to the same video decoder and video data points with different video identifiers to different video decoders.
[0078] Thus, this embodiment can adapt the video decoder according to the video identifier of the log data to be processed, so that video data with different video identifiers can be decoded concurrently using different video decoders, and video data with the same video identifier can be decoded using the same video decoder. In this way, multi-threaded parallel decoding can be achieved, thereby further improving the return efficiency of the autonomous driving logbook and meeting the consumer node's requirement for smooth playback.
[0079] In one embodiment, each piece of autonomous driving log data corresponds to a log recording timestamp. For a detailed explanation of the log recording timestamps, please refer to the above embodiments; they will not be repeated here.
[0080] The step of reading the log data to be processed corresponding to the current round from the vehicle includes:
[0081] If the current round is the first round corresponding to this data enqueue operation, then the earliest log record timestamp corresponding to this data enqueue operation will be used as the data acquisition timestamp of the current round; otherwise, the log record timestamp that is later than the data acquisition timestamp of the previous round and adjacent to the data acquisition timestamp of the previous round will be used as the data acquisition timestamp of the current round.
[0082] The autonomous driving log data corresponding to the data acquisition timestamp of the current round is read from the vehicle and used as the log data to be processed for the current round.
[0083] After the step of writing the image data into the data queue, the method further includes: sorting the log data of each queue in the data queue according to the log record timestamp from earliest to latest.
[0084] Specifically, in some cases, consumer nodes have requirements regarding the order in which they acquire autonomous driving log data. For example, when a consumer node needs to acquire and play the vehicle's driving video, the vehicle needs to return the autonomous driving log data to the consumer node in an orderly manner according to the timestamps of the log entries, so that the consumer node can play the driving video normally.
[0085] To ensure that the vehicle returns autonomous driving log data in the order of the log recording timestamps, thereby achieving complete API transparency and further ensuring that the function execution of the consumer node is not affected, in this embodiment, the vehicle can read autonomous driving log data and retrieve queue log data in the order of the log recording timestamps.
[0086] Specifically, during each round of reading log data to be processed, the vehicle can determine whether the current round is the first round corresponding to this data enqueue operation. If so, the earliest log record timestamp corresponding to this data enqueue operation can be used as the data acquisition timestamp for the current round. This earliest log record timestamp is the log record timestamp corresponding to the first log data to be processed read in this data enqueue operation. If the current round is not the first round corresponding to this data enqueue operation, the log record timestamp that is later than the data acquisition timestamp of the previous round and adjacent to the data acquisition timestamp of the previous round can be used as the data acquisition timestamp for the current round.
[0087] For example, a local storage device stores five autonomous driving log data sets. In chronological order, the timestamps for each log data set are T1, T2, T3, T4, and T5. If the current round is not the first round corresponding to this data enqueue operation, and the timestamp for the data retrieval in the previous round is T2, then the timestamp for the data retrieval in the current round is T3. It should be noted that this example is only for illustrating the process of determining the data retrieval timestamp and does not impose a limit on the amount of autonomous driving log data stored. It can be understood that the amount of autonomous driving log data stored on the local storage device can be determined based on the actual situation.
[0088] When determining the data acquisition timestamp corresponding to the current round, the vehicle can read the autonomous driving log data corresponding to that timestamp as the log data to be processed for the current round. For example, in the previous example, the vehicle can use the autonomous driving log data corresponding to T3 as the log data to be processed for the current round. In this way, the autonomous driving log data can be read in an orderly manner according to the chronological order of the log recording timestamps.
[0089] To achieve ordered retrieval of queued log data according to the chronological order of log record timestamps, and thus ordered output of autonomous driving log data, after writing image data to the data queue, the vehicle can sort the queued log data according to the chronological order of log record timestamps. After sorting, the queued log data corresponding to the earliest log record timestamp is located at the head of the data queue, and the queued log data corresponding to the latest log record timestamp is located at the tail of the data queue. In this way, when retrieving data from the data queue, the vehicle can retrieve the queued log data in ascending order of log record timestamps and return it, thus achieving ordered output. In one example, the data queue can be implemented using a priority queue.
[0090] This embodiment ensures the sequential return of autonomous driving log data by reading the log data to be processed in an orderly manner and sorting the log data in each queue according to the order of the log record timestamps. This achieves complete API transparency and further ensures that the function execution of consumer nodes is not affected.
[0091] In one embodiment, the step of determining whether a data enqueue operation needs to be performed on the data queue according to a preset enqueue rule includes:
[0092] When the data queue includes at least one queue log data, the data time span of the data queue is determined; wherein, the data time span is the time interval between the latest log record timestamp and the earliest log record timestamp corresponding to each of the queue log data;
[0093] If the data time span is less than a first threshold, then it is determined that a data enqueue operation needs to be performed; wherein the first threshold is greater than 0.
[0094] Specifically, when it is determined that autonomous driving log data needs to be returned, the vehicle can determine whether the data queue stores at least one queue log data. When the data queue includes at least one queue log data, the vehicle can further determine the time interval between the latest log record timestamp and the earliest log record timestamp corresponding to the data queue, that is, determine the data time span of the data queue. If the data time span is less than a first threshold, it can be determined that a data enqueue operation needs to be performed, and this data enqueue operation is performed according to the process described in any of the above embodiments. Since the first threshold is greater than 0, the vehicle will perform the data enqueue operation when the data queue still stores at least one queue log data. This ensures the orderliness between the newly enqueued queue log data and the already returned queue log data, ensuring that the vehicle outputs autonomous driving log data in an orderly manner at any time.
[0095] It is understandable that the specific value of the first threshold can be determined based on the actual situation. In one example, the first threshold could be 0.5 seconds.
[0096] Furthermore, in one example, if the data time span is greater than or equal to a first threshold, it can be determined that no data enqueue operation is needed, and the vehicle can retrieve the target queue log data from the data queue and return. In another example, if the data queue does not store queue log data, it indicates that the data queue is empty, and it is determined that a data enqueue operation needs to be performed.
[0097] In one embodiment, the steps of reading log data to be processed from the vehicle, decoding video data in the log data to be processed into image data, and writing the image data and non-video data in the log data to be processed into the data queue further include:
[0098] The time interval between the log record timestamp of the log data to be processed in the current round and the earliest log record timestamp of the data enqueue operation is taken as the read time span of the current round.
[0099] If the read time span corresponding to the current round is greater than or equal to the custom first threshold, it is determined that the log data to be processed has been read completely; otherwise, it is determined that the log data to be processed has not been read completely.
[0100] In this embodiment, during each data enqueueing operation, the vehicle can write autonomous driving log data with a time span equal to a first threshold into the data queue. Since the value of the first threshold can be customized, the cache size occupied by the data queue can be adjusted by changing the value of the first threshold, thereby improving the flexibility of data return.
[0101] In one embodiment, the step of acquiring the image data output by the video decoder and writing the image data into the data queue includes:
[0102] Determine the total number of decoder calls corresponding to this data enqueue operation;
[0103] Based on the total number of decoder calls, each image data output by each video decoder is obtained and written into the data queue.
[0104] The total number of decoder calls in this data queuing operation refers to the number of times the vehicle calls the video decoder during this data queuing operation. Since the vehicle can write multiple autonomous driving log data points to the data queue in each data queuing operation, in some cases, the vehicle may call the video decoder multiple times in a single data queuing operation, such as calling the same video decoder multiple times and / or calling at least two video decoders. To avoid data omissions, the vehicle can obtain each image data point output by each video decoder based on the total number of decoder calls and write each image data point to the data queue. This ensures that complete messages are enqueued and avoids data omissions.
[0105] The autonomous driving log data return device provided in the embodiments of this application is described below. The autonomous driving log data return device described below can be referred to in correspondence with the autonomous driving log data return method described above.
[0106] In one embodiment, this application also provides an autonomous driving log data return device 400, which is applied to a vehicle. The vehicle stores various autonomous driving log data, including camera data, which is video data. Figure 4 As shown, the device 400 specifically includes the following steps:
[0107] The enqueue determination module 410 is used to determine whether to perform a data enqueue operation on the data queue according to a preset enqueue rule when it is determined that autonomous driving log data needs to be returned.
[0108] The data queuing module 420 is used to read log data to be processed from the vehicle when a data queuing operation is required, decode the video data in the log data to be processed into image data, and write the image data and the non-video data in the log data to be processed into the data queue.
[0109] The data return module 430 is used to retrieve and return target queue log data from the data queue.
[0110] In one embodiment, the data queuing module 420 of this application includes a data reading unit, a decoding unit, a first queuing unit, and a second queuing unit. The data reading unit reads the log data to be processed corresponding to the current round from the vehicle. The decoding unit, if the log data to be processed corresponding to the current round is video data, calls the video decoder based on the log data to be processed corresponding to the current round and determines whether the log data to be processed corresponding to this data queuing operation has not been completely read. The first queuing unit, if the log data to be processed corresponding to the current round is not video data, writes the log data to be processed corresponding to the current round into the data queue and determines whether the log data to be processed corresponding to this data queuing operation has not been completely read. The second queuing unit, if it is determined that the log data to be processed corresponding to this data queuing operation has not been completely read, proceeds to the next round; otherwise, it acquires the image data output by the video decoder and writes the image data into the data queue.
[0111] In one embodiment, the decoding unit of this application includes a decoder invocation unit. This decoder invocation unit is used to take the video identifier of the log data to be processed corresponding to the current round as the target identifier, and invoke the video decoder corresponding to the target identifier to perform video decoding on the log data to be processed corresponding to the current round; wherein, the same video identifier corresponds to the same video decoder, and different video identifiers correspond to different video decoders.
[0112] In one embodiment, each piece of autonomous driving log data corresponds to a log recording timestamp. The data reading unit of this application includes a data acquisition timestamp determination timestamp and a log reading unit. The data acquisition timestamp determination timestamp is used to determine the earliest log recording timestamp corresponding to the current data enqueue operation if the current round is the first round corresponding to this data enqueue operation; otherwise, the log recording timestamp that is later than the previous round and adjacent to the previous round's data acquisition timestamp is used as the current round's data acquisition timestamp. The log reading unit is used to read the autonomous driving log data corresponding to the current round's data acquisition timestamp from the vehicle as the log data to be processed for the current round.
[0113] The data enqueue module 420 of this application also includes a sorting unit. This sorting unit is used to sort the log data of each queue in the data queue according to the log record timestamp from earliest to latest.
[0114] In one embodiment, the enqueue determination module 410 of this application includes a data time span determination unit and an enqueue determination unit. The data time span determination unit is used to determine the data time span of the data queue when the data queue includes at least one queue log data; wherein the data time span is the time interval between the latest log record timestamp and the earliest log record timestamp corresponding to each queue log data. The enqueue determination unit is used to determine that a data enqueue operation needs to be performed if the data time span is less than a first threshold; wherein the first threshold is greater than 0.
[0115] In one embodiment, the data queuing module 420 of this application further includes a read time span determination unit and a read judgment unit. The read time span determination unit is used to determine the read time span corresponding to the current round of log data to be processed, using the time interval between the log record timestamp of the current round's log data and the earliest log record timestamp corresponding to the current data queuing operation as the read time span for the current round. The read judgment unit is used to determine that the log data to be processed has been completely read if the read time span corresponding to the current round is greater than or equal to a custom first threshold; otherwise, it determines that the log data to be processed has not been completely read.
[0116] In one embodiment, the second queuing unit of this application includes a count determination unit and an image data writing unit. The count determination unit is used to determine the total number of decoder calls corresponding to this data queuing operation. The image data writing unit is used to acquire each image data output by each video decoder based on the total number of decoder calls, and write each image data into the data queue.
[0117] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the autonomous driving log data return method as described in any embodiment.
[0118] In one embodiment, this application also provides a computer device that can be installed in a vehicle, such as a vehicle controller or a microcontroller unit (MCU) of an autonomous vehicle. The computer device of this application stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform steps of the autonomous driving log data return method as described in any embodiment.
[0119] Indicatively, Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. (Refer to...) Figure 5The computer device 900 includes a processing component 902, which further includes one or more processors, and memory resources represented by memory 901 for storing instructions, such as application programs, that can be executed by the processing component 902. The application programs stored in memory 901 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 902 is configured to execute instructions to perform the steps of the autonomous driving log data return method described in any of the above embodiments.
[0120] The computer device 900 may also include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate on an operating system stored in memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.
[0121] Those skilled in the art will understand that the internal structure of the computer device shown in this application is merely a block diagram of a portion of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0122] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, "a," "an," "the," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. "Multiple" refers to at least two, such as 2, 3, 5, or 8, etc. "And / or" includes any and all combinations of the related listed items.
[0123] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
[0124] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for returning autonomous driving log data, characterized in that, Applied to a vehicle, the vehicle stores various autonomous driving log data, each of the autonomous driving log data including camera data, the camera data being video data; the method includes: If it is determined that autonomous driving log data needs to be returned, determine whether a data enqueue operation needs to be performed on the data queue according to the preset enqueue rules. When a data queuing operation is required, log data to be processed is read from the vehicle, video data in the log data to be processed is decoded into image data, and the image data and non-video data in the log data to be processed are written into the data queue. Retrieve and return the target queue log data from the data queue.
2. The method according to claim 1, characterized in that, The steps of reading log data to be processed from the vehicle, decoding video data in the log data to be processed into image data, and writing the image data and non-video data in the log data to be processed into the data queue include: Read the log data to be processed corresponding to the current round from the vehicle; If the log data to be processed in the current round is video data, then the video decoder is called based on the log data to be processed in the current round, and it is determined whether the log data to be processed corresponding to this data enqueue operation has not been completely read. If the log data to be processed in the current round is non-video data, then the log data to be processed in the current round is written into the data queue, and it is determined whether the log data to be processed corresponding to this data enqueue operation has not been completely read. If it is determined that the pending log data corresponding to this data enqueue operation has not been completely read, then proceed to the next round; otherwise, obtain the image data output by the video decoder and write the image data into the data queue.
3. The method according to claim 2, characterized in that, The step of calling the video decoder based on the log data to be processed corresponding to the current round includes: The video identifier of the log data to be processed in the current round is used as the target identifier, and the video decoder corresponding to the target identifier is called to decode the log data to be processed in the current round. The same video identifier corresponds to the same video decoder, and different video identifiers correspond to different video decoders.
4. The method according to claim 2, characterized in that, Each of the autonomous driving log data corresponds to a log record timestamp; the step of reading the log data to be processed corresponding to the current round from the vehicle includes: If the current round is the first round corresponding to this data enqueue operation, then the earliest log record timestamp corresponding to this data enqueue operation will be used as the data acquisition timestamp of the current round; otherwise, the log record timestamp that is later than the data acquisition timestamp of the previous round and adjacent to the data acquisition timestamp of the previous round will be used as the data acquisition timestamp of the current round. Read the autonomous driving log data corresponding to the timestamp of the current round from the vehicle as the log data to be processed for the current round; After the step of writing the image data into the data queue, the method further includes: The log data in each queue of the data queue is sorted according to the log record timestamp from earliest to latest.
5. The method according to claim 4, characterized in that, The step of determining whether a data enqueue operation needs to be performed on the data queue according to a preset enqueue rule includes: When the data queue includes at least one queue log data, the data time span of the data queue is determined; wherein, the data time span is the time interval between the latest log record timestamp and the earliest log record timestamp corresponding to each of the queue log data; If the data time span is less than a first threshold, then it is determined that a data enqueue operation needs to be performed; wherein the first threshold is greater than 0.
6. The method according to any one of claims 2 to 5, characterized in that, The steps of reading log data to be processed from the vehicle, decoding video data in the log data to be processed into image data, and writing the image data and non-video data in the log data to be processed into the data queue further include: The time interval between the log record timestamp of the log data to be processed in the current round and the earliest log record timestamp of the data enqueue operation is taken as the read time span of the current round. If the read time span corresponding to the current round is greater than or equal to the custom first threshold, it is determined that the log data to be processed has been read completely; otherwise, it is determined that the log data to be processed has not been read completely.
7. The method according to any one of claims 2 to 5, characterized in that, The step of acquiring the image data output by the video decoder and writing the image data into the data queue includes: Determine the total number of decoder calls corresponding to this data enqueue operation; Based on the total number of decoder calls, each image data output by each video decoder is obtained and written into the data queue.
8. An autonomous driving log data return device, characterized in that, Applied to a vehicle, the vehicle storing various autonomous driving log data, each of the autonomous driving log data including camera data, the camera data being video data; the device includes: The enqueue determination module is used to determine whether to perform a data enqueue operation on the data queue based on preset enqueue rules when it is determined that autonomous driving log data needs to be returned. The data queuing module is used to read log data to be processed from the vehicle when a data queuing operation is required, decode the video data in the log data to be processed into image data, and write the image data and the non-video data in the log data to be processed into the data queue. The data return module is used to retrieve and return target queue log data from the data queue.
9. A storage medium, characterized in that, The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the autonomous driving log data return method as described in any one of claims 1 to 7.
10. A computer device, characterized in that, The computer device, located in a vehicle, includes one or more processors and memory. The memory stores computer-readable instructions that, when executed by the one or more processors, perform the steps of the autonomous driving log data return method as described in any one of claims 1 to 7.