A method, device and storage medium for reading and writing vehicle-mounted data

Abstract

The application discloses a kind of vehicle-mounted data reading and writing method, equipment and storage medium, the method includes: receiving the original data detected in the process of automatic driving;Create data packet in line with binary log storage format RecordIO;With the packet metadata related to vehicle as header information is written into data packet;Under the condition of unbinding with packet metadata, the original data is written into data packet as message;With the key information related to message as foot information is written into data packet.This embodiment multiplexes RecordIO to realize custom data packet, can guarantee the reliability of data packet on storage format, unbinds message and packet metadata, improves the compatibility of message, when the format of message changes, old data can still be read normally, the ecological coupling degree of data packet and ROS is low, can be developed to middleware, improve the matching degree with middleware, reduce the cost of development.

Description

Technical Field

[0001] This invention relates to the technical field of vehicles, and more particularly to a method, device, and storage medium for reading and writing vehicle data. Background Technology

[0002] As vehicles continue to evolve towards intelligence and connectivity, their electronic and electrical architecture is shifting from traditional distributed architecture to domain architecture and centralized computing architecture. At the software level, due to increasingly shorter software iteration cycles, vehicle software architecture is also gradually upgrading from signal-oriented architecture to service-oriented architecture (SOA) to better achieve hardware-software decoupling and rapid software iteration.

[0003] Currently, most vehicles are equipped with ROS (Robot Operating System), which is an open-source software framework and toolset used to help developers build robot applications. It provides many functions such as hardware abstraction, device drivers, function libraries, visualization tools, messaging, and package management.

[0004] ROS has poor compatibility; changes in the format of data packets can cause old data to fail to be read. Furthermore, ROS data packets are tightly coupled with the ROS ecosystem and have a low degree of compatibility with middleware, which restricts development. Summary of the Invention

[0005] This invention provides a method, device, and storage medium for reading and writing vehicle data, in order to solve the problems of improving data packet compatibility and ecological decoupling of data packets.

[0006] According to one aspect of the present invention, a method for writing vehicle data is provided, applied in a vehicle, the method comprising:

[0007] Receive raw data detected during autonomous driving;

[0008] Create data packets that conform to the RecordIO binary log storage format;

[0009] The packet metadata related to the vehicle is written as header information into the data packet;

[0010] Under the condition of unbinding from the packet metadata, the original data is written as a message into the data packet;

[0011] Key information related to the message is written into the data packet as footer information.

[0012] According to another aspect of the present invention, a method for reading vehicle data is provided, applied in a vehicle, the method comprising:

[0013] Receive data packet read requests from a unified data interface;

[0014] If the read request is a single read request, then open a data packet conforming to the RecordIO binary log storage format;

[0015] Read the header information from the data packet to obtain packet metadata related to the vehicle;

[0016] Foot information is read from the data packet to obtain key information related to the message;

[0017] Under the condition of unbinding from the packet metadata, messages are filtered from the data packet based on the key information and output through the data interface.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the vehicle data writing method according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, the computer program being configured to cause a processor to execute and implement the method for writing vehicle data according to any embodiment of the present invention.

[0023] In this embodiment, raw data detected during autonomous driving is received; a data packet conforming to the RecordIO binary log storage format is created; vehicle-related packet metadata is written as header information into the data packet; with the binding to the packet metadata removed, the raw data is written as a message into the data packet; and key information related to the message is written as footer information into the data packet. This embodiment reuses RecordIO to implement a custom data packet, which ensures the reliability of the data packet in terms of storage format. Debinding the message from the packet metadata improves message compatibility, allowing the old data to be read normally even when the message format changes. The data packet has low coupling with the ROS ecosystem, allowing for development targeting middleware, improving compatibility with middleware, and reducing development costs.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a method for writing vehicle data according to Embodiment 1 of the present invention;

[0027] Figure 2 This is a vehicle architecture diagram provided according to Embodiment 1 of the present invention;

[0028] Figure 3 This is a flowchart of a method for reading vehicle data according to Embodiment 2 of the present invention;

[0029] Figure 4 This is a flowchart of a method for reading vehicle data according to Embodiment 3 of the present invention;

[0030] Figure 5 This is a flowchart of a method for reading vehicle data according to Embodiment 4 of the present invention;

[0031] Figure 6 This is a schematic diagram of the structure of a vehicle data writing device according to Embodiment 5 of the present invention;

[0032] Figure 7 This is a schematic diagram of the structure of a vehicle data reading device according to Embodiment Six of the present invention;

[0033] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] Figure 1 This is a flowchart illustrating a method for writing vehicle data according to Embodiment 1 of the present invention. This embodiment is applicable to writing custom vehicle data packets based on RecordIO. The method can be executed by a vehicle data writing device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0038] Step 101: Receive the raw data detected during the autonomous driving process.

[0039] In practical applications, vehicles with autonomous driving programs can be deployed according to business needs, such as garbage trucks, container trucks, taxis, and buses.

[0040] Autonomous driving programs can refer to control programs that support autonomous driving. Autonomous driving can mean that the vehicle itself has the ability to perceive the environment, plan the path, and autonomously control the vehicle, which is human-like driving using electronic technology to control the vehicle.

[0041] Based on the degree of control over vehicle operation, autonomous vehicles can be classified into L0 (No Automotion), L1 (Driver Assistance), L2 (Partial Automation), L3 (Conditional Automation), L4 (High Automation), and L5 (Full Automation).

[0042] like Figure 2As shown, vehicle 200 may include driving control equipment 201, body bus 202, ECU (Electronic Control Unit) 203, ECU 204, ECU 205, sensor 206, sensor 207, sensor 208, and actuator 209, actuator 210, actuator 211.

[0043] The driving control device (also known as the vehicle's brain) 201 is responsible for the overall intelligent control of the entire vehicle 200. The driving control device 201 can be a standalone controller, such as a CPU, heterogeneous processor (e.g., GPU, TPU, NPU), programmable logic controller (PLC), microcontroller, industrial control computer, etc.; it can also be a device composed of other electronic components with input / output ports and computational control functions; or it can be a computer device equipped with vehicle driving control applications. The driving control device can analyze and process data received from various ECUs and / or sensors on the body bus 202, make corresponding decisions, and send the corresponding instructions to the body bus.

[0044] The body bus 202 can be a bus used to connect driving control devices 201, ECUs 203, 204, and 205, sensors 206, 207, and 208, and other devices not shown in the vehicle 200. Because the high performance and reliability of the CAN (Controller Area Network) bus are widely recognized, the CAN bus is currently the most commonly used body bus in motor vehicles. Of course, it is understandable that the body bus can also be other types of buses.

[0045] The vehicle bus 202 can send the commands issued by the driving control device 201 to ECU 203, ECU 204, and ECU 205. ECU 203, ECU 204, and ECU 205 then analyze and process the commands and send them to the corresponding actuators for execution.

[0046] Sensors 206, 207, and 208 include, but are not limited to, multi-beam lidar, cameras, etc.

[0047] It should be understood that Figure 2 The number of vehicles, driving control devices, body buses, ECUs, actuators, and sensors shown is merely illustrative. Any number of vehicles, driving control devices, body buses, ECUs, and sensors can be included depending on implementation requirements.

[0048] During the autonomous driving process, the vehicle can call on various sensors to detect various raw data. This raw data can be data of the vehicle itself, such as speed and acceleration, or data of the external environment of the vehicle, such as image data and point cloud data. This embodiment does not limit this.

[0049] Step 102: Create a data packet that conforms to the RecordIO binary log storage format.

[0050] RecordIO is configured in the vehicle's in-vehicle system. RecordIO is a highly reliable binary log storage format. The native RecordIO functionality is relatively simple and does not support features such as timestamp, topic, and indexing.

[0051] RecordIO offers several specific physical storage formats. Riegeli, for example, is a file format used to store sequences of string records, typically serialization protocol buffers. It supports dense compression, fast decoding, lookup, detection, and optional skipping of data corruption, filtering of raw message fields for faster decoding, and parallel encoding.

[0052] This embodiment designs a logical data format and data interface based on RecordIO (such as Riegeli), which not only makes full use of RecordIO's reliability and compatibility, but also provides metadata, indexing and other functions.

[0053] When writing raw data by calling the data interface, the writer of the data packet Mbag can be started. The writer generates the data packet Mbag according to the data format.

[0054] Step 103: Write the vehicle-related packet metadata as header information into the data packet.

[0055] In this embodiment, the storage format of the data packet Mbag includes a header, which can be used to store vehicle-related packet metadata (Mbag metadata), including static metadata and user-defined metadata, such as vehicle model, country / region code, city code, etc.

[0056] In one example, the header information of the data packet Mbag is as follows:

[0057] 1. vehicle_id:GZU_SPY_10026

[0058] 2. country_code:CN

[0059] 3. city_code: Guangzhou

[0060] In the specific implementation, the bag metadata related to the vehicle is queried, the name of the message type is queried, and the name of the message type is written into the bag metadata. For robustness, the name of the message type can also exist in the footer information and scattered appendix in the message messages.

[0061] Ignore message type definitions, that is, do not store message type definitions, in order to decouple messages from message types and achieve better compatibility.

[0062] Write the packet metadata as header information into the data packet Mbag.

[0063] Step 104: Under the condition of unbinding from the packet metadata, write the original data as a message into the data packet.

[0064] In this embodiment, the storage format of the data packet Mbag includes messages. A data packet Mbag can include multiple messages, forming a sequence.

[0065] Messages are the main content of the data packet Mbag, including the original data itself. Under the condition of unbinding the message from the packet metadata, the generator writer can write the original data as messages into the data packet Mbag according to the currently specified format.

[0066] In one embodiment of the present invention, the format of the messages includes<topic,timestamp,messagemetadata,payload> (Topic, timestamp, message metadata, payload), in this embodiment, step 104 may include the following steps:

[0067] Step 1041: For each piece of raw data, generate a message that is independent of the message type.

[0068] Since the bag metadata stores the name of the message type, not the definition of the message type, messages unrelated to the message type can be generated for each piece of raw data.

[0069] Step 1042: Write the subject, timestamp, and message metadata of the original data into the message respectively.

[0070] In this embodiment, for a given set of raw data, the topic to which the raw data belongs can be queried.

[0071] If the topic is a new topic, topic metadata is generated for the topic, and the topic name is mapped to a topic ID, which is then stored in the package metadata.

[0072] If the topic is an existing topic, query the topic ID (topicid) of the topic and write the topic ID (topicid) into the messages of the data packet Mbag.

[0073] For a given set of raw data, you can query the timestamp of the raw data and write the timestamp of the raw data into the messages of the data packet Mbag.

[0074] Message metadata is key information related to a message and can be defined by the user according to business needs. Examples include message ID, publish time, receive time, main sensor time, and so on.

[0075] For a given set of raw data, the message metadata of the raw data can be queried, and the message metadata of the raw data can be written sequentially into the messages of the data packet Mbag.

[0076] Step 1043: Convert the raw data into structured target data.

[0077] Step 1044: Write the target data as the payload into the message.

[0078] The payload is the message body of a message, and its format can be user-defined or third-party defined.

[0079] Given raw data, it can be converted into structured target data according to this format, and the target data can be written into the message as the payload.

[0080] In one format, the original data can be serialized using the Protobuf (Protocol Buffers) structured data serialization method to obtain a byte array, denoted as the target data. Serialization can serialize the structure of the original data into a format that can be easily stored and transmitted. Furthermore, Protobuf supports backward and forward compatibility, which can improve compatibility.

[0081] For example, the Protobuf format is defined as follows:

[0082] message CompressedImage{

[0083] Header header = 1;

[0084] string format = 2;

[0085] bytes data = 3;

[0086] bool is_metadata = 4;

[0087] uint32 status = 5;

[0088] }

[0089] Furthermore, when writing a message to the data packet Mbag, if the payload format is Protobuf, the original data can be serialized according to Protobuf first.

[0090] In one example, the message looks like this:

[0091] 1.topic: / camera_center / image_color / compressed

[0092] 2.timestamp:1669036927776876970#0.151547s,2022-11-21 05:22:07.776876

[0094] 3. message metadata:

[0095] a.publish_time:1669036927776811220

[0096] b.receive_time:1669036927776876970

[0097] c.main_sensor_time:1669036927776052992

[0098] d.message_id:29996

[0099] 4.message type:CompressedImage

[0100] 5.message payload:an array of bytes

[0101] Step 1045: Write each message into the data packet in sequence.

[0102] When there are multiple messages, each message can be written into the data packet Mbag in a certain order.

[0103] In the specific implementation, check whether a sorting method is set. This sorting method is generally user-defined and sorted according to a certain parameter (such as the push time publish_time). The sorting method can be left as default.

[0104] If no sorting method is set, the messages will be written to the data packet Mbag in the order they were received at the time of receipt of the original message by default.

[0105] If a sorting method has been set, then each message can be written into the data packet Mbag in the order specified by the sorting method.

[0106] Step 105: Write the key information related to the message as footer information into the data packet.

[0107] In this embodiment, the storage format of the data packet Mbag includes a footer, which is used to record key information related to the message.

[0108] If the original data's topic is a new topic, then topic metadata is generated for the original data's topic.

[0109] If the timestamp of the original data exceeds a preset time interval, the physical address of the message will be written into the index information indices.

[0110] For example, if a data packet Mbag contains 10 messages, the addresses and timestamps of these 10 messages in the data packet Mbag are respectively...<address1,t1> ,<address2,t2> , ...,

[0111] <address10,t10> Therefore, the index information `indices` includes a set of arrays [<t1,address1> ,<t2,address2> , ...,<t10,address10> ].

[0112] Write at least one of the topic metadata, the statistics of the raw data, and the indices as a special record into the footer of the data packet Mbag.

[0113] In this embodiment, raw data detected during autonomous driving is received; a data packet conforming to the RecordIO binary log storage format is created; vehicle-related packet metadata is written as header information into the data packet; with the binding to the packet metadata removed, the raw data is written as a message into the data packet; and key information related to the message is written as footer information into the data packet. This embodiment reuses RecordIO to implement a custom data packet, which ensures the reliability of the data packet in terms of storage format. Debinding the message from the packet metadata improves message compatibility, allowing the old data to be read normally even when the message format changes. The data packet has low coupling with the ROS ecosystem, allowing for development targeting middleware, improving compatibility with middleware, and reducing development costs.

[0114] Example 2

[0115] Figure 3 This is a flowchart of a method for reading vehicle data according to Embodiment 2 of the present invention. This embodiment is applicable to reading custom vehicle data packets based on RecordIO. This method can be executed by a vehicle data reading device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 3 As shown, the method includes:

[0116] Step 301: Receive a data packet read request from the unified data interface.

[0117] RecordIO is configured in the vehicle's onboard system. This embodiment designs a set of logical data formats and data interfaces based on RecordIO (such as Riegeli), which not only makes full use of RecordIO's reliability and compatibility, but also provides metadata, indexing and other functions.

[0118] The vehicle can use the method of Embodiment 1 to call the data interface to generate a data packet Mbag. This embodiment provides multiple modes for reading the data packet Mbag. Different modes all call a unified data interface to read the data packet Mbag. Therefore, when receiving a read request for the data packet Mbag from the unified data interface, the type of the read request can be identified, thereby determining the mode for reading the data packet Mbag.

[0119] Step 302: If the read request is a single read request, then open the data packet that conforms to the RecordIO binary log storage format.

[0120] This embodiment provides a mode for reading data packets Mbag individually. If the read request is a single read request, indicating a mode for reading data packets Mbag individually, then the reader for data packets Mbag can be started. The reader reads data packets Mbag individually according to a predetermined data format.

[0121] At this point, the reader can open a data packet Mbag that conforms to the RecordIO binary log storage format (such as Riegeli).

[0122] Step 303: Read the header information from the data packet to obtain the packet metadata related to the vehicle.

[0123] The reader reads the header information from the position specified in the packet Mbag. The header information records the packet metadata (Mbag metadata) related to the vehicle, including static metadata and user-defined metadata, such as the vehicle model, country / region code, city code, etc.

[0124] Step 304: Read the foot information from the data packet to obtain key information related to the message.

[0125] The reader jumps to the last record of RecordIO and reads the key information related to the message from the footer.

[0126] Step 305: After unbinding from the packet metadata, filter messages from the data packet based on key information and output the messages through the data interface.

[0127] In this embodiment, the binding between the message and the packet metadata (Mbag metadata) is removed. In this case, each message can be traversed from the data packet Mbag based on key information, and the appropriate message can be selected and output through a unified data interface.

[0128] In one embodiment of the present invention, if the packet metadata (Mbag metadata) contains the name of the message type, then in this embodiment, step 305 may include the following steps:

[0129] Step 3051: Set filter criteria.

[0130] In this embodiment, one or more filtering conditions can be set according to the business requirements of autonomous driving.

[0131] For example, the filtering criteria include a topic whitelist and / or a timestamp range, where the timestamp range is in the format <start time, end time>.<start_time,end_time> express.

[0132] Step 3052: Read messages unrelated to message type from the data packet based on key information.

[0133] Debinding the message from the packet metadata (Mbag metadata) can mean ignoring the message type definition, i.e., not storing the message type definition. In this case, the message unrelated to the message type definition can be read from the packet Mbag based on key information (such as index information).

[0134] If the filtering criteria include the timestamp range, then a binary search can be used in the index information (indices) to search for the start time of the timestamp range (start_time), jump to the last record (Record) whose start time is less than the start time (start_time), and thus obtain the message regardless of the message type.

[0135] Step 3053: If the reading is complete, filter out messages that do not meet the filtering criteria.

[0136] Iterate through the read messages, compare the parameters of each message with the filtering conditions. If the parameters of a message meet the filtering conditions, keep the message; otherwise, filter it out.

[0137] In one example, the topic of the raw data can be read from the message and compared with the topic whitelist.

[0138] If the topic of the original data is in the topic whitelist, then the message will be retained.

[0139] If the topic of the original data is not in the topic whitelist, the message will be filtered out.

[0140] In another example, the timestamp of the raw data can be read from the message and compared with the timestamp range.

[0141] If the timestamp of the original data is within the timestamp range, then the message is retained.

[0142] If the timestamp of the original data is not within the timestamp range, the message will be filtered out.

[0143] Of course, the above-described screening conditions and filtering methods are merely examples. When implementing the embodiments of the present invention, other screening conditions and filtering methods can be set according to actual circumstances, and the embodiments of the present invention do not impose any limitations on this. Furthermore, in addition to the above-described screening conditions and filtering methods, those skilled in the art can also employ other screening conditions and filtering methods according to actual needs, and the embodiments of the present invention do not impose any limitations on this.

[0144] Step 3054: If filtering is complete, output a message through the data interface to read the payload containing the original data from the message.

[0145] For the remaining messages after filtering, the message can be output through a unified data interface. Devices that need the message can read the payload from the message, parse the payload according to a preset structure, and obtain the original data.

[0146] For example, a message can be output through a unified data interface, so that the device that needs the message can read the payload from the message and perform deserialization on the payload according to the Protobuf structured data serialization method to obtain the original data.

[0147] This embodiment receives data packet read requests from a unified data interface. If the read request is a single read request, a data packet conforming to the RecordIO binary log storage format is opened. Header information is read from the data packet to obtain vehicle-related metadata. Footer information is read from the data packet to obtain key information related to the message. With the binding to the metadata removed, messages are filtered from the data packet based on the key information and output through the data interface. This embodiment reuses RecordIO to implement custom data packets, ensuring the reliability of the data packet's storage format. Debinding messages from metadata improves message compatibility; even when the message format changes, old data can still be read normally. The data packet has low coupling with the ROS ecosystem, allowing for development targeting middleware, improving compatibility with middleware, and reducing development costs.

[0148] Example 3

[0149] Figure 4 This is a flowchart of a method for writing vehicle data according to Embodiment 3 of the present invention. This embodiment adds an operation to jointly read data packets Mbag generated by multiple devices, based on the previous embodiment. For example... Figure 4 As shown, the method includes:

[0150] Step 401: Receive a data packet read request from the unified data interface.

[0151] Step 402: If the read request is a multi-read request, then open multiple data packets that conform to the RecordIO binary log storage format simultaneously.

[0152] This embodiment provides a mode for jointly reading data packets Mbag generated by multiple devices. For example, a vehicle may contain a camera and a LiDAR. The camera generates image data and the LiDAR generates point cloud data. When detecting obstacles, the image data generated by the camera (stored in the form of a data packet Mbag) and the point cloud data generated by the LiDAR (stored in the form of a data packet Mbag) are read together.

[0153] If the read request is a multi-read request, indicating a mode of jointly reading data packets Mbag generated by multiple devices, then the reader of the data packets Mbag can be started. The reader reads multiple data packets Mbag in combination according to the predetermined data format to meet the business requirements.

[0154] Step 403: Read the header information from the data packet to obtain the packet metadata related to the vehicle.

[0155] The reader reads the header information from the position specified in the packet Mbag. The header information records the packet metadata (Mbag metadata) related to the vehicle, including static metadata and user-defined metadata, such as the vehicle model, country / region code, city code, etc.

[0156] Step 404: Read the foot information from the data packet to obtain key information related to the message.

[0157] The reader jumps to the last record of RecordIO and reads the key information related to the message from the footer.

[0158] Step 405: After unbinding from the packet metadata, filter messages from the data packet based on key information.

[0159] In this embodiment, the binding between the message and the packet metadata (Mbag metadata) is removed. In this case, each message can be traversed from the data packet Mbag based on key information, and the appropriate message can be selected from it.

[0160] In one embodiment of the present invention, if the packet metadata (Mbag metadata) contains the name of the message type, then in this embodiment, step 405 may include the following steps:

[0161] Step 4051: Set filter criteria.

[0162] In this embodiment, one or more filtering conditions can be set according to the business requirements of autonomous driving.

[0163] For example, the filtering criteria include a topic whitelist and / or a timestamp range, where the timestamp range is in the format <start time, end time>.<start_time,end_time> express.

[0164] Step 4052: Read messages unrelated to message type from the data packet based on key information.

[0165] Debinding the message from the packet metadata (Mbag metadata) can mean ignoring the message type definition, i.e., not storing the message type definition. In this case, the message unrelated to the message type definition can be read from the packet Mbag based on key information (such as index information).

[0166] If the filtering criteria include the timestamp range, then a binary search can be used in the index information (indices) to search for the start time of the timestamp range (start_time), jump to the last record (Record) whose start time is less than the start time (start_time), and thus obtain the message regardless of the message type.

[0167] Step 4053: If the reading is complete, filter out messages that do not meet the filtering criteria.

[0168] Iterate through the read messages, compare the parameters of each message with the filtering conditions. If the parameters of a message meet the filtering conditions, keep the message; otherwise, filter it out.

[0169] In one example, the topic of the raw data can be read from the message and compared with the topic whitelist.

[0170] If the topic of the original data is in the topic whitelist, then the message will be retained.

[0171] If the topic of the original data is not in the topic whitelist, the message will be filtered out.

[0172] In another example, the timestamp of the raw data can be read from the message and compared with the timestamp range.

[0173] If the timestamp of the original data is within the timestamp range, then the message is retained.

[0174] If the timestamp of the original data is not within the timestamp range, the message will be filtered out.

[0175] Of course, the above-described screening conditions and filtering methods are merely examples. When implementing the embodiments of the present invention, other screening conditions and filtering methods can be set according to actual circumstances, and the embodiments of the present invention do not impose any limitations on this. Furthermore, in addition to the above-described screening conditions and filtering methods, those skilled in the art can also employ other screening conditions and filtering methods according to actual needs, and the embodiments of the present invention do not impose any limitations on this.

[0176] Step 406: Write the message to the preset first priority queue.

[0177] In this embodiment, a first priority queue is maintained. The key of the first priority queue is a timestamp, etc. When each message packet Mbag reads a message (i.e., the message remaining after filtering), the message and related index information are entered into the first priority queue.

[0178] Step 407: Read messages from the first priority queue and output messages through the data interface.

[0179] Messages are read from the first priority queue in order of timestamp, etc. The message can be output through a unified data interface. Devices that need the message read the payload from the message, parse the payload according to the preset structure, and obtain the raw data.

[0180] For example, a message can be output through a unified data interface, so that the device that needs the message can read the payload from the message and perform deserialization on the payload according to the Protobuf structured data serialization method to obtain the original data.

[0181] Example 4

[0182] Figure 5 This is a flowchart of a method for writing vehicle data according to Embodiment 4 of the present invention. This embodiment adds an operation of reading messages in a custom order based on the previous embodiment. Figure 5 As shown, the method includes:

[0183] Step 501: Receive a data packet read request from the unified data interface.

[0184] Step 502: If the read request is a custom read request, then determine the custom read order.

[0185] Step 503: Open the data packet that conforms to the RecordIO binary log storage format.

[0186] This embodiment provides a user-defined order for reading data packets Mbag. If the read request is a custom read request, it indicates that the user has defined the order for reading data packets Mbag. In this case, the reader of data packets Mbag can be started to query the user-defined reading order. For example, data packets Mbag can be read in the order of push time (publish_time), in the order of receive time (receive_time), in the order of sensor time (main_sensor_time), and so on. The reader reads data packets Mbag in the user-defined order according to the predetermined data format to meet the needs of the business.

[0187] Step 504: Read the header information from the data packet to obtain the packet metadata related to the vehicle.

[0188] The reader reads the header information from the position specified in the packet Mbag. The header information records the packet metadata (Mbag metadata) related to the vehicle, including static metadata and user-defined metadata, such as the vehicle model, country / region code, city code, etc.

[0189] Step 505: Read the foot information from the data packet to obtain key information related to the message.

[0190] The reader jumps to the last record of RecordIO and reads the key information related to the message from the footer.

[0191] Step 506: Under the condition of unbinding from the packet metadata, read the messages located within the preset time period from the data packet based on the key information.

[0192] In this embodiment, the binding between the message and the packet metadata (Mbag metadata) is removed. In this case, messages within a preset time period can be read from the data packet Mbag based on key information.

[0193] Step 507: Write the message to the preset second priority queue.

[0194] In this embodiment, a second priority queue is maintained. For messages that are within a preset time period, the message can be entered into the second priority queue.

[0195] Step 508: Read messages from the second priority queue in the order of reading and output the messages through the data interface.

[0196] Messages are read from the second priority queue in the order they are read. The message can be output through a unified data interface. Devices that need the message read the payload from the message and parse the payload according to a preset structure to obtain the raw data.

[0197] For example, a message can be output through a unified data interface, so that the device that needs the message can read the payload from the message and perform deserialization on the payload according to the Protobuf structured data serialization method to obtain the original data.

[0198] Example 5

[0199] Figure 6 This is a schematic diagram of a vehicle data writing device provided in Embodiment 5 of the present invention. Figure 6 As shown, the device is used in a vehicle and includes:

[0200] The raw data receiving module 601 is used to receive raw data detected during autonomous driving.

[0201] The packet creation module 602 is used to create packets conforming to the RecordIO binary log storage format;

[0202] The header information writing module 603 is used to write packet metadata related to the vehicle as header information into the data packet;

[0203] The message writing module 604 is used to write the original data as a message into the data packet when the binding with the packet metadata is released;

[0204] The foot information writing module 605 is used to write key information related to the message as foot information into the data packet.

[0205] In one embodiment of the present invention, the header information writing module 603 is further configured to:

[0206] Query the packet metadata related to the vehicle;

[0207] When querying the name of a message type, the name of that message type is written into the packet metadata;

[0208] Ignore the definition of the message type to remove the binding relationship between the message and the message type;

[0209] The packet metadata is written into the data packet as header information.

[0210] In one embodiment of the present invention, the message writing module 604 is further configured to:

[0211] For each of the original data, a message unrelated to the message type is generated for the original data;

[0212] The subject, timestamp, and message metadata of the original data are written into the message respectively;

[0213] The raw data is converted into structured target data;

[0214] The target data is written as the payload into the message;

[0215] Each of the aforementioned messages is written into the data packet sequentially.

[0216] In one embodiment of the present invention, the message writing module 604 is further configured to:

[0217] The original data is serialized using the Protobuf structured data serialization method to obtain the target data.

[0218] In one embodiment of the present invention, the message writing module 604 is further configured to:

[0219] Check if a sorting method is set;

[0220] If no sorting method is set, the messages are written into the data packet in sequence according to the time when the original messages were received;

[0221] If a sorting method has been set, then each message is written into the data packet in sequence according to the sorting method.

[0222] In one embodiment of the present invention, the foot information writing module 605 is further configured to:

[0223] If the topic of the original data is a new topic, then topic metadata is generated for the topic of the original data;

[0224] If the timestamp of the original data exceeds a preset time interval, the physical address of the message is written into the index information;

[0225] At least one of the topic metadata, the statistical information of the original data, and the index information is written as a record into the footer information of the data packet.

[0226] The vehicle data writing device provided in this embodiment of the invention can execute the vehicle data writing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the vehicle data writing method.

[0227] Example 6

[0228] Figure 7 This is a schematic diagram of the structure of a vehicle data reading device provided in Embodiment Six of the present invention. Figure 7 As shown, the device is used in a vehicle and includes:

[0229] The read request receiving module 701 is used to receive read requests for data packets from a unified data interface;

[0230] The first data packet opening module 702 is used to open a data packet conforming to the RecordIO binary log storage format if the read request is a single read request.

[0231] The first header information reading module 703 is used to read header information from the data packet to obtain packet metadata related to the vehicle;

[0232] The first foot information reading module 704 is used to read foot information from the data packet to obtain key information related to the message;

[0233] The first message reading module 705 is used to filter messages from the data packet based on the key information and output the messages through the data interface, provided that the binding with the packet metadata is removed.

[0234] In one embodiment of the present invention, the packet metadata contains a message type name, and the first message reading module 705 is further configured to:

[0235] Set filter criteria;

[0236] Based on the key information, read the message, which is independent of the message type, from the data packet;

[0237] If the reading is complete, then filter out messages that do not meet the filtering criteria;

[0238] If filtering is complete, the message is output through the data interface to read the payload containing the original data from the message.

[0239] In one embodiment of the present invention, the filtering conditions include a whitelist and / or a time range; the first message reading module 705 is further configured to:

[0240] The topic of reading raw data from the message;

[0241] If the topic is not in the whitelist, the message is filtered out;

[0242] And / or,

[0243] Read the timestamp of the raw data from the message;

[0244] If the timestamp is not within the time range, the message is filtered out.

[0245] In one embodiment of the present invention, the first message reading module 705 is further configured to:

[0246] The message is output through the data interface to read the payload from the message and perform a deserialization operation on the payload according to the Protobuf structured data serialization method to obtain the original data.

[0247] In one embodiment of the present invention, it further includes:

[0248] The second data packet opening module is used to simultaneously open multiple data packets conforming to the RecordIO binary log storage format if the read request is a multi-read request.

[0249] The second header information reading module is used to read header information from the data packet to obtain packet metadata related to the vehicle;

[0250] The second foot information reading module is used to read foot information from the data packet to obtain key information related to the message;

[0251] The second message reading module is used to filter messages from the data packet based on the key information, provided that the binding with the packet metadata is removed.

[0252] The first priority queue write module is used to write the message into a preset first priority queue;

[0253] The first priority queue read module is used to read the message from the first priority queue and output the message through the data interface.

[0254] In one embodiment of the present invention, it further includes:

[0255] The read order confirmation module is used to determine a custom read order if the read request is a custom read request;

[0256] The third data packet opening module is used to open data packets that conform to the RecordIO binary log storage format;

[0257] The third header information reading module is used to read header information from the data packet to obtain packet metadata related to the vehicle;

[0258] The third-leg information reading module is used to read leg information from the data packet to obtain key information related to the message;

[0259] The third message reading module is used to read messages within a preset time period from the data packet based on the key information, provided that the binding with the packet metadata is removed.

[0260] The second priority queue write module is used to write the message into a preset second priority queue;

[0261] The second priority queue read module is used to read the message from the second priority queue according to the read order and output the message through the data interface.

[0262] The vehicle data reading device provided in this embodiment of the invention can execute the vehicle data reading method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the vehicle data reading method.

[0263] Example 7

[0264] Figure 8A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0265] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0266] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0267] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for writing or reading in-vehicle data.

[0268] In some embodiments, the method for writing or reading vehicle data can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for writing or reading vehicle data described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the method for writing or reading vehicle data by any other suitable means (e.g., by means of firmware).

[0269] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0270] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0271] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0272] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0273] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0274] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0275] Example 8

[0276] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the vehicle data writing method or the vehicle data reading method provided in any embodiment of this invention.

[0277] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0278] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0279] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for writing vehicle-mounted data, characterized in that, When applied to vehicles, the method includes: Receive raw data detected during autonomous driving; Create data packets that conform to the RecordIO binary log storage format; The packet metadata related to the vehicle is written as header information into the data packet; Ignoring the definition of message type, the original data is written as a message into the data packet; Key information related to the message is written into the data packet as footer information; The step of writing the packet metadata related to the vehicle as header information into the data packet includes: Query the packet metadata related to the vehicle, the packet metadata including at least one of the following: vehicle model, country / region code, city code; Query the name of the message type corresponding to the original data, and write the name of the message type into the packet metadata; Ignore the definition of the message type to remove the binding relationship between the message and the message type; The packet metadata is written into the data packet as header information.

2. The method according to claim 1, characterized in that, The step of writing the original data as a message into the data packet while ignoring the definition of the message type includes: For each of the original data, a message unrelated to the message type is generated for the original data; The subject, timestamp, and message metadata of the original data are written into the message respectively; The raw data is converted into structured target data; The target data is written as the payload into the message; Each of the aforementioned messages is written into the data packet sequentially.

3. The method according to claim 2, characterized in that, The process of converting the raw data into structured target data includes: The original data is serialized using the Protobuf structured data serialization method to obtain the target data.

4. The method according to claim 2, characterized in that, The step of sequentially writing each of the messages into the data packet includes: Check if a sorting method is set; If no sorting method is set, the messages are written into the data packet in sequence according to the time when the original data was received; If a sorting method has been set, then each message is written into the data packet in sequence according to the sorting method.

5. The method according to any one of claims 1-4, characterized in that, The step of writing key information related to the message as footer information into the data packet includes: If the topic of the original data is a new topic, then topic metadata is generated for the topic of the original data; If the timestamp of the original data exceeds a preset time interval, the physical address of the message is written into the index information; At least one of the topic metadata, the statistical information of the original data, and the index information is written as a record into the footer information of the data packet.

6. A method for reading vehicle-mounted data, characterized in that, When applied to vehicles, the method includes: Receive data packet read requests from a unified data interface; If the read request is a single read request, then open a data packet conforming to the RecordIO binary log storage format; Read the header information from the data packet to obtain packet metadata related to the vehicle. The packet metadata includes at least one of the following: vehicle model, country / region code, and city code. Foot information is read from the data packet to obtain key information related to the message; Ignoring the definition of message type, messages are filtered from the data packet based on the key information and output through the data interface.

7. The method according to claim 6, characterized in that, The packet metadata contains the name of the message type. The step of filtering messages from the data packet based on the key information and outputting the messages through the data interface, ignoring the definition of the message type, includes: Set filter criteria; Based on the key information, read the message, which is independent of the message type, from the data packet; If the reading is complete, then filter out messages that do not meet the filtering criteria; If filtering is complete, the message is output through the data interface to read the payload containing the original data from the message.

8. The method according to claim 7, characterized in that, The filtering criteria include a whitelist and / or a time range; The filtering of messages that do not meet the filtering criteria includes: The topic of reading raw data from the message; If the topic is not in the whitelist, the message is filtered out; And / or, Read the timestamp of the raw data from the message; If the timestamp is not within the time range, the message is filtered out.

9. The method according to claim 7, characterized in that, The step of outputting the message through the data interface to read the payload containing the original data from the message includes: The message is output through the data interface to read the payload from the message and perform a deserialization operation on the payload according to the Protobuf structured data serialization method to obtain the original data.

10. The method according to any one of claims 6-9, characterized in that, Also includes: If the read request is a multi-read request, then multiple data packets conforming to the RecordIO binary log storage format are opened simultaneously; Read the header information from the data packet to obtain packet metadata related to the vehicle; Foot information is read from the data packet to obtain key information related to the message; Ignoring the definition of message type, messages are filtered from the data packet based on the key information; Write the message into a preset first priority queue; The message is read from the first priority queue and output through the data interface.

11. The method according to any one of claims 6-9, characterized in that, Also includes: If the read request is a custom read request, then determine the custom read order; Open data packets that conform to the RecordIO binary log storage format; Read the header information from the data packet to obtain packet metadata related to the vehicle; Foot information is read from the data packet to obtain key information related to the message; Ignoring the definition of message type, messages within a preset time period are read from the data packet based on the key information; Write the message into a preset second priority queue; The message is read from the second priority queue in the read order and output through the data interface.

12. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for writing vehicle data according to any one of claims 1-5, or the method for reading vehicle data according to any one of claims 6-11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for writing vehicle data as described in any one of claims 1-5, or the method for reading vehicle data as described in any one of claims 6-11.

Images