Autonomous driving data recording system

Abstract

Embodiments of the present disclosure provide an automatic driving data recording system, comprising: a processing unit; a power module connected to the processing unit to supply power to the processing unit; an interface unit electrically connected to the processing unit to receive at least one of video data and non-video data related to automatic driving, and send the received data to the processing unit; a first data storage unit electrically connected to the processing unit to cache at least one of the video data and the non-video data; and a second data storage unit electrically connected to the processing unit to store target data among the data cached by the first data storage unit, which is valid and corresponds to at least one of a timestamp event and a time period event.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of autonomous driving, and particularly to autonomous driving data recording systems. Background Technology

[0002] With the continuous evolution of autonomous driving technology, vehicles equipped with advanced autonomous driving functions are gradually entering the mass production stage. Against this backdrop, the safety and accident traceability of autonomous vehicles, especially the complete preservation and reliable retrieval of relevant data after an accident, have become essential functional requirements for autonomous vehicles.

[0003] Currently, most autonomous driving R&D vehicles are equipped with a device for storing almost all data. This stored data is primarily used for optimizing and iterating autonomous driving algorithms, as well as for identifying and analyzing problems during the R&D process. However, due to differences in the functional positioning and design goals of various manufacturers within the industry, existing devices are generally not designed and manufactured using automotive-grade standards, making it difficult to meet the application requirements of mass-produced vehicles.

[0004] In addition, some of the conventional devices used for storing data during road tests of autonomous vehicles are industrial control computers, whose hardware and software systems do not meet the requirements of automotive-grade products and cannot be mass-produced; others are simple consumer-grade devices that can store a small amount of data, which also do not meet the requirements of automotive-grade products and cannot be mass-produced.

[0005] To meet the mass production functional requirements of advanced autonomous vehicles, there is an urgent need for an onboard data storage device that meets automotive-grade standards, is feasible for mass production, and can ensure the complete preservation of accident data. Utility Model Content

[0006] In one aspect of this disclosure, an autonomous driving data recording system is provided, comprising: a processing unit; a power module connected to the processing unit to supply power to the processing unit; an interface unit electrically connected to the processing unit to receive at least one of video data and non-video data related to autonomous driving, and to send the received data to the processing unit; a first data storage unit electrically connected to the processing unit to cache the at least one of the video data and the non-video data; and a second data storage unit electrically connected to the processing unit to store valid target data from the data cached by the first data storage unit that corresponds to at least one of a timestamp event and a time period event.

[0007] In some embodiments, the interface unit is connected to at least one of the autonomous driving master system and the autonomous driving backup system to receive at least one of the video data and the non-video data.

[0008] In some embodiments, the first data storage unit includes dynamic random access memory, and the second data storage unit includes an embedded multimedia card.

[0009] In some embodiments, the interface unit is connected to a data reading device so that the data reading device reads data stored in the second data storage unit via the interface unit.

[0010] In some embodiments, the interface unit is adapted to transmit data via at least one of the following: Controller Area Network (CAN) bus, Controller Area Network Flexible Data Rate (CAN FD) bus, Local Interconnect Network (LIN) bus, Automotive Ethernet, Industrial Ethernet, Low Voltage Differential Signaling / Gigabit Multimedia Serial Link (LVDS / GMSL), and hardwire.

[0011] In some embodiments, the power module includes: a power interface adapted to connect to a main power supply; a power management module electrically connected between the power interface and the processing unit to supply power to the processing unit using power received via the power interface; and a backup power supply electrically connected to the processing unit to supply power to the processing unit in the event of a power outage at the power interface.

[0012] In some embodiments, the backup power source includes at least one of a lithium battery, a nickel-metal hydride battery, or a button cell battery.

[0013] In some embodiments, the processing unit includes a first sub-processing unit and a second sub-processing unit; the first sub-processing unit is connected to the first data storage unit and the second data storage unit to support operations related to the storage of autonomous driving data; the second sub-processing unit is connected to the first sub-processing unit and supports at least one of software over-the-air upgrades, data uploads, remote diagnostics, and on-site diagnostics; and the autonomous driving data recording system further includes a third data storage unit connected to the second data storage unit.

[0014] In some embodiments, the interface unit includes an Ethernet interface, a Universal Serial Bus (USB) interface, a wake-up interface, and a Controller Area Network (CAN) interface. The autonomous driving data recording system further includes a switch, a wake-up signal detection unit, and a CAN transceiver. The Ethernet interface is electrically connected to the first sub-processing unit via the switch, the USB interface is electrically connected to the first sub-processing unit, the wake-up interface is electrically connected to the second sub-processing unit via the wake-up signal detection unit, and the CAN interface is electrically connected to the second sub-processing unit via the CAN transceiver.

[0015] In some embodiments, the first sub-processing unit includes a system-on-a-chip, the second sub-processing unit includes a microcontroller (MCU), and the third data storage unit includes flash memory.

[0016] The autonomous driving data recording system of the present disclosure not only meets the requirements for autonomous driving data storage, but also meets the requirements for automotive-grade products and is mass-producible.

[0017] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0019] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0020] Figure 2 A schematic diagram of the architecture of an autonomous driving data recording system according to some embodiments of the present disclosure is shown; and

[0021] Figure 3 An example recording process for autonomous driving data according to an embodiment of this disclosure is shown. Detailed Implementation

[0022] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0023] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0024] As briefly mentioned above, in order to meet the mass production functional requirements of advanced autonomous vehicles, there is an urgent need for an onboard data storage device that meets automotive-grade standards, is feasible for mass production, and can ensure the complete preservation of accident data.

[0025] This disclosure provides an autonomous driving data recording system. The system includes a processing unit, a power module, an interface unit, a first data storage unit, and a second data storage unit. The power module is connected to and supplies power to the processing unit. The interface unit is electrically connected to the processing unit and is used to receive at least one of video data and non-video data related to autonomous driving, and to send the received data to the processing unit. The first data storage unit is electrically connected to the processing unit and is used to cache at least one of the video data and non-video data. The second data storage unit is also electrically connected to the processing unit to store valid target data from the data cached by the first data storage unit that corresponds to at least one of a timestamp event and a time period event. In this way, the autonomous driving data recording system of this disclosure meets both autonomous driving data storage requirements and automotive-grade product requirements, and is mass-producible.

[0026] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown. In this example environment, an autonomous driving data recording system 100 is used to record key data related to autonomous driving. The autonomous driving data recording system 100 can receive at least one of video data and non-video data from an autonomous driving main system 21 and an autonomous driving backup system 22.

[0027] The main autonomous driving system 21 can be the primary computing and decision-making unit responsible for the vehicle's autonomous driving functions, used for processing sensor information, planning driving paths, and generating control commands. The backup autonomous driving system 22 serves as redundancy and safety assurance; its architecture is typically independent of the main autonomous driving system 21. It can seamlessly take over vehicle control in the event of a failure or performance degradation in the main autonomous driving system 21 to ensure driving safety. One or more backup autonomous driving systems 22 can be installed in an autonomous vehicle as needed.

[0028] The data transmission method between the autonomous driving data recording system 100 and the autonomous driving main system 21 and the autonomous driving backup system 22 can be any one of the following: Controller Area Network (CAN) bus, Controller Area Network Flexible Data Rate (CAN FD) bus, Local Interconnect Network (LIN) bus, vehicle Ethernet, industrial Ethernet, Low Voltage Differential Signaling / Gigabit Multimedia Serial Link (LVDS / GMSL), hardwire, etc.

[0029] As an example, video data may include real-time road conditions, traffic signs, pedestrians, and other visual information captured by cameras. It should be understood that video data may also include other types of data. Furthermore, as an example, non-video data may include LiDAR data, millimeter-wave radar data, and the vehicle's own real-time status (such as speed, acceleration, steering angle, etc.). It should be understood that non-video data may also include other types of data.

[0030] Upon receiving at least one of video data and non-video data, the autonomous driving data recording system 100 can extract key data from it and store the key data in a data storage unit. In the following text, this will be combined with... Figure 2 and Figure 3 This section describes in detail the example architecture of the autonomous driving data recording system 100 and the example recording process of autonomous driving data.

[0031] like Figure 1 As shown, in this example environment, the autonomous driving data recording system 100 can be connected to the data reading device 30. The data reading device 30 can be any type of device, including handheld and non-handheld devices. The data reading device 30 can read the data stored in the autonomous driving data recording system 100 to facilitate subsequent data analysis and accident investigation. The data transmission method between the data reading device 30 and the autonomous driving data recording system 100 can be any of the following: CAN bus, CAN FD bus, LIN bus, automotive Ethernet, industrial Ethernet, LVDS / GMSL, hardwired, etc.

[0032] like Figure 1 As shown, in this example environment, the autonomous driving data recording system 100 can be connected to the field diagnostic tool 40 to facilitate on-site diagnostics of the autonomous driving data recording system 100. When system maintenance or troubleshooting is required, the operator can connect the field diagnostic tool 40 to the autonomous driving data recording system 100. The field diagnostic tool 40 can send specific diagnostic commands and / or query requests to the autonomous driving data recording system 100 and receive the diagnostic results returned by it.

[0033] like Figure 1 As shown, in this example environment, the autonomous driving data recording system 100 can upload data to a remote device 50, such as a cloud device, to facilitate data analysis and / or remote diagnostics. For example, the autonomous driving data recording system 100 can automatically encapsulate, encrypt, and compress data according to preset trigger conditions, and upload it to the remote device 50 via a wireless communication link.

[0034] like Figure 1As shown, in this example environment, the autonomous driving data recording system 100 can be upgraded via over-the-air (OTA) software updates. This OTA upgrade mechanism enables firmware optimization and functional expansion of the autonomous driving data recording system 100.

[0035] Next, we will refer to Figure 2 and Figure 3 This describes an example architecture of an autonomous driving data recording system 100 and an example process for recording autonomous driving data.

[0036] like Figure 2 As shown, the autonomous driving data recording system 100 described herein generally includes a processing unit 11, a power module 12, an interface unit 13, a first data storage unit 141, and a second data storage unit 142. The power module 12 is connected to the processing unit 11 and provides it with a stable and reliable power supply. The processing unit 11 is the core component of the autonomous driving data recording system 100, responsible for data processing, judgment, and scheduling.

[0037] In some embodiments, such as Figure 2 As shown, the power module 12 includes a power interface 121, a power management module 122, and a backup power supply 123. The power interface 121 is used to connect to the vehicle's main power supply, such as a 12V vehicle power supply. The power management module 122 is electrically connected between the power interface 121 and the processing unit 11, and is used to perform conversion, voltage regulation, filtering, and other processing on the input power to ensure that the processing unit 11 receives the required power. As an example, the power management module 122 can convert 12V voltage to 5V voltage to power the processing unit 11. The backup power supply 123 is electrically connected to the processing unit 11 and is used to provide emergency power to the processing unit 11 in the event of an unexpected power outage of the main power supply connected through the power interface 121, ensuring that the autonomous driving data recording system 100 continues to receive and store data from the autonomous driving main system 21 and the autonomous driving backup system 22, preventing data loss. For example, in the event of a collision, the backup power supply 123 can ensure that the system continues to store data for more than one minute. In some embodiments, the backup power supply 123 can be at least one of a lithium battery, a nickel-metal hydride battery, or a button battery. It should be understood that the backup power supply 123 may also be of other types, and the embodiments of this disclosure do not limit this.

[0038] like Figure 2As shown, interface unit 13 is electrically connected to processing unit 11, serving as a channel for the system to interact with external data sources and devices. Interface unit 13 is adapted to receive at least one of the following: video data related to autonomous driving (such as real-time road conditions, traffic signs, pedestrians, and other visual information captured by cameras, or other types of video data) and non-video data (such as LiDAR data, millimeter-wave radar data, the vehicle's own real-time status, or other types of non-video data), and transmit the received data to processing unit 11. Interface unit 13 can be connected to at least one of the autonomous driving main system 21 and the autonomous driving backup system 22 to receive this data from at least one of the autonomous driving main system 21 and the autonomous driving backup system 22.

[0039] In some embodiments, the interface unit 13 can support multiple communication protocols and physical interfaces. For example, data can be transmitted via at least one of the following methods: CAN bus, CAN FD bus, LIN bus, automotive Ethernet, industrial Ethernet, LVDS / GMSL, and hardwired.

[0040] In addition, the interface unit 13 is also adapted to be connected to an external data reading device 30, so that the data reading device 30 can read the data stored in the second data storage unit 142 through the interface unit 13.

[0041] like Figure 2 As shown, the first data storage unit 141 is electrically connected to the processing unit 11 and typically serves as a cache. The first data storage unit 141 is adapted to temporarily cache raw data received via the interface unit 13, such as at least one of video data and non-video data received from the primary autonomous driving system 21 and the secondary autonomous driving system 22. In some embodiments, the first data storage unit 141 includes dynamic random access memory, such as low-power double data rate (LPDDR) memory, which has fast read and write speeds and is suitable as a buffer for data streams. The first data storage unit 141 may include one or more caches. It should be understood that the first data storage unit 141 may also be any other suitable type for caching data received via the interface unit 13.

[0042] like Figure 2As shown, the second data storage unit 142 is electrically connected to the processing unit 11 to store valid target data from the data cached by the first data storage unit 141 that corresponds to at least one of a timestamp event and a time period event. The second data storage unit 142 can store critical data determined to need to be saved long-term or semi-permanently. In some embodiments, the second data storage unit 142 includes an embedded multimedia card (EMMC), which has the characteristics of large capacity and high reliability, suitable for data recording in an in-vehicle environment. The second data storage unit 142 may include one or more data storage areas. It should be understood that the second data storage unit 142 can also be any other suitable type for storing critical data related to predetermined events of autonomous driving.

[0043] Processing unit 11 can execute the core data recording logic; its example workflow can be found by referring to [reference needed]. Figure 3 .

[0044] like Figure 3 As shown, in block 301, processing unit 11 receives video data via interface unit 13. In block 311, processing unit 11 receives non-video data via interface unit 13. As described above, processing unit 11 can receive at least one of video data and non-video data.

[0045] In block 302, processing unit 11 parses the video data and caches it in the first data storage unit 141. In block 303, processing unit 11 parses the non-video data and caches it in the first data storage unit 141.

[0046] In block 303, processing unit 11 performs a validity check on the video data cached in the first data storage unit 141. In block 313, processing unit 11 performs a validity check on the non-video data cached in the first data storage unit 141. As an example, processing unit 11 can check whether data frames cached in the first data storage unit 141 are duplicated, missing, or have discontinuous sequence numbers. Furthermore, processing unit 11 can perform resolution and accuracy checks on the data cached in the first data storage unit 141. Through validity checks, data that does not meet the validity requirements can be removed, and only data that meets the requirements can be retained as valid data. In this way, it can be ensured that the data is not duplicated, the data is complete, and meets the requirements for resolution and accuracy.

[0047] In box 304, processing unit 11 determines whether the valid data in the cached data corresponds to at least one of a timestamp event and a time period event. A timestamp event refers to an event that, when a triggering condition is met, only records relevant data at the event's starting moment. For example, a timestamp event may include the activation and deactivation of an autonomous driving function, the issuance of an intervention request, a severe failure of the autonomous driving system, or a severe vehicle failure. A time period event refers to an event that, when a triggering condition is met, records relevant data for a period of time before and after the event's starting point. For example, a time period event may include the occurrence of a collision event or a collision risk event. For instance, when a collision event or a collision risk event occurs, data for 250ms before and after the event can be stored.

[0048] It should be noted that the figures, values, etc., mentioned above and elsewhere in this disclosure are exemplary and are not intended to limit the scope of this disclosure in any way. Any other suitable figures or values ​​are possible.

[0049] In blocks 305 and 306, in response to valid data corresponding to at least one of a timestamp event and a time period event, processing unit 11 stores the data corresponding to at least one of the timestamp event and the time period event in the second data storage unit 142 based on predetermined cyclic overwrite logic. In some embodiments, the predetermined cyclic overwrite logic may refer to overwriting old, non-critical, or expired data according to a certain strategy (such as first-in-first-out FIFO) when the storage space of the second data storage unit 142 is about to be full or has been full, in order to continuously record new event data and ensure the effective utilization of storage resources. If the valid data does not correspond to a timestamp event or a time period event, processing unit 11 will not store such valid data in the second data storage unit 142.

[0050] In box 307, the data reading device 30 can read the data stored in the second data storage unit 142.

[0051] In some embodiments, such as Figure 2 As shown, the processing unit 11 may include two sub-units, namely the first sub-processing unit 111 and the second sub-processing unit 112, to achieve better task division and system management.

[0052] like Figure 2As shown, the first sub-processing unit 111 is connected to the first data storage unit 141 and the second data storage unit 142, and is mainly responsible for performing operations related to the storage and retrieval of autonomous driving data, such as data reception, caching, validity judgment, timestamp and time period event judgment, and storage to the second data storage unit 142 based on loop overlay logic. As an example, the first sub-processing unit 111 may include a system-on-a-chip (SoC), which has strong processing capabilities and is suitable for processing large data volume tasks such as video.

[0053] The second sub-processing unit 112 is connected to the first sub-processing unit 111 and can support the basic functions of the autonomous driving data recording system 100, such as at least one of over-the-air (OTA) software upgrades, data uploads (to a cloud server), remote diagnostics, and on-site diagnostics. The second sub-processing unit 112 may include a microcontroller (MCU), which is suitable for tasks such as system management and interaction with the vehicle.

[0054] The SoC is mainly responsible for performing operations related to the storage and retrieval of autonomous driving data. It cannot access the KL15 wake-up signal, but the MCU can receive the KL15 hard-wired wake-up, bring out resource interfaces, perform power-on / off management and sleep / wake-up, and adapt to the whole vehicle for diagnosis and communication.

[0055] In some embodiments, such as Figure 2 As shown, the autonomous driving data recording system 100 may further include a third data storage unit 143, which is connected to the second sub-processing unit 112. The third data storage unit 143 can be used to store system firmware, configuration parameters, diagnostic logs, etc. As an example, the third data storage unit 143 includes flash memory.

[0056] In some embodiments, such as Figure 2 As shown, the interface unit 13 may include multiple independent physical interfaces, such as an Ethernet interface 131, a Universal Serial Bus (USB) interface 132, a wake-up interface 133, and a Controller Area Network (CAN) interface 134. These interfaces can be connected to the first sub-processing unit 111 and the second sub-processing unit 112 through corresponding components.

[0057] like Figure 2 As shown, Ethernet interface 131 can be electrically connected to the first sub-processing unit 111 via switch 151 for receiving high-speed data (such as video streams and point cloud data) and connecting to data reading device 30. For example, Ethernet interface 131 may include an Ethernet T1 interface and an Ethernet TX interface.

[0058] like Figure 2As shown, the USB interface 132 is electrically connected to the first sub-processing unit 111, which can serve as another way to connect to the data reading device 30, or to connect to an external storage device.

[0059] like Figure 2 As shown, the wake-up interface 133 is electrically connected to the second sub-processing unit 112 via the wake-up signal detection unit 152, for receiving the wake-up signal (e.g., KL15 signal) of the vehicle or system, and transmitting the wake-up signal to the second sub-processing unit 112 via the wake-up signal detection unit 152.

[0060] like Figure 2 As shown, the CAN interface 134 is electrically connected to the second sub-processing unit 112 via the CAN transceiver 153. It is used to receive non-video data (such as vehicle speed, steering angle, fault codes, etc.) on the vehicle bus and can be used to transmit diagnostic commands or status information.

[0061] like Figure 2 As shown, the interface unit 13 may also include other reserved IO interfaces 135, which can be used for input and output of other data.

[0062] The autonomous driving data recording system according to the embodiments of this disclosure not only meets the requirements for autonomous driving data storage, but also meets the requirements for automotive-grade products and is mass-producible.

[0063] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. An autonomous driving data recording system, characterized in that, include: Processing unit; A power module is connected to the processing unit to supply power to the processing unit; An interface unit, electrically connected to the processing unit, is used to receive at least one of video data and non-video data related to autonomous driving, and to send the received data to the processing unit. A first data storage unit is electrically connected to the processing unit to cache at least one of the video data and the non-video data; as well as The second data storage unit is electrically connected to the processing unit to store valid target data from the data cached by the first data storage unit that corresponds to at least one of a timestamp event and a time period event.

2. The autonomous driving data recording system according to claim 1, characterized in that, The interface unit is connected to at least one of the autonomous driving main system and the autonomous driving backup system to receive at least one of the video data and the non-video data.

3. The autonomous driving data recording system according to claim 1, characterized in that, The first data storage unit includes dynamic random access memory, and the second data storage unit includes an embedded multimedia card.

4. The autonomous driving data recording system according to claim 1, characterized in that, The interface unit is connected to the data reading device so that the data reading device can read the data stored in the second data storage unit via the interface unit.

5. The autonomous driving data recording system according to claim 1, characterized in that, The interface unit is adapted to transmit data via at least one of the following: Controller Area Network (CAN) bus, Controller Area Network Flexible Data Rate (CAN FD) bus, Local Interconnect Network (LIN) bus, Automotive Ethernet, Industrial Ethernet, Low Voltage Differential Signaling / Gigabit Multimedia Serial Link (LVDS / GMSL), and hardwire.

6. The autonomous driving data recording system according to claim 1, characterized in that, The power module includes: Power interface, suitable for connecting to mains power; A power management module, electrically connected between the power interface and the processing unit, supplies power to the processing unit using power received via the power interface; and A backup power supply is electrically connected to the processing unit to supply power to the processing unit in the event of a power outage at the power interface.

7. The autonomous driving data recording system according to claim 6, characterized in that, The backup power source includes at least one of lithium batteries, nickel-metal hydride batteries, or button batteries.

8. The autonomous driving data recording system according to claim 1, characterized in that, The processing unit includes a first sub-processing unit and a second sub-processing unit; The first sub-processing unit is connected to the first data storage unit and the second data storage unit to support operations related to the storage of autonomous driving data; The second sub-processing unit is connected to the first sub-processing unit and supports at least one of the following: over-the-air software upgrades, data uploads, remote diagnostics, and on-site diagnostics; and The autonomous driving data recording system also includes a third data storage unit, which is connected to the second data storage unit.

9. The autonomous driving data recording system according to claim 8, characterized in that, The interface unit includes an Ethernet interface, a Universal Serial Bus (USB) interface, a wake-up interface, and a Controller Area Network (CAN) interface. The autonomous driving data recording system also includes a switch, a wake-up signal detection unit, and a CAN transceiver. The Ethernet interface is electrically connected to the first sub-processing unit via the switch, the USB interface is electrically connected to the first sub-processing unit, the wake-up interface is electrically connected to the second sub-processing unit via the wake-up signal detection unit, and the CAN interface is electrically connected to the second sub-processing unit via the CAN transceiver.

10. The autonomous driving data recording system according to claim 8, characterized in that, The first sub-processing unit includes a system-on-a-chip, the second sub-processing unit includes a microcontroller (MCU), and the third data storage unit includes flash memory.

Images