Recording and providing data in vehicle
By using a combination of ring buffers and permanent memory in autonomous driving vehicles, sensor data is efficiently stored and managed, and data allocation and encryption is allocated and encrypted through metadata, the problems of data storage and security of autonomous driving vehicles are solved, and efficient and secure data management is achieved.
Patent Information
- Application Number
- CN202411166452.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-10-05
- Filing Date
- 2024-08-23
- Publication Date
- 2025-04-08
AI Technical Summary
Autonomous vehicles need to process and store large amounts of sensor data, and prior art is difficult to store and manage this data efficiently, especially when legal provisions and data security requirements are required.
By temporarily storing sensor data in a vehicle using a ring buffer, and storing the relevant data in a permanent memory when a driving event occurs, using metadata for data allocation and encryption, ensuring the security and efficiency of the data.
It realizes efficient storage and management of sensor data, saves permanent storage space, ensures data security and compliance, and is especially suitable for data recording and provision needs of autonomous vehicles.
Smart Images
Figure CN119785451A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for recording and providing data in a vehicle. Background Art
[0002] Nowadays, many vehicles have multiple sensors to detect sensor data. The sensor data may relate to the environment of the vehicle and / or the operating state of the vehicle. Sensors that detect sensor data about the vehicle environment are, for example, cameras, lidar sensors, radar sensors, and ultrasonic sensors. Sensors that detect the operating state of the vehicle are, for example, rotation speed sensors, speed sensors, and acceleration sensors.
[0003] In particular, the present invention relates to a method for recording and providing data in an autonomous vehicle. An autonomous vehicle is also referred to as an autonomous vehicle. Such a vehicle is understood to be a vehicle that can be driven and controlled without a human driver. In order to be able to drive and control itself with the necessary traffic safety, an autonomous vehicle has a particularly large number of sensors and therefore requires the detection, storage and processing of a particularly large amount of sensor data. On the one hand, in order to check the traffic safety of the autonomous vehicle, legal provisions generally require the detection of certain sensor data. On the other hand, the sensor data are required and used by computer programs that train, check and / or correct the driving behavior of the autonomous vehicle if necessary. The driving behavior of the autonomous vehicle includes, for example, driving operations performed by the autonomous vehicle in response to traffic conditions. These traffic conditions and the response of the autonomous vehicle are detected and evaluated in particular via sensor data. Summary of the invention
[0004] The invention is based on the object of specifying an improved method for recording and providing data in a vehicle.
[0005] According to the invention, this object is achieved by a method for recording and providing data in a vehicle, wherein
[0006] - sensor data is continuously detected by the vehicle's sensors,
[0007] - store detected sensor data in a ring buffer,
[0008] - defining driving events and assigning to each of these driving events an event period from a first time point before the driving event to a second time point after the driving event,
[0009] - when one of these driving events occurs, storing the sensor data stored in the ring buffer detected during the event period associated with the driving event in the persistent memory, wherein:
[0010] - All sensor data detected during the overlapping period where multiple event periods overlap and stored in the ring buffer are stored only once in the permanent memory
[0011] - references to metadata corresponding to the stored data, and
[0012] - Provide data stored in permanent storage to data users.
[0013] In the method according to the invention, the sensor data detected by the sensors of the vehicle are therefore first stored in a ring buffer. A ring buffer, also called a ring memory, refers to a memory area with a fixed memory size in which the sensor data are temporarily stored. In the ring buffer, the sensor data are stored, for example, according to the so-called FIFO principle (FIFO: abbreviation for First In-First Out). In the case of this principle, if there is not enough storage space in the ring buffer to store newly detected sensor data, the oldest sensor data stored in the ring buffer are overwritten by the newly detected sensor data. Therefore, the most recently detected sensor data are always stored in the ring buffer.
[0014] In addition, driving events are defined, and an event period is assigned to each driving event from a first time point before the driving event occurs to a second time point after the driving event occurs. For example, the definition of a driving event includes a change in the driving speed of the vehicle and / or a change in the driving direction of the vehicle. For example, when the vehicle turns or changes lanes at an intersection, the driving event can be an emergency braking of the vehicle or a change in the driving direction. The event period associated with the driving event links the driving event with the sensor data detected during the event period. For example, it is possible to analyze whether and how the driving event is associated with events in the vehicle environment and how the vehicle reacts to these events.
[0015] When a driving event occurs, the sensor data detected during the event period is stored in a permanent memory, because it will be overwritten in the ring buffer sooner or later. Permanent memory is a non-volatile memory in which sensor data is permanently stored and provided to data users, especially for regulatory purposes, such as accident investigations or liability issues. Here, all sensor data detected in the overlapping period where multiple event periods overlap and stored in the ring buffer are stored only once in the permanent memory. Therefore, for example, if the second event period starts during the first event period and ends after the first event period, the sensor data detected from the beginning of the first event period to the end of the second event period are stored in the permanent memory. If more than two event periods overlap, the same applies. In this way, for example, compared with the storage in which a separate storage block is set for each driving event (the sensor data detected during the event period associated with the driving event is stored in the storage block), a large amount of storage space can be saved in the permanent memory.
[0016] In one embodiment of the method according to the invention, the vehicle is an autonomous vehicle. As mentioned above, autonomous vehicles need to detect, store and process a particularly large amount of sensor data. Therefore, the efficient storage of sensor data in a permanent memory according to the invention has a particularly advantageous effect for autonomous vehicles.
[0017] In another embodiment of the method according to the invention, the sensor data includes environmental data related to the vehicle environment and / or vehicle data related to the vehicle operating state. For example, the environmental data is detected by a camera, a lidar sensor, a radar sensor and / or an ultrasonic sensor. For example, the vehicle data is detected by a rotation speed sensor, a speed sensor and / or an acceleration sensor.
[0018] In a further embodiment of the method according to the invention, metadata are assigned to each occurring driving event, these metadata referencing a memory area in the persistent memory in which sensor data detected during an event period associated with the occurring driving event are stored.
[0019] The above-described embodiment of the method according to the invention enables the assignment of sensor data stored in the permanent memory to the driving events that occurred, by means of metadata associated with the driving events. This is essential because, according to the invention, instead of allocating a separate sensor data storage block in the permanent memory for each driving event, in which the sensor data detected during the event period associated with the driving event are stored, in the case of overlapping event periods, the sensor data are stored in the permanent memory, which are detected during the entire period including all these overlapping event periods. The metadata allocates a memory area in the permanent memory to the driving event, in which the sensor data detected during the event period associated with the driving event are stored.
[0020] In another embodiment of the method according to the invention, the metadata is stored in a persistent memory. This enables data users to access the metadata and the sensor data via the same storage medium, ie the persistent memory.
[0021] In another embodiment of the method according to the present invention, the metadata associated with the driving event that occurred includes event attributes that characterize the driving event that occurred. In particular, the metadata associated with the driving event that occurred, for example, includes the time point of occurrence of the driving event. In addition, the metadata associated with the driving event that occurred, for example, includes the end time point of the driving event.
[0022] In a further embodiment of the method according to the invention, at least one data user profile is defined which specifies which sensor data stored in the permanent memory may be made available to a data user of the data user profile.
[0023] The data user profile thus specifies the access authorization for which sensor data stored in the persistent memory are allowed to be accessed by the data user of the data user profile. For example, an organization that has to be granted access rights to certain sensor data due to legal requirements, e.g. for the purpose of investigating the cause of an accident, defines a different data user profile than a manufacturer of software used in a vehicle, the purpose of which is to train and optimize this software.
[0024] In a further embodiment of the method according to the invention, each data user profile specifies a cryptographic data encryption method for encrypting the sensor data provided to the data user of the data user profile. Therefore, a further embodiment of the method according to the invention provides that the sensor data stored in the permanent memory are provided to the data user of the data user profile in an encrypted manner using the data encryption method specified in the data user profile.
[0025] The above-described embodiment of the method according to the invention is dedicated to the data security of the sensor data stored in the permanent memory. Since different data user profiles have different access authorizations to the sensor data stored in the permanent memory, in this embodiment of the method according to the invention, a cryptographic data encryption method is also assigned to the data user profile for encrypting the sensor data provided to the data user of the data user profile. For example, in this way, data users who are allowed to use sensor data with a higher security level can be provided with these sensor data with stronger encryption than data users who are allowed to use sensor data with a lower security level. For example, access to certain sensor data is granted to an organization with a cryptographically weaker encryption than to a software manufacturer of software used in a vehicle.
[0026] In another embodiment of the method according to the invention, in the event that the event periods of multiple driving events overlap, the data user is enabled to call up sensor data of one of these driving events from the permanent memory, which sensor data were detected during a period that includes and increases the event period associated with the driving event.
[0027] The above-described embodiment of the method according to the invention makes use of the fact that according to the invention, the sensor data detected during overlapping event periods of a plurality of driving events are not stored independently of one another in a permanent memory. This makes it possible to extend the event period actually assigned to the driving event beyond this event period. Thus, if necessary, the driving event can be examined more closely by analyzing the period that increases the event period actually assigned to the driving event.
[0028] In a further embodiment of the method according to the invention, at least one driving event is defined for which the acquisition of sensor data is required by law.
[0029] The above-described embodiments of the method according to the invention take into account that, in particular for autonomous vehicles, the detection of sensor data for certain driving events may be required by law. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Embodiments of the present invention are explained in more detail below with reference to the accompanying drawings, in which:
[0031] Figure 1 A flow chart showing an embodiment of the method according to the invention is shown,
[0032] Figure 2 Schematic diagram showing storage of sensor data in persistent memory. DETAILED DESCRIPTION
[0033] Figure 1A flow chart 100 shows an embodiment of a method according to the invention having method steps 101 to 107 for recording and providing data in a vehicle. The vehicle may in particular be an autonomous vehicle.
[0034] In a first method step 101, driving events are defined and an event period is assigned to each of these driving events from a first time point before the driving event occurs to a second time point after the driving event occurs. For example, a driving event includes a change in the driving speed of the vehicle and / or a change in the driving direction of the vehicle. For example, when the vehicle turns or changes lanes at an intersection, the driving event may be an emergency braking of the vehicle or a change in the driving direction. In particular, driving events for which legal provisions are made to store sensor data, which are detected by sensors of the vehicle, may be defined.
[0035] For example, a driving event is assigned an event period that starts 30 seconds before the driving event occurs and ends 5 seconds after the driving event ends.
[0036] In a second method step 102, sensor data is continuously detected by sensors of the vehicle. The sensor data includes environmental data related to the vehicle environment and vehicle data related to the vehicle operating state. For example, the environmental data is detected by at least one camera, at least one lidar sensor, at least one radar sensor and / or at least one ultrasonic sensor of the vehicle. For example, the vehicle data is detected by at least one rotation speed sensor, at least one speed sensor and / or at least one acceleration sensor.
[0037] In a third method step 103, the detected sensor data are stored in a ring buffer arranged in the vehicle. A ring buffer is a memory area with a fixed memory size in which the detected sensor data are temporarily stored. In the ring buffer, the sensor data are stored according to the FIFO principle, in the case of which the oldest sensor data stored in the ring buffer are overwritten by the newly detected sensor data if there is insufficient storage space in the ring buffer to store newly detected sensor data. As a result, the most recently detected sensor data are always stored in the ring buffer.
[0038] In a fourth method step 104, it is checked whether one of the driving events defined in the first method step 101 has occurred. When one of these driving events occurs, the sensor data stored in the ring buffer during the event period associated with the driving event are stored in a permanent memory arranged in the vehicle. Here, all sensor data detected during the overlapping period in which a plurality of event periods overlap and stored in the ring buffer are stored only once in the permanent memory. The permanent memory is a non-volatile memory in which the sensor data is permanently stored. The permanent memory can belong to the same component as the ring buffer, or be a separate component.
[0039] The event period associated with the driving event links the driving event with the sensor data detected during the event period. These sensor data are stored in the permanent memory, because they will be overwritten in the ring buffer and will be lost if they are not stored in the permanent memory. Thus, for example, it can be analyzed whether and how the driving event is associated with events in the vehicle environment and how the vehicle reacts to these events.
[0040] Figure 2 The schematic diagram of storing sensor data in a permanent memory of a vehicle is shown as an example. The permanent memory is only shown schematically here and is denoted by the reference symbol P.
[0041] exist Figure 2 Two event periods T1 and T2 are shown in FIG. 1 . The first event period T1 associated with the first driving event extends from time point t1 at time t to time point t2 at time t. The second event period T2 associated with the second driving event extends from time point t3 to time point t4. Figure 2 In the example shown, the event periods T1 and T2 overlap between the time points t3 and t2. According to the present invention, the sensor data detected between the time points t3 and t2 are stored only once in the permanent memory P, that is, the sensor data detected between the time points t1 and t4 are stored only once in the permanent memory P.
[0042] Therefore, in other words, the sensor data detected during event period T1 and the sensor data detected during event period T2 are not stored in independent memory areas of the permanent memory P, but the sensor data detected between time points t3 and t2 are stored in the same memory area of the permanent memory P.
[0043] In a fifth method step 105 , metadata are assigned to each occurring driving event, which metadata refer to a memory area S1 , S2 in the persistent memory P in which the sensor data detected during the event period T1 , T2 associated with the occurring driving event are stored. Figure 2 In the example shown, the metadata of the first driving event references a first memory area S1 in which sensor data detected during an event period T1 associated with the first driving event is stored. The metadata of the second driving event references a second memory area S2 in which sensor data detected during an event period T2 associated with the second driving event is stored. For example, each of these memory areas S1, S2 is referenced by a storage address stored in the metadata. However, it is important that the memory areas S1, S2 have a non-empty intersection in which those sensor data detected between time points t3 and t2 are stored, and the event periods T1, T2 overlap between time points t3 and t2.
[0044] Metadata is also stored in the persistent storage P. Figure 2 In the example shown, metadata for a first driving event is stored in a memory area M1 of the persistent memory P, and metadata for a second driving event is stored in a memory area M2 of the persistent memory P.
[0045] By the reference of the metadata associated with the driving event that occurred to the memory area S1, S2 of the persistent memory P, in which the sensor data detected during the event period T1, T2 associated with the driving event that occurred is stored, it is possible to assign the sensor data stored in the persistent memory P to the driving event that occurred. This is important because according to the invention, instead of each driving event being assigned a separate sensor data storage block, in the case of overlapping event periods T1, T2, the sensor data are stored in the persistent memory P, which are detected during the entire period including all these overlapping event periods. The metadata assigns the storage location in the persistent memory P where the relevant sensor data is stored to the driving event.
[0046] The metadata associated with the driving event that occurred also includes event attributes that characterize the driving event that occurred. For example, the metadata associated with the driving event that occurred includes the time point when the driving event occurred and the time point when the driving event ended.
[0047] Additionally, the metadata includes the cryptographic keys used to encrypt the sensor data.
[0048] In another embodiment of the invention, the metadata of consecutive driving events form a blockchain. This means that each block automatically contains the metadata of the previous driving event stored in the permanent storage P. In addition, each metadata block contains a so-called hash or fingerprint ("#") of the previous metadata block and provides a new hash as a reference for the metadata blocks of subsequent driving events.
[0049] In a sixth method step 106, different data user profiles are defined. Each data user profile specifies which sensor data stored in the permanent memory P may be provided to the data user of the data user profile. In addition, each data user profile specifies a cryptographic data encryption method for encrypting the sensor data provided to the data user of the data user profile.
[0050] The data user profile thus specifies the access authorization for which sensor data stored in the permanent memory P may be accessed by the data user of the data user profile. Furthermore, the data user profile specifies a data encryption method, with which the sensor data are provided in encrypted form to the data user of the data user profile.
[0051] In this context, the data user profiles are adapted to the requirements of the respective data users. On the one hand, in particular for autonomous vehicles, for example, the law may provide for the detection and storage of sensor data of certain driving events, for example for the investigation of traffic accidents in which the vehicle is involved. Therefore, corresponding data user profiles can be defined for "official" data users that comply with these legal provisions. On the other hand, sensor data linked to driving events can be useful for software manufacturers who develop and optimize vehicle software. Therefore, data user profiles for software manufacturers can also be defined. Data user profiles for software manufacturers will generally differ from data user profiles for official data users, because software manufacturers use the data, for example, for the purpose of training and optimizing the driving functions of the vehicle, while officials use the data, for example, for the purpose of ensuring the traffic safety of the vehicle.
[0052] In a seventh method step 107 , the sensor data stored in the permanent memory P are made available to the data user. In this context, the data user profile of the respective data user is taken into account. Thus, only the sensor data which their data user profile permits to be made available to them are made available to the data user.
[0053] Furthermore, the sensor data are provided to the data user in an encrypted form using a cryptographic data encryption method specified in the data user profile. In this case, access to the sensor data is achieved via metadata.
[0054] Official data users can access the metadata, for example, via a public cryptographic key. However, access to the sensor data stored in the permanent memory P is restricted, depending on the data user profile for the official data user. For example, access to sensor data detected during event periods T1, T2 of certain driving events for which the sensor data are required to be stored by law is restricted. Furthermore, access to certain data transmitted via the vehicle CAN bus (CAN: abbreviation for Controller Area Network) and / or to data of the vehicle's lidar sensors may be restricted for official data users, as long as there is no provision for these data to be made available to official data users.
[0055] In contrast, the software manufacturer can access the metadata, for example, via a private cryptographic key, which enables the software manufacturer to access all sensor data stored in the permanent memory P. This enables the software manufacturer, for example, to analyze the interaction of the vehicle with events in the vehicle environment and in particular the vehicle's reaction to events in the vehicle environment and to check the functionality of the software used in the vehicle and, if necessary, correct or optimize it. This is particularly advantageous for autonomous vehicles, which require particularly high-quality and stable software to operate the vehicle.
[0056] Reference numerals list
[0057] 100 Flowchart
[0058] 101 to 107 Methods and Steps
[0059] t time
[0060] Time points t1 to t4
[0061] M1, M2, S1, S2 memory areas
[0062] P Persistent storage
[0063] T1, T2 event period
Claims
1. A method for recording and providing data in a vehicle, wherein: - sensor data is continuously detected by the vehicle's sensors, - store detected sensor data in a ring buffer, - defining driving events and assigning to each of said driving events an event period (T1, T2) from a first time point (t1, t3) before the driving event occurs to a second time point (t2, t4) after the driving event occurs, - when one of the driving events occurs, storing in a permanent memory (P) the sensor data detected during the event period (T1, T2) associated with the driving event and stored in the ring buffer, wherein: - all sensor data detected during the overlapping period of a plurality of event periods (T1, T2) and stored in the ring buffer are stored only once in the permanent memory (P), and - providing the data stored in said permanent storage (P) to a data user, Among them, metadata of continuous driving events includes blockchain.
2. The method according to claim 1, wherein: The vehicle is an autonomous vehicle.
3. The method according to claim 1 or 2, wherein: The sensor data include environmental data related to the vehicle environment and / or vehicle data related to the vehicle operating state.
4. The method according to claim 1 or 2, wherein: Metadata is assigned to each occurring driving event, the metadata referencing a memory area (S1, S2) in the persistent memory (P) in which sensor data detected during a relevant event period (T1, T2) of the occurring driving event are stored.
5. The method according to claim 4, wherein: The metadata is stored in the persistent storage (P).
6. The method according to claim 4, wherein: The metadata associated with the occurred driving event includes event attributes characterizing the occurred driving event.
7. The method according to claim 4, wherein: The metadata associated with the driving event includes an occurrence time point and / or an end time point of the driving event.
8. The method according to claim 1 or 2, wherein: At least one data user profile is defined, which specifies which sensor data stored in the persistent memory (P) are allowed to be provided to a data user of the data user profile.
9. The method according to claim 8, wherein: Each data user profile specifies a cryptographic data encryption method for encrypting sensor data provided to a data user of the data user profile.
10. The method according to claim 9, wherein: The sensor data stored in the persistent memory (P) is provided to a data user of the data user profile in an encrypted manner using a data encryption method specified in the data user profile.
11. The method according to claim 1 or 2, wherein: In case event periods (T1, T2) of a plurality of driving events overlap, a data user is enabled to call up sensor data of one of the driving events from the permanent memory (P), the sensor data being detected during a period including and increasing the event period (T1, T2) associated with the driving event.
12. The method according to claim 1 or 2, wherein: At least one driving event is defined for which the detection of sensor data is required by law.