Methods for performing data recording, methods for generating data streams, data processing equipment, computer program products, signal sequences, and storage media.

Abstract

This provides a low-cost, simple, and reliable method for simultaneously recording multiple data streams from different sources at high bandwidth. [Solution] A method for performing data recording, comprising: receiving at least one data stream for data recording 101; dividing each unique data stream, of which at least one unique data stream exists within the data stream, into basic data units based on a data stream descriptor set for each unique data stream 103; and distributing and storing all the basic data units of the unique data streams in at least two storage media 105.

Description

Technical Field

[0001] The present invention relates to a method for performing data recording. The present invention further relates to a method for generating a data stream from basic data units distributedly stored across at least two storage media or source files, particularly by such a data recording method. The present invention also relates to a data processing apparatus, a computer program product, a signal sequence, and a storage medium.

Background Art

[0002] In data-driven development of applications for driver assistance systems (ADAS) and autonomous driving (AD), it is necessary to use vehicle data and environmental data that are as realistic as possible. To achieve this, sensor data from, for example, LiDAR sensors, radar sensors, and camera sensors are read at a very high bandwidth during test drives and stored for later use.

[0003] In order to be able to provide the bandwidth required for simultaneous recording of multiple data streams from various sensors, in the past, for example, a RAID controller has been used to distributively store the data arriving in RAID0 mode without redundancy on multiple hard disks. The operation method of such a solution has also been demonstrated conventionally. However, such RAID controllers are relatively expensive and account for most of the cost of the corresponding data recording system.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Therefore, an object of the present invention is to overcome the above-mentioned drawbacks of the prior art, and particularly to provide a means by which simultaneous recording of multiple data streams from different sources at a high bandwidth can be achieved at a low cost, easily and reliably.

Means for Solving the Problems

[0005] The above problems are addressed by the present invention. First aspect A method for performing data recording is provided, wherein at least one data stream is received for data recording, at least one unique data stream exists within the data stream, each unique data stream is divided into basic data units based on a data stream descriptor set for each unique data stream, and the basic data units of all unique data streams are distributed and stored in at least two storage media.

[0006] Thus, the present invention is based on the recognition that each received unique data stream is highly predictable in terms of one or more characteristics. By using data stream descriptors for unique data streams, this prior knowledge can be taken into account when dividing a unique data stream into multiple basic data units. At the same time, this also makes it possible to know in advance, or at least partially, about the amount of basic data units acquired per data stream and per time unit, and consequently, the total amount of data received and stored per time unit.

[0007] This allows resources for processing and storing data in a specific data stream to be pre-planned, at least partially based on and / or originating from the data stream descriptor. For example, storage space (e.g., in main memory) can be pre-planned for the temporary storage of received data (particularly data in individual basic data units), and can be appropriately sized and / or reserved (e.g., with respect to memory size and / or temporal availability). This allows, alternatively or supplementarily, storage space and / or bandwidth on the storage medium for write access to individual storage mediums to be planned, appropriately sized, and / or allocated for the expected basic data units to be distributed and stored.

[0008] The at least partial predictability of the amount of data received and stored here eliminates the need for a RAID controller to dynamically allocate incoming data to the hard disk. Therefore, according to the proposed method, data recording can be performed advantageously without a RAID controller. This reduces the costs traditionally associated with RAID controllers.

[0009] Particularly advantageous is that the distributed storage of the basic data units can be configured to be performed at least temporarily using temporally parallel write access to at least two storage media. Advantageously, during this time, one other basic data unit is stored in each of the at least two storage media. For example, the first basic data unit of the first unique data stream is stored in the first of the at least two storage media, and temporally parallel to this, the second basic data unit (of the first or second unique data stream) is stored in the second of the at least two storage media. Parallel write access ensures that the data of the unique data streams can also be stored with high bandwidth, particularly in real time, because in this case, the allowable total bandwidth for the received unique data streams is advantageously scalable linearly by the bandwidth (for write access) of each storage medium. For example, with S storage media, each allowing write access at a maximum data rate R [bit / s] during the storage process, the received unique data streams theoretically have a total data rate S*R. Therefore, by providing multiple storage media that allow parallel write access, the maximum allowable total data rate of the unique data stream can be increased.

[0010] In the sense of this application, "real-time" is understood to mean that the period from the arrival of a first basic data unit in a data stream to the arrival of a second basic data unit directly following the same data stream, i.e., the period length of the basic data units in the data stream, is longer than the period from the arrival of the first basic data unit to the end of the storage process of the first basic data unit in the storage medium. For example, whether or not the real-time condition here is met or can be met can be checked based on the data stream descriptor and / or another technical descriptor of the data stream (e.g., with respect to the incident frequency and size of the basic data units) and / or based on the write speed of the storage medium.

[0011] Because the proposed method is relatively easy to implement, conventional memory solutions can also be adapted to it quite easily, thereby benefiting from the advantages mentioned above, particularly cost savings. This makes the proposed method particularly interesting from an economic standpoint.

[0012] Preferably, the basic data units of all unique data streams are distributed and stored across two, three, four, five, or six or more (e.g., 10 or 11 or more, or 20 or 21 or more) storage media.

[0013] Each storage medium is advantageously requestable individually (for example, to write data to and / or read data from each storage medium during data recording). Each storage medium may be located within a memory unit. Preferably, each storage medium is protected against loss. For example, the storage mediums are interconnected in such a way that they cannot be separated from one another without destruction. For this purpose, the storage mediums may be formed, for example, in the form of closed-lock memory. Each storage medium is preferably implemented by a solid-state disk (SSD).

[0014] Preferably, the proposed method is performed entirely or partially on a data logger. The data logger may be, for example, part of an in-vehicle data detection system and / or prototyping system. The method is also advantageously executable in a distributed system.

[0015] The proposed method can be fully or partially implemented on a computer, for example.

[0016] Preferably, the proposed method makes data from sensors such as LiDAR sensors, radar sensors, and / or camera sensors, and / or data from high-bandwidth automatic buses and networks, respectively, readable, processable, and / or recordable. Sensor data may be, for example, raw sensor data. In the spirit of this application, high bandwidth in the received unique data stream means, for example, a data rate greater than 1 Gbit / s. Therefore, preferably, at least one data stream of at least one unique data stream has a data rate greater than 1 Gbit / s.

[0017] Therefore, in summary, the proposed method enables the storage of (unique) data streams themselves across multiple storage media with high bandwidth. This provides the bandwidth necessary for the simultaneous recording of multiple data streams from different sensors, even during test runs. Thus, the proposed method is particularly advantageous for data-driven development of driver assistance systems and applications for autonomous driving. For example, the stored data can then be used to train neural networks and stimulate driver assistance systems, or as a sample of virtual traffic scenarios for testing driver assistance systems in a virtual environment.

[0018] It should be noted that one or more of the received data streams may be processed and / or stored in a manner different from that described herein, in which case they would no longer be unique data streams.

[0019] Alternatively or supplementarily, it is also conceivable to receive each unique data stream sequentially and divide them into multiple basic data units.

[0020] Continuous reception and partitioning eliminate or minimize the need for resources to temporarily store data in a unique data stream. This makes the method particularly economical to implement. Furthermore, it reduces latency from data reception to storage, and consequently, allows for more efficient data recording.

[0021] Alternatively or supplementally, two or more data streams may be received, and within those data streams, two or more unique data streams may be present, and these data streams may be received in particular at least partially in parallel in time.

[0022] This allows data streams from multiple sources (especially sensors) to be received and stored simultaneously. Preferably, the number of unique data streams corresponds to the number of received data streams.

[0023] Alternatively or additionally, each data stream descriptor set for an individual unique data stream can be configured to have one or more technical specifications for each unique data stream, and to perform at least partially the division of each unique data stream into basic data units based on those technical specifications.

[0024] This allows for the division of individual unique data streams into basic data units and / or data processing to be performed while taking into account the technical specifications of each unique data stream.

[0025] This enables each unique data stream to be processed individually, divided into basic data units taking into account their respective characteristics, and the resulting basic data units to be processed. Unique data streams from various sources with at least partially different technical specifications can also be processed (especially divided into a plurality of basic data units respectively) and stored in this way efficiently.

[0026] The technical specifications of the unique data streams can be fully or partially known respectively. For example, such technical specifications can be set manually. The technical specifications of the unique data streams can alternatively or additionally be calculated automatically, fully or partially respectively. For example, these can be calculated by communicating with the respective sources (such as sensors) of the unique data streams. The automatic calculation is preferably performed before and / or at the start of data recording. The data stream descriptors of the individual unique data streams can be calculated at least partially based on the technical specifications, at least partially manually and / or at least partially automatically (especially subsequently).

[0027] Advantageously, the same and / or common data stream descriptors can be used for at least two unique data streams. This is particularly possible when each unique data stream is derived from sources of the same structural type.

[0028] Alternatively or additionally, the data stream descriptors set for the individual unique data streams each have (i) information about the memory size of the basic data units of each unique data stream, and / or (ii) information about the frequency at which data of sequentially consecutive basic data units within each unique data stream is received, and based on this information, the division of each unique data stream into basic data units can be configured to be at least partially executed.

[0029] Based on this information, the data of the basic data units can be processed efficiently with particularly high reliability. In this case, the memory size is, advantageously, the required memory space for the data of the basic data units in each unique data stream. This ensures that the memory space required to temporarily store the data of such basic data units is reliably and accurately adapted and reserved (for example, for at least the period during which the data of the basic data units is received and / or until the received data is stored in the storage medium of the basic data units). The frequency may be, for example, the frequency at which the first byte of the data stored in sequentially consecutive basic data units within each unique data stream is received. If the data rate of the unique data stream (in particular, at least temporarily constant) is known, the frequency can advantageously be calculated based on the memory size, and / or conversely, the memory size can be calculated based on the frequency for at least a given time section.

[0030] Furthermore, each data stream descriptor set for individual unique data streams can contain information about control data, which is used, for example, to separate sequentially consecutive basic data units within each unique data stream. Based on such control data, the division of each unique data stream into basic data units can be performed particularly easily and reliably. This will be discussed in more detail below.

[0031] Alternatively or supplementally, individual data stream descriptors can be configured to be read from separate memory locations.

[0032] This allows for easy and reliable access to the data stream descriptor. The separate memory can be remote memory, such as cloud memory.

[0033] Alternatively or supplementally, the division of unique data streams into base data units may be configured to include grouping the data of each unique data stream based on a data stream descriptor set for each unique data stream, thereby obtaining a base data unit.

[0034] For example, for each unique data stream, the presence of control data defined in the associated data stream descriptor can be analyzed, and / or each unique data stream can be divided into basic data units based on the control data identified in each unique data stream. This makes it particularly easy and reliable to identify sequentially consecutive amounts of data in each unique data stream based on the control data and to acquire them as individual basic data units. For example, each unique data stream can be divided at multiple points where defined control data exists. The control data does not need to be part of the acquired basic data units. The control data can, for example, define division codes (i.e., markings) that are advantageously inserted into the unique data stream from each source (especially a sensor) of the unique data stream.

[0035] Alternatively or supplementarily, the division of each unique data stream into basic data units can be performed based on frequency and / or memory size. The data rate of each unique data stream can also be used as an optional means. For example, each unique data stream can be monitored, and a new basic data unit can be obtained after a predetermined period according to the frequency has elapsed, or after reaching a further data throughput corresponding to each memory size.

[0036] Alternatively or additionally, the basic data units of a unique data stream may be configured to have, or consist of, one data group for each unique data stream.

[0037] In other words, each unique data stream's data group is a subset of the data in each unique data stream. These groups are configurable and / or computable by the data stream descriptor of each unique data stream. The basic data unit of a unique data stream may be, for example, a grouping of the data in the unique data stream, which is preferably selected based, for example, on the technical configuration of each unique data stream stored in each data stream descriptor.

[0038] Preferably, each received unique data stream can be completely and / or at least partially reconstructed from the data of unique data streams grouped into basic data units, particularly by connecting the individual basic data units in parallel. In embodiments, it may be necessary to add additional data not included in the basic data units, such as the control data described above, for the complete reconstruction of each unique data stream.

[0039] Alternatively or additionally, the data of the basic data units in the unique data stream may represent a representation of the sensor's environment as recorded by the sensor at a specific point in time and / or over a specific period of time.

[0040] In this case, the “specific period” may be the period required to record data using sensors to display the environment. For example, the “specific period” for a radar sensor may be the period required to continuously drive and control multiple radar sensor elements of the radar sensor array. For example, in the case of a LiDAR sensor, the “specific period” may be the period required to scan the environment area using the LiDAR sensor's laser and / or to rotate the laser at least 360° around its axis of rotation.

[0041] Alternatively or supplementarily, the data from the basic data unit can represent the spatiotemporal resolution of the sensor data.

[0042] Alternatively or supplementally, the basic data units of the unique data stream may be configured to contain data from individual images recorded by the sensor, or each row or pixel of such individual images.

[0043] This means that the basic data unit may, advantageously, be "self-contained," i.e., it may contain all the information, especially for instantaneous recording of the sensor environment.

[0044] In particular, the basic data units of the unique data stream are, solely , including data from individual images recorded by the sensor, or one row or pixel of such individual images.

[0045] Alternatively or supplementally, distributed storage of the basic data units of a unique data stream across at least two storage media can also be performed based on a distributed protocol.

[0046] The distributed protocol can, advantageously, be calculated based on the data stream descriptor of the unique data stream. This allows for the at least implicit use of the known properties of the unique data stream to distribute the basic data units to individual storage media. This is because, as already mentioned above, the basic data units obtained according to the defined schema advantageously yield the amount of data per unit of time which is essentially known in any case and / or the amount of data which is, in principle, a priori obtained in terms of the properties known in any case.

[0047] This allows resources to be pre-configured to store data for specific data streams, at least partially, based on and / or originating from a distributed protocol. For example, the storage space and / or bandwidth on individual storage media can be adjusted, appropriately sized, and / or allocated for the basic data units to be distributed and stored for the purpose of write access to each storage medium.

[0048] It is particularly advantageous when the distributed protocol is calculated based on the results of an algorithm for solving the "container problem." This advantage is that the basic data units of the unique data streams generated per unit of time are already known beforehand, allowing the distribution of these basic data units to the storage medium to be planned a priori using such an algorithm. Such algorithms are well known to those skilled in the art, and therefore will not be discussed in further detail here. Instead, relevant literature can be consulted.

[0049] Here, the distributed protocol can be computed entirely or partially automatically, for example, based on data stream descriptors and / or considering algorithms for solving the "container problem." The automated computed protocol is preferably performed before and / or at the start of data recording.

[0050] This allows for the distributed storage of unique data streams to the base data unit to be performed at least partially based on information prior to and derived from the unique data streams. This advantageously eliminates the need for RAID controllers used in conventional forms.

[0051] Alternatively or supplementally, based on the distribution protocol of each basic data unit of the unique data stream, at least one storage medium, preferably exactly one storage medium, of at least two storage media may be allocated as a storage location, and the individual basic data units may be stored in the storage media allocated to them, respectively.

[0052] This allows the distribution of individual basic data units within a unique data stream to be performed advantageously according to a fixed pattern and / or a pre-known schema. This makes the storage process particularly efficient, as it eliminates the need for further (especially content-based) analysis of the data to be stored. These temporal advantages are particularly important from a real-time processing perspective.

[0053] Distributed protocols can also implicitly configure at least two temporally parallel write accesses to storage media.

[0054] Alternatively or supplementally, a configuration is also possible in which at least one, preferably all, of the at least one unique data streams are distributed and stored on at least two storage media, particularly according to a distributed protocol. This allows for particularly good maximization of the available input bandwidth.

[0055] Alternatively or supplementally, a distribution protocol for at least one unique data stream, preferably for each unique data stream, sets a particular periodic order for the storage medium, and in accordance with this order, the basic data units of each unique data stream are sequentially stored in the storage medium.

[0056] Therefore, in this case, the order of the storage media is the same for the basic data units of each corresponding unique data stream.

[0057] In this case, the configuration order of the storage media is advantageously identical for each specific data stream in question. In embodiments, the order may be partially different, and in particular at least partially relating to different selections of storage media and / or the order of storage media.

[0058] This makes it possible to achieve a defined distribution of at least one (and identical as an optional means) of the basic data unit to at least two storage media for the relevant unique data stream.

[0059] For example, if there are three storage media (S1 to S3), the basic data units for each corresponding unique data stream are stored sequentially in S1, S2, S3, and then back to the beginning, i.e., they can be stored in the order S1, S2, S3, ... Considering the case where only one unique data stream is received, the three basic data units of that single unique data stream can always be stored in the three storage media S1, S2, S3 in parallel in time. Herein lies a means of increasing the input bandwidth by providing another storage medium.

[0060] Alternatively or supplementally, and through distributed protocols, the order of storage media, particularly periodic order, can be set, and the basic data units of the unique data stream can be configured to be stored in the storage media in order according to this order.

[0061] Therefore, the order of the storage media applies here to the basic data units of all relevant unique data streams.

[0062] This makes it possible to distribute the basic data units across at least two storage media for each relevant unique data stream.

[0063] For example, if there are three storage media (S1 to S3), and two unique data streams, each containing the basic data units D1.1, D1.2, D1.3, ... for the first unique data stream and D2.1, D2.2, D2.3, ... for the second unique data stream, then the temporally sequentially consecutive basic data units D1.1, D2.1, D1.2, D2.2, D1.3, D2.3, ... can be stored sequentially in S1, S3, S5, and then back to the beginning, i.e., in the order S1, S2, S3, etc. Even if, for the sake of illustration, the basic data units of the first unique data stream and the basic data units of the second unique data stream always appear alternately, in this embodiment, the basic data units of individual data streams may also appear in other orders / frequencyes (e.g., D1.1, D1.2, D2.1, D1.3, D1.4, D2.2, etc.) depending on when the data of each unique data unit is received.

[0064] Alternatively or supplementally, the system can be configured to compute the distributed protocol before data recording begins.

[0065] This allows distributed storage to be performed based on rules that are clearly defined a priori. The distributed protocol can be calculated particularly easily based on the data stream descriptors of each unique data stream.

[0066] Alternatively or supplementally, the distribution protocol can be configured to calculate such that, when distributing the unique data stream of a unique data unit across at least two storage media, the maximum data rate allowed for write access to each individual storage medium is not exceeded.

[0067] This enables particularly efficient storage of the basic data unit.

[0068] Alternatively or supplementally, when distributing the basic data units across at least two storage media, the distribution protocol can be configured to calculate such that uniform data distribution across each storage medium is achieved.

[0069] In this way, a reduction in the allowable input bandwidth can be avoided as soon as one of the storage media becomes full. Furthermore, this allows for particularly efficient use of the available memory space. This enables particularly efficient storage of basic data units. The storage media are "filled with data" by storing data in the storage media. Such uniform filling can be achieved, particularly advantageously, by using algorithms that solve the "container problem" already described above.

[0070] Alternatively or supplementally, the system may be configured to automatically calculate, for each of at least two storage media, the maximum data rate that can describe each storage medium as bandwidth information and / or the available memory space as capacity information, in which case the distributed protocol is preferably calculated based on the bandwidth information and / or capacity information of the storage media.

[0071] This allows for particularly efficient loading of basic data units into the storage medium.

[0072] Preferably, the automatic calculation of bandwidth information and / or capacity information includes querying the corresponding information from individual storage media.

[0073] Alternatively or supplementally, the distributed protocol can also be configured to calculate based on data stream descriptors set for the unique data streams and / or the information underlying these data stream descriptors, in particular based on (i) the memory size of the basic data units of each unique data stream and / or (ii) the frequency at which the data of sequentially consecutive basic data units within each unique data stream is received.

[0074] This, particularly advantageously, allows for the consideration of the technical characteristics of unique data streams during distribution.

[0075] Particularly advantageous is that the distributed protocol is calculated based on the data stream descriptor set for the unique data stream and / or the information underlying that data stream descriptor, as well as on bandwidth information and / or capacity information. This is particularly advantageous because it allows for consideration of the technical characteristics of the unique data stream (source) and storage medium (target) during distribution.

[0076] Alternatively or supplementally, the same distributed protocol can be configured to apply to the entire duration of data recording.

[0077] This makes it possible to execute memory processes particularly efficiently.

[0078] Alternatively or supplementally, the unique data streams may be configured such that (i) each contains data from the application-related layer, and / or (ii) each does not contain protocol information and / or carrier information.

[0079] The layer relevant to the application is, for example, one of layers 5 through 7 in the OSI model.

[0080] In other words, the unique data stream advantageously does not contain the information required to transmit this data over the transmission channel (e.g., address information for each source or address information for each target of the data).

[0081] Preferably, the information used to map data from a unique data stream to a basic data unit (e.g., the control data described above) is understood as data from an application-related layer and / or not as protocol information and / or carrier information.

[0082] Alternatively or supplementally, the unique data streams can be configured to originate from sensor signals, specifically from the image generation sensor and / or different sensors.

[0083] For example, at least one sensor is selected from a group of sensors that include camera sensors, radar sensors, LiDAR sensors, and / or ultrasonic sensors.

[0084] The sensor signal may be, for example, sensor data, particularly raw sensor data.

[0085] In other words, the unique data stream may contain, for example, image data from an image generation sensor. For this purpose, each sensor can supply a continuous and uniform stream to the image frame, each of which is received as a unique data stream.

[0086] Alternatively or supplementally, at least one unique data stream may be configured to represent sensor signals from a camera sensor, radar sensor, LiDAR sensor, and / or ultrasonic sensor.

[0087] In this case, the sensor can supply a time-dependent signal.

[0088] Alternatively or supplementally, at least one sensor may be configured to operate in a sequentially consecutive sequence of different operating modes, and depending on the operating mode, at least one technical specification of the unique data stream received from the sensor, such as (i) data rate, (ii) data size of the basic data unit and / or (iii) frequency at which a new basic data unit is received, may change.

[0089] For example, the sensor can be operated first in a first operating mode, and then in a second operating mode some time later. For instance, the resolution of the image recorded by the sensor, and consequently the data rate of the unique data stream received by the sensor, may differ in the two operating modes, and in particular, the resolution in the second operating mode may be higher than that in the first operating mode.

[0090] For example, an automatic camera sensor can increase the image rate or resolution when critical traffic conditions are present, thereby enabling switching between different operating modes.

[0091] In several embodiments, time intervals in which the operating mode does not change can be stored as separate records, each having its own unique start and end time and unique ID. Therefore, the individual intervals can be characterized as related based on a higher-level record ID. This allows these intervals to be combined into a single continuous record, for example, during reconstruction.

[0092] Alternatively or supplementally, the distributed protocol can be configured to calculate such that the basic data units are distributed and stored across at least two storage media in each operating mode of the sensor.

[0093] In other words, in this case, the distributed protocol is advantageously calculated so that the distributed storage of the basic data units is guaranteed independently of the time-dependent selection of the operating mode.

[0094] Alternatively or supplementally, the data stream descriptor and / or distributed protocol of the unique data stream received from the sensor may be configured, at least partially, to calculate for the operating mode in which the unique data stream received from the sensor has the maximum data rate.

[0095] This directs data recording towards a "worst-case" scenario, which is still feasible. For example, if the "worst-case" scenario is designed during periods when the data rate is lower than the maximum data rate, the memory space reserved for temporary storage can be oversized. Similarly, the write access bandwidth reserved for storing the basic data units of a unique data stream can be oversized during periods when the data rate is lower than the maximum data rate. Likewise, the memory space of one of the multiple storage media reserved for storing the basic data units of a unique data stream can be oversized during periods when the data rate is lower than the maximum data rate. These compromises are accepted in these cases to ensure that the processing of received data and the distributed storage of basic data units continue to be reliably performed during periods when the data rate corresponds to the maximum data rate.

[0096] Alternatively or supplementally, the system can be configured to switch between different data stream descriptors and / or distributed protocols for the unique data stream received from the sensor, depending on the operating mode of the selected sensor.

[0097] This allows for flexible consideration of modified boundary conditions, which in turn enables particularly efficient handling of resources, especially storage media.

[0098] Alternatively or supplementally, for at least one unique data stream arriving from the sensor, at least one technical specification for each unique data stream can be automatically calculated through communication with the sensor and used in the data stream descriptor.

[0099] This allows, for example, the automatic calculation of the aforementioned information for the data stream descriptor, which can then be used when calculating the data stream descriptor. For further information, refer to the embodiments described above.

[0100] Alternatively or supplementally, the system can be configured such that at least one unique data stream, preferably the basic data units of all unique data streams, are distributed and stored in at least two of the at least two storage media.

[0101] This is particularly advantageous because it allows for maximizing the allowable bandwidth of the received unique data stream.

[0102] Alternatively or supplementally, the basic data units of at least one unique data stream, preferably all unique data streams, can be configured to be stored without redundancy in at least two of at least two storage media.

[0103] This makes it possible to execute memory processes particularly efficiently.

[0104] Alternatively or supplementally, the data rate of at least one unique data stream, particularly at least one of all unique data streams received in parallel, may be configured to have a maximum value greater than the maximum allowable data rate that can describe at least one of the at least two storage media, preferably all of the storage media.

[0105] Distributed storage of basic data units (executed in parallel in time) across at least two storage media allows the total bandwidth of the received unique data stream to be greater than the maximum allowable bandwidth for write access to just one of the aforementioned storage media.

[0106] For example, each storage medium can perform writes at a maximum data rate of 1 Mbit / s. If only one unique data stream is received for data recording, this could theoretically have a bandwidth of 2 Mbit / s based on parallel distributed memory.

[0107] Therefore, advantageously, this method allows for an increase in input bandwidth by distributing and storing, preferably in real time, one or more incoming unique data streams in parallel across multiple storage media.

[0108] The data rate of all unique data streams received in parallel is, here preferably, understood as the sum of the data rates of the individual unique data streams.

[0109] Alternatively or supplementally, the system can be configured to perform data recording in real time and / or to avoid temporary storage of the received unique data stream data in non-volatile memory.

[0110] If temporary storage in non-volatile memory is not performed, data processing (e.g., grouping data within a unique data stream into its basic data units) can be performed particularly quickly. For example, temporary storage can be performed in main memory (RAM).

[0111] Alternatively or supplementally, the data recording can also be configured to generate metadata containing information about the basic data units stored on individual storage media, particularly information for reconstructing individual unique data streams from basic data units distributed across at least two storage media.

[0112] Metadata can be advantageously used to identify the basic data units stored on individual storage media, retrieve them from each storage medium, map them to unique data streams, and / or arrange them in the correct order relative to each other for reconstruction of the original unique data streams.

[0113] Metadata is advantageously associated with the distributed protocol used. Therefore, metadata and the distributed protocol used can complement each other and provide a common information base. Preferably, based on this information base, the reconstruction of unique data streams can be achieved at least partially (preferably completely).

[0114] Alternatively or supplementally, metadata may include the following information: - Information for data recording, such as a unique identifier for data recording and / or a unique identifier for the system used for data recording. - The memory address regions of individual basic data units, in particular the pointers to the first and last bytes of each basic data unit, - A unique identifier for identifying individual basic data units, - Mapping between individual basic data units and sources, especially sensors, - For example, information about the temporal sorting of basic data units in the form of timestamps, one or more lists of unique identifiers in a time series. - A unique identifier for a data recording session, - A unique identifier for the storage medium involved in data recording. - A unique identifier for a tag placed on a storage medium. - A list containing the source of the unique data stream, - A list including storage media involved in data recording, specifically for checking whether the recorded data is completely present. - The name and path of the target file where the reconstructed datastream data should be stored, and / or - Start and end points of data recording, It can be configured to include the selection of [something].

[0115] The tags may be, for example, RFID tags. These allow for wireless detection of the presence of a storage medium. Each individual storage medium may be equipped with one such tag for this purpose. Each tag may contain a unique identifier for each storage medium, which can be obtained, in particular, by reading the tag wirelessly.

[0116] Preferably, the metadata contains information about the operating modes of one or more sensors present during data recording. This makes it known which data stream descriptors and / or distributed protocols were applied and when. Alternatively or supplementally, the metadata may contain information about data stream descriptors and / or distributed protocols applied in a time-dependent manner during data recording.

[0117] Alternatively or supplementally, the metadata can also be configured to be stored entirely in at least one of at least two storage media, preferably all of them, and / or in separate memories.

[0118] When metadata is stored along with the remaining basic data units on one or more storage media, reliable data reconstruction becomes possible.

[0119] The separate memory can be remote memory, such as cloud memory.

[0120] Alternatively or supplementally, each storage medium can be configured to store metadata only for the basic data units stored on that particular storage medium.

[0121] In this case, metadata can be retrieved and evaluated from all involved storage media for the purpose of data reconstruction. Subsequently, reconstruction can be performed favorably based on the selected and collected metadata.

[0122] Alternatively or supplementally, each storage medium can be configured to store metadata for all basic data units.

[0123] Therefore, advantageously, it is possible to identify which storage media are needed for reconstruction. This allows for the rapid detection of missing storage media for reconstruction.

[0124] The above problems are addressed by the present invention. Second aspect A method for forming at least one data stream from basic data units distributed across at least two storage media or source files, particularly by a method according to a first aspect of the present invention, wherein at least two basic data units are invoked and temporally ordered based on metadata from different storage media or different source files, thereby generating at least one data stream, at least partially.

[0125] This has the advantage of allowing data streams to be reconstructed from data distributed as fragments across multiple storage media or source files. In this case, the interaction between the stored metadata and the stored base data units has the advantage of eliminating the need for additional prior knowledge about the original data stream or other contexts during data recording. This makes reconstruction particularly reliable.

[0126] The means by which data streams can also be generated from source files provides an advantage in terms of independence from the storage medium. For example, in the data recording framework according to the first aspect of the present invention, data stored in at least two storage media can be transmitted to a remote computer (e.g., a cloud computer) in the form of files (particularly the contents of each storage medium as unique files), where a data stream can be generated from the transmitted source files as described above. Subsequently, the generated data stream may be returned by an optional means. In several embodiments, source files can be transmitted to the remote computer from various entities, and / or the generation of the data stream can be started only when all the necessary source files are present. The presence of all source files, for example, all storage media, can be checked using information stored in metadata, which will be described in more detail below.

[0127] Here, the generated data stream may be a data stream from a sensor (e.g., a radar sensor, a LiDAR sensor, or a camera sensor). For example, the data stream may be a previously received data stream (e.g., a unique data stream received in the method according to the first aspect of the present invention).

[0128] Advantageously, all features described in relation to the method according to the first aspect of the present invention can be configured in a particularly mirror-image manner in the method according to the second aspect of the present invention, unless otherwise stated in relation to the present invention.

[0129] Alternatively or supplementally, the generated data stream can be configured to be transmitted over the network and / or stored in separate memory.

[0130] The separate memory can be remote memory, such as cloud memory.

[0131] Alternatively or supplementally, at least two data streams may be generated at least partially from at least two basic data units based on metadata, preferably configured to store each of the generated data streams in a unique target file or to store at least two of the generated data streams in a common file.

[0132] The generated data stream can be stored in individual files, particularly advantageously, by storing the data as files.

[0133] Alternatively or supplementally, metadata can also be configured to be retrieved from at least one storage medium or from at least one operator, from which at least one of the basic data units may also be retrieved.

[0134] Alternatively or supplementally, the system may be configured to check, based on metadata, whether all of at least two storage media or all of at least two source files are available in particular for calling basic data units stored in or within them.

[0135] For example, the metadata can include information about tags placed on individual storage media. In this case, the existence of individual tags, and consequently the existence of individual storage media, can be checked. Similarly, each source file may contain a unique identifier, allowing the existence of all source files to be checked using metadata containing the identifiers of all source files. If the existence of at least one storage medium or at least one source file cannot be identified, an error signal can be generated, and optionally, the generation of the data stream can be interrupted.

[0136] The above problems are addressed by the present invention. Third aspectThe problem is solved by a data processing device, particularly a data recording device, having means configured to perform a method according to a first aspect of the present invention, which is data-technically coupled to or can be coupled to one or more sensors and / or data-technically coupled to or can be coupled to at least two storage media.

[0137] In this case, all the advantages described in relation to the method according to the first aspect of the present invention also apply to the present apparatus accordingly. Therefore, the above description can be referenced here.

[0138] In this case, the features described in relation to the method according to the first aspect of the present invention can be appropriately configured in the apparatus or in relation to the apparatus, individually and in any combination, unless otherwise stated in relation. In particular, the apparatus can be configured to perform the method steps described in relation to the method according to the first aspect of the present invention, particularly individually and in any combination.

[0139] Preferably, the device has one or more interfaces through which data-technical coupling with one or more sensors is manufactured or can be manufactured. Preferably, the device has one or more interfaces that form or can form data-technical coupling with a storage medium.

[0140] Alternatively or supplementally, the device may have memory, a processor, a receiver, a transmitter, or any combination thereof. Alternatively or supplementally, the device may have all of the devices it possesses, in particular all necessary resources, such as software resources and / or hardware resources that are available and / or can be supplied and / or included. Advantageously, the device may have means for receiving, transmitting, storing, processing, evaluating and / or analyzing data of each unique data stream.

[0141] This device can be configured as a data logger. Alternatively, or supplementally, this device may be a distributed system.

[0142] The above problems are addressed by the present invention. Fourth aspect The problem is solved by proposing a data processing device for reconstructing a data stream that is data-technically coupled or can be coupled to at least two storage media, particularly a data stream recorded by the apparatus of the present invention according to a third aspect, the apparatus having means configured to perform the method according to a second aspect of the present invention.

[0143] In this case, all the advantages described in relation to the method of the second aspect of the present invention also apply to the apparatus of the fourth aspect of the present invention. Therefore, the above description can be referenced here.

[0144] In this case, the features described in relation to the method according to the second aspect of the present invention can be appropriately configured in the apparatus or in relation to the apparatus, individually and in any combination, unless otherwise stated in relation. In particular, the apparatus can be configured to perform the method steps described in relation to the method according to the second aspect of the present invention, particularly individually and in any combination.

[0145] Preferably, the device has one or more interfaces that form or are capable of forming a data-technical coupling with a storage medium.

[0146] Alternatively or supplementally, the device may have memory, a processor, a receiver, a transmitter, or any combination thereof. Alternatively or supplementally, the device may have everything it possesses, in particular all necessary resources, such as software resources and / or hardware resources that are available and / or can be supplied and / or included. Advantageously, the device may have means for transmitting each data stream (e.g., to another entity).

[0147] The data processing device may be, for example, a cloud computer. Alternatively, or supplementarily, the data processing device may form a distributed system. For example, a portion of the data processing device can be located in the cloud.

[0148] In several embodiments, the data processing device according to the third aspect of the present invention and the data processing device according to the fourth aspect of the present invention are considered to be an integrated data processing device.

[0149] The above problems are addressed by the present invention. Fifth aspect This problem is solved by proposing a computer program product that includes instructions to cause a data processing device to execute a method according to the first and / or second aspect of the present invention when the program is executed by the data processing device according to the third and / or fourth aspect of the present invention.

[0150] Computer program products can be stored, for example, on computer-readable media or configured as files that can be stored.

[0151] The above problems are addressed by the present invention. Sixth aspect This is solved by proposing a signal sequence that can be transmitted, particularly over a computer network, which represents data from a data stream received in a method according to a first aspect of the present invention, and / or represents data from a data stream generated in a method according to a second aspect of the present invention.

[0152] This signaling sequence advantageously allows data from each data stream to be exchanged between different subscribers.

[0153] For example, this signal sequence may be receivable by a remote computer and / or transmittable to a remote computer.

[0154] The above problems are addressed by the present invention. Seventh aspect This is resolved by a storage medium used in an apparatus according to a second and / or fourth aspect of the present invention, wherein at least some data of at least one data stream is stored in the storage medium, in particular in a plurality of storage areas of the storage medium that do not have a contiguous address space, and metadata is stored in the storage medium, the metadata defining in which storage area of ​​the storage medium the data of the data stream is stored.

[0155] Therefore, metadata as functional data allows for the complete or partial reconstruction of fragmented and stored data from a data stream into a single data stream.

[0156] All the advantages described in relation to the method according to the second aspect of the present invention also apply to the storage medium according to the seventh aspect of the present invention. Therefore, the above description can be referenced here.

[0157] At least two storage media data-technically coupled to a data processing device according to a fourth aspect of the present invention and / or at least two storage media used in a method according to a second aspect of the present invention may each be storage media according to a seventh aspect of the present invention.

[0158] Another feature and advantage of the present invention will become apparent from the following description, which illustrates preferred embodiments of the invention with reference to schematic drawings. Each figure shows the following: [Brief explanation of the drawing]

[0159] [Figure 1] This is a schematic diagram showing a data processing device according to a third aspect of the present invention, which includes a sensor and a storage medium that are data-technically coupled. [Figure 2] This is a schematic diagram showing the data stream received from the sensor in Figure 1. [Figure 3]Figure 1 is a schematic diagram showing the storage medium in which the data stream and the basic data units of a portion of the data stream are stored. [Figure 4] This is a schematic diagram showing a file containing metadata. [Figure 5] This is a schematic diagram showing a data processing device according to a fourth aspect of the present invention, which includes a storage medium that is data-technically coupled. [Figure 6] This is a flowchart showing a method according to a first aspect of the present invention. [Figure 7] This is a flowchart showing a method according to a second aspect of the present invention. [Modes for carrying out the invention]

[0160] Figure 1 A schematic diagram of a data processing device 1 according to a third aspect of the present invention is shown.

[0161] The data processing device 1 is data-technically coupled to three sensors 5a to 5c via data lines 3a to 3c. Sensors 5a to 5c may be vehicle sensors. Each of the sensors 5a to 5c can generate sensor data, and the data processing device 1 can receive this sensor data via data lines 3a to 3c.

[0162] The data processing device 1 continuously receives a unique data stream containing sensor data from the sensor 5a via the data line 3a. The sensor 5a is, for example, a camera sensor. The camera sensor can capture images of its environment at, for example, a specific image repetition rate (frame rate). In this case, the unique data stream contains data from individual images of the camera sensor that are recorded in chronological order.

[0163] The data processing device 1 continuously receives a unique data stream containing sensor data from sensor 5b via data line 3b. Sensor 5b is, for example, a radar sensor. A radar sensor may have, for example, multiple pairs of transmitting and receiving antennas and may capture images of the radar environment at a specific capture rate. In this case, the transmitted sensor data may be, for example, an image representing the environment in space, already reconstructed from sensor raw data by sensor 5b. Alternatively, only sensor raw data (e.g., existing in Fourier space) may be transmitted.

[0164] The data processing device 1 continuously receives a unique data stream containing sensor data from the sensor 5c via the data line 3c. The sensor 5c is, for example, a LiDAR sensor. The LiDAR sensor can capture images of its environment at, for example, a specific image repetition rate (frame rate). For this purpose, the laser of the LiDAR sensor can scan its environment. In this case, the unique data stream includes data from individual images recorded by the LiDAR sensor in a temporally sequential manner.

[0165] In multiple embodiments, the data processing device 1 may be data-technically coupled to multiple sensors, or to some sensors.

[0166] Figure 2 This shows a schematic diagram of the unique data streams received from sensors 5a to 5c.

[0167] The data processing device 1 receives a unique data stream 7a from the sensor 5a. The unique data stream 7a received by the sensor 5a is known in advance with respect to several technical characteristics. These characteristics are stored in a data stream descriptor created in advance for the unique data stream 7a. In the case of the camera sensor mentioned as an example, the technical characteristics of the unique data stream 7a stored in the data stream descriptor may be the frame rate and memory size of one frame. Each such group of data in one frame can be processed as a single basic data unit. The data of sequentially consecutive frames (i.e., basic data units) are logically separated from each other within the data stream 7a, identifiable by the control data contained in the unique data stream 7a. For example, the control data can mark the start and / or end of the amount of data belonging to a basic data unit (each sensor can, for example, insert the corresponding marking into the data stream itself). Information about this control data is advantageously stored in the data stream descriptor of the unique data stream 7a. In Figure 2, the unique data stream 7a is shown as temporally sequential basic data units D1.1 to D1.N. The first basic data unit D1.1 of the unique data stream 7a is received at time T0, and subsequent basic data units D1.2 to D1.N are received in time in response to the inversion of the frame rate. It is obvious that each basic data unit D1.1 to D1.N itself is constructed from multiple data, such as the values ​​of individual pixels in a captured image frame.

[0168] Similarly, the data processing device 1 receives a unique data stream 7b from sensor 5b and a unique data stream 7c from sensor 5c. The unique data streams 7b and 7c are also known in advance with respect to several technical characteristics. These characteristics are stored in pre-created data stream descriptors for each unique data stream 7b and 7c. In the case of the radar and LiDAR sensors mentioned as examples, the technical characteristics stored in each data stream descriptor for each unique data stream 7b or 7c could be the frame rate and memory size of the radar or LiDAR recording frames. In this case as well, such a data group for a single frame can be referred to as a basic data unit. Therefore, in Figure 2, the unique data streams 7b and 7c are also shown as temporally sequential basic data units D2.1~D2.N or D3.1~D3.N, respectively. The first basic data unit D2.1 of the unique data stream 7b and the first basic data unit D3.1 of the unique data stream 7c are received at time T0, and further basic data units D2.2-D2.N and D3.2-D3.N are received subsequently in accordance with the inversion of the frame rate. In this case as well, it is obvious that each basic data unit D2.1-D2.N and D3.1-D3.N itself is constructed from multiple data, such as the values ​​of individual pixels in the captured image frame.

[0169] The basic data units of the further unique data streams 7a to 7c along the positive time axis t in Figure 2 described above are received at a later time than the basic data units of each unique data stream 7a to 7c shown below. Specifically, for example, within unique data stream 7a, basic data unit D1.1 is received temporally earlier than basic data unit D1.2, and basic data unit D1.2 is received temporally earlier than basic data units D1.3 and later.

[0170] The unique data streams 7a-7c are received from sensors 5a-5c in parallel in time. Therefore, the basic data units of the various sensors 5a-5c are also received in parallel in time. In the case shown in Figure 2, all nth [n=1...N] basic data units of the three unique data streams 7a-7c are received in parallel in time. That is, all first basic data units D1.1, D2.1 and D3.1 are received in parallel in time, all second basic data units D1.2, D2.2 and D3.2 are received in parallel in time, and so on. Such special cases are selected in this specification solely for the reason of better displayability. In general, the nth basic data units of different unique data streams can be received with arbitrary staggerations relative to each other. The time difference between the same number n basic data units (i.e., D1.4 and D2.4) in two unique data streams depends, in particular, on the amount of data contained within each basic data unit and the data rate at which the data processing unit 1 receives data from each sensor via which data line. In the special case shown in Figure 2, it is assumed that the amount of data in each basic data unit for each sensor 5a to 5c is transmitted simultaneously from each sensor to the data processing unit. Furthermore, the number N of basic data units in each unique data stream 7a to 7c may be different.

[0171] For example, sensor data is generated by sensors 5a-5c during test runs of the vehicle mentioned at the beginning, and this sensor data is subsequently used for data-driven development of driver assistance systems and autonomous driving applications. For this purpose, unique data streams must be stored during the test runs.

[0172] For this purpose, the data processing device 1 (Figure 1) is data-technically coupled to the four storage media 11a to 11d of the memory unit 13 via four data lines 9a to 9d. Data received from the three sensors 5a to 5c can be stored in these storage media 11a to 11d.

[0173] Here, as already described in relation to Figure 2, the data processing device 1 divides each unique data stream 7a to 7c into basic data units D1.n, D2.n, and D3.n [n=1...N], respectively. That is, the division of unique data stream 7a into basic data unit D1.n is performed based on the data stream descriptor created for unique data stream 7a (in particular by dividing unique data stream 7a at locations where markings stored as control data within the data stream descriptor are detected). Similarly, the division of unique data stream 7b into basic data unit D2.n is performed based on the data stream descriptor created for unique data stream 7b (in particular by dividing unique data stream 7b at locations where markings stored as control data within the data stream descriptor are detected). Furthermore, the division of unique data stream 7c into basic data unit D3.n is performed based on the data stream descriptor created for unique data stream 7c (in particular by dividing unique data stream 7c at locations where markings stored as control data within the data stream descriptor are detected). Therefore, dividing the basic data streams 7a-7c into basic data units D1.n, D2.n, and D3.n means here that the data from each data stream 7a-7c is grouped, thereby obtaining individual data units D1.n, D2.n, and D3.n. Each basic data unit obtained in this way can, advantageously, accurately correspond to the data of each sensor 5a-5c (e.g., camera sensor, radar sensor, or LiDAR sensor).

[0174] To enable the individual unique data streams 7a-7c to be divided into basic data units based on each control data, the unique data streams 7a-7c can be analyzed sequentially, for example, according to corresponding markings. In some embodiments, the data of the unique data streams 7a-7c can be grouped as individual basic data units based on the frequency or memory size of the data frame indicated in each data stream descriptor.

[0175] In this case, the data of each basic data unit D1.n, D2.n, and D3.n of the unique data streams 7a to 7c is temporarily stored in volatile RAM memory before being finally stored. For this reason, the memory size reserved for the basic data units D1.n, D2.n, and D3.n of the unique data streams 7a to 7c can be predetermined based on the data stream descriptor, thereby optimizing resource utilization.

[0176] The individual basic data units D1.n, D2.n, and D3.n of the unique data streams 7a to 7c are distributed and stored across four storage media 11a to 11d. The data processing device 1 performs distributed storage according to a distribution protocol. This distribution protocol is created before the start of data recording based on three data stream descriptors (see the description above) that exist for the unique data streams 7a to 7c. This allows the distribution of basic data units to the individual storage media 11a to 1d to depend, for example, on the data sizes of the different basic data units D1.n, D2.n, and D3.n.

[0177] In this embodiment, the nth basic data unit of each of the three unique data streams 7a to 7c is received in parallel in time, but these basic data units can each have different amounts of data (for example, the data speeds on the data lines 3a to 3c are different, but the nth data unit is received in parallel in time), and consequently, different memory space requirements for each of the storage media 11a to 11d and different write times to each of the storage media 11a to 11d can be required.

[0178] The distributed protocol, as described herein, sets a periodic order in which the basic data units D1.n, D2.n, and D3.n of the unique data streams 7a to 7c are stored in the individual storage media 11a to 11d. For example, without limiting generality, the basic data units are allocated to storage media 11a in order, then to storage media 11b, then to storage media 11c, then to storage media 11d, then back to storage media 11a, and so on. Furthermore, the distributed protocol, as described herein, sets the order in which, when different basic data units of unique data streams 7a to 7c arrive in parallel in time, the basic data units of unique data stream 7a are allocated to the storage media first, then to the storage media, then to the storage media, and then to the storage media, and so on.

[0179] Figure 3 Figure 1 schematically shows the storage media 11a to 11d, which have unique data streams 7a to 7c stored in the storage media 11a to 11d, and basic data units of parts 7a and 7b of said data streams.

[0180] At the point shown in Figure 3, the basic data units D1.1-D1.4 of the unique data stream 7a, D2.1-D2.4 of the unique data stream 7b, and D3.1-D3.4 of the unique data stream 7c are already distributed and stored across the four storage media 11a-11d. Therefore, in the upper part of Figure 3, the already stored basic data units are no longer shown as part of the unique data streams 7a-7c.

[0181] In this way, a distributed protocol is applied (storage locations in which incoming basic data units are periodically stored in storage media 11a-11b-11c-11d; for basic data units of different unique data streams arriving in parallel in time, unique data stream 7a has a higher priority than the amount of data in unique data stream 7b, and data stream 7b has a higher priority than the amount of data in unique data stream 7c), The storage medium 11a is occupied by the basic data unit D1.1 of the unique data stream 7a, the basic data unit D2.2 of the unique data stream 7b, and the basic data unit D3.3 of the unique data stream 7c. The storage medium 11b is occupied by the basic data unit D2.1 of the unique data stream 7b, the basic data unit D3.2 of the unique data stream 7c, and the basic data unit D1.4 of the unique data stream 7a. A state in which the storage medium 11b is occupied by the basic data unit D3.1 of the unique data stream 7c, the basic data unit D1.3 of the unique data stream 7a, and the basic data unit D2.4 of the unique data stream 7b, and The storage medium 11a is occupied by the basic data units D1.2 of the unique data stream 7a, D2.3 of the unique data stream 7b, and D3.4 of the unique data stream 7c. You can obtain this.

[0182] Here, the storage of various basic data units into individual storage media 11a to 11d is performed by writing to storage media 11a to 11d in parallel in time. That is, for example, if individual data already exists, basic data units D1.1, D2.1, D3.1 and D1.2 are always stored in parallel in time. By writing to storage media 11a to 11d in parallel, the allowable total input bandwidth, i.e., the sum of the bandwidths of the individual unique data streams 7a to 7c, can be increased.

[0183] Next, in the following step, if the next basic data unit has already been received, or if more will be received, it will be stored in the storage medium according to the distributed protocol.

[0184] At the bottom of Figure 3, the different memory sizes (and therefore data amounts) of the basic data units of the three unique data streams 7a to 7c are shown by their respective different levels (stored in the storage media 11a to 11d). Therefore, the memory size of the basic data unit D2.n of unique data stream 7b is larger than the memory size of the basic data unit D3.n of unique data stream 7c, and the memory size of the basic data unit D3.n of unique data stream 7c is larger than the memory size of the basic data unit D1.n of unique data stream 7a. However, in this case, the memory sizes of the individual basic data units D1.n, D2.n, and D3.n of each unique data stream 7a to 7c are the same.

[0185] The distributed protocol is designed to ensure that the individual data units fill each storage medium 11a-11d as uniformly as possible. For this purpose, algorithms for solving the "container problem" can also be used when creating the distributed protocol. Furthermore, the distributed protocol guarantees that the maximum bandwidth of each storage medium 11a-11d is not exceeded during write access to them. Since the predicted amount of data per unit time and per unique data stream (and thus the number of basic data units) is known in advance based on each data stream descriptor, the distributed protocol can be generated to ensure the most efficient distributed storage of the basic data units. Thus, the distributed protocol can take into account the maximum allowable bandwidth of each storage medium 11a-11d (during writing), just as the required memory amount per basic data unit is optimized for various unique data streams and, consequently, for the storage process.

[0186] By dividing the unique data streams 7a to 7c into multiple basic data units based on data stream descriptors and distributing the basic data units across storage media 11a to 11d based on a distributed protocol, the data of the unique data streams 7a to 7c can be stored in real time even at high total bandwidth. This is because the maximum allowable bandwidth of each individual storage medium 11a to 11d becomes advantageously available as a sum, based on the division into basic data units and the time-parallel writing to the storage media 11a to 11d. In other words, the input data can be advantageously stored with the total bandwidth scaled to the individual bandwidths of the storage media 11a to 11d.

[0187] It is assumed that the memory size of each basic data unit in each unique data stream 7a-7c is the same as described above. However, in some embodiments, the memory size of each basic data unit in at least one unique data stream 7a-7c may also change over time. This can occur, for example, when one of the multiple sensors 5a-5c operates in different operating modes over time, causing the amount of data in the basic data units or the time intervals between basic data units in each unique data stream to change over time. For example, a camera sensor (sensor 5a) can be operated sequentially over time, first in a standard mode with a first resolution, and then in an alarm mode with a second resolution higher than the first resolution. For this purpose, the data stream descriptor and / or distribution protocol can be directed to the operating mode with the highest data rate ("worst case"). In this case, it is acceptable that the reserved resources may not be fully utilized during one operating mode (i.e., the normal mode in this example), depending on the circumstances. Alternatively, different data stream descriptors and / or distribution protocols can be selected and applied depending on the operating mode. This allows for optimized resource utilization for each operating mode.

[0188] In either case, after all the basic data units of all unique data streams 7a to 7c are stored in the storage media 11a to 11d, a file 15 containing metadata is stored in each storage medium 11a to 11d.

[0189] Figure 4 A schematic diagram of File 15 is shown there.

[0190] The first metadata area 15a of file 15 contains information about the storage media 11a to 11d involved. For example, a unique identifier for each storage media 11a to 11d can be stored in the metadata area 15a.

[0191] The second metadata area 15b of file 15 contains information about the basic data units stored in each storage medium 11a to 11d in which file 15 is stored. For example, for this purpose, for each basic data unit that partially contains the data of each basic data unit from a unique data stream stored in each storage medium, the number n of the basic data unit in the unique data stream and / or the start address and / or end address of the storage area that stores the data of the basic data unit in each storage medium can be stored.

[0192] The third metadata area 15c of file 15 stores information about the data recording, such as a timestamp indicating the start of the data recording and a timestamp indicating the end of the data recording.

[0193] Therefore, by using the data processing device 1 (and the method performed by the device as described above), the storage of data streams themselves to multiple storage media can be performed efficiently with higher bandwidth. This makes it possible to provide the bandwidth necessary for the simultaneous recording of multiple data streams from various sensors, even during test drives. This is extremely advantageous for data-driven development of driver assistance systems and autonomous driving applications.

[0194] The data of individual unique data streams 7a to 7c, which are distributed and stored in the storage media 11a to 11d, can be used, for example, later to train a neural network. For this purpose, each unique data stream must be reconstructed from the distributed and stored basic data units.

[0195] Figure 5 A schematic diagram of a data processing device 17 according to a fourth aspect of the present invention is shown. The data processing device 17 can be used to reconstruct the original unique data streams 7a to 7c.

[0196] For this purpose, the data processing unit 17 is data-technically coupled to the four storage media 11a to 11d of the memory unit 13 (which were coupled to the data processing unit 1 in the previous case) via four data lines 19a to 19d. For example, this allows the memory unit 13 to be moved from one location on the data processing unit 1 to the other location on the data processing unit 17.

[0197] Unlike what was described above in relation to the situation shown at the bottom of Figure 3, here all basic data units D1.n, D2.n, and D3.n of all unique data streams 7a to 7c are stored in storage media 11a to 11d (in Figure 5, only the basic data units already shown at the bottom of Figure 3 are shown, but the remaining basic data units at the top of Figure 3 are not shown). Furthermore, each storage medium 11a to 11d stores files M1 to M4 containing metadata. Each file can be constructed similarly to file 15 described in relation to Figure 4 and can contain corresponding information. In the four files M1 to M4, the information in the first metadata area and the information in the third metadata area are identical, but the information about the basic data units stored in each second metadata area differs for each storage medium.

[0198] To reconstruct the original data streams (unique data streams 7a-7c), metadata from files M1-M4 is first evaluated. Based on the (identical) information in the first metadata area of ​​one of the files M1-M4, the existence of all four storage media 11a-11d can be checked and confirmed. Based on the information from the second metadata areas of all four files M1-M4, for each basic data unit D1.n of unique data stream 7a, both the start and end addresses of the storage areas of the storage media 11a-11d and the storage media in which the data is stored can be calculated. The unique data stream 7a can be reconstructed by reading the data of each basic data unit D1.n from each storage area and arranging them in the correct order n=1...N. The same procedure for each basic data unit D2.n of unique data stream 7b and each basic data unit D3.n of unique data stream 7c enables the reconstruction of unique data streams 7b and 7c. The reconstructed data stream can be stored, for example, on a data carrier, or transmitted via a computer network, such as the Internet, for further use.

[0199] In some embodiments, the data processing device 17 may be the same as the data processing device 1.

[0200] The data of the received and reconstructed unique data streams 7a to 7c may each be a signal sequence according to a sixth aspect of the present invention.

[0201] The storage media 11a to 11d described in relation to Figure 5 may each be a single storage media according to the seventh aspect of the present invention.

[0202] Figure 6 A flowchart 100 of a method according to a first aspect of the present invention is shown.

[0203] This method can be used to perform data recording, such as the data recording described above in relation to Figures 1 to 4.

[0204] In 101, at least one data stream is received to record data, and within that data stream, there is at least one unique data stream.

[0205] In 103, each unique data stream is divided into basic data units based on the data stream descriptor set for each unique data stream.

[0206] In 105, the basic data units of all unique data streams are distributed and stored across at least two storage media.

[0207] For example, in 105, distributed storage of the basic data units of a unique data stream across at least two storage media can be performed based on a distributed protocol. Based on the distributed protocol, each basic data unit of a unique data stream can be assigned a storage location to at least one storage medium and / or exactly one of at least two storage media, and each basic data unit can be stored in the storage medium assigned to it.

[0208] This method can be executed, for example, by a data processing device 1 described in relation to Figure 1, and can be configured to correspond to said data processing device 1.

[0209] Figure 7 A flowchart 200 of a method according to a second aspect of the present invention is shown.

[0210] This method can be used, in particular, to generate at least one data stream from basic data units distributed across at least two storage media or source files, using the method according to the first aspect of the present invention. This could be, for example, the reconstruction described above in relation to Figure 5.

[0211] In 201, based on metadata, at least two basic data units are invoked, each from a different storage medium or a different source file.

[0212] In 203, at least two basic data units are temporally ordered, thereby generating at least one data stream, at least partially.

[0213] This method can be executed, for example, by a data processing device 17 described in relation to Figure 5, and can be configured to correspond to said data processing device 17.

[0214] The features disclosed in the prior art, drawings, and claims may be important to various embodiments of the present invention, either individually or in any combination. [Explanation of symbols]

[0215] 1 Data Processing Device 3a,3b,3c data line 5a, 5b, 5c sensors 7a, 7b, 7c Unique data streams 9a, 9b, 9c, 9d Data lines 11a,11b,11c,11d Storage medium 13 memory units 15. Files containing metadata 15a, 15b, 15c Metadata area 17 Data Processing Devices 19a, 19b, 19c, 19d Data lines D1.n The nth basic data unit of the unique data stream 7a D2.n The nth basic data unit of the unique data stream 7b D3.n The nth basic data unit of the unique data stream 7c Files containing M1, M2, M3, and M4 metadata n is the number of the basic data unit in the unique data stream. N is the number of basic data units in a unique data stream. t Time axis Time T0 100 flowcharts 101 At least one data stream is received to record data; where at least one unique data stream exists within that data stream. 103 Based on the data stream descriptor set for each unique data stream, each unique data stream is divided into basic data units. 105 Distribute and store the basic data units of all unique data streams across at least two storage media. 200 flowcharts 201 Based on metadata, call at least two basic data units, each from a different storage medium or from a different source file. 203. Temporarily order at least two basic data units, thereby generating at least one data stream, at least partially.

Claims

1. A method for performing data recording, At least one data stream is received for data recording, and at least one unique data stream exists within the data stream, and each unique data stream is divided into basic data units based on a data stream descriptor set for each unique data stream, and the basic data units of all unique data streams are distributed and stored on at least two storage media. The distributed storage of the basic data units of the unique data stream to the at least two storage media is performed based on a distributed protocol. Based on the distributed protocol, for each basic data unit of a unique data stream, at least one and / or exactly one of the at least two storage media is allocated as a storage location, and each basic data unit is stored in the storage media allocated to each of the basic data units. method.

2. Each of the aforementioned unique data streams is received sequentially and divided into basic data units, and / or Two or more data streams are received, and among the data streams, there are two or more unique data streams that are received at least partially in parallel in time. The method according to claim 1.

3. (a) A data stream descriptor set for each individual data stream has one or more technical specifications for each individual data stream, and based on the technical specifications, the division of each individual data stream into basic data units is performed at least partially. (b) The data stream descriptor set for each individual data stream is, (i) Information about the memory size of the basic data unit of each unique data stream, and / or (ii) Information about the frequency at which sequentially consecutive basic data units within each unique data stream are received. The information provided is such that, based on this information, the division of each unique data stream into the basic data unit is performed at least partially, and / or (c) The division of the unique data streams into basic data units includes grouping the data of each unique data stream based on the data stream descriptor set for each unique data stream, thereby obtaining the basic data units. The method according to claim 1 or 2.

4. Each basic data unit of a unique data stream has either data from an individual image recorded by a sensor or rows or pixels of the individual image. The method according to any one of claims 1 to 3.

5. At least one of the at least one unique data streams, preferably all of the unique data streams, has a basic data unit, in particular according to the distributed protocol. The information is distributed and stored across multiple storage media, and / or The distributed protocol sets a particularly periodic order for storage media for at least one unique data stream, preferably for each unique data stream, and the basic data units of each unique data stream are stored in the storage media according to the order. The method according to any one of claims 1 to 4.

6. (i) The distributed protocol is calculated such that, when distributing the basic data units of the unique data stream to the at least two storage media, the maximum data rate permitted for write access to each individual storage medium is not exceeded. (ii) The distributed protocol is calculated such that uniform data filling of each storage medium is achieved when distributing the basic data units of the unique data stream to the at least two storage mediums, and / or (iii) For each of the at least two storage media, the maximum data rate that each storage medium can write to is known as bandwidth information and / or the free memory space is known as capacity information, and / or is automatically calculated, and preferably the distributed protocol is calculated based on the bandwidth information and / or capacity information of the storage media. The method according to any one of claims 1 to 5.

7. Each of the aforementioned unique data streams represents a sensor signal originating from, in particular, the image generation sensor and / or a different sensor. Preferably, (a) at least one unique data stream represents a sensor signal from a camera sensor, radar sensor, LiDAR sensor and / or ultrasonic sensor, and / or (b) at least one sensor operates in different operating modes sequentially in time, and depending on the operating mode, at least one technical specification of the unique data stream received by the sensor, e.g., (i) data rate, (ii) data size of a basic data unit and / or (iii) frequency when a new basic data unit is received is changed. Preferably, the distributed protocol is calculated so that in each operating mode of the sensor, the basic data units are distributed and stored across at least two storage media. The method according to any one of claims 1 to 6.

8. The data stream descriptor and / or the distributed protocol of the unique data stream received from the sensor are each calculated for an operating mode in which the unique data stream received from the sensor has the maximum data rate, at least partially. The method according to any one of claims 1 to 7.

9. Depending on the selected operating mode of the sensor, switching occurs between different data stream descriptors and / or between the distributed protocols of the unique data streams received by the sensor. The method according to any one of claims 1 to 8.

10. At least one unique data stream, preferably all unique data streams, each has its basic data units distributed and stored in at least two of the at least two storage media, without any redundancy. The method according to any one of claims 1 to 9.

11. The aforementioned data recording is performed in real time, and / or the data of the received unique data stream is not temporarily stored in non-volatile memory. The method according to any one of claims 1 to 10.

12. The data recording includes generating metadata that includes information about basic data units stored on individual storage media, and information for reconstructing individual unique data streams from basic data units distributed across at least two storage media. Preferably, the metadata includes the following information, namely: - Information about data recording, such as a unique identifier for the data recording and / or a unique identifier for the system used for data recording. - The memory address areas of each basic data unit, in particular the pointers to the first and last bytes of each basic data unit, - A unique identifier for identifying individual basic data units, - Mapping between individual basic data units and sources, especially sensors, - For example, information about the temporal sorting of basic data units in the form of timestamps, a list of one or more unique identifiers in a time series, - A unique identifier for the data recording session, - A unique identifier for the storage medium involved in data recording, - A unique identifier for tags placed on a storage medium, - A list containing the source of the unique data stream, - A list including storage media involved in data recording, specifically for checking whether the recorded data is completely present. - The name and path of the target file where the reconstructed data stream data should be stored, and / or - Start and end points of data recording, Including the selection, The method according to any one of claims 1 to 11.

13. A method for generating at least one data stream from basic data units distributed across at least two storage media or source files, particularly according to any one of claims 1 to 12, Based on metadata, at least two basic data units are invoked and temporally ordered from different storage media or different source files, thereby generating at least one data stream, at least partially. Preferably, based on the metadata, (i) At least two data streams are generated at least partially from each of at least two basic data units, preferably each of the generated data streams is stored in a specific target file, or at least two of the generated data streams are stored in a common file, and / or (ii) At least two storage media or at least two source files are checked to see whether they are available in particular for calling basic data units stored on the storage media or in the source files or within the storage media or in the source files. method.

14. A data processing device, particularly a data recording device, that is data-technically coupled or can be coupled to one or more sensors and / or data-technically coupled or can be coupled to at least two storage media, The apparatus includes means and / or data processing devices configured to perform the method described in any one of claims 1 to 12, in particular a data recording device that reconstructs a data stream recorded by an apparatus according to the first alternative means of this claim, which is data-technically coupled to or can be coupled to at least two storage media. The apparatus further comprises means configured to carry out the method described in claim 13, Data processing device.

15. A computer program product, When the program is executed, a data processing device, particularly one of the data processing devices described in claim 14, includes an instruction to cause the data processing device to execute the method described in any one of claims 1 to 12 or the method described in claim 13. Computer program products.

Images