File formats for efficient storage and access of data

Abstract

The systems and methods may include a computing system that obtains data including a plurality of data elements associated with a plurality of indexes, each index associated with one or more data elements, generates a data file that stores the plurality of data elements according to a predetermined file format including a header and a plurality of data rows, each data row corresponding to a respective index and storing one or more data elements associated with the respective index, the header including, for each row, an association between the respective index and a corresponding data offset indicating a memory location of the data row, uses an indication of a first index and the header to determine a first memory location of the data row corresponding to the first index, and uses the first memory location to access at least one data element associated with the first index.

Description

[Technical Field]

[0001] [Cross-reference to related patent applications] This application claims priority to U.S. Provisional Application No. 63 / 377,954, filed September 30, 2022, and U.S. Provisional Application No. 63 / 378,012, filed September 30, 2022, which are incorporated by reference in their entireties for all purposes.

[0002] FIELD OF THE DISCLOSURE This disclosure relates generally to efficient techniques for storing and accessing data. In particular, this disclosure relates to new file formats and systems and methods using file formats for more efficient storage of data and accelerated data access. [Background technology]

[0003] Due to rapid advances in computer technology, autonomous navigation techniques used for autonomous vehicles and robots (collectively, egos) have become widespread. These advances enable safer and more reliable autonomous navigation for ego's. Egos often must navigate through complex and dynamic environments and terrain that may include vehicles, traffic, pedestrians, cyclists, and a variety of other static or dynamic obstacles. Understanding the ego's surroundings is necessary for making informed and appropriate decisions to avoid collisions. Summary of the Invention

[0004] The systems, devices, and methods described herein provide efficient techniques for storing and accessing data. In particular, the systems, devices, and methods described herein enable a new file format that results in more efficient use of memory resources when storing data and faster access to stored data. The new file format can be used in applications involving relatively large amounts of data and a relatively large number of system I / O operations, such as training machine learning (ML) models. In such applications, the amount of memory used to store data can be reduced, and the number of system I / O operations used and the processing time of the system I / O operations can be significantly reduced. For example, in the case of ML or artificial intelligence (AI) models used to predict or detect ego surroundings, such as occupancy networks, training such models can be very time-consuming. The systems, devices, and methods described herein significantly accelerate the training and / or validation of such models by reducing the time used to read and / or write pieces of training data. Additionally, the systems, devices, and methods described herein result in more efficient use of processing and memory resources.

[0005] According to at least one aspect, a method may include: acquiring, by a processor, ego navigation data including a plurality of data elements associated with a plurality of indexes, each index associated with one or more data elements, the ego navigation data being determined based on data captured by at least one sensor of the ego; generating, by the processor, a data file storing data according to a predetermined file format including a header and a plurality of data rows, such that each data row corresponds to a respective index and stores one or more data elements associated with the respective index, the header including, for each data row, an association between the respective index and a corresponding data offset indicating a memory location of the data row; determining, by the processor, a first memory location of the data row corresponding to the first index using an indication of the first index and the header; accessing, by the processor, at least one data element associated with the first index using the first memory location; and using, by the processor, the at least one data element associated with the first index to train a machine learning (ML) model.

[0006] In some implementations, the ego navigation data can be used to train the ego's occupancy network, which is used to predict or sense the ego's surroundings. In some implementations, the multiple indexes can include multiple timestamps or multiple time values. In some implementations, the data file can be a read-only data file.

[0007] In some implementations, the data can further include at least one other data element associated with all of the plurality of indexes. The method can further include determining, by the processor, that the at least one other data element is associated with all of the plurality of indexes, and storing, by the processor, the at least one other data element before the plurality of data rows in the data file.

[0008] In some implementations, the data file can store a plurality of data elements according to a plurality of columns, each column representing a corresponding data field, and the method can further include arranging the plurality of columns according to an ascending order of data size associated with the plurality of data columns.

[0009] In some implementations, the data elements can include multiple tensors, and each data row can store one or more tensors associated with a respective index corresponding to the data row. The method can further include at least one of transposing, by the processor, at least a subset of the tensors before storing them in the data file, or encrypting, by the processor, at least a subset of the tensors before storing them in the data file.

[0010] In some implementations, the method may further include receiving, by the processor, a request for at least one data element associated with a first index, the request including an indication of the first index, and determining, by the processor, the first index using the indication.

[0011] In some implementations, accessing at least one data element associated with the first index may include, by the processor, skipping from a second memory location associated with the header to the first memory location when determining a first memory location of the data row corresponding to the first index.

[0012] According to at least one other aspect, a computing system may include a processor and a memory storing executable instructions that, when executed by the processor, cause the computer system to obtain ego navigation data including a plurality of data elements associated with a plurality of indexes, each index associated with one or more data elements, the ego navigation data determined based on data captured by at least one sensor of the ego, generate a data file storing the plurality of data elements according to a predetermined file format including a header and a plurality of data rows, each data row corresponding to a respective index and storing one or more data elements associated with the respective index, the header including, for each data row, an association between the respective index and a corresponding data offset indicating a memory location of the data row, determine a first memory location of the data row corresponding to the first index using an indication of the first index and the header, access at least one data element associated with the first index using the first memory location, and use the at least one data element associated with the first index to train a machine learning (ML) model.

[0013] In some implementations, the ego navigation data can be used to train the ego's occupancy network, which is used to predict or sense the ego's surroundings. In some implementations, the multiple indexes can include multiple timestamps or multiple time values. In some implementations, the data file can be a read-only data file.

[0014] In some implementations, the data can further include at least one other data element associated with all of the plurality of indexes. The executable instructions, when executed by the processor, further cause the computer system to determine that the at least one other data element is associated with all of the plurality of indexes and store the at least one other data element before the plurality of data rows in the data file.

[0015] In some implementations, the executable instructions, when executed by a processor, cause a computer system to store a plurality of data elements according to a plurality of columns in a data file, each column representing a corresponding data field, and arrange the plurality of columns according to ascending order of data size associated with the plurality of data columns.

[0016] In some implementations, the multiple data elements can include multiple tensors, and each data row can store one or more tensors associated with a respective index corresponding to the data row. The executable instructions, when executed by a processor, cause a computer system to store the multiple data elements according to at least one of transposing at least a subset of the tensors before storing them in a data file or encrypting at least a subset of the tensors before storing them in a data file.

[0017] In some implementations, the executable instructions, when executed by the processor, further cause the computer system to receive a request for at least one data element associated with the first index, the request including an indication of the first index, and determine the first index using the indication.

[0018] In some implementations, when accessing at least one data element associated with the first index, the executable instructions, when executed by the processor, cause the computer system to skip from a second memory location associated with the header to the first memory location when determining a first memory location of the data row corresponding to the first index.

[0019] According to yet another aspect, a non-transitory computer-readable medium can store computer code instructions that, when executed by a processor, cause the processor to obtain ego navigation data including a plurality of data elements associated with a plurality of indexes, each index associated with one or more data elements, the ego navigation data determined based on data captured by at least one sensor of the ego, generate a data file storing the plurality of data elements according to a predetermined file format including a header and a plurality of data rows, each data row corresponding to a respective index and storing one or more data elements associated with the respective index, the header including, for each data row, an association between the respective index and a corresponding data offset indicating a memory location of the data row, determine a first memory location of the data row corresponding to the first index using an indication of the first index and the header, access at least one data element associated with the first index using the first memory location, and use the at least one data element associated with the first index to train a machine learning (ML) model. [Brief explanation of the drawings]

[0020] Non-limiting embodiments of the present disclosure are described by way of example with reference to the accompanying drawings, which are schematic and not intended to be drawn to scale, and unless indicated as representing background art, the figures represent aspects of the present disclosure.

[0021] [Figure 1A]1 illustrates components of an AI-enabled visual data analysis system, according to one embodiment.

[0022] [Figure 1B] 1 illustrates various sensors associated with an ego according to one embodiment.

[0023] [Figure 1C] 1 illustrates components of a vehicle, according to one embodiment.

[0024] [Figure 2] 1 illustrates a block diagram of a video training system according to one embodiment.

[0025] [Figure 3] 3 shows a flowchart of a method for efficient storage and access of data according to one embodiment.

[0026] [Figure 4] FIG. 1 shows a diagram illustrating an example of the arrangement of data elements in a data file according to a file format with multiple rows, according to one embodiment.

[0027] [Figure 5] 5 illustrates an exemplary layout diagram of the memory layout of the file described in connection with FIG. 4 according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0028] Reference will now be made to the exemplary embodiments illustrated in the drawings, and specific language will be used herein to describe the embodiments. It will nevertheless be understood that no limitation of the scope of the claims or this disclosure is intended thereby. Alterations and further modifications of the features of the invention shown herein, and further applications of the principles of the subject matter shown herein that will occur to those skilled in the art in possession of this disclosure, are to be considered within the scope of the subject matter disclosed herein. Other embodiments may be used and / or other changes may be made without departing from the spirit or scope of the present disclosure. The exemplary embodiments described in the detailed description are not meant to limit the presented subject matter.

[0029] Training a relatively complex ML or AI model, such as an ML model for detecting or predicting ego surroundings, typically takes a significant amount of time and involves the use of a large amount of training data. Training typically consumes a significant amount of processing power, bandwidth, and memory capacity. In particular, training involves a relatively large number of training iterations. In each iteration, a significant amount of data may be provided as input to the training module. Thus, training typically involves a large amount of system input / output operations, which adds to the complexity of training the ML model.

[0030] In this disclosure, a new file format is introduced to improve the efficiency of data storage and access. Although described herein primarily in the context of training ML models, the file format can be used for other applications. According to the file format, data can be stored in multiple indexed data rows, or more generally, multiple indexed data segments. A file header can store an index of a data row in relation to the corresponding data offset of the data row (or data segment). The header enables random access of any data row or data segment of a data file generated according to the file format, without having to read the data file completely to the data row of interest. In other words, when reading a data file, a processor or computing system can parse the header to determine the data offset of the data row (or data segment) of interest and accordingly move or skip to the memory location corresponding to the data offset to read the data row or data segment. As described in further detail below, the file format and systems, devices, and methods described herein enable efficient storage of data, relatively fast access of data, and reduced processing power usage.

[0031] FIG. 1A is a non-limiting example of system components capable of implementing the methods and systems discussed herein. For example, an analytics server may train an AI model and use the trained AI model to generate occupancy datasets and / or maps for one or more egos. FIG. 1A illustrates components of an AI-enabled visual data analytics system 100. System 100 may include analytics server 110a, system database 110b, administrator computing device 120, egos 140a-b (collectively ego 140), ego computing devices 141a-c (collectively ego computing devices 141), and server 160. System 100 is not limited to the components described herein and may include additional or other components not shown for the sake of brevity, which components should be considered within the scope of the embodiments described herein.

[0032] The above components may be connected via a network 130. Examples of network 130 may include, but are not limited to, a private or public LAN, a WLAN, a MAN, a WAN, and the Internet. Network 130 may include wired and / or wireless communications according to one or more standards and / or over one or more transport media.

[0033] Communications over network 130 may occur according to various communication protocols, such as Transmission Control Protocol / Internet Protocol (TCP / IP), User Datagram Protocol (UDP), IEEE communications protocols, etc. In one example, network 130 may include wireless communications according to the Bluetooth® set of specifications or another standard or proprietary wireless communications protocol. In another example, network 130 may also include communications over cellular networks, including, for example, Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), or Enhanced Data for Global Evolution (EDGE) networks.

[0034] System 100 illustrates an example of a system architecture and components that may be used to train and execute one or more AI models, such as AI model 110c. Specifically, as depicted in FIG. 1A and described herein, analytics server 110a may train AI model 110c using data retrieved from ego 140 (e.g., by using data streams 172 and 174) using methods described herein. Once AI model 110c is trained, each ego 140 may access the trained AI model 110c and execute it. For example, vehicle 141a with ego computing device 140a may send its camera feed to trained AI model 110c and determine the occupancy status of its surroundings (e.g., data stream 174). Furthermore, data captured and / or predicted by AI model 110c for ego 140 (during inference) may be used to improve AI model 110c. Thus, system 100 describes a continuous loop that can periodically improve the accuracy of AI model 110c. Furthermore, the system 100 loops in such a way that the data received by the ego 140 can be used in the learning phase as well as the inference phase.

[0035] Analytics server 110a may be configured to collect, process, and analyze navigation data (e.g., images captured while navigating) and various sensor data collected from ego 140. The collected data may then be processed and prepared into a training dataset. The training dataset may then be used to train one or more AI models, such as AI model 110c. Analytics server 110a may also be configured to collect visual data from ego 140. Using AI model 110c (trained using the methods and systems discussed herein), analytics server 110a may generate a dataset and / or an occupancy map for ego 140. Analytics server 110a may display the occupancy map on ego 140 and / or transmit the occupancy map / dataset to ego computing device 141, administrator computing device 120, and / or server 160.

[0036] Although FIG. 1A shows AI model 110c as a component of system database 110b, AI model 110c may be stored in a different or separate component, such as cloud storage or any other data repository accessible to analytics server 110a.

[0037] The analytics server 110a may also be configured to display an electronic platform showing various training attributes for training the AI ​​model 110c. The electronic platform may be displayed on the administrator computing device 120 to allow an analyst to monitor the training of the AI ​​model 110c. An example of an electronic platform generated and hosted by the analytics server 110a may be a web-based application or website configured to display the training dataset collected from the ego 140 and / or the training status / metrics of the AI ​​model 110c.

[0038] Analysis server 110a may be any computing device equipped with a processor and non-transitory machine-readable storage capable of performing the various tasks and processes described herein. Non-limiting examples of such computing devices may include workstation computers, laptop computers, server computers, etc. Although system 100 includes a single analysis server 110a, system 100 may include any number of computing devices operating in a distributed computing environment, such as a cloud environment.

[0039] Ego 140 may represent various electronic data sources that transmit data associated with a previous or current navigation session to analytics server 110a. Ego 140 may be any device configured for navigation, such as vehicle 140a and / or truck 140c. Ego 140 is not limited to being a vehicle and may include robotic devices as well. For example, ego 140 may include robot 140b, which may represent a general-purpose, bipedal, autonomous humanoid robot capable of navigating various terrains. Robot 140b may be provided with software that enables balance, navigation, perception, or interaction with the physical world. Robot 140b may also include various cameras configured to transmit visual data to analytics server 110a.

[0040] Although referred to herein as “ego,” ego 140 may or may not be an autonomous device configured for automatic navigation. For example, in some embodiments, ego 140 may be controlled by a human operator or by a remote processor. ego 140 may include various sensors, such as those depicted in FIG. 1B . The sensors may be configured to collect data as ego 140 navigates various terrains (e.g., roads). Analytics server 110a may collect data provided by ego 140. For example, analytics server 110a may obtain navigation session and / or road / terrain data (e.g., images of ego 140 navigating roads) from various sensors, such that the collected data is ultimately used by AI model 110c for training purposes.

[0041] As used herein, a navigation session corresponds to a trip in which ego 140 travels a route, regardless of whether the trip was autonomous or controlled by a human. In some embodiments, the navigation session may be for data collection and model training purposes. However, in some other embodiments, ego 140 may refer to a vehicle purchased by a consumer, and the purpose of the trip may be classified as daily use. A navigation session may begin when ego 140 travels more than a threshold distance (e.g., 0.1 miles, 100 feet) from a non-moving location or exceeds a threshold speed (e.g., greater than 0 mph, greater than 1 mph, greater than 5 mph). A navigation session may end when ego 140 is returned to a non-moving location and / or turned off (e.g., when the driver exits the vehicle).

[0042] Ego 140 may correspond to a group of egos monitored by analytics server 110a to train AI model 110c. For example, drivers of vehicles 140a may authorize analytics server 110a to monitor data associated with their respective vehicles. As a result, analytics server 110a may collect sensor / camera data using various methods discussed herein and generate a training dataset for training AI model 110c accordingly. Analytics server 110a may then apply the trained AI model 110c to analyze data associated with ego 140 and predict an occupancy map for ego 140. Furthermore, additional / ongoing data associated with ego 140 may be processed and added to the training dataset, allowing analytics server 110a to recalibrate AI model 110c accordingly. Thus, system 100 performs a loop in which navigation data received from ego 140 can be used to train AI model 110c. Ego 140 may include a processor that executes the trained AI model 110c for navigation purposes. During navigation, ego 140 can collect additional data about their navigation session and can use the additional data to calibrate AI model 110c. That is, ego 140 corresponds to an ego that can be used to train, run / use, and recalibrate AI model 110c. In a non-limiting example, ego 140 corresponds to a vehicle purchased by a customer that can navigate autonomously using AI model 110c and simultaneously improve AI model 110c.

[0043] Egos 140 may be equipped with various technologies that enable them to gather data from their surroundings and (potentially) navigate autonomously. For example, ego 140 may be equipped with an inference chip for running self-driving software.

[0044] Various sensors for each ego 140 may monitor and transmit collected data associated with different navigation sessions to analytics server 110a. FIGS. 1B-1C show block diagrams of sensors incorporated within ego 140, according to one embodiment. The number and location of each sensor discussed with respect to FIGS. 1B-1C may depend on the type of ego discussed in FIG. 1A. For example, robot 140b may include different sensors than vehicle 140a or truck 140c. For example, robot 140b may not include airbag activation sensor 170q. Additionally, the sensors for vehicle 140a and truck 140c may be in different locations than those shown in FIG. 1C.

[0045] As discussed herein, various sensors incorporated within each ego 140 may be configured to measure various data associated with each navigation session. Analytics server 110a may periodically collect the data monitored and collected by these sensors, which is processed according to methods described herein and used to train and / or execute AI model 110c to generate an occupancy map.

[0046] Ego 140 may include user interface 170a. User interface 170a may refer to the user interface of an ego computing device (e.g., ego computing device 141 of FIG. 1A). User interface 170a may be implemented as a display screen integrated with or coupled to a vehicle's interior, a head-up display, a touchscreen, etc. User interface 170a may include input devices such as a touchscreen, knobs, buttons, a keyboard, a mouse, a gesture sensor, a steering wheel, etc. In various embodiments, user interface 170a may be adapted to provide user input (e.g., as types of signals and / or sensor information) to other devices or sensors of ego 140 (e.g., the sensors shown in FIG. 1B), such as controller 170c.

[0047] User interface 170a may also be implemented with one or more logic devices that may be adapted to execute instructions, such as software instructions, that implement any of the various processes and / or methods described herein. For example, user interface 170a may be adapted to form a communication link, send and / or receive communications (e.g., sensor signals, control signals, sensor information, user input, and / or other information), or perform various other processes and / or methods. In another example, a driver may use user interface 170a to control the temperature of ego 140 or activate its features (e.g., autonomous driving or steering system 170o). Accordingly, user interface 170a may monitor and collect driving session data in conjunction with other sensors described herein. User interface 170a may also be configured to display various data generated / predicted by analytics server 110a and / or AI model 110c.

[0048] Orientation sensor 170b may be implemented as one or more of a compass, float, accelerometer, and / or other digital or analog device capable of measuring the orientation of ego 140 (e.g., the magnitude and direction of roll, pitch, and / or yaw relative to one or more reference orientations, such as gravity and / or magnetic north). Orientation sensor 170b may be adapted to provide orientation measurements at ego 140. In other embodiments, orientation sensor 170b may be adapted to provide roll, pitch, and / or yaw rate of ego 140 using a time series of orientation measurements. Orientation sensor 170b may be positioned and / or adapted to provide orientation measurements relative to a particular coordinate frame of ego 140.

[0049] Controller 170c may be implemented as any suitable logic device (e.g., a processing device, microcontroller, processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), memory storage device, memory reader, or other device or combination of devices) that can be adapted to execute, store, and / or receive appropriate instructions, such as software instructions that implement control loops for controlling various operations of ego 140. Such software instructions may also process sensor signals, determine sensor information, provide user feedback (e.g., via user interface 170a), interrogate devices regarding operating parameters, select operating parameters for devices, or implement methods for performing any of the various operations described herein.

[0050] Communications module 170e may be implemented as any wired and / or wireless interface configured to communicate sensor data, configuration data, parameters, and / or other data and / or signals to any feature shown in FIG. 1A (e.g., analytics server 110a). As described herein, in some embodiments, communications module 170e may be implemented in a distributed manner, such that portions of communications module 170e are implemented within one or more elements and sensors shown in FIG. 1B. In some embodiments, communications module 170e may delay communication of sensor data. For example, when ego 140 does not have network connectivity, communications module 170e may store sensor data in temporary data storage and transmit the sensor data when ego 140 is identified as having adequate network connectivity.

[0051] Speed ​​sensor 170d may be implemented as an electronic pitot tube, a metering gear or wheel, a water speed sensor, a wind speed sensor, a wind speed sensor (e.g., direction and magnitude), and / or other device capable of measuring or determining the linear velocity of ego 140 (e.g., within the surrounding medium and / or aligned with the longitudinal axis of ego 140) and providing such measurement as a sensor signal that can be communicated to various devices.

[0052] Gyroscope / accelerometer 170f may be implemented as one or more of an electronic sextant, a semiconductor device, an integrated chip, an accelerometer sensor, or other system or device capable of measuring angular velocity / acceleration and / or linear acceleration (e.g., direction and magnitude) of ego 140 and providing such measurements as sensor signals that can be communicated to other devices, such as analytics server 110a. Gyroscope / accelerometer 170f may be positioned and / or adapted to make such measurements relative to a particular coordinate frame of ego 140. In various embodiments, gyroscope / accelerometer 170f may be mounted in a common housing and / or module with other elements shown in FIG. 1B to ensure a common frame of reference or known transformations between frames of reference.

[0053] Global Navigation Satellite System (GNSS) 170h may be implemented as a global positioning satellite receiver and / or another device capable of determining the absolute and / or relative position of ego 140 based on, for example, radio signals received from space-born and / or terrestrial sources and providing such measurements as sensor signals that can be communicated to various devices. In some embodiments, GNSS 170h may be adapted to determine the velocity, speed, and / or yaw rate of ego 140 (e.g., using a time series of position measurements), such as the absolute velocity and / or yaw component of angular velocity of ego 140.

[0054] Temperature sensor 170i may be implemented as a thermistor, an electrical sensor, an electrical thermometer, and / or other device capable of measuring a temperature associated with ego 140 and providing such measurement as a sensor signal. Temperature sensor 170i may be configured to measure an environmental temperature associated with ego 140, such as a cockpit or dash temperature, which may be used to estimate the temperature of one or more elements of ego 140.

[0055] Humidity sensor 170j may be implemented as a relative humidity sensor, an electrical sensor, an electrical relative humidity sensor, and / or another device capable of measuring the relative humidity associated with ego 140 and providing such measurement as a sensor signal.

[0056] Steering sensor 170g may be adapted to physically adjust the orientation of ego 140 according to one or more control signals provided by a logic device, such as controller 170c, and / or user input. Steering sensor 170g may include one or more actuators and control surfaces of ego 140 (e.g., rudders or other types of steering or trim mechanisms) and may be adapted to physically adjust the control surfaces to various positive and / or negative steering angles / positions. Steering sensor 170g may also be adapted to sense the current steering angles / positions of such steering mechanisms and provide such measurements.

[0057] Propulsion system 170k may be implemented as a propeller, turbine, or other thrust-based propulsion system, a mechanical wheeled and / or tracked propulsion system, a wind / sail-based propulsion system, and / or other type of propulsion system that may be used to provide motive power to ego 140. Propulsion system 170k may also monitor the direction of motive power and / or thrust of ego 140 relative to a coordinate frame of reference of ego 140. In some embodiments, propulsion system 170k may be coupled to and / or integrated with steering sensor 170g.

[0058] Occupant restraint sensor 170l may monitor the seat belt detection and lock / unlock assembly, as well as other occupant restraint subsystems. Occupant restraint sensor 170l may include various environmental and / or status sensors, actuators, and / or other devices that facilitate operation of safety mechanisms associated with operation of ego 140. For example, occupant restraint sensor 170l may be configured to receive movement and / or status data from other sensors shown in FIG. 1B. Occupant restraint sensor 170l may determine whether a safety mechanism (e.g., a seat belt) is engaged.

[0059] Camera 170m may refer to one or more cameras integrated into ego 140, as depicted in FIG. 1C, or may include multiple cameras integrated into (or retrofitted to) ego 140. Camera 170m may be an inward-facing or outward-facing camera of ego 140. For example, as depicted in FIG. 1C, ego 140 may include one or more inward-facing cameras that can monitor and collect video of occupants of ego 140. Ego 140 may include eight outward-facing cameras. For example, ego 140 may include front camera 170m-1, forward-looking side camera 170m-2, forward-looking side camera 170m-3, rearward-looking side camera 170m-4 on each front fender, camera 170m-5 on each side (e.g., integrated into the B-pillar), and rear camera 170m-6.

[0060] 1B, radar 170n and ultrasonic sensor 170p may be configured to monitor the distance of ego 140 to other objects, such as other vehicles or immovable objects (e.g., trees or garage doors). Ego 140 may also include an automatic driving or steering system 170o configured to autonomously navigate ego 140 using data collected via various sensors (e.g., radar 170n, speed sensor 170d, and / or ultrasonic sensor 170p).

[0061] Thus, automated driving or steering system 170o may analyze various data collected by one or more sensors described herein to identify driving data. For example, automated driving or steering system 170o may calculate the risk of a forward collision based on ego 140's speed and its distance to another vehicle on the road. Autonomous driving or steering system 170o may also determine whether the driver is touching the steering wheel. Autonomous driving or steering system 170o may transmit the analyzed data to various features discussed herein, such as an analytics server.

[0062] Airbag deployment sensor 170q may predict or detect a crash and cause one or more airbags to deploy or inflate. Airbag deployment sensor 170q may transmit data regarding the deployment of the airbags, including data associated with the event that caused the deployment.

[0063] 1A , administrator computing device 120 may represent a computing device operated by a system administrator. Administrator computing device 120 may be configured to display data retrieved or generated by analytics server 110a (e.g., various analytics metrics and risk scores), allowing the system administrator to monitor various models utilized by analytics server 110a, review feedback, and / or facilitate training of AI models 110c maintained by analytics server 110a.

[0064] Ego 140 may be any device configured to navigate various routes, such as vehicle 140a or robot 140b. As discussed with respect to FIGS. 1B-1C, ego 140 may include various telemetry sensors. Ego 140 may also include ego computing device 141. Specifically, each ego may have its own ego computing device 141. For example, truck 140c may have ego computing device 141c. For simplicity, ego computing devices are collectively referred to as ego computing device 141. Ego computing device 141 may control the presentation of content on ego 140's infotainment system, process commands related to the infotainment system, aggregate sensor data, manage communication of data to electronic data sources, receive updates, and / or send messages. In one configuration, ego computing device 141 communicates with an electronic control unit. In another configuration, ego computing device 141 is an electronic control unit. Ego computing device 141 may include a processor and non-transitory machine-readable storage media capable of performing the various tasks and processes described herein. For example, AI model 110c described herein may be stored and executed (or directly accessed) by ego computing device 141. Non-limiting examples of ego computing device 141 may include a vehicle multimedia and / or display system.

[0065] In one example of a highly efficient method for processing data, such as training data for training and / or validating AI model 110c and / or other ML models, the data can be stored and / or accessed using a file format having a header and multiple indexed data segments. Given a dataset including multiple data elements associated with multiple indices, e.g., timestamps or time values, where each index is associated with one or more corresponding data elements, the file format allows the dataset to be stored in multiple data rows or data segments in a data file. Each data row or data segment can correspond to a corresponding index or timestamp and can store the data elements associated with the corresponding index or timestamp. When storing the dataset in a data file, the computer system can store the index of the data row or data segment in the header in association with a data offset of the data row or data segment. Specifically, for each data row or data segment, the computer system can store a corresponding index (or timestamp) in the header in association with the corresponding data offset. For data fields or data variables that are the same in all indexes, the computer system may store a single instance of the data field or data variable in the data file, for example, before the data row or data segment, to avoid unnecessary redundancy and reduce the amount of data written or stored in the data file.

[0066] A file format may be associated with or have system read and write operations specific to the file format. A write operation may be configured to write or store data in a data file of the file format according to a predetermined arrangement including a header and data rows or data segments. Specifically, a write operation may be capable of writing a data element associated with a given index within a data row or data segment corresponding to a data index. A write operation may also be capable of generating or writing a header, for example, at the beginning of a data file, to include an index of a data row or data segment in relation to a corresponding data offset of the data row or data segment.

[0067] A read operation may first read a header to identify data offsets of data rows of interest, and then move to a memory location corresponding to the data offsets to read or parse the data rows or data segments of interest. In other words, a read operation may receive as input one or more indexes (or representations thereof) of one or more rows, and use the input one or more indexes to determine the data offsets of the one or more rows from the header. A read operation may be configured to move directly to a memory location corresponding to the data offsets to read or parse the one or more rows indicated by the input one or more indexes.

[0068] The file format and systems, devices, and methods described herein reduce the size of data files in the new file format by approximately 11% compared to other file formats. Also, the number of system input / output operations used per unit time, e.g., IOPS, can be reduced by a factor of four. This means that memory usage is reduced by approximately 11% and processing time is reduced by at least a factor of four.

[0069] 2 shows a block diagram of a computing environment 200 for training an ML model, according to one embodiment. The computing environment 200 may include a training system 202 for training the ML model and a data storage system 204 for storing training data and / or validation data. The training system 202 may include multiple training nodes (or processing nodes) 206. Each training node 206 may include a respective data loader (or data loading device) 208 and a respective graphical processing unit (GPU) 210. Each GPU 210 may include memory, such as a cache memory 212, processing circuitry 214, and one or more video decoders 216.

[0070] Data storage system 204 can include or be a distributed storage system. For example, data storage system 204 can have an infrastructure that can split data across multiple physical servers, such as a supercomputer. Data storage system 204 can include one or more storage clusters of storage units, along with mechanisms and infrastructure for parallel and accelerated access of data from multiple nodes or storage units of the storage cluster. For example, data storage system 204 can include sufficient data links and bandwidth to deliver data to training nodes 206 in parallel or simultaneously.

[0071] The data storage system 204 may include sufficient memory capacity to store millions or even billions of video frames, e.g., in compressed format. For example, the data storage system 204 may have memory capacity to store multiple petabytes of data, e.g., 10, 20, or 30 petabytes. The data storage system 204 may allow thousands of video sequences to move in and out of the data storage system 204 at any given time instance. The relatively large size and bandwidth of the data storage system 204 enables parallel training of one or more ML models, as discussed below.

[0072] Training system 202 may be implemented as one or more physical servers, such as server 110a. For example, training system 202 may be implemented as one or more supercomputers. Each supercomputer may include thousands of processing or training nodes 206. Training nodes 206 may be configured or designed to support parallel training of one or more ML models, such as AI model 110c. Each training node 206 may be communicatively coupled to data storage system 204 to access training data and / or validation data stored in data storage system 204.

[0073] Each training node 206 may include a respective data loader 208 and a respective GPU 210 that are communicatively coupled to each other. The data loader 208 may be (or may include) a processor or central processing unit (CPU) for handling data requests or data transfers between a corresponding GPU 210, e.g., a GPU 210 within the same training node 206, and the data storage system 204. For example, the GPU 210 may request one or more video sequences captured by one or more of the cameras 170m described in connection with FIG. 1C . For example, the front or forward-looking cameras 170m-1, 170m-2, and 170m-3, the rear-looking side camera 170m-4, the side camera 170m-5, and the rear camera 170m-6 may simultaneously capture video sequences and send the video sequences to the data storage system 204 for storage in the data storage system 204. In some implementations, data storage system 204 can store video sequences captured simultaneously by multiple cameras, such as camera 170m of ego 140, as bundles or combinations of video sequences that can be distributed together to training node 206. For example, data storage system 204 can maintain additional data indicating which video sequences were captured simultaneously by camera 170m or represent the same scene from different camera angles. Data storage system 204 may, for example, maintain data indicating, for each stored video sequence, an ego identifier, a camera identifier, and a time instance associated with the video sequence.

[0074] The training node 206 can simultaneously train one or more ML models, e.g., in parallel, using video data captured by the ego 140's cameras 170m and stored in the data storage system 204. In some implementations, the video data can be captured by multiple ego 140's cameras 170m. In the training node 206, a corresponding data loader 208 can request video data for one or more video sequences captured simultaneously during a time interval by one or more ego 140's cameras 170m from the data storage system 204 and send the received video data to a corresponding GPU 210 for use in performing a training step (or validation step) when training the ML model. The video data can be in a compressed format. For example, the video sequences can be encoded by an encoder built into or implemented in the ego 140. Each data loader 208 can have sufficient processing power and bandwidth to deliver video data to a corresponding GPU 210 so as to keep the GPU 210 busy. In other words, GPU 210 may be configured or designed, for example, in terms of processing power and bandwidth, to request video data for a bundle of compressed video sequences from data storage system 204 and deliver the video to GPU 210 in a period of time that is equal to or less than the average time consumed by GPU 210 to process the bundle of video sequences.

[0075] Each GPU 210 may include a corresponding internal memory 212, such as a cache memory, for storing executable instructions for performing processes that may be described herein, received video data of one or more video sequences, decoded video frames, features extracted from the decoded video frames, parameters or data of a trained ML model, and / or other data used to train the ML model. The memory 212 may be large enough to store all data necessary to perform a single training step. As used herein, a training step may include receiving and decoding video data of one or more video sequences (e.g., a bundle of video sequences simultaneously captured by one or more cameras 170m of ego 140), extracting features from the decoded video data, and using the extracted features to update parameters of an ML model being trained or validated.

[0076] Each GPU 210 may include processing circuitry 214 for performing the processes or methods described herein. Processing circuitry 214 may include one or more microprocessors, multi-core processors, digital signal processors (DSPs), one or more logic circuits, or a combination thereof. Processing circuitry 214 may execute computer code instructions stored, for example, in memory 212, to perform the processes or methods described herein.

[0077] The GPU 210 may include one or more video decoders 216 for decoding video data received from the data storage system 204. The one or more video decoders 216 may include hardware video decoders integrated into the GPU 110 to accelerate video decoding. The one or more video decoders 216 may be part of the processing circuitry 214 or may include separate electronic circuitry communicatively coupled to the processing circuitry 214.

[0078] Each GPU 210 may be configured or designed to handle or perform the training steps without the use of external resources. The GPUs 210 may include sufficient memory capacity and processing power to perform the training steps. The processes for storing and accessing training data performed by a training node 206 or corresponding GPU 210 are described in further detail below in connection with FIGS. 3-56.

[0079] Referring now to FIG. 3, a flowchart diagram of a method 300 for efficient storage and access of data is shown, according to one embodiment. Briefly, the method 300 can include steps of: acquiring data including a plurality of data elements associated with a plurality of timestamps (STEP 302), where each timestamp can be associated with one or more data elements; and generating a data file storing the data according to a predetermined file format having a plurality of rows, where each row corresponds to a respective timestamp and stores one or more data elements associated with the respective timestamps (STEP 304), the data file including a header storing, for each row, an association between the respective timestamps and corresponding data offsets indicating the memory location of the row. The method 300 can also include steps of: determining a first memory location of the row corresponding to the first timestamp using an indication of the first timestamp and the header of the data file (STEP 306); and accessing at least one data element associated with the first timestamp using the first memory location (STEP 308). Method 300 may optionally include using at least one data element associated with the first timestamp to train a training model (step 310).

[0080] Method 300 may be implemented, performed, or executed entirely by either the CPU or loader 208, either the GPU 210, and / or other computing device (or computer system) of computer environment 200 or system 100. For example, GPU 210 may execute method 300 after decoding a selected (or indicated) image frame and extracting corresponding image features. GPU 210 may store the extracted features in a data file along with other training data in accordance with method 300. Method 300, or any steps thereof, may be implemented as executable instructions that may be stored in memory and executed by one or more processors.

[0081] Method 300 may include a step (STEP 302) in which a computer system including a memory and one or more processors, such as GPU 210, acquires data including a plurality of data elements associated with a plurality of timestamps, where each timestamp may be associated with one or more data elements. In some implementations, the data may include training data for training or validating one or more machine learning (ML) models. For example, the data may include data for training and / or validating AI model 110c or an occupancy network. For example, the data may include ground truth data, features extracted from image frames decoded by video decoder 216, interference output, a map of the geographic location of ego 140, a speed or speed value of ego 140, attributes of ego 140 (e.g., make and model, fuel level, battery charge state, etc.), a maximum allowed speed, sensor data generated by one or more sensors of ego 140, other types of data, and / or combinations thereof. Obtaining data may include receiving the data or a portion thereof (e.g., from data storage system 204), generating the data or a portion thereof, and / or retrieving the data or a portion thereof from a memory, e.g., memory 212.

[0082] In some implementations, the data may include or be ego navigation data determined based on data captured by at least one sensor of the ego, such as the sensor described in connection with FIG. 1B . The data may include multiple data elements. As used herein, a data element may be, among other things, a parameter value, a tensor, or a text string. For example, the GPU 210 may group features extracted from a single image frame into a single tensor. In some implementations, the GPU 210 may group features extracted from multiple image frames captured by the camera 170m at substantially the same time instance (taking into account asynchrony between the camera 170m and / or corresponding encoders) into a single tensor. Also, road data such as traffic signs, traffic lights, and speed limits at a given time instance may be grouped or arranged into corresponding tensors. Some sensor data, such as the ego speed, may be processed as a single parameter value. Ego attributes, such as the vehicle make and model and engine type, among other things, may be represented as text strings or numerical values.

[0083] Multiple data elements can be associated with multiple timestamps or time values. Each timestamp (or time value) can be associated with one or more data elements. For example, GPU 210 can associate each set of features extracted from a decoded image frame with the timestamp of the image frame. Also, ego velocity values, other sensor data from ego sensors, map data, and other received training data can be associated with timestamps, for example, depending on the time instance at which each piece or set of training data was captured or recorded. For example, each velocity value can be associated with a timestamp or time value indicating the time instance at which the velocity value was recorded at ego 140. The association between timestamps and data elements can be performed at ego 140, for example, when the data elements are captured or recorded at training system 202, or a combination of both. The timestamp (or time value) can be equal to the timestamp of an image frame of a video sequence captured by camera 170m, or some other time value.

[0084] Method 300 may include a step (step 304) in which the GPU 210 or other computer system generates a data file storing data according to a predetermined file format having a plurality of rows, each row corresponding to a respective timestamp and storing one or more data elements associated with the respective timestamp, the data file including a header storing, for each row, an association between the respective timestamp and a corresponding data offset indicating the memory location of the row. The GPU 210 or some other computer system may generate the data file and store the data elements in corresponding rows of the data file. The GPU 210 or some other computer system may generate the data file and store data in the data file according to a predetermined file format including a header and one or more indexed rows. The indexing or index of the rows along with the corresponding memory location may be specified in the header.

[0085] When generating the data file, the GPU 210 or other computer system may determine the total number of rows in the data file based on, for example, the total number of timestamps (or time values) and each row associated with a corresponding timestamp (or time value). The GPU 210 or other computer system may arrange the rows according to ascending order of the corresponding timestamps (or time values). The GPU 210 or other computer system may determine or identify data elements to be stored in each row based on the association between the data elements and the timestamps and the association between the rows and the timestamps. In particular, the GPU 210 or other computer system may determine to store data elements associated with a given timestamp in rows corresponding to the same timestamp. When determining the arrangement of data elements in the data file, the GPU 210 or other computer system may determine or create multiple columns for the data file. Each column may correspond to or represent a separate data field, data category, or data type. For example, each column in the data file may correspond to a data parameter or a tensor type.

[0086] Referring now to FIG. 4 , an exemplary arrangement 400 of data elements in a data file according to a file format with multiple rows is shown, according to an exemplary embodiment. The GPU 210 or other computer system may configure the data file to include n rows, e.g., rows 402-1 through 402-n, where n is an integer. Rows 402-1 through 402-n are also referred to herein, individually or in combination, as rows 402. Each row 402 corresponds to a distinct timestamp in the timestamps 404. The GPU 210 or other computer system may configure the data file to include m columns, e.g., columns 406-1 through 406-m, where m is an integer. Columns 406-1 through 406-m are also referred to herein, individually or in combination, as columns 406. The GPU 210 or other computer system may assign a corresponding data field or data element type or category to each column 406. For example, column 406-1 may store ego velocity values ​​at different timestamps or time values. Column 406-2 can store a first type of tensor, referred to herein as "tensor A," associated with various timestamps or time values. For example, a "tensor A" type tensor can carry road or traffic data at various timestamps or time values. In particular, a tensor to be stored in a cell at the intersection of row 402-1 and column 406-2 can carry road or traffic data corresponding to timestamp 1337.37. Column 406-3 can store a second type of tensor, referred to herein as "tensor B," associated with various timestamps or time values. For example, a "tensor B" type tensor can carry extracted features associated with various timestamps or time values.

[0087] The GPU 210 or other computer system can index the rows 402 using the timestamp 404. In other words, the timestamp 404 does not have to be stored in a separate column 406. Instead, the GPU 210 or other computer system can store the timestamp 404 in the header in association with a data offset of the row that indicates the memory location of the row. For each row, the GPU 210 or other computer system can store a corresponding timestamp and a corresponding data offset in the header. For example, the GPU 210 or other computer system can store the timestamps and data offsets according to the order of the corresponding rows. The GPU 210 or other computer system can then store, in each row of the data file, the data element associated with the same timestamp that corresponds to that row.

[0088] For each row 402, the corresponding data offset may be expressed in data units, such as bytes. The data offset of a given row 402 may represent the distance or separation in data units between the start of the row in memory and some reference memory location. The reference memory location may be a storage location representing the start of a data file. In other words, the data offset of a row 402 may be considered the amount of data (in data units) that precedes the row 402 in the data file.

[0089] In some implementations, the data may include at least one data element associated with all of the multiple timestamps. In other words, at least one data element may be the same for all of the multiple timestamps. For example, the vehicle make and model of ego 140 may not change over time. Also, a map of ego 140's geographic location may not change over some time interval spanning the multiple timestamps, depending, for example, on ego 140's speed. For such data elements, it may not be efficient to repeatedly store the same data across all rows 402 in one or more columns.

[0090] In some implementations, when storing data in rows, the GPU 210 or other computer system may store a first data element having a first size preceding a second data element having a second size greater than the first size. For example, the GPU 210 or other computer system may sort the columns 406 according to ascending order of corresponding data size. For example, the GPU 210 or other computer system may select the data field (or data element type) with the smallest size to be stored in the first column 406-1, select the data field with the next smallest size to be stored in column 406-2, and so on, up to column 406 n, where the data field with the largest size is stored. Ordering the columns or data elements within each data row according to order of increasing data size further reduces the amount of data to be read or parsed if not all data elements in a row are retrieved.

[0091] In some implementations, GPU 210 or other computer systems may apply a dimensional transposition to one or more tensors before storing them in a data file. In other words, GPU 210 or other computer systems may store a tensor, such as tensor A or tensor B, in a transposed form. In some implementations, GPU 210 or other computer systems may encrypt one or more data elements of one or more types before storing them in a data file. For example, GPU 210 or other computer systems may store one or more tensors in an encrypted form.

[0092] The GPU 210 or other computer system may determine that at least one other data element is associated with or is the same for all of the multiple timestamps. For example, the data may include an indication that data associated with a particular data field or data element type has not changed or is the same across all timestamps. For data fields or data element types that have the same data across multiple timestamps, there may be only a single corresponding data element in the data that indicates that the same data element applies across all timestamps. The GPU 210 or other computer system may parse the data to determine which data fields or data element types have static data across multiple timestamps.

[0093] The GPU 210 or other computer system stores at least one data element that is associated with or is the same for all of the multiple timestamps before multiple rows in the data file. In particular, the GPU 210 or other computer system may store static data elements once in the data file, for example, before any of the rows 402, to avoid unnecessary redundancy and efficiently use memory resources. The GPU 210 or other computer system may store invariant data elements within the header, between the header and the rows 402, or before the header.

[0094] Referring now to FIG. 5, an exemplary layout 500 of the memory layout of the file described in connection with FIG. 4 is shown, according to an exemplary embodiment. As stored in memory, data or a data file can be viewed as multiple contiguous data segments. A first data segment 502 can represent a header. A second data segment 504, referred to herein as a column, can represent data elements with static or identical values ​​across multiple timestamps 402. The second data segment 504 can be followed by a series of data segments 506-1 through 506-n, corresponding to rows 402-1 through 402-n, respectively. For example, data segment 506-1 can store a velocity value 508-1, an instance of tensor A 510-1, and an instance of tensor B 512-1 associated with row 402-1 in the data file. Data segment 506-2 can store velocity values ​​508-2, instances of tensor A 510-2 and instances of tensor B 512-2 associated with row 402-2 of the data file, and data segment 506-n can store velocity values ​​508-n, instances of tensor A 510-n and instances of tensor B 512-n associated with row 402-n of the data file.

[0095] A data segment 502 representing a header may include a series of timestamps 402 denoted as T1, T2, ..., Tn, and data offsets denoted as O1, O2, ..., On. In some implementations, the GPU 210 or other computer system may store the timestamps 402 and data offsets in order of the rows 402-1, starting with timestamp T1 and data offset O1 of row 402-1, followed by timestamp T2 and data offset O2 of row 402-2, etc., until timestamp Tn and data offset On of the last row 402-n. In some implementations, the header or data segment 502 may further include a data offset of the data segment 502, for example, at the beginning or end of the data segment 504.

[0096] In some implementations, the file formats described above in connection with FIGS. 4-5 can have corresponding extensions. In some implementations, the file format extension can be ".smol." In some implementations, other extensions can be used. Certain write and read operations that take into account the data placement and memory layout as described above in FIGS. 4-5 can be used with the file formats. A write operation can be configured to generate a data file in a data format having memory layout 500 as discussed in connection with FIG. 5. The write operation can be configured to take into account memory layout 500 and read a header to identify a data offset for a data row of interest, and then navigate directly to a memory location corresponding to the data offset to read or parse the data row of interest.

[0097] 3 , method 300 may include the GPU 210 or other computer system using an indication of the first timestamp and a header of the data file to determine a memory location of a row corresponding to the timestamp (STEP 306), and using the determined memory location to access at least one data element associated with the first timestamp (STEP 308). In some implementations, the GPU 210 or other computer system may receive a request (e.g., an application programming interface) for at least one data element. The request or call may include an indication of the first timestamp. The indication may include a time value. The GPU 210 or other computer system may use the indication to determine the first timestamp, for example, as the timestamp closest to the indication or time value in the request.

[0098] The GPU 210 or other computer system may parse the header of the data file to determine a data offset corresponding to a first timestamp. The data offset corresponding to the first timestamp may represent a memory location of row 402 containing at least one data element, e.g., relative to the start of the data file. In response to determining the data offset, the GPU 210 or other computer system may skip to the memory location indicated by the data offset. For example, if the data offset indicates the memory location of data segment 506n corresponding to row 402n, the GPU 210 or other computer system may skip directly to the memory location indicated by the data offset and read data segment 506n corresponding to row 402n without reading or parsing data segments 506-1 through 506n-1. In reading data segment 506n, the GPU 210 or other computer system may determine the requested at least one data element.

[0099] In some implementations, when multiple data elements from multiple rows 402 are to be read, the GPU 210 or other computer system can parse the header or portions thereof until the data offsets for all rows of interest have been determined. It should be noted that the GPU 210 or other computer system need not parse the entire header and can stop once all relevant or necessary data offsets have been determined. Once the data offsets have been determined, the GPU 210 or other computer system can navigate to the corresponding memory locations and begin sequentially reading the corresponding data segments representing the corresponding rows. For example, when the GPU 210 or other computer system is to read instances of tensor A in rows 402-1 and 402-n-1, the GPU 210 or other computer system can parse the header to determine data offsets O1 and On-1, which correspond to timestamps T1 and Tn-1, respectively. The GPU 210 or other computer system can navigate to the memory location corresponding to data offset O1 to read data segment 506-1, and then navigate to the memory location corresponding to data offset On-1 to read data segment 506-(n-1).

[0100] In some implementations, a data file may be a read-only or non-editable data file. Preventing editing of a data file maintains the layout structure depicted in FIG. 5. Generally, when a data file is edited, data is added to the end of the data file. As a result, the added data is not indexed in the header. For example, if a data file is edited to add one or more rows, the header does not include the timestamp and data offset of the added row, which causes confusion regarding reading data from the data file. In particular, a read operation (as described above) to read a data element from any of the added rows will fail because such data rows are not reflected in the header.

[0101] Storing the timestamp and data offset of each row in the header allows for fast random access of any of the rows 402 of the data file. Specifically, the amount of data that must be parsed when reading one or more data elements from the data file is significantly reduced, thereby reducing the execution time of the read operation. Furthermore, the number of system input / output operations is reduced because multiple data elements in different locations within the data file can be read with a single read operation. Also, when creating a data file, the number of write operations is reduced by writing data elements that are the same with respect to all timestamps to the data file only once.

[0102] At step 310, method 300 may include GPU 210 or other computer system training an ML model using data stored and retrieved according to the methods and systems discussed herein. In particular, GPU 210 or other computer system may use at least one data element associated with a first timestamp to train an ML model, such as an occupancy network. GPU 210 or other computer system may perform one or more training steps of the ML model using data stored in a data file.

[0103] Although method 300 has been described above with reference to a predetermined file format, the same steps can be applied to store data in memory and read data from memory. In other words, data can be written to or stored in a memory area according to memory layout 500 of FIG. 5 and read by considering the same memory layout without a predetermined file format. Data can be stored according to multiple indexed data segments, for example, rather than necessarily data rows. Also, while method 300 is described with reference to timestamps, any type of index can be used to index data segments or data rows. Finally, the embodiments described herein can be used in other applications and should not be limited or restricted to training or validating ML models.

[0104] The various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure or the claims.

[0105] Computer software-implemented embodiments may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0106] The actual software code or specialized control hardware used to implement these systems and methods is not a limitation of the claimed features or this disclosure. Accordingly, the operation and behavior of the systems and methods are described without reference to specific software code, with the understanding that software and control hardware can be designed to implement the systems and methods based on the description herein.

[0107] If implemented in software, the functions may be stored as one or more instructions or code on a non-transitory, computer-readable, or processor-readable storage medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module, which may reside on a computer-readable or processor-readable storage medium. Non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transfer of a computer program from one place to another. Non-transitory processor-readable storage media may be any available medium that can be accessed by a computer. By way of example, and not limitation, such non-transitory processor-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible storage medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer or processor. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), Blu-ray discs, and floppy disks, with a "disc" typically reproducing data magnetically and a "disc" reproducing data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media. Furthermore, the operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable medium and / or computer-readable medium, which may be incorporated into a computer program product.

[0108] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the embodiments described herein and variations thereof. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

[0109] While various aspects and embodiments have been disclosed, other aspects and embodiments are contemplated. The various disclosed aspects and embodiments are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

1. acquiring, by a processor, ego navigation data including a plurality of data elements associated with a plurality of indexes, each index being associated with one or more data elements, the ego navigation data being determined based on data captured by at least one sensor of the ego; generating, by a processor, a data file storing the data according to a predetermined file format including a header and a plurality of data lines, each data line corresponding to a respective index and storing the one or more data elements associated with the respective index, the header including, for each data line, an association between the respective index and a corresponding data offset indicating a memory location of the data line; determining, by the processor, a first memory location of a data row corresponding to the first index using an indication of the first index and the header; accessing, by the processor, at least one data element associated with the first index using the first memory location; using, by the processor, at least one data element associated with the first index to train a machine learning (ML) model; A method comprising:

2. The method of claim 1 , wherein the ego navigation data is used to train an occupancy network of the ego, and the occupancy network is used to predict or sense the ego's surroundings.

3. The data further includes at least one other data element associated with all of the plurality of indexes, and the method further comprises: determining, by the processor, that the at least one other data element is associated with all of the plurality of indexes; storing, by the processor, the at least one other data element before the plurality of data rows in the data file; The method of claim 1 further comprising:

4. The method of claim 1 , wherein the data file is a read-only data file.

5. 2. The method of claim 1, wherein the data file stores the plurality of data elements according to a plurality of columns, each column representing a corresponding data field, the method further comprising the step of arranging the plurality of columns according to ascending order of data size associated with the plurality of data columns.

6. The method of claim 1 , wherein the plurality of indexes comprises a plurality of timestamps or a plurality of time values.

7. The method of claim 1 , wherein the plurality of data elements comprises a plurality of tensors, and each data row stores one or more tensors associated with the respective index corresponding to the data row.

8. transposing, by the processor, at least a subset of the tensors before storing them in the data file; or encrypting, by the processor, at least a subset of the tensors before storing them in the data file; The method of claim 7 , further comprising at least one of:

9. receiving, by the processor, a request for the at least one data element associated with the first index, the request including the indication of the first index; determining, by the processor, the first index using the indication; The method of claim 1 further comprising:

10. The step of accessing the at least one data element associated with the first index comprises: skipping, by the processor, from a second memory location associated with the header to the first memory location when determining the first memory location of the data row corresponding to the first index; The method of claim 1 , comprising:

11. In a computer system, a processor; a memory storing executable instructions that, when executed by the processor, cause the computer system to: acquiring ego-navigation data including a plurality of data elements associated with a plurality of indexes, each index being associated with one or more data elements, the ego-navigation data being determined based on data captured by at least one sensor of the ego; generating a data file storing the plurality of data elements according to a predetermined file format including a header and a plurality of data rows, each data row corresponding to a respective index and storing the one or more data elements associated with the respective index, the header including, for each row, an association between the respective index and a corresponding data offset indicating a memory location of the data row; determining a first memory location of a data row corresponding to the first index using an indication of the first index and the header; accessing at least one data element associated with the first index using the first memory location; using at least one data element associated with the first index to train a machine learning (ML) model; Memory and A computer system comprising:

12. 12. The computer system of claim 11, wherein the ego navigation data is used to train an occupancy network of the ego, and the occupancy network is used to predict or sense the ego's surroundings.

13. The data further includes at least one other data element associated with all of the plurality of indexes, and the executable instructions, when executed by the processor, further cause the computer system to: determining that the at least one other data element is associated with all of the plurality of indexes; storing said at least one other data element before said plurality of data rows in said data file; 12. The computer system of claim 11.

14. 12. The computer system of claim 11, wherein the data file is a read-only data file.

15. The executable instructions, when executed by the processor, cause the computer system to: storing the plurality of data elements according to a plurality of columns in the data file, each column representing a corresponding data field; arranging the plurality of data strings in ascending order of data size associated with the plurality of data strings; 12. The computer system of claim 11.

16. The computer system of claim 11 , wherein the plurality of indexes comprises a plurality of timestamps or time values.

17. 12. The computer system of claim 11, wherein the plurality of data elements comprises a plurality of tensors, and each data row stores one or more tensors associated with the respective index.

18. The executable instructions, when executed by the processor, further cause the computer system to: transposing at least a first subset of the tensors before storing them in the data file; or encrypting at least a second subset of the tensors prior to storing them in the data file; and causing the device to perform at least one of the following:

18. The computer system of claim 17.

19. 2. The computer system of claim 1, wherein when accessing the at least one data element associated with the first index, the executable instructions, when executed by the processor, cause the computer system to skip from a second memory location associated with the header to the first memory location when determining the first memory location of the row that corresponds to the first timestamp.

20. A non-transitory computer-readable medium storing computer code instructions that, when executed by a processor, cause the processor to: acquiring ego-navigation data including a plurality of data elements associated with a plurality of indexes, each index being associated with one or more data elements, the ego-navigation data being determined based on data captured by at least one sensor of the ego; generating a data file storing the plurality of data elements according to a predetermined file format including a header and a plurality of data rows, each data row corresponding to a respective index and storing the one or more data elements associated with the respective index, the header including, for each row, an association between the respective index and a corresponding data offset indicating a memory location of the data row; determining a first memory location of a data row corresponding to the first index using an indication of the first index and the header; accessing at least one data element associated with the first index using the first memory location; using at least one data element associated with the first index to train a machine learning (ML) model; Non-transitory computer-readable medium.

Images