File formats for efficient storage and access of data

Abstract

Systems and methods may include a computing system that: obtains data including a plurality of data elements associated with a plurality of indices, where each index is associated with one or more data elements; a data file storing a plurality of data elements is generated according to a predefined file format, the predefined file format comprising a header and a plurality of data rows, where each data row corresponds to a respective index and stores one or more data elements associated with the respective index, and for each row, the header is associated with one or more data elements associated with the respective index. The header includes an association between a respective index and a corresponding data offset indicating a memory location of the data row; determining a first memory location of a data row corresponding to the first index using the indication of the first index and the header; and accessing at least one data element associated with the first index using the first memory location.

Description

[0001] Cross - reference to related patent applications

[0002] This application claims the benefit of priority of U.S. Provisional Application No. 63 / 377,954, filed Sep. 30, 2022, and U.S. Provisional Application No. 63 / 378,012, filed Sep. 30, 2022, the entire contents of which are incorporated herein by reference for all purposes. TECHNICAL FIELD

[0003] The present disclosure generally relates to efficient techniques for storing and accessing data. Specifically, the present disclosure relates to new file formats and systems and methods for using such file formats to more efficiently store data and accelerate data access. BACKGROUND ART

[0004] Due to the rapid development of computer technology, autonomous navigation technologies for autonomous vehicles and robots (collectively referred to as self-bodies) have become ubiquitous. These advancements allow for safer and more reliable autonomous navigation of self-bodies. Self-bodies typically need to navigate in complex and dynamic environments and terrains, which can include vehicles, traffic, pedestrians, cyclists, and various other static or dynamic obstacles. Understanding the surrounding environment of the self-body is necessary for informed and capable decision-making to avoid collisions. SUMMARY OF THE INVENTION

[0005] The systems, devices, and methods described herein provide efficient and effective techniques for storing and accessing data. Specifically, the systems, devices, and methods described herein implement new file formats that more efficiently utilize memory resources when storing data and provide faster access to stored data. The new file formats can be used in applications involving relatively large amounts of data and relatively large numbers of system input / output operations, such as the training of machine learning (ML) models. In such applications, the amount of memory used to store data can be reduced, and a significant reduction in the number of system input / output operations used and the processing time of system input / output operations can be achieved. For example, training ML models or artificial intelligence (AI) models, such as occupancy networks, used to predict or sense the surrounding environment of a self-body, is extremely time-consuming. The systems, devices, and methods described herein significantly accelerate the training and / or validation of such models by reducing the amount of time used to read and / or write training data slices. In addition, the systems, devices, and methods described herein result in more efficient use of processing and memory resources.

[0006] According to at least one aspect, a method may include obtaining, by a processor, ego-navigation data including a plurality of data elements associated with a plurality of indices, each index being associated with one or more data elements, the ego-navigation data being determined based on data captured by at least one sensor of an ego; generating, by the processor, a data file storing the data according to a predefined file format, the predefined file format including a header and a plurality of data rows such that each data row corresponds to a respective index and stores one or more data elements associated with the respective index, and for each data row, the header includes an association between the respective index and a corresponding data offset indicating a memory location of the data row; determining, by the processor, a first memory location of a data row corresponding to a first index using an indication of the first index and the header; accessing, by the processor, at least one data element associated with the first index using the first memory location; and training, by the processor, a machine learning (ML) model using the at least one data element associated with the first index.

[0007] In some implementations, the ego-navigation data may be used to train an occupancy network of the ego, which is used to predict or perceive the surrounding environment of the ego. In some implementations, the plurality of indices may include a plurality of timestamps or a plurality of time values. In some implementations, the data file may be a read-only data file.

[0008] In some implementations, the data may further include at least one other data element associated with all of the plurality of indices. The method may further include: determining, by the processor, that the at least one other data element is associated with all of the plurality of indices; and storing, by the processor, the at least one other data element before the plurality of data rows in the data file.

[0009] In some implementations, the data file may store the plurality of data elements according to a plurality of columns, where each column represents a corresponding data field. The method may further include arranging the plurality of columns in an increasing order of data sizes associated with the plurality of data columns.

[0010] In some implementations, the plurality of data elements may include a plurality of tensors, and each data row may store one or more tensors associated with the respective index corresponding to the data row. The method may further include at least one of the following: transposing, by the processor, at least a subset of the tensors before storing the at least a subset of the tensors in the data file; or encrypting, by the processor, at least a subset of the tensors before storing the at least a subset of the tensors in the data file.

[0011] In some implementations, the method may further include: receiving, by the processor, a request for at least one data element associated with the first index, the request including an indication of the first index; and determining, by the processor, the first index using the indication.

[0012] In some implementations, accessing at least one data element associated with a first index can include: when determining a first memory location of a data row corresponding to the first index, the processor jumps from a second memory location associated with a header to the first memory location.

[0013] According to at least one other aspect, a computing system can include a processor and a memory storing executable instructions. The executable instructions, when executed by the processor, cause the computer system to: obtain self-navigating data including a plurality of data elements associated with a plurality of indexes, each index associated with one or more data elements, the self-navigating data being determined based on data captured by at least one sensor of the self; generate a data file storing the plurality of data elements according to a predefined file format, the predefined file format including a header and a plurality of data rows, where each data row corresponds to a respective index and stores one or more data elements associated with the respective index, and for each row, the header includes an association between the respective index and a corresponding data offset indicating the memory location of the data row; use an indication of the first index and the header to determine a first memory location of a data row corresponding to the first index; use the first memory location to access at least one data element associated with the first index; and use the at least one data element associated with the first index to train a machine learning (ML) model.

[0014] In some implementations, the self-navigating data can be used to train an occupancy network of the self, which is used to predict or sense the surrounding environment of the self. In some implementations, the plurality of indexes can include a plurality of timestamps or a plurality of time values. In some implementations, the data file can be a read-only data file.

[0015] In some implementations, the data can further include at least one other data element associated with all the plurality of indexes. The executable instructions, when executed by the processor, further cause the computer system to: determine that the at least one other data element is associated with all the plurality of indexes; and store the at least one other data element before the plurality of data rows in the data file.

[0016] In some implementations, the executable instructions, when executed by the processor, cause the computer system to: store the plurality of data elements in the data file according to a plurality of columns, each column representing a corresponding data field; and arrange the plurality of columns in an increasing order of data sizes associated with the plurality of data columns.

[0017] In some implementations, the multiple data elements can include multiple tensors, and each data row can store one or more tensors associated with a respective index corresponding to the data row. The executable instructions, when executed by a processor, cause the computer system to store the multiple data elements according to at least one of the following: transposing at least a subset of the tensors before storing at least the subset of the tensors in a data file; or encrypting at least a subset of the tensors before storing at least the subset of the tensors in a data file.

[0018] In some implementations, the executable instructions, when executed by a processor, also cause the computer system to: receive a request for at least one data element associated with a first index, wherein the request includes an indication of the first index; and use the indication to determine the first index.

[0019] In some implementations, when accessing at least one data element associated with a first index, the executable instructions, when executed by a processor, cause the computer system to: jump from a second memory location associated with a header to a first memory location when determining a first memory location of a data row corresponding to the first index.

[0020] According to yet another aspect, a non-transitory computer-readable medium can store computer code instructions thereon. The computer code instructions, when executed by a processor, can cause the processor to: obtain ego-navigation data including multiple data elements associated with multiple indexes, each index associated with one or more data elements, the ego-navigation data being determined based on data captured by at least one sensor of the ego; generate a data file storing the multiple data elements according to a predefined file format, the predefined file format including a header and multiple data rows, wherein each data row corresponds to a respective index and stores one or more data elements associated with the respective index, and for each row, the header includes an association between the respective index and a corresponding data offset indicating a memory location of the data row; use an indication of a first index and the header to determine a first memory location of a data row corresponding to the first index; use the first memory location to access at least one data element associated with the first index; and use the at least one data element associated with the first index to train a machine learning (ML) model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Non-limiting embodiments of the present disclosure are described by way of examples in relation to the accompanying drawings, which are schematic and not intended to be drawn to scale. Unless indicated as representing the background art, the drawings represent aspects of the present disclosure.

[0022] Figure 1A Illustrates components of an AI-enabled visual data analysis system according to an embodiment.

[0023] Figure 1BIllustrates various sensors associated with an entity according to an embodiment.

[0024] Figure 1C Illustrates components of a vehicle according to an embodiment.

[0025] Figure 2 Illustrates a block diagram of a video training system according to an embodiment.

[0026] Figure 3 Illustrates a flowchart of method 300 for efficient storage and access of data according to an embodiment.

[0027] Figure 4 Illustrates a diagram depicting an example arrangement of data elements in a data file having multiple rows according to a file format according to an embodiment.

[0028] Figure 5 Illustrates according to an embodiment the Figure 4 example layout of a memory layout of a file described in relation to Detailed Description

[0029] Reference will now be made to the illustrative embodiments depicted in the drawings, and specific language will be used to describe them. However, it should be understood that no limitation of the claims or the scope of the disclosure is thereby intended. Changes and further modifications of the features of the inventions shown herein, and additional applications of the principles of the subject matter shown herein, which would be obvious to those of the relevant art and having the present disclosure, will be considered to be within the scope of the subject matter disclosed herein. Other embodiments may be used and / or other changes may be made without departing from the spirit or scope of the disclosure. The illustrative embodiments described in the detailed description are not meant to limit the subject matter presented.

[0030] Training of relatively complex ML models or AI models, such as ML models for sensing or predicting the surrounding environment of an entity, is typically very time-consuming and involves the use of a large amount of training data. Training typically consumes a significant amount of processing power, bandwidth, and memory capacity. Specifically, training includes a relatively large number of training iterations. At each iteration, a significant amount of data can be fed as input into the training module. As such, training typically involves a large number of system input / output operations, which increases the complexity of training the ML model.

[0031] In the present disclosure, a new file format is introduced to enhance the efficiency of data storage and access. Although the description herein is mainly related to the training of ML models, the file format can be used in other applications. According to the file format, data can be stored in multiple indexed data rows, or more generally, in multiple indexed data segments. The file header can store the index of a data row in association with the corresponding data offset of the data row (or data segment). The header allows random access to any data row or data segment of a data file generated according to the file format without having to read the data file all the way to the data row of interest. In other words, when reading a data file, a processor or computing system can parse the header, determine the data offset of the data row (or data segment) of interest, and in response, move or jump to the memory location corresponding to the data offset to read the data row or data segment. As discussed further below, the file format and the systems, devices, and methods described herein achieve efficient storage of data, relatively fast access to data, and reduced use of processing power.

[0032] Figure 1A is a non-limiting example of a component of a system in which the methods and systems discussed herein can be implemented. For example, an analytics server can train an AI model and use the trained AI model to generate occupancy datasets and / or maps for one or more avatars. Figure 1A Illustrates the components of an AI-enabled visual data analysis system 100. The system 100 can include an analytics server 110a, a system database 110b, an administrator computing device 120, avatars 140a to 140b (collectively referred to as (multiple) avatars 140), avatar computing devices 141a to 141c (collectively referred to as avatar computing devices 141), and a server 160. The system 100 is not limited to the components described herein and can include additional or other components not shown for the sake of brevity, which will be considered within the scope of the embodiments described herein.

[0033] The above components can be connected via a network 130. Examples of the network 130 can include but are not limited to private or public LANs, WLANs, MANs, WANs, and the Internet. The network 130 can include wired and / or wireless communication according to one or more standards and / or via one or more transmission media.

[0034] Communication on network 130 can be performed according to various communication protocols such as Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), and IEEE communication protocols. In one example, network 130 can include wireless communication according to a Bluetooth specification set or another standard or proprietary wireless communication protocol. In another example, network 130 can also include communication on a cellular network, including for example GSM (Global System for Mobile Communications), CDMA (Code Division Multiple Access), or EDGE (Enhanced Data for Global Evolution) network.

[0035] System 100 illustrates an example of a system architecture and components that can be used to train and execute one or more AI models such as (multiple) AI models 110c. Specifically, as Figure 1A depicted and described herein, the analysis server 110a can use the methods discussed herein to train (multiple) AI models 110c using data retrieved from the self 140 (e.g., by using data streams 172 and 174). When (multiple) AI models 110c have been trained, each self in the self 140 can access and execute (multiple) trained AI models 110c. For example, a vehicle 140a having a self computing device 141a can transmit its camera feed to (multiple) trained AI models 110c and can determine the occupancy status of its surrounding environment (e.g., data stream 174). In addition, data ingested and / or predicted by (multiple) AI models 110c with respect to the self 140 (at inference time) can also be used to improve (multiple) AI models 110c. Thus, system 100 depicts a continuous loop that can periodically improve the accuracy of (multiple) AI models 110c. In addition, system 100 depicts a loop in which, in addition to the inference phase, data received by the self 140 can also be used in the training phase.

[0036] The analysis server 110a can be configured to collect, process, and analyze navigation data (e.g., images captured during navigation) and various sensor data collected from the self 140. Then, the collected data can be processed and prepared into a training dataset. Then, the training dataset can be used to train one or more AI models such as AI model 110c. The analysis server 110a can also be configured to collect visual data from the self 140. Using AI model 110c (trained using the methods and systems discussed herein), the analysis server 110a can generate a dataset and / or an occupancy map for the self 140. The analysis server 110a can display the occupancy map on the self 140 and / or transmit the occupancy map / dataset to the self computing device 141, the administrator computing device 120, and / or the server 160.

[0037] In Figure 1AIn [the figure], the AI model 110c is illustrated as a component of the system database 110b, but the AI model 110c can be stored in a different or separate component, such as a cloud storage device or any other data repository accessible by the analytics server 110a.

[0038] The analytics server 110a can also be configured to display an electronic platform that illustrates various training attributes for training the AI model 110c. The electronic platform can be displayed on the administrator computing device 120 such that an analyst can monitor the training of the AI model 110c. An example of an electronic platform generated and hosted by the analytics server 110a can be a web-based application or website configured to display the training data set collected from the agent 140 and / or the training status / metrics of the AI model 110c.

[0039] The analytics server 110a can be any computing device including a processor and a non-transitory machine-readable storage device capable of performing the various tasks and processes described herein. Non-limiting examples of such computing devices can include workstation computers, laptop computers, server computers, etc. Although the system 100 includes a single analytics server 110a, the system 100 can include any number of computing devices operating in a distributed computing environment, such as a cloud environment.

[0040] The agent 140 can represent various electronic data sources that transmit data associated with its previous or current navigation session to the analytics server 110a. The agent 140 can be any device configured for navigation, such as a vehicle 140a and / or a truck 140c. The agent 140 is not limited to being a vehicle and can also include robotic devices. For example, the agent 140 can include a robot 140b, which can represent a general-purpose, bipedal, autonomous humanoid robot capable of navigating various terrains. The robot 140b can be equipped with software for achieving balance, navigation, perception, or interacting with the physical world. The robot 140b can also include various cameras configured to transmit visual data to the analytics server 110a.

[0041] Although referred to herein as an “agent,” the agent 140 may or may not be an autonomous device configured for autonomous navigation. For example, in some embodiments, the agent 140 can be controlled by a human operator or a remote processor. The agent 140 can include various sensors, such as Figure 1BThe sensors depicted in. The sensors can be configured to collect data while the vehicle 140 navigates various terrains (e.g., roads). The analysis server 110a can collect the data provided by the vehicle 140. For example, the analysis server 110a can obtain navigation sessions and / or road / terrain data (e.g., images of the vehicle 140 navigating on a road) from various sensors, such that the collected data is ultimately used by the AI model 110c for training purposes.

[0042] As used herein, a navigation session corresponds to the journey of the vehicle 140's travel route, regardless of whether the journey is autonomous or human-controlled. In some embodiments, the navigation session can be used for data collection and model training purposes. However, in some other embodiments, the vehicle 140 can refer to a vehicle purchased by a consumer, and the purpose of the journey can be classified as daily use. The navigation session can start when the vehicle 140 moves from a non-moving position by more than a threshold distance (e.g., 0.1 miles, 100 feet) or at a rate greater than a threshold rate (e.g., greater than 0 mph, greater than 1 mph, greater than 5 mph). The navigation session can end when the vehicle 140 returns to a non-moving position and / or shuts down (e.g., when the driver exits the vehicle).

[0043] The vehicle 140 can represent a collection of vehicles monitored by the analysis server 110a to train the AI model(s) 110c. For example, the driver of the vehicle 140a can authorize the analysis server 110a to monitor the data associated with their respective vehicle. As a result, the analysis server 110a can utilize the various methods discussed herein to collect sensor / camera data and generate a training dataset to train the AI model(s) 110c accordingly. The analysis server 110a can then apply the trained AI model(s) 110c to analyze the data associated with the vehicle 140 and predict an occupancy map for the vehicle 140. Additionally, additional / ongoing data associated with the vehicle 140 can also be processed and added to the training dataset so that the analysis server 110a can recalibrate the AI model(s) 110c accordingly. Thus, the system 100 depicts a cycle in which the navigation data received from the vehicle 140 can be used to train the AI model(s) 110c. The vehicle 140 can include a processor that executes the trained AI model(s) 110c for navigation purposes. During navigation, the vehicle 140 can collect additional data about its navigation session, and this additional data can be used to calibrate the AI model(s) 110c. That is, the vehicle 140 represents a vehicle that can be used to train, execute / use, and recalibrate the AI model(s) 110c. In a non-limiting example, the vehicle 140 represents vehicles purchased by customers that can use the AI model(s) 110c for autonomous navigation while improving the AI model(s) 110c.

[0044] The self 140 can be equipped with various technologies that allow the self to collect data from its surrounding environment and (possibly) navigate autonomously. For example, the self 140 can be equipped with an inference chip to run autonomous driving software.

[0045] The various sensors for each self 140 can monitor the data collected associated with different navigation sessions and transmit the data to the analysis server 110a. Figures 1B to 1C A block diagram of sensors integrated within the self 140 according to an embodiment is illustrated. The number and location of each sensor discussed with respect to Figures 1B to 1C can depend on the type of self discussed in Figure 1A For example, the robot 140b can include different sensors than the vehicle 140a or the truck 140c. For example, the robot 140b may not include the airbag activation sensor 170q. Additionally, the sensors of the vehicle 140a and the truck 140c can be positioned differently than shown in Figure 1C As discussed herein, the various sensors integrated within each self 140 can be configured to measure various data associated with each navigation session. The analysis server 110a can periodically collect the data monitored and collected by these sensors, where the data is processed according to the methods described herein and is used to train the AI model 110c and / or execute the AI model 110c to generate an occupancy map.

[0046] The self 140 can include a user interface 170a. The user interface 170a can refer to the user interface of the self - computing device (e.g., the self - computing device 141 in

[0047] The user interface 170a can be implemented as a display screen, a head - up display, a touch - screen, etc. integrated with or coupled to the interior of the vehicle. The user interface 170a can include input devices such as a touch - screen, a knob, a button, a keyboard, a mouse, a gesture sensor, a steering wheel, etc. In various embodiments, the user interface 170a can be adapted to provide user input (e.g., as a kind of signal and / or sensor information) to other devices or sensors of the self 140 (e.g., the sensors shown in Figure 1A such as the controller 170c). Figure 1B As shown).

[0048] The user interface 170a can also be implemented with one or more logic devices that can be adapted to execute instructions, such as software instructions, to implement any of the various processes and / or methods described herein. For example, the user interface 170a can be adapted to form a communication link, transmit and / or receive communications (e.g., sensor signals, control signals, sensor information, user input, and / or other information) or perform various other processes and / or methods. In another example, a driver can use the user interface 170a to control the temperature of the vehicle 140 or activate its features (e.g., the autonomous driving or steering system 170o). Thus, the user interface 170a can be combined with other sensors described herein to monitor and collect driving session data. The user interface 170a can also be configured to display various data generated / predicted by the analysis server 110a and / or the AI model 110c.

[0049] The orientation sensor 170b can be implemented as a compass, a plumb bob, an accelerometer, and / or one or more of other digital or analog devices capable of measuring the orientation of the vehicle 140 (e.g., the magnitude and direction of roll, pitch, and / or yaw relative to one or more reference orientations such as gravity and / or magnetic north). The orientation sensor 170b can be adapted to provide heading measurements to the vehicle 140. In other embodiments, the orientation sensor 170b can be adapted to provide roll, pitch, and / or yaw rates to the vehicle 140 using a time series of orientation measurements. The orientation sensor 170b can be positioned and / or adapted to make orientation measurements relative to a particular coordinate system of the vehicle 140.

[0050] The controller 170c can be implemented as any suitable logic device (e.g., a processing device, a microcontroller, a processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a memory storage device, a memory reader, or other device or combination of devices) that can be adapted to execute, store, and / or receive appropriate instructions, such as software instructions that implement control loops for controlling various operations of the vehicle 140. Such software instructions can also implement methods for processing sensor signals, determining sensor information, providing user feedback (e.g., via the user interface 170a), querying devices for operating parameters, selecting operating parameters for devices, or performing any of the various operations described herein.

[0051] The communication module 170e can be implemented as any wired and / or wireless interface that is configured to transfer sensor data, configuration data, parameters, and / or other data and / or signals to Figure 1A any of the features shown (e.g., the analysis server 110a). As described herein, in some embodiments, the communication module 170e can be implemented in a distributed manner such that portions of the communication module 170e are inFigure 1B implemented within one or more of the illustrated elements and sensors. In some embodiments, the communication module 170e may delay the transmission of sensor data. For example, when the vehicle 140 does not have a network connection, the communication module 170e may store the sensor data in a temporary data storage device and transmit the sensor data when the vehicle 140 is identified as having an appropriate network connection.

[0052] The speed sensor 170d may be implemented as an electronic pitot tube, a metering gear or wheel, a water speed sensor, a wind speed sensor, a wind rate sensor (e.g., direction and magnitude), and / or other devices capable of measuring or determining the linear speed of the vehicle 140 (e.g., in the surrounding medium and / or aligned with the longitudinal axis of the vehicle 140) and providing such measurement as a sensor signal, which may be transmitted to various devices.

[0053] The gyroscope / accelerometer 170f may be implemented as one or more electronic sextants, semiconductor devices, integrated chips, accelerometer sensors, or other systems or devices capable of measuring the angular velocity / acceleration and / or linear acceleration of the vehicle 140 (e.g., direction and magnitude) and providing such measurement as a sensor signal, which may be transmitted to various devices, such as the analysis server 110a. The gyroscope / accelerometer 170f may be positioned and / or adapted to make such measurements relative to a particular coordinate system of the vehicle 140. In various embodiments, the gyroscope / accelerometer 170f may be implemented in a common housing and / or module with Figure 1B the other elements depicted to ensure a common reference frame or a known transformation between reference frames.

[0054] The Global Navigation Satellite System (GNSS) 170h may be implemented as a global positioning satellite receiver and / or other devices capable of determining the absolute and / or relative position of the vehicle 140 based on wireless signals received, for example, from space and / or ground sources and capable of providing measurements such as sensor signals, which may be transmitted to various devices. In some embodiments, the GNSS 170h may be adapted to determine the rate, speed, and / or yaw rate of the vehicle 140 (e.g., using a time series of position measurements), such as the yaw component of the absolute rate and / or angular rate of the vehicle 140.

[0055] The temperature sensor 170i may be implemented as a thermistor, an electrical sensor, an electrical thermometer, and / or other devices capable of measuring the temperature associated with the vehicle 140 and providing such measurement as a sensor signal. The temperature sensor 170i may be configured to measure the ambient temperature associated with the vehicle 140, such as the cockpit or dashboard temperature. For example, this ambient temperature may be used to estimate the temperature of one or more elements of the vehicle 140.

[0056] The humidity sensor 170j can be implemented as a relative humidity sensor, an electrical sensor, an electrical relative humidity sensor, and / or another device capable of measuring the relative humidity associated with the body 140 and providing such measurement as a sensor signal.

[0057] The steering sensor 170g can be adapted to physically adjust the heading of the body 140 based on one or more control signals provided by a logic device (such as the controller 170c) and / or user input. The steering sensor 170g can include one or more actuators and control surfaces (e.g., a rudder or other type of steering or trim mechanism) of the body 140, and can be adapted to physically adjust the control surface to various positive and / or negative steering angles / positions. The steering sensor 170g can also be adapted to sense the current steering angle / position of such steering mechanism, and provide such measurement.

[0058] The propulsion system 170k can be implemented as a propeller, a turbine, or other thrust-based propulsion systems, mechanical wheeled and / or tracked propulsion systems, wind / sail-based propulsion systems, and / or other types of propulsion systems that can be used to power the body 140. The propulsion system 170k can also monitor the direction of the motive power and / or thrust of the body 140 relative to the reference coordinate system of the body 140. In some embodiments, the propulsion system 170k can be coupled to the sensor 170g and / or integrated with the steering sensor 170g.

[0059] The passenger restraint sensor 170l can monitor seatbelt detection and locking / unlocking components and other passenger restraint subsystems. The passenger restraint sensor 170l can include various environmental and / or status sensors, actuators, and / or other devices that facilitate the operation of the safety mechanisms associated with the operation of the body 140. For example, the passenger restraint sensor 170l can be configured to receive motion and / or status data from Figure 1B the other sensors depicted. The passenger restraint sensor 170l can determine whether a safety measurement (e.g., seatbelt) is being used.

[0060] As Figure 1C depicted, the camera 170m can refer to one or more cameras integrated within the body 140, and can include multiple cameras integrated (or retrofitted) into the body 140. The camera 170m can be an internal or external facing camera of the body 140. For example, as Figure 1CAs depicted, the vehicle 140 may include one or more inward-facing cameras that can monitor and collect footage of the passengers in the vehicle 140. The vehicle 140 may include eight outward-facing cameras. For example, the vehicle 140 may include a front camera 170m-1, a front side camera 170m-2, a front side camera 170m-3, rear side cameras 170m-4 on each front fender, cameras 170m-5 on each side (e.g., integrated within the B-pillar), and a rear camera 170m-6.

[0061] Reference Figure 1B , the radar 170n and ultrasonic sensors 170p may be configured to monitor the distance of the vehicle 140 to other objects, such as other vehicles or immovable objects (e.g., trees or garage doors). The vehicle 140 may also include an autonomous driving or steering system 170o configured to self-navigate the vehicle 140 using data collected via various sensors, such as the radar 170n, speed sensors 170d, and / or ultrasonic sensors 170p.

[0062] Thus, the autonomous driving or steering system 170o may analyze various data collected by one or more of the sensors described herein to identify driving data. For example, the autonomous driving or steering system 170o may calculate the risk of a front collision based on the speed of the vehicle 140 and its distance to another vehicle on the road. The autonomous driving or steering system 170o may also determine whether the driver is touching the steering wheel. The autonomous driving or steering system 170o may transmit the analyzed data to various features discussed herein, such as an analytics server.

[0063] The airbag activation sensor 170q may predict or detect a collision and cause the activation or deployment of one or more airbags. The airbag activation sensor 170q may transmit data regarding the airbag deployment, including data associated with the event that caused the deployment.

[0064] Again referring Figure 1A , the administrator computing device 120 may represent a computing device operated by a system administrator. The administrator computing device 120 may be configured to display data retrieved or generated by the analytics server 110a (e.g., various analytics metrics and risk scores), where the system administrator may monitor the various models utilized by the analytics server 110a, review feedback, and / or facilitate the training of the AI models 110c maintained by the analytics server 110a.

[0065] (The) vehicles 140 may be any device configured to navigate various routes, such as a vehicle 140a or a robot 140b. As regarding Figures 1B to 1CAs discussed, the vehicle 140 can include various telemetry sensors. The vehicle 140 can also include a vehicle computing device 141. Specifically, each vehicle can have its own vehicle computing device 141. For example, truck 140c can have a vehicle computing device 141c. For simplicity, the vehicle computing devices are collectively referred to as the vehicle computing device(s) 141. The vehicle computing device 141 can control content presentation on the infotainment system of the vehicle 140, process commands associated with the infotainment system, aggregate sensor data, manage communication of data to an electronic data source, receive updates, and / or transmit messages. In one configuration, the vehicle computing device 141 communicates with an electronic control unit. In another configuration, the vehicle computing device 141 is an electronic control unit. The vehicle computing device 141 can include a processor and a non-transitory machine-readable storage medium capable of performing the various tasks and processes described herein. For example, the AI model(s) 110c described herein can be stored and executed (or directly accessed) by the vehicle computing device 141. Non-limiting examples of the vehicle computing device 141 can include vehicle multimedia and / or display systems.

[0066] In one example of how to enhance and efficiently process data, such as training data for training and / or validating the AI model(s) 110c and / or other ML models, a file format with a header and multiple index data segments can be used to store and / or access the data. Given a data set including multiple data elements associated with multiple indexes (e.g., timestamps or time values), where each index is associated with one or more corresponding data elements, the file format enables the data set to be stored in multiple data rows or data segments of a data file. Each data row or data segment can correspond to a corresponding index or timestamp and can store the data elements associated with the corresponding index or timestamp. When storing the data set in the data file, the computer system can store the index of the data row or data segment in association with the data offset of the data row or data segment in the header. Specifically, for each data row or data segment, the computer system can store the corresponding index (or timestamp) in the header in association with the corresponding data offset. For a data field or data variable that is the same for all indexes, the computer system can store a single instance of the data field or data variable in the data file, e.g., before the data row or data segment, to avoid unnecessary redundancy and reduce the amount of data written or stored in the data file.

[0067] A file format may be associated with, or may have, file format-specific system read and write operations. A write operation may be configured to write or store data in a data file in a file format according to a predefined layout including a header and data rows or data segments. Specifically, a write operation may enable writing a data element associated with a given index into a data row or data segment corresponding to the data index. A write operation may also (e.g., at the beginning of the data file) enable generating or writing a header to include an index of the data row or data segment in association with a corresponding data offset of the data row or data segment.

[0068] A read operation may enable first reading a header to identify the data offset(s) of the data row(s) of interest, and then moving to the memory location corresponding to the data offset to read or parse the data row or data segment of interest. In other words, a read operation may receive one or more indexes (or indications thereof) of one or more rows as input, and use the input one or more indexes to determine the data offset(s) between the one or more rows and the header. A read operation may be configured to move directly to the memory location(s) corresponding to the data offset(s) in order to read or parse the one or more rows indicated by the input one or more indexes.

[0069] The file format and the systems, devices, and methods described herein result in a data file in the new file format being approximately 11% smaller in size compared to other file formats. Additionally, the number of system input / output operations per unit time (e.g., IOPS) may be reduced by a factor of four. This means that there is a savings of approximately 11% in memory usage and a reduction in processing time by at least a factor of four.

[0070] Figure 2 A block diagram illustrating a computer environment 200 for training an ML model according to an embodiment is shown. The computer environment 200 may include a training system 202 for training an ML model and a data storage system 204 for storing training data and / or validation data. The training system 202 may include a plurality of training nodes (or processing nodes) 206. Each training node 206 may include a corresponding data loader (or data loading device) 208 and a corresponding graphics processing unit (GPU) 210. Each GPU 210 may include a memory (e.g., cache memory) 212, processing circuitry 214, and one or more video decoders 216.

[0071] The data storage system 204 can include, or can be, a distributed storage system. For example, the data storage system 204 can have an infrastructure that can split data across multiple physical servers, such as supercomputers. The data storage system 204 can include one or more storage clusters of storage units, having mechanisms and infrastructure for parallel and accelerated access to data from multiple nodes or storage units of the (multiple) storage clusters. For example, the data storage system 204 can include sufficient data links and bandwidth to deliver data to the training nodes 206 in parallel or simultaneously.

[0072] The data storage system 204 can include sufficient memory capacity to store, for example, millions or even billions of video frames in a compressed form. For example, the data storage system 204 can have a memory capacity to store data of multiple petabytes (e.g., 10, 20, or 30 petabytes). The data storage system 204 can allow thousands of video sequences to be moved into and / or out of the data storage system 204 at any time instance. The relatively large size and bandwidth of the data storage system 204 allow for parallel training of one or more ML models, as discussed below.

[0073] The training system 202 can be implemented as one or more physical servers, such as server 110a. For example, the training system 202 can be implemented as one or more supercomputers. Each supercomputer can include thousands of processing or training nodes 206. The training nodes 206 can be configured or designed to support parallel training of one or more ML models, such as the (multiple) AI models 110c. Each training node 206 can be communicatively coupled to the data storage system 204 to access training data and / or validation data stored therein.

[0074] Each training node 206 can include a corresponding data loader 208 and a corresponding GPU 210 that are communicatively coupled to each other. The data loader 208 can be (or can include) a processor or a central processing unit (CPU) for handling data requests or data transfers between the corresponding GPU 210 (e.g., the GPU 210 in the same training node 206) and the data storage system 204. For example, the GPU 210 can request data by Figure 1COne or more video sequences captured by one or more of the cameras 170m described. For example, the front or forward cameras 170m-1, 170m-2, and 170m-3, the rear side camera 170m-4, the side camera 170m-5, and the rear camera 170m-6 can capture video sequences simultaneously and send the video sequences to the data storage system 204 and store the video sequences in the data storage system 204 for storage. In some implementations, the data storage system 204 can store the video sequences captured simultaneously by multiple cameras of the vehicle 140 (such as the cameras 170m) as a set of video sequences or a combination of video sequences, and these video sequences can be delivered together to the training node 206. For example, the data storage system 204 can maintain additional data that indicates which video sequences were captured simultaneously by the cameras 170m, or represents the same scene from different camera angles. The data storage system 204 can maintain (e.g., for each stored video sequence) data indicating the vehicle identifier, the camera identifier, and the time instance associated with the video sequence.

[0075] The training node 206 can use the video data captured by the cameras 170m of the vehicle 140 and stored in the data storage system 204 to train one or more ML models simultaneously (e.g., in parallel). In some implementations, the video data can be captured by the cameras 170m of multiple vehicles 140. In the training node 206, the corresponding data loader 208 can request from the data storage system 204 the video data of one or more video sequences captured simultaneously by one or more of the cameras 170m of the vehicle 140 during a time interval, and send the received video data to the corresponding GPU 210 for use in performing training steps (or validation steps) when training the ML model. The video data can be in a compressed form. For example, the video sequences can be encoded by an encoder integrated in or implemented in the vehicle 140. Each data loader 208 can have sufficient processing power and bandwidth to deliver the video data to the corresponding GPU 210 in a way that keeps the GPU 210 busy. In other words, the GPU 210 can be configured or designed to, for example, in terms of processing power and bandwidth, request the video data of a set of compressed video sequences from the data storage system 204 and deliver the video to the GPU 210 within a duration less than or equal to the average time consumed by the GPU 210 to process a set of video sequences.

[0076] Each GPU 210 may include a corresponding internal memory 212, such as a cache memory, to store executable instructions for performing the processes described herein, received video data of one or more video sequences, decoded video frames, features extracted from the decoded video frames, data of parameters or trained ML models, and / or other data used for training the ML models. The memory 212 may be large enough to store all data required to perform a single training step. As used herein, a training step may include receiving and decoding video data of one or more video sequences (e.g., a set of video sequences simultaneously captured by one or more cameras 170m of the ego vehicle 140), extracting features from the decoded video data, and using the extracted features to update the parameters of the ML model being trained or validated.

[0077] Each GPU 210 may include processing circuitry 214 to perform the processes or methods described herein. The processing circuitry 214 may include one or more microprocessors, multi-core processors, digital signal processors (DSPs), one or more logic circuits, or a combination thereof. The processing circuitry 214 may execute computer code instructions (e.g., stored in the memory 212) to perform the processes or methods described herein.

[0078] The GPU 210 may include one or more video decoders 216 for decoding video data received from the data storage system 204. The one or more video decoders 216 may include (multiple) hardware video decoders integrated in the GPU 110 to accelerate video decoding. The one or more video decoders 216 may be part of the processing circuitry 214 or may include separate (multiple) electronic circuits communicatively coupled to the processing circuitry 214.

[0079] Each GPU 210 may be configured or designed to process or execute a training step without using any external resources. The GPU 210 may include sufficient memory capacity and processing power to execute the training step. The processes for storing and accessing training data performed by the training node 206 or the corresponding GPU 210 will be described in further detail below in conjunction with Figures 3 to 5 6.

[0080] Now refer to Figure 3, according to an embodiment, a flowchart of method 300 for efficient storage and access of data. Briefly, method 300 may include obtaining data including a plurality of data elements associated with a plurality of timestamps, where each timestamp may be associated with one or more data elements (step 302), and generating a data file storing the data according to a predefined file format having a plurality of rows, where each row corresponds to a respective timestamp and stores one or more data elements associated with the respective timestamp, and the data file includes a header that stores an association between the respective timestamp and a corresponding data offset indicating a memory location of that row for each row (step 304). Method 300 may include using an indication of a first timestamp and the header of the data file to determine a first memory location of a row corresponding to the first timestamp (step 306), and using the first memory location to access at least one data element associated with the first timestamp (step 308). Method 300 may optionally include using at least one data element associated with the first timestamp to train a training model (step 310).

[0081] Method 300 may be fully implemented, executed, or carried out by any device in the CPU or loader 208 of computer environment 200 or system 100, any CPU in GPU 210, and / or other computing devices (or computer systems). For example, GPU 210 may execute method 300 after decoding a selected (or indicated) image frame and extracting corresponding image features. GPU 210 may store the extracted features in a data file together with other training data according to method 300. Method 300 or any of its steps may be implemented as executable instructions that may be stored in a memory and executed by one or more processors.

[0082] Method 300 may include a computer system that includes a memory and one or more processors, such as GPU 210, that obtains data that includes a plurality of data elements associated with a plurality of timestamps, where each timestamp may be associated with one or more data elements (step 302). In some implementations, the data may include training data for training or validating one or more machine learning (ML) models. For example, the data may include data for training and / or validating AI model(s) 110c or an occupancy network. For example, the data may include ground truth data, features extracted from image frames decoded by video decoder(s) 216, interference output, a map of the geographical location of ego 140, rate or speed values of ego 140, attributes of ego 140 (e.g., make and model, fuel level, battery charge status, etc.), maximum allowable speed, sensor data generated by one or more sensors of ego 140, other type(s) of data, and / or combinations thereof. Obtaining the data may include receiving the data or a portion thereof (e.g., from data storage system 204), generating the data or a portion thereof, and / or retrieving the data or a portion thereof from a memory (e.g., memory 212).

[0083] In some implementations, the data may include or may be based on ego navigation data determined from data captured by at least one sensor of the ego, such as the sensors described with respect to Figure 1B the data may include a plurality of data elements. As used herein, a data element may be a parameter value, a tensor, a text string, etc. For example, GPU 210 may group features extracted from a single image frame into a single tensor. In some implementations, GPU 210 may group features extracted from multiple image frames (e.g., captured by camera 170m at substantially the same time instance, taking into account the desynchronization between camera 170m and / or the corresponding encoder) into a single tensor. Additionally, road data (such as traffic signs, traffic lights, speed limits, etc.) at a given time instance may be grouped or arranged as a corresponding tensor. Some sensor data (such as ego rate) may be processed as a single parameter value. Ego attributes (such as vehicle make and model, engine type, etc.) may be expressed as a text string or a numerical value.

[0084] Multiple data elements can be associated with multiple timestamps or time values. Each timestamp (or time value) can be associated with one or more data elements. For example, GPU 210 can associate each set of features extracted from a decoded image frame with the timestamp of the image frame. Additionally, other received training data (such as ego rate values, other sensor data from ego sensors, map data, etc.) can be associated with timestamps, depending on, for example, the time instance at which each piece or set of training data is captured or recorded. For example, each rate value can be associated with a timestamp or time value that indicates the time instance at which the rate value is recorded at ego 140. The association between the timestamp and the data element can be performed at ego 140 (e.g., when the data element is captured or recorded), in the training system 202, or a combination of both. The timestamp (or time value) can be equal to the timestamp of the image frame of the video sequence captured by the (multiple) cameras 170m or some other time value.

[0085] Method 300 can include the GPU 210 or other computer system generating a data file storing data according to a predefined file format having multiple rows, where each row corresponds to a respective timestamp and stores one or more data elements associated with the respective timestamp, and where the data file includes a header that stores the association between the respective timestamp and a corresponding data offset indicating the memory location of the row for each row (step 304). The GPU 210 or some other computer system can generate the data file and store the data elements in the corresponding rows of the data file. The GPU 210 or other computer system can generate a data file according to a predefined file format including a header and one or more index rows and store the data therein. The indexing or indexing of the rows and the corresponding memory locations can be specified in the header.

[0086] When generating the data file, the GPU 210 or other computer system can determine the total number of rows of the data file (e.g., based on the total number of timestamps (or time values)) and associate each row with a corresponding timestamp (or time value). The GPU 210 or other computer system can arrange the rows in ascending order of the corresponding timestamps (or time values). The GPU 210 or other computer system can determine or identify the data elements to be stored in each row based on the association between the data elements and the timestamps and the association between the rows and the timestamps. Specifically, the GPU 210 or other computer system can determine to store the data elements associated with a given timestamp in the row corresponding to the same timestamp. When determining the arrangement of the data elements in the data file, the GPU 210 or other computer system can determine or create multiple columns of the data file. Each column can correspond to or represent a separate data field, data category, or data type. For example, each column of the data file can correspond to a type of data parameter or tensor.

[0087] Now referring to Figure 4 ,and according to the example embodiment, an example arrangement 400 of data elements in a data file according to a file format having multiple lines is shown. The GPU 210 or other computer system may configure the data file to include n lines, e.g., lines 402-1 to 402-n, where n is an integer. The lines 402-1 to 402-n are also referred to herein individually or collectively as the (multiple) lines 402. Each line 402 corresponds to a separate timestamp 404. The GPU 210 or other computer system may configure the data file to include m columns, e.g., columns 406-1 to 406-m, where m is an integer. The columns 406-1 to 406-m are also referred to herein individually or collectively as the (multiple) columns 406. The GPU 210 or other computer system may assign a corresponding data field or data element type or category to each column 406. For example, column 406-1 may store autologous rate values for different timestamps or time values. Column 406-2 may store a first type of tensor for various timestamps or time values, referred to herein as "tensor A". For example, a tensor of the "tensor A" type may carry road or traffic data at various timestamps or time values. Specifically, the tensor to be stored in the cell at the intersection of row 402-1 and column 406-2 may carry road or traffic data corresponding to the timestamp 1337.37. Column 406-3 may store a second type of tensor for various timestamps or time values, referred to herein as "tensor B". For example, a tensor of the "tensor B" type may carry extracted features associated with various timestamps or time values.

[0088] The GPU 210 or other computer system may use the timestamp 404 to index the rows 402. In other words, the timestamp 404 may not be stored in a separate column 406. Instead, the GPU 210 or other computer system may store the timestamp 404 in the header in association with the data offset of the row, where the data offset of the row indicates the memory location of the row. The GPU 210 or other computer system may store the corresponding timestamp and the corresponding data offset in the header for each row. For example, the GPU 210 or other computer system may store the timestamp and the data offset according to the order of the corresponding rows. Then, the GPU 210 or other computer system may store the data elements associated with the same timestamp corresponding to the row in each row of the data file.

[0089] For each row 402, the corresponding data offset can be expressed in data units such as bytes. The data offset for a given row 402 can represent the distance or separation in data units between the start of the row in memory and some reference memory location. The reference memory location can be the memory location representing the start of the data file. In other words, the data offset for row 402 can be regarded as the amount of data (in data units) before row 402 in the data file.

[0090] In some implementations, the data can include at least one data element associated with all timestamps of a plurality of timestamps. In other words, for each timestamp of the plurality of timestamps, at least one data element can be the same. For example, the vehicle make and model of the vehicle body 140 do not change over time. Additionally, a map of the geographical location of the vehicle body 140 may not change over some time intervals spanned by the plurality of timestamps, e.g., depending on the speed of the vehicle body 140. For such data elements, it would be inefficient to repeatedly store the same data across all rows 402 within one or more columns.

[0091] In some implementations, when storing data in rows, the GPU 210 or other computer system can store a first data element having a first size before a second data element having a second size (greater than the first size). For example, the GPU 210 or other computer system can store the columns 406 in increasing order of the corresponding data sizes. For example, the GPU 210 or other computer system can select the data field (or data element type) having the smallest size to be stored in the first column 406-1, select the data field having the next smallest size to be stored in the column 406-2, and so on, until the column 406-n in which the data field having the largest size is stored. When not all data elements in a row are to be retrieved, sorting the columns or data elements in each data row in increasing order of data size further reduces the amount of data read or parsed.

[0092] In some implementations, the GPU 210 or other computer system can apply a dimension transpose to one or more tensors before storing them in the data file. In other words, the GPU 210 or other computer system can store some tensors in a transposed form, such as tensor A or tensor B. In some implementations, the GPU 210 or other computer system can encrypt one or more data elements of one or more types before storing them in the data file. For example, the GPU 210 or other computer system can store one or more tensors in an encrypted form.

[0093] The GPU 210 or other computer system may determine that at least one other data element is associated with or is the same for all of the plurality of timestamps. For example, the data may indicate that data associated with a particular data field or data element type does not change or is the same across all timestamps. For a data field or data element type having the same data across multiple timestamps, there may be only a single corresponding data element in the data, indicating that the same data element applies to all timestamps. The GPU 210 or other computer system may parse the data to determine which data field(s) or data element type(s) have static data across multiple timestamps.

[0094] The GPU 210 or other computer system stores at least one data element that is associated with or is the same for all of the plurality of timestamps before the plurality of rows in the data file. Specifically, the GPU 210 or other computer system may store the static data element once in the data file, e.g., before any of the rows in row 402, to avoid unnecessary redundancy and to use memory resources efficiently. The GPU 210 or other computer system may store the unchanging data element in the header, between the header and row 402, or before the header.

[0095] Now referring to Figure 5 , an example layout 500 of the memory layout of a file described with respect to Figure 4 is shown according to an example embodiment. When stored in memory, the data or data file may be viewed as a plurality of contiguous data segments. A first data segment 502 may represent a header. A second data segment 504 (referred to herein as a column) may represent the (multiple) data elements that have static or the same values across the plurality of timestamps 402. A series of data segments 506-1 to 506-n corresponding to rows 402-1 to 402-n, respectively, may follow the second data segment 504. For example, data segment 506-1 may store a rate value 508-1, an instance 510-1 of tensor A, and an instance 512-1 of tensor B associated with row 402-1 in the data file. Data segment 506-2 may store a rate value 508-2, an instance 510-2 of tensor A, and an instance 512-2 of tensor B associated with row 402-2 of the data file, and data segment 506-n may store a rate value 508-n, an instance 510-n of tensor A, and an instance 512-n of tensor B associated with row 402-n of the data file.

[0096] The data segment 502 representing the header may include a sequence of timestamps 402 (indicated as T1, T2,..., T n ) and data offsets (indicated as O1, O2,..., O n)。In some implementations, the GPU 210 or other computer system may store the timestamps 402 and data offsets in the order of line 402, i.e., starting with the timestamp T1 and data offset O1 of line 402-1, followed by the timestamp T2 and data offset O2 of line 402-2, and so on, until the timestamp T of the last line 402-n n and data offset O n 。In some implementations, the header or data segment 502 may also include the data offset of the data segment 504, for example, at the start or end of the data segment 502.

[0097] In some implementations, the file format described above regarding Figures 4 to 5 may have a corresponding extension. In some implementations, the file format extension may be ".smol". In some implementations, another extension may be used. Specific write and read operations that take into account the data arrangement and memory layout described above in Figures 4 to 5 may be used with the file format. The write operation may be configured to generate a data file with a data format regarding the Figure 5 memory layout 500 discussed. The write operation may be configured to consider the memory layout 500 and read the header to identify the data offset of the data row of interest, and then directly move to the memory location corresponding to the data offset in order to read or parse the data row of interest.

[0098] Return reference Figure 3 , method 300 may include the GPU 210 or other computer system using an indication of a first timestamp and the header of the data file to determine the memory location of the row corresponding to the timestamp (step 306), and using the determined memory location to access at least one data element associated with the first timestamp (step 308). In some implementations, the GPU 210 or other computer system may receive a request (e.g., an application programming interface) for at least one data element. The request or call may include an indication of the first timestamp. The indication may include a time value. The GPU 210 or other computer system may use the indication to determine the first timestamp, e.g., as the timestamp closest to the indication or time value in the request.

[0099] The GPU 210 or other computer system may parse the header of the data file to determine the data offset corresponding to the first timestamp. The data offset corresponding to the first timestamp represents the memory location of line 402 including at least one data element, e.g., relative to the starting position of the data file. In response to determining the data offset, the GPU 210 or other computer system may jump to the memory location indicated by the data offset. For example, if the data offset indicates the memory location of data segment 506-n corresponding to line 402-n, the GPU 210 or other computer system will not read or parse data segments 506-1 to 506-n-1 and will directly move to the memory location indicated by the data offset to read data segment 506-n corresponding to line 402-n. When reading data segment 506-n, the GPU 210 or other computer system may determine the at least one requested data element.

[0100] In some implementations, when multiple data elements from multiple lines 402 are to be read, the GPU 210 or other computer system may parse the header or portions thereof until the data offsets for all lines of interest are determined. Note that the GPU 210 or other computer system does not need to parse the entire header and may stop at the point where all relevant or required data offsets are determined. Once the data offsets are determined, the GPU 210 or other computer system may start moving to the corresponding memory locations and sequentially read the corresponding data segments representing the corresponding lines. For example, if the GPU 210 or other computer system is to read instances of tensor A in lines 402-1 and 402-n-1, the GPU 210 or other computer system may parse the header to determine the data offsets O1 and O n-1 corresponding respectively to timestamps T1 and T n-1 . The GPU 210 or other computer system may move to the memory location corresponding to data offset O1 to read data segment 506-1, and then move to the memory location corresponding to data offset O n-1 to read data segment 506-(n-1).

[0101] In some implementations, the data file may be a read-only or non-editable data file. Preventing the data file from being edited maintains the Figure 5 layout structure depicted in. Generally, when a data file is edited, data is added to the end of the data file. As a result, any additional data will not be indexed in the header. For example, if the data file is edited to add one or more additional lines, the header will not include the timestamps and data offsets of the added lines, which can create confusion regarding reading data from the data file. Specifically, a read operation (as described above) to read a data element from any of the added lines will fail because such data lines are not reflected in the header.

[0102] Storing the timestamp and data offset of each line in the header enables fast random access to any line in line 402 of the data file. Specifically, when reading one or more data elements from the data file, the amount of data to be parsed is significantly reduced, resulting in a reduction in the execution time of the read operation. Additionally, since multiple data elements at different positions in the data file can be read in a single read operation, the number of system input / output operations is reduced. Further, when generating the data file, the number of write operations is reduced by writing data elements that are the same for all timestamps only once in the data file.

[0103] At step 310, method 300 may include the GPU 210 or other computer system training an ML model using the data stored and retrieved according to the methods and systems discussed herein. Specifically, the GPU 210 or other computer system may use at least one data element associated with the first timestamp to train the ML model, such as an occupancy network. The GPU 210 or other computer system may use the data stored in the data file to perform one or more training steps of the ML model.

[0104] Although method 300 has been described above with respect to a predefined file format, the same steps can be applied to store data in and read data from memory. In other words, data can be written or stored in a memory region according to the Figure 5 memory layout 500, and data can be read by considering the same memory layout, regardless of the predefined file format. Data can be stored according to multiple index data segments (e.g., not necessarily data lines). Additionally, although method 300 is described according to timestamps, any type of index can be used to index data segments or data lines. Finally, the embodiments described herein can be used in other applications and should not be limited or confined to the training or validation of ML models.

[0105] The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality. Whether this functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure or the claims.

[0106] Embodiments implemented in computer software can be implemented in software, firmware, middleware, microcode, hardware description language, or any combination thereof. Code segments or machine-executable instructions can represent a process, function, subroutine, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. By passing and / or receiving information, data, arguments, parameters, or memory contents, a code segment can be coupled to another code segment or hardware circuit. Information, arguments, parameters, data, etc. can be passed, forwarded, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0107] The actual software code or special control hardware used to implement these systems and methods does not limit the claimed features or this disclosure. Thus, without referring to specific software code, the operation and behavior of the systems and methods can be described, and it can be understood that the software and control hardware can be designed to implement the systems and methods based on the description herein.

[0108] When implemented in software, the functions can be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of the methods or algorithms disclosed herein can be embodied in processor-executable software modules, which can reside on a computer-readable or processor-readable storage medium. The non-transitory computer-readable or processor-readable medium includes both computer storage media and tangible storage media, and the tangible storage media facilitates the transfer of a computer program from one place to another. The non-transitory processor-readable storage medium can be any available medium accessible by a computer. By way of example and not limitation, such non-transitory processor-readable media can include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other tangible storage medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer or processor. The disks and optical disks used herein include optical discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), Blu-ray discs, and floppy disks, where "disks" generally reproduce data magnetically, while "optical discs" reproduce data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of the methods or algorithms can reside as one or any combination or collection of code and / or instructions on the non-transitory processor-readable medium and / or computer-readable medium, which can be incorporated into a computer program product.

[0109] The foregoing description of the disclosed embodiments is provided to enable a person skilled in the art to make or use the embodiments and their variations described herein. Various modifications to these embodiments will be apparent to those skilled in the art, and the principles defined herein can be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Thus, the present disclosure is not intended to be limited to the embodiments shown herein, but should be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

[0110] Although various aspects and embodiments have been disclosed, other aspects and embodiments are also contemplated. The disclosed aspects and embodiments are for illustrative purposes only and are not intended to be limiting, and the true scope and spirit are indicated by the following claims.

Claims

1. A method, comprising: Autonomous navigation data including a plurality of data elements associated with a plurality of indices is obtained by a processor, each index being associated with one or more data elements, and the autonomous navigation data being determined based on data captured by at least one sensor of the autonomous entity; A data file storing the data is generated by the processor according to a predefined file format, the predefined file format including a header and a plurality of data rows, each data row corresponding to a respective index and storing the one or more data elements associated with the respective index, and for each data row, the header including an association between the respective index and a corresponding data offset indicating the memory location of the data row; The processor uses an indication of a first index and the header to determine a first memory location of a data row corresponding to the first index; The processor uses the first memory location to access at least one data element associated with the first index; And The processor uses the at least one data element associated with the first index to train a machine learning (ML) model.

2. The method according to claim 1, wherein the ego-navigation data is used to train an occupancy network of the ego, and the occupancy network is used to predict or sense the surrounding environment of the ego.

3. The method according to claim 1, wherein the data further includes at least one other data element associated with all of the plurality of indices, and the method further comprises: The processor determines that the at least one other data element is associated with all of the plurality of indices; And The processor stores the at least one other data element before the plurality of data rows in the data file.

4. The method according to claim 1, wherein the data file is a read-only data file.

5. The method according to claim 1, wherein the data file stores the plurality of data elements according to a plurality of columns, each column representing a corresponding data field, and the method further comprises arranging the plurality of columns in an increasing order of data sizes associated with the plurality of data columns.

6. The method according to claim 1, wherein the plurality of indices includes a plurality of timestamps or a plurality of time values.

7. The method according to claim 1, wherein the plurality of data elements includes a plurality of tensors, and each data row stores one or more tensors associated with the corresponding index corresponding to the data row.

8. The method according to claim 7, further comprising at least one of the following: transposing at least a subset of the tensors by the processor before storing at least the subset of the tensors in the data file; or encrypting at least a subset of the tensors by the processor before storing at least the subset of the tensors in the data file.

9. The method according to claim 1, further comprising: The processor receives a request for the at least one data element associated with the first index, the request including the indication of the first index; The processor uses the indication to determine the first index.

10. The method according to claim 1, wherein accessing the at least one data element associated with the first index includes: When determining the first memory location of the data row corresponding to the first index, the processor jumps from a second memory location associated with the header to the first memory location.

11. A computer system, comprising: A processor; And A memory storing executable instructions which, when executed by the processor, cause the computer system to: Obtain autonomous navigation data including a plurality of data elements associated with a plurality of indices, each index being associated with one or more data elements, and the autonomous navigation data being determined based on data captured by at least one sensor of the autonomous entity; Generate a data file storing the plurality of data elements according to a predefined file format, the predefined file format including a header and a plurality of data rows, each data row corresponding to a respective index and storing the one or more data elements associated with the respective index, and for each row, the header including an association between the respective index and a corresponding data offset indicating the memory location of the data row; Use an indication of a first index and the header to determine a first memory location of a data row corresponding to the first index; Use the first memory location to access at least one data element associated with the first index; And Use the at least one data element associated with the first index to train a machine learning (ML) model.

12. The computer system according to claim 11, wherein the self-navigation data is used to train an occupancy network of the self, and the occupancy network is used to predict or sense the surrounding environment of the self.

13. The computer system according to claim 11, wherein the data further includes at least one other data element associated with all of the plurality of indices, and the executable instructions, when executed by the processor, further cause the computer system to: determine that the at least one other data element is associated with all of the plurality of indices; and store the at least one other data element before the plurality of data rows in the data file.

14. The computer system according to claim 11, wherein the data file is a read-only data file.

15. The computer system according to claim 11, wherein the executable instructions, when executed by the processor, cause the computer system to: store the plurality of data elements in the data file according to a plurality of columns, each column representing a corresponding data field; and arrange the plurality of columns in an increasing order of data sizes associated with the plurality of data columns.

16. The computer system according to claim 11, wherein the plurality of indices includes a plurality of timestamps or time values.

17. The computer system according to claim 11, wherein the plurality of data elements includes a plurality of tensors, and each data row stores one or more tensors associated with the corresponding index.

18. The computer system according to claim 17, wherein the executable instructions, when executed by the processor, further cause the computer system to perform at least one of the following: transpose at least a first subset of the tensors before storing the at least a first subset of the tensors in the data file; or encrypt at least a second subset of the tensors before storing the at least a second subset of the tensors in the data file.

19. The computer system according to claim 1, wherein when accessing the at least one data element associated with the first index, the executable instructions, when executed by the processor, cause the computer system to jump from a second memory location associated with the header to the first memory location when determining the first memory location of the row corresponding to the first timestamp.

20. A non-transitory computer-readable medium having computer code instructions stored thereon, the computer code instructions, when executed by a processor, cause the processor to: obtain self-navigating data including a plurality of data elements associated with a plurality of indexes, each index being associated with one or more data elements, the self-navigating data being determined based on data captured by at least one sensor of the self; Generate a data file storing the multiple data elements according to a predefined file format, the predefined file format including a header and multiple data lines, each data line corresponding to a respective index and storing the one or more data elements associated with the respective index, and for each line, the header including an association between the respective index and a corresponding data offset indicating the memory location of the data line; Use an indication of a first index and the header to determine a first memory location of a data line corresponding to the first index; Use the first memory location to access at least one data element associated with the first index; And Use the at least one data element associated with the first index to train a machine learning (ML) model.

Images