Extended learning model for autonomous civil engineering vehicles

Abstract

Disclosed are a system and method for using an extended learning model for autonomous civil engineering vehicles. The method may include receiving a second set of sensor data, generating a first compressed vector from the second set of sensor data by processing the second set of sensor data with a first machine learning model at least partially, and selecting an action to be performed by the vehicle by processing the first compressed vector with a second machine learning model at least partially. The method may further include taking one or more samples of sensor data from the first set of sensor data, generating a second compressed vector by fine-tuning the first machine learning model by processing one or more samples of sensor data at least partially, and fine-tuning the second machine learning model by processing the second compressed vector at least partially.

Description

Technical Field

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 500,227, filed May 4, 2023, which is hereby incorporated by reference in its entirety.

Background Art

[0002] Earthmoving vehicles (EMVs) are heavy machines designed to move large amounts of earth, rock, soil, or rubble during construction, mining, agriculture, or any other earthmoving operation. These vehicles are designed to perform various tasks such as excavation, grading, leveling, hauling, and demolition. For example, an excavator can be used to dig earth and other materials out of the ground. A backhoe loader is similar to an excavator but is smaller and has an adjustable shovel at the front and a bucket at the back for more precise digging. A bulldozer can be used to move earth over a large area of land to level it. These EMVs are typically controlled by human operators, are dangerous, and can cause injuries due to accidents. Furthermore, using human operators can be costly and may limit the operational time of EMVs.

Summary of the Invention

[0003] The present disclosure provides systems and methods related to an extended learning model for autonomous EMVs that can improve the operation of EMVs. The systems and methods described herein can help reduce or even eliminate injuries caused at construction sites when using EMVs. The systems and methods described herein can also help automate the process of using EMVs at construction sites. The systems and methods described herein can further help reduce the time required for training and retraining machine learning models of EMVs operated by artificial intelligence.

[0004] One embodiment is a method for autonomous operation of a vehicle, comprising: (a) maintaining a first set of sensor data in a computer data store; and (b) executing a first set of instructions and a second set of instructions in parallel by one or more processors until a convergence condition is reached, wherein the first set of instructions (1) receives a second set of sensor data, wherein the second set of sensor data is not included in the first set of sensor data; and (2) processes the second set of sensor data at least partially with a first machine learning model to obtain a first compressed vector from the second set of sensor data. A method comprising (i) generating a first compressed vector and (3) selecting an action to be performed by the vehicle by processing the first compressed vector with a second machine learning model at least partially, and (ii) executing a second set of instructions which includes (1) taking one or more samples of sensor data from a first set of sensor data and (2) generating a second compressed vector by fine-tuning the first machine learning model by processing one or more samples of sensor data at least partially, and (3) fine-tuning the second machine learning model by processing the second compressed vector at least partially.

[0005] In some embodiments, the first machine learning model includes a first set of model weights, and the second machine learning model includes a second set of model weights.

[0006] In some embodiments, the fine-tuning further includes changing a first set of weights for the model and / or a second set of weights for the model based on the results of the above action.

[0007] In some embodiments, the vehicle includes a civil engineering vehicle or heavy machinery.

[0008] In some embodiments, the civil engineering vehicle or heavy machinery includes earth movers, bulldozers, backhoes, excavators, tractors, snowmobiles, excavators, cranes, forklifts, boring machines, mowing machines, compaction machines, drilling machines, pile drivers, street sweepers, snowplows, aerial work platforms, or dump trucks.

[0009] In some embodiments, a first set of instructions is executed on a first thread, and a second set of instructions is executed on a second thread.

[0010] In some embodiments, the method further includes, prior to (b), (i) training the first machine learning model with a first set of sensor data using one or more processors, and (ii) training the second machine learning model at least partially with the first compressed vector generated by the first machine learning model using one or more processors.

[0011] In some embodiments, the method further includes training the second machine learning model on the first set of sensor data using one or more processors.

[0012] In some embodiments, the method further includes compressing the first set of sensor data by sampling before (b).

[0013] In some embodiments, the method further includes compressing a second set of sensor data by sampling before (2).

[0014] In some embodiments, the fine-tuning further includes calculating the difference between a first set of sensor data and a second set of sensor data.

[0015] In some embodiments, the first set of sensor data includes light detection and ranging (LIDAR) data, GPS data, vehicle status data, or a combination thereof, and the second set of sensor data includes LIDAR data, GPS data, vehicle status data, or a combination thereof.

[0016] In some embodiments, the vehicle status data includes location data or operation data.

[0017] In some embodiments, the position data is associated with a part of the vehicle.

[0018] In some embodiments, the position data is associated with the angle or orientation of the vehicle component.

[0019] In some embodiments, the vehicle components include arms, blades, or tools. In some embodiments, the tools include excavation buckets, hammers, hydraulic thumbs, couplers, crushers, compactors, grading buckets, demolition grapples, and tiltrotators.

[0020] In some embodiments, motion data is related to speed or acceleration. In some embodiments, motion data is the speed of the vehicle. In some embodiments, motion data is the acceleration of the vehicle. In some embodiments, motion data is related to a part of the vehicle. In some embodiments, motion data is related to speed or acceleration.

[0021] In some embodiments, in (b)(ii)(1), one or more samples of the sensor data are sampled from a memory buffer.

[0022] In some embodiments, the first or second compression vector is a learned representation, encoding, or embedding.

[0023] In some embodiments, the action is selected from a finite set of actions.

[0024] In some embodiments, the finite set of actions includes movement of the vehicle, operation of a part of the vehicle, increasing the output of the vehicle, decreasing the output of the vehicle, and / or reversing the previous operation of the vehicle.

[0025] In some embodiments, the movement of the vehicle includes movement of the vehicle forward, backward, or laterally.

[0026] In some embodiments, the movement of a part of the vehicle includes movement of the arm of the vehicle upward, downward, forward, or laterally.

[0027] In some embodiments, the movement of a part of the vehicle includes movement of a tool connected to the arm of the vehicle outward or inward.

[0028] In some embodiments, the convergence condition is associated with some actions performed by the vehicle.

[0029] In some embodiments, the convergence condition is further associated with the performance of the vehicle measured by a reward function.

[0030] In some embodiments, the completion of the some actions includes a task.

[0031] In some embodiments, the first set of instructions and the second set of instructions are configured to be executed in parallel or sequentially.

[0032] Another embodiment is a system for autonomously operating a vehicle, comprising one or more sensors configured to output a first set of sensor data and a second set of sensor data, wherein the second set of sensor data comprises one or more sensors not included in the first set of sensor data, a computer data store configured to store the first set of sensor data, and one or more processors, wherein the one or more processors are configured to execute a first set of instructions and a second set of instructions in parallel until a convergence condition is reached, wherein the first set of instructions comprises (1) receiving the second set of sensor data and (2) at least partially processing the second set of sensor data by a first machine learning model (b) a system comprising: (a) generating a first compressed vector from a second set of sensor data by processing; (b) selecting an action to be performed by the vehicle by processing the first compressed vector at least partially with a second machine learning model; and (c) a second set of instructions comprising: (a) generating a second compressed vector by taking one or more samples of sensor data from the first set of sensor data; (b) generating a second compressed vector by fine-tuning the first machine learning model by processing one or more samples of sensor data at least partially; and (c) fine-tuning the second machine learning model by processing the second compressed vector at least partially.

[0033] In some embodiments, the first machine learning model includes a first set of model weights, and the second machine learning model includes a second set of model weights.

[0034] In some embodiments, the second set of instructions further includes changing the first set of weights for the model and / or the second set of weights for the model based on the result of the Action.

[0035] In some embodiments, the vehicle includes a civil engineering vehicle or heavy machinery.

[0036] In some embodiments, the civil engineering vehicle or heavy machinery includes earth movers, bulldozers, backhoes, excavators, tractors, snowmobiles, excavators, cranes, forklifts, boring machines, mowing machines, compaction machines, drilling machines, pile drivers, street sweepers, snowplows, aerial work platforms, or dump trucks.

[0037] In some embodiments, a first set of instructions is executed on a first thread, and a second set of instructions is executed on a second thread.

[0038] In some embodiments, one or more processors are configured to execute a third set of instructions before a first set of instructions and a second set of instructions, the third set of instructions including (i) training a first machine learning model with a first set of sensor data, and (ii) training a second machine learning model with a first compressed vector generated by the first machine learning model, at least in part.

[0039] In some embodiments, the third set of instructions further includes training the second machine learning model with the first set of sensor data.

[0040] In some embodiments, the third set of instructions further includes compressing the first set of sensor data by sampling the first set of sensor data.

[0041] In some embodiments, the first set of instructions further includes compressing a second set of sensor data by sampling the second set of sensor data before (a)(2).

[0042] In some embodiments, the second set of instructions further includes calculating the difference between the first set of sensor data and the second set of sensor data.

[0043] In some embodiments, the first set of sensor data includes light detection and ranging (LIDAR) data, GPS data, vehicle status data, or a combination thereof, and the second set of sensor data includes LIDAR data, GPS data, vehicle status data, or a combination thereof.

[0044] In some embodiments, the vehicle state data includes position data or motion data. In some embodiments, the position data is associated with a part of the vehicle. In some embodiments, the position data is associated with the angle or orientation of a part of the vehicle.

[0045] In some embodiments, the vehicle components include arms, blades, or tools. In some embodiments, the tools include excavation buckets, hammers, hydraulic thumbs, couplers, crushers, compactors, grading buckets, demolition grapples, and tiltrotators.

[0046] In some embodiments, motion data is related to speed or acceleration. In some embodiments, motion data is the speed of the vehicle. In some embodiments, motion data is the acceleration of the vehicle. In some embodiments, motion data is related to a part of the vehicle. In some embodiments, motion data is related to speed or acceleration.

[0047] In some embodiments, one or more samples of the sensor data are sampled from the computer data store.

[0048] In some embodiments, the first or second compression vector is a learned representation, encoding, or embedding.

[0049] In some embodiments, the above action is selected from a finite set of actions.

[0050] In some embodiments, the finite set of actions includes moving the vehicle, activating a part of the vehicle, increasing the output of the vehicle, decreasing the output of the vehicle, and / or reversing the previous operation of the vehicle. In some embodiments, the movement of the vehicle includes moving the vehicle forward, backward, or laterally.

[0051] In some embodiments, the movement of a portion of the vehicle includes upward, downward, forward, or lateral movement of the vehicle's arms.

[0052] In some embodiments, some of the movement of the vehicle includes outward or inward movement of a tool connected to an arm of the vehicle.

[0053] In some embodiments, the convergence conditions are associated with several actions performed by the vehicle.

[0054] In some embodiments, the completion of several actions includes a task.

[0055] Additional aspects and advantages of the Disclosure will be readily apparent to those skilled in the art from the following detailed description. Only exemplary embodiments of the Disclosure are shown and described in the following detailed description. It will be recognized that various other embodiments are possible, and some of their details are modifiable in various obvious ways, all of which can be done without departing from the Disclosure. Therefore, the drawings and description should be considered illustrative and not restrictive.

[0056] Embedding by reference All publications, patents, and patent applications described herein are incorporated by reference to the same extent as each individual publication, patent, or patent application is specifically and individually indicated as being incorporated by reference. This specification is intended to supersede and / or take precedence over any publications, patents, or patent applications incorporated by reference to the extent that they conflict with any disclosures contained herein. [Brief explanation of the drawing]

[0057] Novel features of the present invention are specified in the appended claims. The features and advantages of the present invention will be better understood by referring to the following detailed description which describes exemplary embodiments. The principles of the present invention are utilized in these embodiments, and the accompanying drawings (also referred to herein as "Figure" and "FIG.") are as follows.

[0058] [Figure 1] This figure shows a non-limiting example of a system and data flow that includes some of the components of EMV. [Figure 2] This figure shows a non-restrictive example of a flowchart for manipulating autonomous EMV using augmented learning. [Figure 3] This figure shows a non-limiting example of a computing device according to several embodiments, in this case a device having one or more processors, memory, storage devices, and network interfaces. [Figure 4] This figure shows a non-limiting example of a web / mobile application delivery system according to several embodiments, in this case a system that provides a browser-based and / or native mobile user interface. [Figure 5]This figure shows a non-limiting example of a cloud-based web / mobile application delivery system according to several embodiments, in which case the system comprises elastically load-distributed, auto-scaling web server and application server resources, as well as a synchronously replicated database. [Modes for carrying out the invention]

[0059] overview This disclosure includes systems and methods that offer advantages over conventional EMV and also provide a localization control algorithm for EMV. Machines or devices running on machine learning models are typically not equipped to retrain or update their models with new information, because retraining a model can be time-consuming and require significant processing power. Actions performed by machines are not efficient or performant, because the control algorithms are based on localization or freezing models, which can lead to inefficiency and performance degradation.

[0060] The systems and methods disclosed herein can improve and / or fine-tune machine learning models by employing augmented learning, thereby increasing the efficiency and / or effectiveness of the machine. The machine may be deployed with two machine learning models, one model of the external environment and the other model of behavior for predicting or generating the next action. Both models stored in the EMV can be fine-tuned based on the external environment. Augmented learning of the models can result in higher efficiency and performance, which can lead to reduced project time and cost savings.

[0061] Machine learning methodology The systems, methods, computer-readable media, and techniques described herein may employ machine learning. In some cases, machine learning may generally involve identifying and recognizing patterns in existing data to facilitate predictions for subsequent data. Machine learning may include machine learning models (which may include, for example, machine learning algorithms). Whether inherently analytical or statistical, machine learning may provide deductive or hypothetical reasoning based on real or simulated data.

[0062] A machine learning model may be a trained model. Machine learning (ML) may include one or more supervised, semi-supervised, self-supervised, or unsupervised machine learning techniques. For example, an ML model may be a trained model that is trained through supervised learning (for example, various parameters are determined as weights or multipliers).

[0063] Training a machine learning model may, in some cases, involve selecting one or more initialized data models to be trained using a training dataset. The selected initialized data models may include any kind of untrained machine learning model for supervised, semi-supervised, self-supervised, or unsupervised machine learning. The selected initialized data models may be identified based on inputs (e.g., user inputs) that identify relevant parameters to be used as predictor variables or other variables to be used as potential explanatory variables. For example, predictor variables or other variables may be variables that the model is trained to predict, and explanatory variables may be variables used to predict or explain the difference between the predictor / other variables. For example, the selected initialized data models may be identified to produce an output (e.g., a prediction) based on the inputs. Conditions for training a machine learning model from the selected initialized data models may similarly be selected, such as limits on the complexity of the machine learning model or limits on improving the machine learning model beyond a certain point. The machine learning model may be trained using the training dataset (e.g., via a computer system such as a server). In some cases, a first subset of the training dataset may be selected to train the machine learning model. The selected initialized data model may then be trained on a first subset of the training dataset using appropriate machine learning techniques, based on the type of machine learning model selected and any conditions identified for training the machine learning model. In some cases, due to the processing power requirements for training the machine learning model, the selected initialized data model may be trained using additional computing resources (e.g., cloud computing resources). Such training may continue indefinitely, in some cases, until at least one aspect of the machine learning model is validated and meets the selection criteria for use as a predictive model.

[0064] In some cases, one or more aspects of a machine learning model may be validated using a second subset of the training dataset (e.g., separate from a first subset of the training dataset) to determine the accuracy and robustness of the machine learning model. Such validation may involve applying the machine learning model to the second subset of the training dataset to make predictions derived from the second subset of training data. The machine learning model may then be evaluated to determine whether its performance is sufficient based on the obtained predictions. The sufficiency criteria applied to the machine learning model may vary depending on the size of the training dataset available for training, the performance of the trained model in previous iterations, or the performance requirements specified by the user. If the machine learning model does not achieve sufficient performance, additional training may be performed. Additional model improvements may include refining the machine learning model or retraining it on a different first subset of the training dataset, after which the new machine learning model may be validated and evaluated again. When the machine learning model achieves sufficient performance, in some cases, the machine learning model may be saved for current or future use. A machine learning model may be stored as a set of parameter values ​​or weights for analysis of further inputs (e.g., further relevant parameters used as further predictor variables, further explanatory variables, further user interaction data, etc.), which may also, in some examples, include analysis logic or representation of model validity. In some cases, multiple machine learning models may be stored to generate predictions under various sets of input data conditions. In some embodiments, machine learning models may be stored in a database (e.g., associated with a server).

[0065] ML may include one or more of the following: regression analysis, regularization, classification, dimensionality reduction, set learning, meta-learning, associative rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning. ML may include k-means, k-means clustering, k-nearest neighbors, learned vector quantization, linear regression, nonlinear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principal component regression, minimum absolute contraction and selection operations, minimum angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal component analysis, principal coordinate analysis, projection tracking, summon mapping, t-distribution stochastic nearest neighbor embedding, ADA boosting, boosting, gradient boosting, bootstrap aggregation, ensemble mean, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stack generalization, Bayesian networks. This may include, but is not limited to, WORK, Bayesian belief networks, Naive Bayes, Gaussian Naive Bayes, Multinomial Naive Bayes, Hidden Markov Models, Hidden Markov Models, Support Vector Machines, Encoders, Decoders, Autoencoders, Stacked Autoencoders, Perceptrons, Multilayer Perceptrons, Artificial Neural Networks, Feedforward Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, Long- and Short-Term Memory, Deep Belief Networks, Deep Boltzmann Machines, Deep Convolutional Neural Networks, Deep Recurrent Neural Networks, or Generative Adversarial Networks.

[0066] Reinforcement learning is a type of machine learning in which an agent learns to make decisions in a given environment in order to maximize a cumulative reward signal. In reinforcement learning, the agent interacts with the environment and takes action based on its current state. The environment then responds with a reward signal that reflects the quality of the action taken by the agent. The agent's goal is to learn a policy or strategy that maximizes the cumulative reward over time. The reinforcement learning framework involves three main components: the agent, the environment, and the reward signal. The agent takes action in the environment, and the environment provides feedback in the form of rewards or penalties. The agent uses this feedback to adjust its actions and improve its performance over time. Reinforcement learning algorithms are typically trial-and-error and can be computationally intensive.

[0067] Fine-tuning involves taking a pre-trained machine learning model and further training it on a new task or dataset. In this process, the pre-trained model's weights are updated based on the new data or task. Fine-tuning can be useful when a model trained on a large dataset is used as a starting point for solving a similar task with a smaller dataset. The pre-trained model has already learned useful features applicable to the new task, saving time and resources compared to training a new model from scratch. The fine-tuning process involves freezing some layers of the pre-trained model and retraining only the final layer, or adding new layers to adapt the model to the new task. The amount of fine-tuning required depends on the original pre-training task, the size and quality of the new dataset, and the similarity of the new task to the complexity of the pre-trained model.

[0068] system Figure 1 shows a non-limiting example of a system 100 and data flow, including some of the components of EMV. System 100 includes one or more sensors 102, an EMV memory 114 mounted on the EMV, and one or more components / tools 116 of the EMV. While certain elements of system 100 are shown in Figure 1, embodiments are not limited to these, and additional elements (e.g., one or more processors) can be added or removed, as will be recognized by those skilled in the art. Furthermore, the arrows indicating the data flow are drawn for brevity and clarity, and those skilled in the art will recognize that the sensors first provide data to the processor before the processor stores the data in memory. Similarly, those skilled in the art will recognize that, in order to train a machine learning model, the sensor data needs to be read from memory, processed (or processed before the sensor data is stored in memory), and then the machine learning model needs to be trained or fine-tuned with the sensor data.

[0069] In some embodiments, the vehicle includes an EMV or heavy machinery. In some embodiments, the civil engineering vehicle or heavy machinery includes earth movers, bulldozers, backhoes, excavators, tractors, snowmobiles, excavators, cranes, forklifts, boring machines, mowing machines, compaction machines, drilling machines, pile drivers, street sweepers, snowplows, aerial work platforms, or dump trucks.

[0070] Sensor 102 may include one or more types of sensors, and one or more sensors of the same type. Sensor types may include light detection and ranging (LIDAR) sensors, global positioning system (GPS) sensors, cameras, radar sensors, inertial measurement unit (IMU) sensors, vehicle sensors, tire pressure sensors, temperature sensors, battery level sensors, fuel level sensors, wind speed sensors, infrared sensors, and the like. Each of the sensors 102 described herein may provide sensor data to memory 114. The sensors may be located around the EMV and / or in the environment in which the EMV is operating (e.g., a construction site). Each sensor may communicate with the EMV via a wired or wireless connection.

[0071] Sensor 102 can generate and output sensor data including first sensor data (or first set of sensor data) 104 and second sensor data (or second set of sensor data) 106. Memory 114 can store the output of sensor 102. The first set of sensor data may include offline data, which may include EMV data such as vehicle type, past actions, operating time, and other driving data. The offline data may also include EMV environmental data such as weather, time, and material. The offline data may include data previously detected and stored by the sensor. Both the first set of sensor data and the second set of sensor data may include vehicle status data.

[0072] A first set of sensor data may include offline data used to train and fine-tune the model. A second set of sensor data may include additional data received and / or detected and / or detected after the ML has been initially trained. The second set of sensor data, or a subset thereof, may be used to fine-tune the model as described herein.

[0073] In some embodiments, vehicle state data includes location data or motion data. In some embodiments, location data is associated with a part of the vehicle. For example, location data may include a history of the part's position at various points in time. Location data may be stored at intervals (e.g., every second) or whenever the part moves.

[0074] In some embodiments, the position data is associated with the angle or orientation of a vehicle part. For example, the angle may be measured relative to the x, y, or z axis of the EMV. In some embodiments, the angle may be measured relative to the x-axis of the ground. The position data may also include the history of the part's angle and / or orientation, as discussed above.

[0075] In some embodiments, motion data is related to velocity or acceleration. In some embodiments, motion data includes the velocity of the EMV (e.g., how fast the EMV is moving along the terrain) or the velocity of a component of the EMV (e.g., how fast the arm is moving). In some embodiments, motion data includes the acceleration and / or deceleration of the EMV or a component of the EMV. In some embodiments, motion data is associated with a component of the vehicle. Motion data may also include how fast individual components of the EMV are moving (e.g., arm, blade, and / or tool).

[0076] Memory 114 may also store a first machine learning model 110 and a second machine learning model 112. The first machine learning model 110 may include a world model, and the second machine learning model 112 may include a behavior model. The first machine learning model 110 and the second machine learning model 112 will be described in more detail later.

[0077] The EMV component / tool ​​116 may include one or more different components and / or tools mounted on the EMV. In some embodiments, the component of the vehicle includes an arm, blade, or tool. In some embodiments, the tool includes a digging bucket, hammer, hydraulic thumb, coupler, crusher, compactor, grading bucket, demolition grapple, and tiltrotator.

[0078] Since the parts and / or tools are driven based on the output of the second machine learning model 112, the parts / tools 116 may affect the EMV environment. The changed environment may be detected by the sensor 102 as described above, and the data may be stored in memory 114 as additional second sensor data 106. For example, as the digger excavates the ground to a certain depth, a depth sensor may detect that the depth has changed, and this information may be provided to memory 114 in the form of second sensor data 106. The first machine learning algorithm 110 and the second machine learning algorithm 112 can then be fine-tuned using the additional second sensor data 106.

[0079] Extended learning method using EMV Figure 2 shows a non-limiting example of a flowchart 200 for operating an autonomous EMV by augmented learning. The flowchart 200 includes a series of steps that can be performed sequentially and / or in parallel. Each step of the flowchart 200 may be performed by one or more processors, which may be located within the EMV and / or away from the EMV (e.g., in one or more servers). Those skilled in the art will recognize that in some embodiments the steps of the flowchart 200 may include additional steps and / or some steps may be eliminationable.

[0080] In some embodiments, the EMV may receive a first set of sensor data from a computer data store (e.g., memory 114) (step 202). The EMV data or sensor data may include a first set of sensor data (e.g., first sensor data 104) and a second set of sensor data (e.g., second sensor data 106). The first set of sensor data may include offline data, which may include EMV data such as vehicle type, past actions, operating hours, and other driving data. The offline data may also include data about the EMV environment, such as weather, time, and material. The offline data may include data previously detected and stored by the sensor. The first set of sensor data may include data used to train a model.

[0081] The second set of sensor data may include additional data received and / or detected and / or sensed after the machine learning model has been trained. A second set of sensor data, or a subset thereof, may be used to fine-tune the model as described herein.

[0082] In some embodiments, the method further includes compressing the first set of sensor data by sampling. For example, memory may store a subset of the sensor data rather than the entire first set of sensor data. In some embodiments, the first set of sensor data may be sampled by using a sampling function. For example, if GPS data indicates that the EMV did not move the previous day, and the GPS sensor is configured to detect the location of the EMV every minute, memory does not need to contain 60 minutes / hour × 24 hours / day = 1440 records of GPS data showing the same thing. Thus, the sampling function can reduce the amount of redundancy stored in memory. In some embodiments, the sampling function may include randomly selecting data to be stored in memory.

[0083] EMV initial training In some embodiments, the method further includes training (or initial training) a first machine learning model (e.g., a first ML model 110) on a first set of sensor data using one or more processors (step 204). The training of the first machine learning model may occur before the first and second threads of the instruction are executed. The first machine learning model may include a world model trained on a first set of sensor data, or on all of the offline data from a memory buffer. The first machine learning model may be trained to predict or output a first compressed vector. In some embodiments, the first compressed vector is or includes a learned representation, encoding, or embedding.

[0084] A learned representation includes a set of features or properties of input data learned through the training process by a neural network or other machine learning model. Encoding includes the process of transforming data from one form to another form suitable for a machine learning model. Embedding includes mapping discrete and categorical variables, such as words or categories, into a continuous vector space that is usually much lower dimensional than the original space.

[0085] In some embodiments, the method further includes training (or initial training) a second machine learning model (e.g., a second ML model 112) at least partially with a first compressed vector generated by a first machine learning model (step 206) using one or more processors. In some embodiments, the second machine learning model may include an action model trained with a first compressed vector output by a world model (e.g., a first machine learning model). In some embodiments, the second machine learning model may be trained to output actions for EMVs.

[0086] In some embodiments, the method further includes training a second machine learning model on a first set of sensor data using one or more processors.

[0087] In a neural network, for example, model parameters include the network's weights and biases. In some embodiments, a first machine learning model includes a first set of weights (or model weights) and biases, and a second machine learning model includes a second set of weights (or model weights) and biases. During the training process, the model parameters may be initialized randomly and then iteratively updated using an optimization algorithm such as gradient descent. During training, the models and biases may be modified to find values ​​of model parameters that minimize a chosen loss function, which measures the difference between the model's predicted output and the ground truth or true state of the training examples, summed across all training examples, which may include measured actual outputs that are either the same as or different from the predicted output of the second machine learning model. In some embodiments, the goal is to minimize the loss function to improve the model. Once the model is trained and the optimal values ​​for the model parameters are found, the model may be used to make predictions about new, unseen data. The first set of weights may be modified during training based at least in part on a first set of sensor data. A second set of weights can be modified during training based at least partially on the first compressed vector and / or the first set of sensor data.

[0088] EMV Action Decision and Augmentation Learning In some embodiments, methods for autonomous operation of a vehicle are disclosed. The method includes maintaining a first set of sensor data in a computer data store. The method further includes executing a first set of instructions and a second set of instructions in parallel by one or more processors until a convergence condition is reached. The first set of instructions may include (1) receiving a second set of sensor data, the second set of sensor data not included in the first set of sensor data (step 212), (2) generating a first compressed vector from the second set of sensor data by processing the second set of sensor data at least partially with a first machine learning model (step 214), and (3) selecting an action to be performed by the vehicle by processing the first compressed vector at least partially with a second machine learning model (step 216). A second set of instructions may include (1) taking one or more samples of sensor data from a first set of sensor data and / or a second set of sensor data (step 222), (2) generating a second compressed vector by fine-tuning the first machine learning model by processing one or more samples of sensor data at least partially (step 224), and (3) fine-tuning the second machine learning model by processing the second compressed vector at least partially (step 226).

[0089] In some embodiments, the first thread 210 may include a series of steps (e.g., steps 212, 214, and 216) in which actions are determined by a second machine learning model. The actions may be actuated or performed by EMV, or by an EMV component or tool (e.g., component / tool ​​116).

[0090] The step 212 of receiving a second set of sensor data includes reading the second set of sensor data from memory. The second set of sensor data may include inputs provided to the first machine learning model. The second set of sensor data may include data about the environment, and / or EMV and / or its components / tools, as discussed herein.

[0091] Step 214, which generates a first compressed vector from a second set of sensor data, includes a first machine learning model for generating the first compressed vector. In some embodiments, the method further includes compressing the second set of sensor data by sampling. Similar to the sampling of the first set of sensor data described above, the second set of sensor data may be sampled and compressed to save space in memory.

[0092] Step 216, which involves selecting an action to be performed by EMV, includes generating an action based at least in part on a second machine learning model. The input to the second machine learning model may include a first compressed vector, and the output may include a learned representation, an embedding or coding of an action to be performed by EMV or its components / tools.

[0093] Once an output is provided (and optionally after the action is completed), thread 210 may repeat by returning to process 212. Thread 210 may repeat until the convergence condition is reached.

[0094] In some embodiments, the second thread 220 may include a series of steps (e.g., steps 222, 224, and 226) that may be performed to fine-tune the first and second machine learning models. This fine-tuning makes the models more accurate and adaptable to the surrounding environment. Thus, EMV can perform its assigned tasks more accurately and efficiently.

[0095] Step 222, which involves taking one or more samples of sensor data from a first set of sensor data, may include using a sampling function. In some embodiments, one or more samples of sensor data are sampled from memory or a memory buffer.

[0096] Step 224 for fine-tuning the first machine learning model may include fine-tuning the first machine learning model by processing samples of sensor data from a first set of sensor data. The fine-tuned first machine learning model can output a second compressed vector.

[0097] Step 226 for fine-tuning the second machine learning model may include receiving a second compressed vector as input for fine-tuning the second machine learning model.

[0098] Once process 226 is completed, thread 220 can be repeated by an instruction to repeat processes 222, 224, and 226 until a convergence condition is reached.

[0099] In some embodiments, fine-tuning may further include changing a first set of model weights and / or a second set of model weights based on the outcome of an action. In some embodiments, fine-tuning may further include calculating the difference between a first set of sensor data and a second set of sensor data. In some embodiments, the model weights, or a subset of the model weights, are tunable to more accurately generate a first and a second compression vector.

[0100] In some embodiments, the action is selected from a finite set of actions. In some embodiments, the finite set of actions includes moving the vehicle, activating a part of the vehicle, increasing the output of the vehicle, decreasing the output of the vehicle, and / or reversing the previous operation of the vehicle. In some embodiments, the vehicle movement includes forward, backward, or lateral movement of the vehicle. In some embodiments, the movement of a part of the vehicle includes upward, downward, forward, or lateral movement of the vehicle's arm. In some embodiments, the movement of a part of the vehicle includes outward or inward movement of a tool connected to the vehicle's arm.

[0101] In some embodiments, the convergence condition is associated with several actions performed by the vehicle. In some embodiments, the completion of several actions constitutes a task. In some embodiments, the convergence condition is further associated with the vehicle's performance as measured by a reward function. The reward function may instruct an agent (e.g., one or more machine learning models discussed herein) to attempt to achieve a particular objective using rewards and penalties. For example, a higher reward score obtained by receiving a reward may indicate that the objective is closer to being achieved, while a lower reward score obtained by receiving a penalty may indicate that the objective is far from being achieved. The convergence condition may be set by the user or predetermined. For example, in a construction site, if a digger is tasked with digging in a certain area, the convergence condition may include the completion of digging in that area. One or more sensors can be used to determine whether convergence has been reached.

[0102] Computing systems Referring to Figure 3, a block diagram is shown depicting an exemplary machine (e.g., EMV) including a computer system 300 (e.g., a processing or computing system), within which a set of instructions is executable to cause a device to perform or execute one or more of the embodiments and / or methodologies for static code scheduling of the present disclosure. The components in Figure 3 are illustrative and do not limit the scope of use or functionality of any hardware, software, embedded logic components, or combinations of two or more such components that implement a particular embodiment.

[0103] The computer system 300 may include one or more processors 301, memory 303, and storage devices 308 that communicate with each other and with other components via a bus 340. The bus 340 may also connect a display device 332, one or more input devices 333 (which may include, for example, a keypad, keyboard, mouse, stylus, etc.), one or more output devices 334, one or more storage devices 335, and various tangible storage media 336. All of these elements may interface directly with the bus 340 or through one or more interfaces or adapters. For example, the various tangible storage media 336 may interface with the bus 340 via a storage media interface 326. The computer system 300 may have any suitable physical form, including but not limited to one or more integrated circuits (ICs), printed circuit boards (PCBs), portable handheld devices (such as mobile phones or PDAs), laptops or notebook computers, distributed computer systems, computing grids, or servers.

[0104] The computer system 300 includes one or more processors 301 that perform functions (e.g., a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), or a quantum processing unit (QPU)). The processors 301 optionally include a cache memory device 302 for temporary local storage of instructions, data, or computer addresses. The processors 301 are configured to assist in the execution of computer-readable instructions. The computer system 300 may provide functionality to the components depicted in Figure 3 as a result of the processors 301 executing non-temporary processor-executable instructions embodied in one or more tangible computer-readable storage media, such as memory 303, storage device 308, storage device 335, and / or storage medium 336. The computer-readable media may store software that implements a particular embodiment, and the processors 301 may execute such software. Memory 303 may read software from one or more other computer-readable media (such as mass storage devices 335, 336) or from one or more other sources via a suitable interface, such as a network interface 320. The software may cause the processor 301 to perform one or more processes, or one or more steps of one or more processes, as described or illustrated herein. Performing such processes or steps may include defining a data structure stored in memory 303 and modifying said data structure as instructed by the software.

[0105] Memory 303 may include, but is not limited to, various components (e.g., machine-readable media) including random access memory components (e.g., RAM 304) (e.g., static RAM (SRAM), dynamic RAM (DRAM), ferroelectric random access memory (FRAM®), phase-change random access memory (PRAM), etc.), read-only memory components (e.g., ROM 305), and any combination thereof. ROM 305 may act to communicate data and instructions unidirectionally to the processor 301, and RAM 304 may act to communicate data and instructions bidirectionally with the processor 301. ROM 305 and RAM 304 may include any suitable tangible computer-readable media described below. In one example, a basic input / output system 306 (BIOS) containing basic routines useful for transferring information between elements within the computer system 300 during startup, etc., may be stored in memory 303.

[0106] The fixed storage device 308 is optionally bidirectionally connected to the processor 301 via a storage control device 307. The fixed storage device 308 may also include any suitable tangible computer-readable medium described herein, which provides additional data storage capacity. The storage device 308 may be used to store the operating system 309, executable files 310, data 311, applications 312 (application programs), etc. The storage device 308 may also include an optical disk drive, a solid memory device (e.g., a flash-based system), or any combination of the above. The information in the storage device 308 may, where appropriate, be incorporated as virtual memory in memory 303.

[0107] In one example, the storage device 335 may be detachably interfaced with the computer system 300 (for example, via an external port connector (not shown)) via a storage device interface 325. In particular, the storage device 335 and its associated machine-readable medium may provide the computer system 300 with machine-readable instructions, data structures, program modules, and / or other data as non-volatile and / or volatile storage devices. In one example, the software may reside entirely or partially in the machine-readable medium on the storage device 335. In another example, the software may reside entirely or partially in the processor 301.

[0108] Bus 340 connects a wide range of subsystems. Here, a reference to a bus may, where appropriate, encompass one or more digital signal lines that provide common functionality. Bus 340 may be any of several types of bus structures, including, but not limited to, memory buses, memory controllers, peripheral buses, local buses, and any combination thereof, using any of various bus architectures. For example, and without limitation, such architectures include the Industry Standard Architecture (ISA) bus, Expansion ISA (EISA) bus, Microchannel Architecture (MCA) bus, Video Electronics Standards Association Local Bus (VLB), Peripheral Interconnect (PCI) bus, PCI Express (PCI-X) bus, Accelerated Graphics Port (AGP) bus, Hypertransport (HTX) bus, Serial Advanced Technology Attachment (SATA) bus, and any combination thereof.

[0109] The computer system 300 also includes an input device 333. In one example, a user of the computer system 300 may input commands and / or other information to the computer system 300 via the input device 333. Examples of input devices 333 include, but are not limited to, alphanumeric input devices (e.g., keyboards), pointing devices (e.g., mice or touchpads), touchpads, touchscreens, multitouchscreens, joysticks, styluses, gamepads, audio input devices (e.g., microphones, voice response systems, etc.), optical scanners, video or still image capture devices (e.g., cameras), and any combination thereof. In some embodiments, the input device is Kinect, Leap Motion, etc. The input device 333 may interface with the bus 340 via one of various input interfaces 323 (e.g., input interface 323) including, but not limited to, serial, parallel, game port, USB, FIREWIRE®, THUNDERBOLT®, or any combination thereof.

[0110] In certain embodiments, when computer system 300 is connected to network 330, computer system 300 may communicate with other devices connected to network 330, specifically, mobile devices and enterprise systems, distributed computing systems, cloud storage systems, and the like. Communication to and from computer system 300 may be transmitted via network interface 320. For example, network interface 320 may receive incoming calls (such as requests or responses from other devices) from network 330 in the form of one or more packets (such as Internet Protocol (IP) packets), and computer system 300 may store the incoming calls in memory 303 for processing. Similarly, computer system 300 may store outgoing calls (such as requests or responses to other devices) in memory 303 in the form of one or more packets and communicate them from network interface 320 to network 330. Processor 301 may access these communication packets stored in memory 303 for processing.

[0111] Examples of network interface 320 include, but are not limited to, network interface cards, modems, and any combination thereof. Examples of network 330 or network segment 330 include, but are not limited to, distributed computing systems, cloud computing systems, wide area networks (WANs) (e.g., the Internet, enterprise networks), local area networks (LANs) (e.g., networks associated with offices, buildings, campuses, or other relatively small geographical spaces), telephone networks, direct connections between two computing devices, peer-to-peer networks, and any combination thereof. Networks such as network 330 may use wired and / or wireless communication modes. In general, any network topology can be used.

[0112] Information and data can be displayed via the display device 332. Examples of the display device 332 include, but are not limited to, cathode ray tubes (CRTs), liquid crystal displays (LCDs), thin-film transistor liquid crystal displays (TFT-LCDs), organic liquid crystal displays (OLEDs) such as passive-matrix OLEDs (PMOLEDs) or active-matrix OLEDs (AMOLEDs), plasma displays, and any combination thereof. The display device 332 can interface with other devices such as a processor 301, memory 303, fixed storage device 308, and input device 333 via bus 340. The display device 332 is connected to bus 340 via video interface 322, and the transport of data between the display device 332 and bus 340 can be controlled via graphics control 321. In some embodiments, the display device is a video projector. In some embodiments, the display device is a head-mounted display (HMD), such as a VR headset. In further embodiments, suitable VR headsets include, but are not limited to, HTC Vive, Oculus Rift, Samsung Gear VR, Microsoft HoloLens, Razer OSVR, FOVE VR, Zeiss VR One, Avegant Glyph, and Freefly VR headsets. In further embodiments, the display device is a combination of devices such as those disclosed herein.

[0113] In addition to the display device 332, the computer system 300 may include, but is not limited to, one or more other peripheral output devices 334, including, but not limited to, a voice speaker, a printer, a storage device, and any combination thereof. Such peripheral output devices may be connected to the bus 340 via an output interface 324. Examples of the output interface 324 include, but are not limited to, a serial port, a parallel connection, a USB port, a FIREWIRE® port, a THUNDERBOLT® port, and any combination thereof.

[0114] In addition to or instead of the above, the computer system 300 may provide functionality as a result of logic embedded in the hardware or otherwise embodied in the circuitry, which, in place of or in conjunction with software, may perform one or more processes or one or more steps of one or more processes described or illustrated herein. References to software in this disclosure may include logic, and references to logic may include software. Furthermore, references to computer-readable media may, as appropriate, include circuitry (such as an IC) storing software for execution, circuitry embodying logic for execution, or both. This disclosure encompasses hardware, software, or any suitable combination of both.

[0115] Those skilled in the art will understand that various exemplary logic blocks, modules, circuits, and algorithmic processes described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly demonstrate this hardware- and software compatibility, various exemplary components, blocks, modules, circuits, and processes have been generally described above in terms of their functionality.

[0116] Various exemplary logic blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented by a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discretized gate or transistor logic, discretized hardware component, or any combination thereof, designed to perform the functions described herein. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may be implemented as a combination of computing devices, e.g., a DSP core, or any other such configuration, as a DSP and a microprocessor, multiple microprocessors, or a combination of one or more microprocessors.

[0117] The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by one or more processors, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. The exemplary storage medium is coupled to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and storage medium may reside within an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor and storage medium may reside as separate components in the user terminal.

[0118] Suitable computing devices, as described herein, include, in non-limiting examples, server computers, desktop computers, laptop computers, note computers, subnotebook computers, netbook computers, netpad computers, set-top computers, media streaming devices, handheld computers, internet devices, mobile smartphones, tablet computers, personal digital assistants, and means of communication. Those skilled in the art will also recognize that selected televisions, video players, and digital music players with optional computer network connectivity are also suitable for use in the systems described herein. In various embodiments, suitable tablet computers include tablet computers having booklet, slate, and convertible configurations known to those skilled in the art.

[0119] In some embodiments, a computing device includes an operating system configured to execute executable instructions. The operating system is, for example, software including programs and data that manages the device's hardware and provides services for running applications. A person skilled in the art will recognize that suitable server operating systems include, in non-limiting examples, FreeBSD, OpenBSD, NetBSD®, Linux®, Apple® Mac OS X® Server®, Oracle® Solaris®, Windows Server®, and Novell® NetWare®. A person skilled in the art will recognize that suitable personal computer operating systems include, in non-limiting examples, Microsoft® Windows®, Apple® Mac OS X®, UNIX®, and UNIX®-like operating systems such as GNU / Linux®. In some embodiments, the operating system is provided through cloud computing. Those skilled in the art will also recognize that suitable mobile smartphone operating systems include, but are not limited to, Nokia® Symbian® OS, Apple® iOS®, Research In Motion® BlackBerry OS®, Google® Android®, Microsoft® Windows Phone® OS, Microsoft® Windows Mobile® OS, Linux®, and Palm® WebOS®.

[0120] Non-temporary computer-readable storage medium In some embodiments, the platforms, systems, media, and methods disclosed herein include one or more non-temporary computer-readable storage media coded with programs that include instructions executable by the operating system of a networked computing device, which is optional. In further embodiments, the computer-readable storage media is a tangible component of the computing device. In even further embodiments, the computer-readable storage media is optionally removable from the computing device. In some embodiments, the computer-readable storage media includes, in non-limiting examples, CD-ROMs, DVDs, flash memory devices, solid memory, magnetic disk drives, magnetic tape drives, optical disk drives, cloud computing systems, and services, as well as distributed computing systems. In some cases, the programs and instructions are coded permanently, substantially permanently, semi-permanently, or non-temporarily on the medium.

[0121] Computer program In some embodiments, the platforms, systems, media, and methods disclosed herein include at least one computer program or use thereof. A computer program includes a sequence of instructions, executable by one or more processors of the CPU of a computing device, written to perform a particular task. Computer-readable instructions may be implemented as program modules, such as functions, objects, application programming interfaces (APIs), and computing data structures, which perform a task or implement a certain abstract data type. In light of the disclosures provided herein, those skilled in the art will recognize that computer programs may be written in various versions of various languages.

[0122] The functionality of computer-readable instructions may be combined or distributed as desired in various environments. In some embodiments, a computer program includes one sequence of instructions. In some embodiments, a computer program includes multiple sequences of instructions. In some embodiments, a computer program is provided from one location. In other embodiments, a computer program is provided from multiple locations. In various embodiments, a computer program includes one or more software modules. In various embodiments, a computer program includes, in part or in whole, one or more web applications, one or more mobile applications, one or more standalone applications, one or more web browser plugins, extensions, add-ins or add-ons, or a combination thereof.

[0123] Web application In some embodiments, a computer program includes a web application. Those skilled in the art will recognize, in light of the disclosures provided herein, that in various embodiments, a web application utilizes one or more software frameworks and one or more database systems. In some embodiments, a web application is written on a software framework such as Microsoft® .NET or Ruby on Rails (RoR). In some embodiments, a web application utilizes one or more database systems, including, in non-limiting examples, relational, non-relational, object-oriented, associative, XML, and document-oriented database systems. In further embodiments, suitable relational database systems include, in non-limiting examples, Microsoft® SQL Server, MySQL®, and Oracle®. Those skilled in the art will also recognize that, in various embodiments, a web application is written in one or more versions of one or more languages. A web application may be written in one or more markup languages, presentation-definition languages, client-side scripting languages, server-side coding languages, database query languages, or a combination thereof. In some embodiments, web applications are written to some extent in a markup language such as Hypertext Markup Language (HTML), Extensible Hypertext Markup Language (XHTML), or Extensible Markup Language (XML). In some embodiments, web applications are written to some extent in a presentation-definition language such as Cascading Style Sheets (CSS). In some embodiments, web applications are written to some extent in a client-side scripting language such as Asynchronous JavaScript and XML (AJAX), Flash® ActionScript, JavaScript, or Silverlight®.In some embodiments, the web application is written to some extent in a server-side coding language such as Active Server Pages (ASP), ColdFusion®, Perl, Java®, JavaServer Pages (JSP), Hypertext Preprocessor (PHP), Python®, Ruby, Tickle, Smalltalk, WebDNA®, or Groovy. In some embodiments, the web application is written to some extent in a database query language such as Structured Query Language (SQL). In some embodiments, the web application integrates with enterprise server products such as IBM® Lotus Domino®. In some embodiments, the web application includes a media player element. In various further embodiments, the media player element utilizes one or more of a number of suitable multimedia technologies, including, but not limited to, Adobe® Flash®, HTML5, Apple® QuickTime®, Microsoft® Silverlight®, Java®, and Unity®.

[0124] Referring to Figure 4, in a particular embodiment, the application delivery system includes one or more databases 400 accessed by a relational database management system (RDBMS) 410. Suitable RDBMSs include Firebird, MySQL®, PostgreSQL, SQLite, Oracle Database, Microsoft SQL Server, IBM DB2, IBM Informix, SAP Sybase, Teradata, etc. In this embodiment, the application delivery system further includes one or more application servers 420 (such as a Java server, .NET server, PHP server, etc.) and one or more web servers 430 (such as Apache, IIS, GWS, etc.). The web servers optionally reveal one or more web services via an app application programming interface (API) 440. Over a network such as the Internet, the system provides a browser-based and / or mobile native user interface.

[0125] Alternatively, referring to Figure 5, in a particular embodiment, the application delivery system has a distributed cloud-based architecture 500 and includes elastically load-balanced, auto-scaling web server resources 510 and application server resources 520, as well as a synchronously replicated database 530.

[0126] Mobile application In some embodiments, the computer program includes a mobile application provided to a mobile computing device. In some embodiments, the mobile application is provided to the mobile computing device during manufacturing. In other embodiments, the mobile application is provided to the mobile computing device via a computer network as described herein.

[0127] With reference to the disclosures provided herein, mobile applications are created using techniques known to those skilled in the art, using hardware, languages, and development environments known in the art. Those skilled in the art will recognize that mobile applications are written in several languages. Suitable programming languages ​​include, but are not limited to, C, C++, C#, Objective-C, Java®, JavaScript, Pascal, Object Pascal, Python®, Ruby, VB.NET, WML, and XHTML / HTML with or without CSS, or a combination thereof.

[0128] Suitable mobile application development environments are available from several sources. Commercial development environments, in non-exclusive examples, include AirplaySDK, alcheMo, Appcelerator®, Celsius, Bedrock, Flash Lite, .NET Compact Framework, Rhomobile, and WorkLight Mobile Platform. Other development environments, in non-exclusive examples, include Lazarus, MobiFlex, MoSync, and PhoneGap, which are available free of charge. Mobile device manufacturers also distribute software developer kits, in non-exclusive examples, including iPhone® and iPad® (iOS) SDKs, Android® SDK, BlackBerry® SDK, BREW SDK, Palm®OS SDK, Symbian SDK, webOS SDK, and Windows® Mobile SDK.

[0129] Those skilled in the art will recognize, as non-limiting examples, that several commercial forums are available for the distribution of mobile applications, including Apple® App Store, Google® Play, Chrome WebStore, BlackBerry® App World, App Store for Palm devices, App Catalog for webOS, Windows® Marketplace for Mobile, Ovi Store for Nokia® devices, Samsung® Apps, and Nintendo® DSi Shop.

[0130] Standalone application In some embodiments, a computer program includes a standalone application, which is a program that runs as an independent computer process, not as an add-on to an existing process, for example, as a plug-in. Those skilled in the art will recognize that standalone applications are often compiled. A compiler is a computer program that translates source code written in a programming language into binary object code, such as assembly language or machine code. Suitable compiled programming languages, in non-limiting examples, include C, C++, Objective-C, COBOL, Delphi, Eiffel, Java®, Lisp, Python®, Visual Basic, and VB .NET, or combinations thereof. Compilation is often performed, at least in part, to create an executable program. In some embodiments, a computer program includes one or more executable compiled applications.

[0131] Web browser plugin In some embodiments, a computer program includes web browser plugins (e.g., extensions). In computing, a plugin is one or more software components that add specific functionality to a larger software application. Software application manufacturers support plugins to create the ability for third-party developers to extend the application, support new features that can be easily added, and reduce the size of the application. When supported, plugins allow for customization of the functionality of the software application. For example, plugins are commonly used in web browsers to play videos, create interactivity, scan for viruses, and display specific file types. Those skilled in the art will be familiar with several web browser plugins, including Adobe® Flash® Player, Microsoft® Silverlight®, and Apple® QuickTime®. In some embodiments, a toolbar includes one or more web browser extensions, add-ins, or add-ons. In some embodiments, a toolbar includes one or more explorer bars, toolbands, or deskbands.

[0132] A person skilled in the art will recognize, in non-limiting examples, that several plugin frameworks are available that enable the development of plugins in a variety of programming languages, including C++, Delphi, Java®, PHP, Python®, and VB .NET, or combinations thereof.

[0133] A web browser (also called an internet browser) is a software application designed for use with a networked computing device to retrieve, present, and examine information resources on the World Wide Web. Suitable web browsers include, but are not limited to, Microsoft® Internet Explorer®, Mozilla® Firefox®, Google® Chrome, Apple® Safari®, Opera Software® Opera®, and KDE Konqueror. In some embodiments, a web browser is a mobile web browser. Mobile web browsers (also called microbrowsers, minibrowsers, and wireless browsers) are designed for use on mobile computing devices, including, but are not limited to, handheld computers, tablet computers, netbooks, subnotebook computers, smartphones, music players, personal digital assistants (PDAs), and handheld video game systems. Suitable mobile web browsers include, in non-exclusive embodiments, the Google® Android® Browser, RIM BlackBerry® Browser, Apple® Safari®, Palm® Blazer, Palm® WebOS® Browser, Mozilla® Firefox® for mobile, Microsoft® Internet Explorer® Mobile, Amazon® Kindle® Basic Web, Nokia® Browser, Opera Software® Opera® Mobile, and Sony® PSP® Browser.

[0134] Software Module In some embodiments, the platforms, systems, media, and methods disclosed herein include software, servers, and / or database modules, or the use thereof. With reference to the disclosures provided herein, a software module is created by techniques known to those skilled in the art using machines, software, and languages ​​known in the art. The software modules disclosed herein are implemented in a variety of ways. In various embodiments, a software module includes files, code sections, programming objects, programming structures, distributed computing resources, cloud computing resources, or a combination thereof. In further various embodiments, a software module includes multiple files, multiple code sections, multiple programming objects, multiple programming structures, multiple distributed computing resources, multiple cloud computing resources, or a combination thereof. In various embodiments, one or more software modules include, in non-limiting examples, web applications, mobile applications, standalone applications, and distributed or cloud computing applications. In some embodiments, a software module resides within one computer program or application. In other embodiments, a software module resides within two or more computer programs or applications. In some embodiments, a software module is hosted on one machine. In other embodiments, a software module is hosted on two or more machines. In further embodiments, the software module is hosted on a distributed computing platform, such as a cloud computing platform. In some embodiments, the software module is hosted on one or more machines at one location. In other embodiments, the software module is hosted on one or more machines at two or more locations.

[0135] database In some embodiments, the platforms, systems, media, and methods disclosed herein include one or more databases, or the use thereof. By considering the disclosures provided herein, those skilled in the art will recognize that many databases are suitable for storing and retrieving, for example, user information, media information, prompt information, summary information, curriculum information, review information, survey information, check-in information, happiness index information, token information, and market information. In various embodiments, suitable databases include, in non-limiting examples, relational databases, non-relational databases, object-oriented databases, object databases, entity-relational model databases, associative databases, XML databases, document-oriented databases, and graph databases. Further non-limiting examples include SQL, PostgreSQL, MySQL®, Oracle, DB2, Sybase, and MongoDB. In some embodiments, the database is internet-based. In further embodiments, the database is web-based. In even further embodiments, the database is cloud computing-based. In certain embodiments, the database is a distributed database. In other embodiments, the database is based on one or more local computer storage devices.

[0136] definition As used herein and in the appended claims, the terms “artificial intelligence,” “artificial intelligence technology,” “artificial intelligence operation,” and “artificial intelligence algorithm” generally refer to any system or computational procedure that can take one or more actions that increase or maximize the likelihood of achieving a goal. The term “artificial intelligence” may include “generative modeling,” “machine learning” (ML), or “reinforcement learning” (RL).

[0137] As used herein and in the appended claims, the terms “machine learning,” “machine learning technique,” ​​“machine learning operation,” and “machine learning model” generally refer to any system or analytical or statistical procedure that can progressively improve the computer performance of a task.

[0138] When used in this specification and the appended claims, “several embodiments,” “further embodiments,” or “a particular embodiment” means that a particular feature, structure, or characteristic described in relation to such embodiment is included in at least one embodiment. Thus, the phrases “in several embodiments,” “further embodiments,” or “a particular embodiment” found in various places throughout this specification do not necessarily all refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be appropriately combined in one or more embodiments.

[0139] Whenever the terms "at least," "greater than," or "greater than or equal to" precede the first number in a series of two or more numbers, the terms "at least," "greater than," or "greater than or equal to" apply to each number in that series. For example, 1, 2, or 3 or more is equivalent to 1 or more, 2 or more, or 3 or more.

[0140] Whenever the terms "no more than," "less than," or "less than or equal to" precede the first number in a series of two or more numbers, the terms "no more than," "less than," or "less than or equal to" apply to each number in that series. For example, 3, 2, or 1 or less is equivalent to 3 or less, 2 or less, or 1 or less.

[0141] When used interchangeably in this specification, the terms “real time” or “real-time” generally refer to events (e.g., operations, processes, methods, techniques, calculations, computations, analyses, visualizations, optimizations, etc.) performed using recently obtained (e.g., collected or received) data. In some cases, real-time events may occur almost instantly or within a sufficiently short time, such as at least 0.0001 milliseconds (ms), 0.0005 ms, 0.001 ms, 0.005 ms, 0.01 ms, 0.05 ms, 0.1 ms, 0.5 ms, 1 ms, 5 ms, 0.01 seconds, 0.05 seconds, 0.1 seconds, 0.5 seconds, 1 second, or longer. In some cases, real-time events may occur almost instantly, or within a sufficiently short time, such as at most 1 second, 0.5 seconds, 0.1 seconds, 0.05 seconds, 0.01 seconds, 5 ms, 1 ms, 0.5 ms, 0.1 ms, 0.05 ms, 0.01 ms, 0.005 ms, 0.0005 ms, 0.0001 ms, or even shorter.

[0142] While preferred embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are given only as examples. Numerous variations, modifications, and substitutions will be conceivable herein without departing from the present invention. It should be understood that various alternatives to the embodiments of the present invention described herein may be used in carrying out the present invention. The following claims define the scope of the present invention, and the methods and configurations within the scope of those claims and their equivalents are intended to be encompassed within the scope of the present invention.

Claims

1. A method for autonomous operation of a vehicle, (a) Maintaining a first set of sensor data in a computer data store, and (b) Until the convergence condition is reached, (i) One or more processors execute a first set of instructions and a second set of instructions in parallel, wherein the first set of instructions is (1) Receiving a second set of sensor data, wherein the second set of sensor data is not included in the first set of sensor data. (2) At least partially, the first compressed vector is generated from the second set of sensor data by processing the second set of sensor data with the first machine learning model, (3) Selecting an action to be performed by the vehicle by processing the first compressed vector with a second machine learning model, at least in part. Includes, (ii) The second set of the above instructions is (1) Taking one or more samples of sensor data from the first set of sensor data and / or the second set of sensor data, (2) To generate a second compressed vector by fine-tuning the first machine learning model by processing at least partially one or more samples of the sensor data, (3) to fine-tune the second machine learning model by processing the second compression vector at least partially. Including, to perform A method that includes this.

2. The method according to claim 1, wherein the first machine learning model includes a first set of model weights, and the second machine learning model includes a second set of model weights.

3. The method according to claim 2, further comprising fine-tuning by changing a first set of weights for the model and / or a second set of weights for the model based on the result of the action.

4. The method according to any one of claims 1 to 3, wherein the vehicle includes a civil engineering vehicle or heavy machinery.

5. The method according to any one of claims 1 to 4, wherein the civil engineering vehicle or heavy machinery includes an earth mover, bulldozer, backhoe, excavator, tractor, snow vehicle, excavator, crane, forklift, boring machine, mowing machine, compaction machine, drilling machine, pile driver, street sweeper, snow removal machine, aerial work platform, or dump truck.

6. The method according to any one of claims 1 to 5, wherein a first set of instructions is executed on a first thread, and a second set of instructions is executed on a second thread.

7. The method according to any one of claims 1 to 6, further comprising, before (b), (i) training the first machine learning model with a first set of the sensor data using one or more processors, and (ii) training the second machine learning model at least partially with the first compressed vector generated by the first machine learning model using one or more processors.

8. The method according to claim 7, further comprising training the second machine learning model on the first set of sensor data using one or more processors.

9. The method according to any one of claims 1 to 8, further comprising compressing the first set of sensor data by sampling before (b).

10. The method according to claim 9, further comprising compressing a second set of sensor data by sampling before (2).

11. The method according to any one of claims 9 to 10, further comprising fine-tuning by calculating the difference between a first set of sensor data and a second set of sensor data.

12. The method according to any one of claims 1 to 11, wherein the first set of sensor data includes light detection and ranging (LIDAR) data, GPS data, vehicle status data, or a combination thereof, and the second set of sensor data includes LIDAR data, GPS data, vehicle status data, or a combination thereof.

13. The method according to claim 12, wherein the vehicle status data includes location data or operation data.

14. The method according to any one of claims 12 to 13, wherein the position data is associated with a part of the vehicle.

15. The method according to any one of claims 12 to 14, wherein the position data is associated with the angle or orientation of the vehicle part.

16. The method according to any one of claims 12 to 15, wherein the vehicle component includes an arm, a blade, or a tool.

17. The method according to any one of claims 12 to 16, wherein the tool includes a drilling bucket, a hammer, a hydraulic thumb, a coupler, a crusher, a compactor, a grading bucket, a demolition grapple, and a tilt rotator.

18. The method according to any one of claims 12 to 17, wherein the motion data is related to velocity or acceleration.

19. The method according to any one of claims 12 to 18, wherein the operation data includes the speed of the vehicle.

20. The method according to any one of claims 12 to 19, wherein the operation data includes the acceleration of the vehicle.

21. The method according to any one of claims 12 to 20, wherein the operation data is associated with a part of the vehicle.

22. The method according to any one of claims 12 to 21, wherein the motion data is related to velocity or acceleration.

23. (b)(ii)(1), the method according to any one of claims 12 to 22, wherein one or more samples of the sensor data are sampled from a memory buffer.

24. The method according to any one of claims 12 to 23, wherein the first compression vector or the second compression vector is a learned representation, encoding, or embedding.

25. The method according to any one of claims 12 to 24, wherein the action is selected from a finite set of actions.

26. The method according to claim 25, wherein the finite set of actions includes moving the vehicle, operating a part of the vehicle, increasing the output of the vehicle, decreasing the output of the vehicle, and / or reversing the previous operation of the vehicle.

27. The method according to any one of claims 25 to 26, wherein the movement of the vehicle includes forward, backward, or lateral movement of the vehicle.

28. The method according to any one of claims 25 to 27, wherein the movement of a part of the vehicle includes upward, downward, forward, or lateral movement of the arm of the vehicle.

29. The method according to any one of claims 25 to 28, wherein the movement of a part of the vehicle includes outward or inward movement of a tool connected to an arm of the vehicle.

30. The method according to any one of claims 1 to 29, wherein the convergence condition is associated with several actions performed by the vehicle.

31. The method according to claim 30, wherein the convergence condition is further associated with the performance of the vehicle as measured by the reward function.

32. The method according to any one of claims 30 to 31, wherein the completion of the aforementioned several actions includes a task.

33. The method according to any one of claims 1 to 32, wherein the first set of instructions and the second set of instructions are configured to be executed in parallel or sequentially.

34. A system for autonomously operating a vehicle, One or more sensors configured to output a first set of sensor data and a second set of sensor data, wherein the second set of sensor data is not included in the first set of sensor data, A computer data store configured to store a first set of the aforementioned sensor data, and One or more processors The process includes, and the one or more processors continue until the convergence condition is reached. (a) A configuration that executes a first set of instructions and a second set of instructions in parallel, wherein the first set of instructions is (1) Receiving a second set of the sensor data, (2) At least partially, the first compressed vector is generated from the second set of sensor data by processing the second set of sensor data with the first machine learning model, (3) Selecting an action to be performed by the vehicle by processing the first compressed vector with a second machine learning model, at least in part. Includes, (b) The second set of instructions is (1) Taking one or more samples of sensor data from the first set of sensor data, (2) To generate a second compressed vector by fine-tuning the first machine learning model by processing at least partially one or more samples of the sensor data, (3) to fine-tune the second machine learning model by processing the second compression vector at least partially. A system that includes this.

35. The system according to claim 34, wherein the first machine learning model includes a first set of model weights, and the second machine learning model includes a second set of model weights.

36. The system according to claim 35, further comprising a second set of instructions that modifies a first set of weights for the model and / or a second set of weights for the model based on the result of the action.

37. The system according to any one of claims 34 to 36, wherein the vehicle includes a civil engineering vehicle or heavy machinery.

38. The system according to any one of claims 34 to 37, wherein the civil engineering vehicles or heavy machinery include earth movers, bulldozers, backhoes, excavators, tractors, snowmobiles, excavators, cranes, forklifts, boring machines, mowing machines, compaction machines, drilling machines, pile drivers, street sweepers, snowplows, aerial work platforms, or dump trucks.

39. The system according to any one of claims 34 to 38, wherein a first set of instructions is executed on a first thread, and a second set of instructions is executed on a second thread.

40. The system according to any one of claims 34 to 39, wherein one or more processors are configured to execute a third set of instructions before a first set of instructions and a second set of instructions, the third set of instructions comprising (i) training a first machine learning model with a first set of sensor data, and (ii) training a second machine learning model at least in part with a first compressed vector generated by the first machine learning model.

41. The system according to claim 40, wherein a third set of the instructions further comprises training the second machine learning model with the first set of sensor data.

42. The system according to any one of claims 34 to 41, wherein a third set of instructions further comprises compressing the first set of sensor data by sampling the first set of sensor data.

43. The system according to claim 42, wherein the first set of instructions further includes compressing the second set of sensor data by sampling the second set of sensor data before (a)(2).

44. The system according to any one of claims 42 to 43, wherein the second set of instructions further includes calculating the difference between the first set of sensor data and the second set of sensor data.

45. The system according to any one of claims 34 to 44, wherein the first set of sensor data includes light detection and ranging (LIDAR) data, GPS data, vehicle status data, or a combination thereof, and the second set of sensor data includes LIDAR data, GPS data, vehicle status data, or a combination thereof.

46. The system according to claim 45, wherein the vehicle status data includes location data or operation data.

47. The system according to any one of claims 45 to 46, wherein the location data is associated with a part of the vehicle.

48. The system according to any one of claims 45 to 47, wherein the position data is associated with the angle or orientation of the vehicle's components.

49. The system according to any one of claims 45 to 48, wherein the vehicle component includes an arm, a blade, or a tool.

50. The system according to any one of claims 45 to 49, wherein the tools include a drilling bucket, a hammer, a hydraulic thumb, a coupler, a crusher, a compactor, a grading bucket, a demolition grapple, and a tiltrotator.

51. The system according to any one of claims 45 to 50, wherein the motion data is related to velocity or acceleration.

52. The system according to any one of claims 45 to 51, wherein the operating data includes the speed of the vehicle.

53. The system according to any one of claims 45 to 52, wherein the operation data includes the acceleration of the vehicle.

54. The system according to any one of claims 45 to 53, wherein the operation data is associated with a part of the vehicle.

55. The system according to any one of claims 45 to 54, wherein the motion data is related to velocity or acceleration.

56. The system according to any one of claims 45 to 55, wherein one or more samples of the sensor data are sampled from the computer data store.

57. The system according to any one of claims 45 to 56, wherein the first compression vector or the second compression vector is a learned representation, encoding, or embedding.

58. The system according to any one of claims 45 to 57, wherein the action is selected from a finite set of actions.

59. The system according to claim 58, wherein the finite set of actions includes moving the vehicle, operating a part of the vehicle, increasing the output of the vehicle, decreasing the output of the vehicle, and / or reversing the previous operation of the vehicle.

60. The system according to any one of claims 58 to 59, wherein the movement of the vehicle includes forward, backward, or lateral movement of the vehicle.

61. The system according to any one of claims 58 to 60, wherein the movement of a part of the vehicle includes upward, downward, forward, or lateral movement of the arm of the vehicle.

62. The system according to any one of claims 58 to 61, wherein the movement of a part of the vehicle includes outward or inward movement of a tool connected to an arm of the vehicle.

63. The system according to any one of claims 34 to 62, wherein the convergence condition is associated with several actions performed by the vehicle.

64. The system according to claim 63, wherein the convergence condition is further associated with the performance of the vehicle as measured by the reward function.

65. The system according to any one of claims 63 to 64, wherein the completion of some of the aforementioned actions includes a task.

66. The system according to any one of claims 34 to 65, wherein the first set of instructions and the second set of instructions are configured to be executed in parallel or sequentially.

Images