Large recurrent action models for controlling robots and other technical systems

Abstract

A technical system, such as a robot, is controlled by a large recurrent action model implemented on a data processing apparatus. The model receives inputs representing user commands or system state information and generates action commands to control the technical system. The model includes a recurrent inference module, which can be a recurrent neural network such as xLSTM, to enable the system to make informed decisions based on the input data.

Description

TECHNICAL FIELDThe disclosed technology relates to the field of machine-learning control systems, and more specifically to recurrent-model-based action generation for robots and other technical systems.BACKGROUNDNeural networks have become increasingly important in the field of artificial intelligence and automation, with significant applications in controlling complex technical systems. The ability of neural networks to learn patterns and make decisions based on input data has made them valuable tools in a wide range of applications, from industrial automation to robotics.Reinforcement Learning (RL) has demonstrated impressive capabilities in areas such as game playing, navigation, and control systems. Traditional RL approaches have typically involved training agents online with specific reinforcement learning objectives. However, the field has been evolving toward offline reinforcement learning settings and sequence models trained via behavior cloning on large-scale datasets with causal sequence modeling objectives.Control of technical systems, particularly robotics, presents unique challenges that demand sophisticated machine learning approaches. Robots must operate in dynamic, often unpredictable environments, requiring models that can process multimodal inputs (visual, positional, tactile) and generate appropriate control commands in real-time. The ability to translate high-level commands into precise sequences of actions is advantageous for autonomous systems.In many technical control applications, particularly robotics, inference latency is a critical constraint. Control theory principles dictate that controller sample rates should match the sample rate of the sensors. For example, typical robots such as drones or industrial robot arms require control rates of 100Hz-1000Hz to maintain stability, which necessitates inference times below 10ms. At 1000Hz, even a brief 15-second robotic movement generates sequences of 15,000 steps, creating substantial processing demands.The computational complexity of machine learning models used for control presents significant challenges when deployed in resource-constrained environments. Many state-of-the-art models exhibit computational bottlenecks that limit their practical application in scenarios requiring real-time processing. This is particularly problematic for embedded systems with limited hardware capacity, such as robots or mobile devices.Memory usage presents another constraint in resource-limited applications. Technical control systems often operate with tight memory budgets, especially in embedded environments. High memory consumption can lead to reduced performance, increased costs, or even make deployment impossible on certain hardware platforms.Action prediction, particularly for continuous control tasks, introduces additional complexity. Technical systems frequently require precise control across multiple continuous dimensions simultaneously. Traditional approaches to action prediction may struggle to balance accuracy with the speed requirements of real-time control.Challenges remain in addressing these issues while maintaining the performance and reliability required for practical applications in technical control systems.SUMMARY OF THE DISCLOSUREIt is an object of embodiments of the present disclosure to enable efficient sequence modeling for real-time control applications. This objective is achieved by the subject-matter defined in the independent claims. Advantageous modifications of embodiments of the present disclosure are defined in the dependent claims as well as in the description and the drawings.In recent years, there has been a trend in the field of Reinforcement Learning (RL) towards large action models trained offline on large-scale datasets via sequence modeling. Existing models are primarily based on the Transformer architecture, which result in powerful agents. However, due to slow inference times, Transformer-based approaches are impractical for real-time applications, such as robotics. Recently, modern recurrent architectures, such as XLSTM and Mamba, have been proposed that exhibit parallelization benefits during training similar to the Transformer architecture while offering fast inference.Certain embodiments of the present disclosure demonstrate the aptitude of these modern recurrent architectures for large action models. Certain embodiments provide a Large Recurrent Action Model (LRAM) with an XLSTM at its core that comes with linear-time inference complexity and natural sequence length extrapolation abilities. Experiments on various tasks from different domains show that LRAM compares favorably to Transformers in terms of performance and speed.In a first aspect, the present disclosure relates to a system for controlling a technical system. The system is designed to facilitate automated or semi-automated control of a technical system, such as a machine, device, or process, by employing advanced machine learning techniques. The system may comprise a large recurrent action model implemented on a data processing apparatus. The data processing apparatus may include one or more processors, memory, and associated hardware capable of executing machine learning models and interfacing with the technical system. The technical system may be any system requiring dynamic control, such as an industrial robot, a vehicle, a manufacturing process, or a building automation system.In embodiments, the large recurrent action model may be configured to receive inputs representing one or more of user commands or state information of the technical system, and to generate action commands for controlling the technical system. The large recurrent action model may be implemented in software, hardware, or a combination thereof. The inputs to the model may include user commands such as operator instructions, and / or state information such as sensor readings, status indicators, or environmental data relating to the technical system. The model processes these inputs to generate action commands, which are output to the technical system to effect control actions, such as adjusting actuators, changing operational parameters, or initiating specific processes.In embodiments, the large recurrent action model may comprise a recurrent inference module, which is responsible for modeling temporal dependencies and sequential relationships in the input data. The recurrent inference module may receive sequences of input data and update its internal state accordingly. Based on the current internal state and the received inputs, the recurrent inference module may generate action commands that are suitable for controlling the technical system in accordance with the desired objectives.In embodiments, the system may further comprise interfaces for receiving input data from external sources, such as user interfaces, sensors, or communication networks, and for transmitting action commands to the technical system. The data processing apparatus may be configured to execute the large recurrent action model continuously or in response to specific events or triggers. The architecture of the system enables flexible and adaptive control, as the recurrent inference module can learn and adapt to changes in the technical system or its environment over time.In embodiments, the recurrent inference module may comprise a recurrent neural network (RNN), such as a long short-term memory (LSTM) network, in particular an extended LSTM (xLSTM) or a state space model (SSM). These architectures enable the system to maintain an internal representation of the system state over time, allowing for context-aware and history-dependent decision making. The use of a recurrent neural network or a state space model allows the system to capture complex temporal patterns and dependencies, which are often present in technical systems with dynamic behavior. This approach enables more accurate, robust, and context-sensitive control compared to conventional rule-based or feedforward control systems. The system is particularly advantageous in applications where the technical system operates in dynamic or uncertain environments, or where user commands and system states evolve over time.In embodiments, the recurrent neural network may be an extended long short-term memory (XLSTM). This allows for efficient processing of sequential data with advanced memory capabilities.In embodiments, the state space model may be a Mamba model. This provides an alternative architecture with linear-time complexity for processing sequential inputs.In embodiments, the extended long short-term memory may comprise a plurality of blocks. The plurality of blocks may comprise a mixture of mLSTM-based blocks and sLSTM-based blocks. This combination of block types enhances the model's representational capacity while maintaining computational efficiency.In embodiments, a ratio of mLSTM-based blocks to sLSTM-based blocks in the plurality of blocks may be 7:1. This specific ratio provides an optimal balance between performance and computational requirements.In embodiments, the extended long short-term memory may comprise a plurality of blocks, wherein all blocks in the plurality of blocks are mLSTM-based blocks. This homogeneous architecture simplifies implementation while maintaining high performance.In embodiments, the large recurrent action model may comprise a plurality of encoders, each encoder configured to process inputs of a different modality. This multi-modal capability allows the system to process diverse input types seamlessly.In embodiments, the plurality of encoders may comprise a convolutional neural network configured to encode image inputs and / or a fully connected network configured to encode low-dimensional inputs. This specialized encoder architecture optimizes processing for different input modalities.In embodiments, the large recurrent action model may be configured to process the image inputs from the convolutional neural network without patchifying the images. This approach reduces sequence length and improves inference speed.In embodiments, the large recurrent action model may be configured to process continuous state inputs from the fully connected network without tokenizing the continuous states. This direct processing approach preserves information integrity and reduces computational overhead.In embodiments, the large recurrent action model may be configured to discretize continuous action dimensions into a predefined number of bins. This discretization strategy enables efficient action representation and prediction. In one example, the predefined number of bins may be 256. This specific bin count provides sufficient resolution for accurate control while maintaining computational efficiency.In embodiments, the system may further comprise a shared action head. The shared action head may be configured to jointly predict a plurality of action dimensions. The plurality of action dimensions may comprise the discretized continuous action dimensions. This shared architecture reduces inference time compared to separate prediction heads.In embodiments, the shared action head may be configured to perform non-autoregressive prediction of the plurality of action dimensions. This parallel prediction approach significantly improves inference speed for multi-dimensional action spaces.In embodiments, the recurrent inference module may be configured to operate in a recurrent inference mode by maintaining a hidden state from a previous timestep to enable inference with linear-time complexity with respect to a sequence length of the inputs. This hidden state persistence enables efficient sequential processing.In embodiments, the recurrent inference module may be configured for chunkwise inference, wherein a single forward pass processes a chunk of tokens corresponding to a single timestep. This optimization further reduces inference latency.In embodiments, the large recurrent action model may not use timestep encodings. This design choice improves generalization across varying episode lengths.In embodiments, the data processing apparatus on which the large recurrent action model is implemented may be embedded in the technical system that is controlled. This embedded implementation enables real-time control applications with minimal latency.In a second aspect, the present disclosure relates to a method for controlling a technical system. The method may comprise receiving, at a large recurrent action model, inputs representing one or more of user commands or state information of the technical system. The method may comprise processing the inputs with the large recurrent action model to generate action commands. The large recurrent action model may comprise a recurrent inference module comprising a recurrent neural network or a state space model. The method may comprise controlling the technical system based on the generated action commands.In embodiments, the large recurrent action model used in the method may be the large recurrent action model of the first aspect described above, optionally in any of the described embodiments.In a third aspect, the present disclosure relates to a robot. The robot may comprise one or more sensors for acquiring state information about the robot and / or its environment. The robot may comprise one or more actuators for performing actions. The robot may comprise a control system comprising the system of the first aspect. The control system may be configured to receive the state information from the one or more sensors as inputs to the large recurrent action model, and control the one or more actuators using the action commands generated by the large recurrent action model.In embodiments, the one or more sensors may comprise at least one of: a camera, a LiDAR sensor, a tactile sensor, a force-torque sensor, or an inertial measurement unit. These sensor types enable comprehensive perception of the robot's environment.In embodiments, the one or more actuators may comprise at least one of: a joint of a robotic arm, a gripper, or a wheel motor. These actuator types provide diverse motion capabilities for different robotic applications.In embodiments, the robot may be one of: a robotic arm, a mobile robot, a humanoid robot, or a drone. This versatility in robot embodiments demonstrates the broad applicability of the control system.In embodiments, the control system may be configured to receive a user command in natural language, and wherein the large recurrent action model is configured to generate the action commands by decomposing the user command into a sequence of actionable steps for the one or more actuators. This natural language interface enhances user interaction and task specification.In embodiments, the action commands may comprise low-level commands, including at least one of: a joint angle command, an actuator voltage command, or a motor velocity command. These low-level command capabilities enable precise control of the robot's physical movements.Another aspect of the disclosed technology provides a data processing apparatus, device or system, comprising means for carrying out (the steps of) the method(s) of any one of the embodiments of the disclosed technology.Another aspect of the disclosed technology provides a data processing apparatus, device or system, comprising: at least one processor; and a memory storing instructions that, when executed by the at least one processor, configure the data processing apparatus, device or system to carry out (the steps of) the method(s) of any one of the embodiments of the disclosed technology.Another aspect of the disclosed technology provides a computer program (product) comprising instructions which, when the program is executed by a computer, data processing apparatus, device or system, cause the computer, data processing apparatus, device or system to carry out (the steps of) the method(s) of any one of the embodiments of the disclosed technology.Another aspect of the disclosed technology provides a non-transitory computer-readable medium on which computer-executable instructions are stored to implement (the steps of) the method(s) of any one of the embodiments of the disclosed technology.Any embodiment of the disclosed technology may be combined with any other embodiment, not just merely with those expressly indicated.It is an advantage of embodiments of the present disclosure that linear-time inference along the sequence length can be provided.It is a further advantage of embodiments of the present disclosure that low-latency operation enabling real-time control can be provided.It is a further advantage of embodiments of the present disclosure that memory consumption during inference can be reduced.It is a further advantage of embodiments of the present disclosure that efficient deployment on resource-constrained hardware can be facilitated.It is a further advantage of embodiments of the present disclosure that long-context sequences can be handled without quadratic computational growth.It is a further advantage of embodiments of the present disclosure that token counts per timestep can be minimized by avoiding unnecessary patchification or discretization.It is a further advantage of embodiments of the present disclosure that integration into embedded control loops of robots and other technical systems can be simplified.It is a further advantage of embodiments of the present disclosure that sensor-to-actuator pipelines can be executed end-to-end within a single model.BRIEF DESCRIPTION OF THE DRAWINGSThe disclosed technology may be better understood by reference to the following drawings:FIG. 1 illustrates a schematic diagram of a system for controlling a technical system according to embodiments.FIG. 2 illustrates a method 200 for controlling a technical system in accordance with one embodiment.FIG. 3 illustrates a comparative graph showing performance comparison between three LRAM variants and a Transformer-based LAM according to embodiments.FIG. 4 illustrates a comparative graph showing latency performance between xLSTM-based LRAM and Transformer-based LAM across various batch sizes according to embodiments.FIG. 5 illustrates a comparative graph showing memory consumption comparison between XLSTM-based LRAM and Transformer-based LAM according to embodiments.FIG. 6 illustrates a comparative graph showing throughput comparison between xLSTM-based LRAM and Transformer-based LAM according to embodiments.FIG. 7 illustrates a schematic block diagram of computer hardware on which embodiments of the disclosed technology can be implemented.DETAILED DESCRIPTIONIn the following, representative embodiments illustrated in the accompanying drawings will be explained. It should be understood that the illustrated embodiments and the following descriptions refer to examples which are not intended to limit the embodiments to one preferred embodiment.TermsThe terms used herein should generally be construed as understood by the average person skilled in the art, unless explicitly indicated otherwise. The following explanations may guide the understanding:As used herein, and unless otherwise specified, the phrase "technical system" refers to any machine or apparatus capable of performing operations in response to inputs that include user commands or state information. Examples of such systems include, but are not limited to, industrial automation equipment, vehicles, robotic arms, drones, home appliances, or computing devices.As used herein, and unless otherwise specified, the phrase "large recurrent action model" refers to a computational model implemented on a data processing apparatus, structured to handle complex or high-dimensional sequential inputs and generate corresponding action commands. Examples of large recurrent action models include multi-layer recurrent neural networks with extensive parameterization, large state space models, or any other recurrent architecture capable of processing time-series data and producing outputs that influence a system's actions. The term "large" may not imply a minimum size of such a model, but may simply refer to its sophisticated design. Accordingly, a "large recurrent action model" may also be referred to as a "recurrent action model".As used herein, and unless otherwise specified, the phrase "recurrent inference module" refers to one or more components within a model that process sequential data using feedback connections or internal hidden states to maintain information across different time steps. Examples include recurrent neural networks, gated recurrent units, or state space models that update an internal representation at each time step to inform subsequent computations.As used herein, and unless otherwise specified, the term "recurrent neural network" refers to a type of neural network with recurrent connections that enable the storage and propagation of hidden states over sequential inputs. Examples include basic RNN cells, gated recurrent units (GRUs), long short-term memory (LSTM) networks, and extended or modified variants thereof.As used herein, and unless otherwise specified, the phrase "extended long short-term memory" (XLSTM) refers to a long short-term memory architecture that is modified or augmented beyond the classical LSTM design, generally to improve performance or adapt to specialized tasks. Examples may include variants with additional gating mechanisms, more complex internal block structures, or custom activation functions. Technological background on the xLSTM architecture can be found in WO 2025 / 168 512 A1, the contents of which are incorporated herein by reference. The document discloses a system implementing a long short-term memory (LSTM) neural network on a data processing apparatus, where the LSTM includes at least one exponential activation function and may comprise input and / or forget gates with exponential activation, as well as normalization mechanisms. The disclosure further describes variants such as scalar memory cell LSTMs (sLSTM), matrix memory cell LSTMs (mLSTM), and architectures with multiple heads or blocks, including residual xLSTM blocks and stacked arrangements. Additional technological background on xLSTM can be found in "xLSTM: Extended Long Short-Term Memory" (arXiv:2405.04517).As used herein, and unless otherwise specified, the phrase "state space model" (SSM) refers to a model that represents system dynamics through internal state variables and equations governing transitions and outputs over time. Examples include linear state space models, nonlinear state space models, and specialized forms such as those that incorporate learned parameters or neural network layers for complex dynamical processes.As used herein, and unless otherwise specified, the phrase "Mamba model" refers to a particular type of state space model designed or configured for complex inference tasks, often incorporating advanced parameterization or specialized update rules. While specific implementations may differ, Mamba models generally maintain a latent state and update it according to observed inputs. Technological background on Mamba can be found in "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" (arXiv:2312.00752).As used herein, and unless otherwise specified, the phrase "encoder" refers to a component within a model architecture responsible for processing a specific type, format, or modality of data. Examples include encoders for images, text, sensor data, or tabular inputs.As used herein, and unless otherwise specified, the term "modality" refers to a particular type or form of input data that a system may process. Examples of modalities include visual data such as images or video frames, textual data such as user commands or documents, audio data such as speech signals, or lower-dimensional signals such as numerical sensor readings.As used herein, and unless otherwise specified, the phrase "convolutional neural network" (CNN) refers to a neural network architecture that applies convolution operations to extract features from spatially or spatiotemporally arranged data. Examples include classic CNN architectures such as LeNet, AlexNet, ResNet, or more specialized variants for processing images or videos.As used herein, and unless otherwise specified, the phrase "encode image inputs" refers to the process of transforming raw image data into a more compact or informative representation, typically using layers that learn spatial features. Examples include applying 2D convolutions, 3D convolutions, or other learned filters that capture edges, shapes, or textures, ultimately resulting in a feature vector or feature map.As used herein, and unless otherwise specified, the phrase "fully connected network configured to encode low-dimensional inputs" refers to a series of densely connected neural layers that process inputs with relatively few dimensions, such as scalar sensor readings or small vectors of numerical values. Examples include feed-forward networks used to process temperature readings, force measurements, or other low-dimensional data sources.As used herein, and unless otherwise specified, the phrase "shared action head" refers to a single output layer or set of output layers that collectively predict multiple action components. Examples include a single network head that simultaneously computes velocity, steering, and acceleration values, or a unified layer that produces positions along multiple axes in parallel.As used herein, and unless otherwise specified, the phrase "non-autoregressive prediction" refers to producing multiple output elements simultaneously rather than predicting each element conditioned on the previously predicted elements. Examples include generating all action dimensions in one step, as opposed to sequentially predicting each axis of motion based on prior outputs.As used herein, and unless otherwise specified, the phrase "recurrent inference mode" refers to an operational setting in which a model processes inputs across time steps by updating and carrying forward a hidden or latent state from one step to the next. Examples include scenarios where new sensor data arrives continuously, and the internal state is repeatedly updated to produce quick inferences at each time step.As used herein, and unless otherwise specified, the phrase "chunkwise inference" refers to performing model computations on segments, or chunks, of sequential data, typically where each chunk corresponds to a discrete time step or a small number of time steps. An example is a system that processes one time step at a time, yet still updates its internal state continuously.System and Process OverviewFIG. 1 illustrates a schematic diagram of a system 100 for controlling a technical system 108 in the form of a robot according to embodiments. The robot 108 may comprise one or more sensors for acquiring state information 112 about the robot 108 and / or its environment, and one or more actuators for performing actions. FIG. 1 also shows a large recurrent action model 102 implemented on a data processing apparatus that acts as a control system. The large recurrent action model 102 is configured to receive inputs representing the state information 112 of the robot 108. In addition, the large recurrent action model 102 may receive user commands 110 from a user 106 as inputs. Based on the provided inputs, the large recurrent action model 102 generates action commands 114 for controlling the robot 108.The robot 108 may be realized as a mobile robot, a stationary robotic manipulator, a humanoid robot, or any other form of robotic system capable of interacting with its environment. The sensors may include, for example, cameras, lidar, radar, ultrasonic sensors, inertial measurement units, force sensors, tactile sensors, or other devices suitable for acquiring information about the robot's internal state and external surroundings. The state information 112 acquired by the sensors may comprise position, orientation, velocity, acceleration, proximity to objects, environmental features, or other relevant data. The actuators may include electric motors, hydraulic or pneumatic actuators, servos, or other mechanisms capable of effecting movement or manipulation. These actuators may be used to drive wheels, joints, arms, grippers, or other movable components of the robot 108, enabling it to perform a wide range of actions such as navigation, object manipulation, assembly, inspection, or interaction with humans and other systems. The action commands 114 issued by the control system may specify target positions, velocities, forces, or other control parameters for the actuators, enabling the robot 108 to perform context-appropriate actions in response to its environment and operational objectives.It should be emphasized that the robot 108 in FIG. 1 is only one specific example of a technical system that can be controlled using embodiments of the disclosed technology. Accordingly, the technologies described next can be applied to any type of technical system, unless indicated otherwise.FIG. 2 illustrates a method 200 for controlling the technical system 108 in accordance with one embodiment. In step 202, method 200 receives, at the large recurrent action model 102, inputs representing one or more of user commands 110 or state information 112 of the technical system 108. In step 204, method 200 processes the inputs with the large recurrent action model 102 to generate action commands 114. As will be explained in more detail below, the large recurrent action model 102 may comprise a recurrent inference module 104 comprising a recurrent neural network or a state space model. In step 206, method 200 controls the technical system 108 based on the generated action commands 114.xLSTM-Based ImplementationsIn embodiments, the recurrent inference module 104 comprises a recurrent neural network (RNN) in the form of an extended long short-term memory (xLSTM). The xLSTM architecture can provide enhanced capability for modeling long-range temporal dependencies and complex sequential patterns in the input data. For example, the xLSTM may incorporate additional gating mechanisms, such as exponential gating, or other architectural modifications beyond those present in conventional LSTM networks. Such enhancements may enable the recurrent inference module 104 to capture more nuanced relationships within sequences of user commands 110 or state information 112, thereby improving the quality of the generated action commands 114 for controlling the technical system 108. Other forms of recurrent neural networks may also be considered, and the use of an xLSTM is merely one possible realization among several alternatives for the recurrent inference module 104. The architecture of the system 100 can thus be adapted to suit a wide range of technical systems 108 and control scenarios by selecting the most appropriate model type for the recurrent inference module 104.In embodiments, the xLSTM may comprise a plurality of blocks. The term "block" in this context may refer to a modular computational unit within the xLSTM, each block being configured to process a portion of the input sequence or to maintain a segment of the internal memory state.In a first configuration, the plurality of blocks comprise a mixture of mLSTM-based blocks and sLSTM-based blocks. A ratio of mLSTM-based blocks to sLSTM-based blocks in the plurality of blocks may be 7:1. The location of sLSTM blocks within the xLSTM may be predetermined according to a fixed pattern to maintain the approximate 7:1 ratio even when the total number of blocks is not a multiple of eight.In a second configuration, the ratio may be 1:0, i.e., the xLSTM comprises a plurality of blocks, wherein all blocks are mLSTM-based blocks.Mamba-Based ImplementationIn another specific embodiment, the recurrent inference module 104 comprises a state space model (SSM) in the form of a Mamba model. The Mamba model represents a particular class of state space models that may be employed to capture temporal dependencies and system dynamics in sequential data. By adopting a Mamba model, the recurrent inference module 104 is optionally able to process input sequences with improved efficiency and scalability, particularly in scenarios involving long-range dependencies or high-dimensional state representations. Other forms of state space models may also be considered, and the use of a Mamba model is merely one possible realization among several alternatives for the recurrent inference module 104. The architecture of the system 100 can thus be adapted to suit a wide range of technical systems 108 and control scenarios by selecting the most appropriate model type for the recurrent inference module 104.Decision TransformerCertain embodiments of the disclosed technology will be benchmarked against the Decision Transformer (DT), a GPT-style model configured to learn to predict actions from offline trajectories via behavior cloning.We assume the standard reinforcement learning (RL) formulation via a Markov Decision Process (MDP) represented by a tuple of (S, A, P, R), where S and A denote state and action spaces, respectively. At every timestep t, the agent observes state s_{t}\in\delta, predicts action a_{t}\in\mathcal{A}, and receives a scalar reward r_{t}. The reward is determined by the reward function \mathcal{R}(r_{t}|s_{t},a_{t}). \mathcal{P}(s_{t+1}|s_{t},a_{t}) defines the transition dynamics and constitutes a probability distribution over next states S_{t+1} when executing action a_{t} in state S_{t}. The goal of RL is to learn a policy \pi(a_{t}|s_{t}) that predicts an action a_{t} in state S_{t} that maximizes r_{t}.The Decision Transformer (DT) casts the RL problem setting as next action prediction task via causal sequence modeling. At training time, DT aims to learn a policy \pi_{\theta} that maps future rewards to actions, which is often referred to as upside-down RL. At inference time, the DT is conditioned via a target return to emit high-reward actions. Consequently, we assume access to a dataset \mathcal{D}=\{\tau_{i}\}_{i=1}^{N} containing N trajectories \tau_{i} consisting of quadruplets \tau_{i}=(s_{1},\hat{R}_{1},a_{1},r_{1},...,s_{T},\hat{R}_{T},a_{T},r_{T}) of state S_{t}, return-to-go (RTG) \hat{R}_{t}=\Sigma_{t^{\prime}=t}^{T}r_{t^{\prime}}, action a_{t}, and reward r_{t}. Here, T refers to the length of the trajectory. The DT \pi_{6} is trained to predict the ground-truth action a_{t} conditioned on sub-trajectories from the dataset:\hat{a}_{t}\sim\pi_{\theta}(\hat{a}_{t}|s_{t-c},\hat{R}_{t-c},a_{t-c},r_{t-c},...,s_{t-1},\hat{R}_{t-1},a_{t-1},r_{t-1},s_{t},\hat{R}_{t}) (1)where C\le T is the size of the context window. In fact, Equation (1) describes the setting of the multi-game DT, which also includes rewards in the sequence representation.Multi-Modal Sequence RepresentationIn embodiments, the large recurrent action model 102 may comprise a plurality of encoders. Each encoder may be configured to process inputs of a different modality. For example, it is possible for the system 100 to include separate encoders for distinct types of input data, such as user commands 110, sensor measurements, audio signals, visual information, or other forms of state information 112 relevant to the technical system 108. Each encoder can be tailored to the characteristics of its respective input modality, employing specialized architectures or preprocessing techniques as appropriate. The use of multiple encoders allows the large recurrent action model 102 to flexibly integrate heterogeneous data sources, thereby enhancing its ability to interpret complex operational contexts.In embodiments, the plurality of encoders may comprise a convolutional neural network (CNN) configured to encode image inputs. The CNN can be designed to process visual data, such as images or video frames, which may be captured from cameras or other imaging sensors associated with the technical system 108. This CNN may extract spatial features and relevant patterns from the image inputs, providing a compact and informative representation suitable for subsequent processing by the large recurrent action model 102.In embodiments, the plurality of encoders may comprise a fully connected network configured to encode low-dimensional inputs. A fully connected network is sometimes also referred to as a multilayer perceptron. The low-dimensional inputs may include, for example, scalar sensor readings, system status indicators, user command vectors, or other numerical data that do not possess a spatial structure. The fully connected network can transform these inputs into feature representations that are compatible with the other encoded modalities within the large recurrent action model 102.In embodiments, the large recurrent action model 102 may be configured to process the image inputs from the convolutional neural network without patchifying the images. In such arrangements, the images can be provided to the convolutional neural network in their original, unsegmented form, allowing the network to extract spatial features directly from the full image data. This approach may be advantageous in scenarios where preserving the global spatial structure of the image is beneficial for downstream processing or for generating action commands that depend on holistic visual context.In embodiments, the large recurrent action model 102 may be configured to process continuous state inputs from the fully connected network without tokenizing the continuous states. In such arrangements, the continuous state inputs such as real-valued sensor measurements, control parameters, or other numerical indicators of the technical system's status-can be provided directly to the fully connected network in their native, unsegmented form. The fully connected network may then encode these continuous values into feature representations suitable for integration with other modalities within the model.In one specific implementation, to encode input from different environments with varying state and action spaces, we use separate encoders per modality that are shared across tasks and domains. For encoding images, we use a CNN, whereas for low-dimensional inputs we use a fully connected network. We refrain from patchifying images and tokenizing continuous states to avoid unnecessarily long sequences. Similarly, we use linear layers to encode rewards and RTGs. We omit actions in our sequence formulation, as we found that this can be detrimental to performance, in particular for continuous control tasks with smoothly changing actions (see Section 4.3). Consequently, our trajectories have the form \tau_{i}= (s_{1},\hat{R}_{1},r_{1},...,s_{T},\hat{R}_{T},r_{T}) and we train our policy \pi_{\rho} to predict the ground-truth action a_{t} as:\hat{a}_{t}\sim\pi_{o}(\hat{a}_{t}|s_{t-c},\hat{R}_{t-c},r_{t-c},...,s_{t-1},\hat{R}_{t-1},r_{t-1},s_{t},\hat{R}_{t}) (2)Shared Action HeadIn embodiments, the system 100 may comprise a shared action head configured to jointly predict a plurality of action dimensions. The plurality of action dimensions may comprise the discretized continuous action dimensions. The shared action head may be realized as a neural network module or another computational component that receives internal representations from the large recurrent action model 102 and produces predictions for multiple action outputs in a unified manner.In embodiments, the large recurrent action model 102 may be configured to discretize continuous action dimensions into a predefined number of bins. This discretization process can be applied to one or more action outputs that are originally represented as continuous variables, such as control signals, actuator positions, or other real-valued commands generated for the technical system 108. By mapping these continuous action dimensions to a finite set of discrete bins, the model can facilitate downstream processing, simplify the action selection mechanism, or enable compatibility with systems or interfaces that require discrete control signals.It is possible for the number of bins to be selected based on the resolution required for the specific application or the characteristics of the technical system. In one configuration, the predefined number of bins may be 256.In embodiments, the shared action head may be configured to perform non-autoregressive prediction of the plurality of action dimensions. In such arrangements, the shared action head can be designed to predict all action dimensions simultaneously, rather than sequentially or in an autoregressive manner. This means that each action dimension may be determined independently of the others within a single forward pass of the model, rather than conditioning the prediction of each action dimension on the previously predicted ones. In embodiments, a unified action head may be implemented to predict a joint probability distribution over both discrete actions and discretized continuous actions in a single non-autoregressive softmax operation.In one specific implementation, action spaces in RL typically vary across environments. For example, in the environments we consider, there are 18 discrete actions and a maximum of 8 continuous dimensions for continuous control environments. Therefore, we employ discretization of continuous action dimensions into 256 uniformly-spaced bins. Unlike prior work, we leverage a shared action head to predict all discrete actions or continuous action dimensions jointly. We found that this setup significantly reduces inference time compared to using autoregressive action prediction of continuous actions.Recurrent Inference ModeIn embodiments, the recurrent inference module 104 may be configured to operate in a recurrent inference mode by maintaining a hidden state from a previous timestep to enable inference with linear-time complexity with respect to a sequence length of the inputs. For example, rather than recomputing the entire sequence history at each inference step, the recurrent inference module 104 may update and carry forward a hidden state that encapsulates relevant information from prior timesteps. This hidden state may be updated incrementally as new input data is received, allowing the system 100 to process long sequences efficiently.In embodiments, the recurrent inference module 104 may be configured for chunkwise inference, such that a single forward pass processes a chunk of tokens corresponding to a single timestep. This arrangement can allow the system 100 to handle input data in discrete segments, or "chunks," where each chunk represents the information available at a particular point in time. By processing a chunk of tokens in one forward computation, the recurrent inference module 104 may efficiently update its hidden state and generate outputs relevant to the current timestep.In embodiments, the large recurrent action model 102 may be realized without the use of timestep encodings. That is, the architecture can be configured such that explicit encodings or representations of the timestep, position, or sequence index are not incorporated into the model's input or internal processing. Instead, the recurrent inference module, such as a recurrent neural network or a state space model, may rely solely on its inherent ability to maintain and update a hidden state across timesteps in order to capture temporal dependencies and sequence information.In embodiments, the large recurrent action model 102 may preprocess continuous state vectors by padding them to a fixed dimensionality before embedding, allowing one universal embedding layer for states of differing sizes and simplifying batching procedures.In embodiments, reward information may be represented by dedicated learnable embeddings produced from scalar signals. Separate linear layers may embed the instantaneous reward and the return-to-go value into compact vectors that convey task objectives without significantly lengthening the sequence.In one specific implementation, at inference time, we leverage the recurrent backbone and maintain the hidden states of the last timestep. This enables fast inference with linear-time complexity along the sequence length. In addition, the recurrent-style inference is well-suited for online fine-tuning via RL objectives, similar to LSTM-based policies in online RL. To speed up inference, we leverage custom kernels for the xLSTM backbone.In one specific implementation, our unified discrete action representation enables consistent training of our agents via the cross-entropy loss as training objective across all tasks and domains. We use separate reward scales per domain and target returns per task. Furthermore, we do not make use of timestep encodings, which are detrimental when episode lengths vary.Model ArchitecturesIn certain specific implementations, we train exemplary models across four model sizes: 16M, 48M, 110M, and 206M. For xLSTM and Mamba, we use twice the number of layers blocks to match the number of parameters of the Transformer, as shown in the following table:ModelLayersHidden DimHeadsParametersTransformer4512816MTransformer67681248MTransformer8102416110MTransformer10128020206MMamba8512-16MMamba12768-48MMamba161024-110MMamba201280-206MxLSTM8512416MxLSTM12768448MxLSTM1610244110MxLSTM2012804206MFor our XLSTM [7:1] variant, which contains sLSTM blocks, we strive to maintain the ratio of 7:1. Not all our model sizes are divisible by 8, and only the 16M and 110M models exhibit the exact 7:1 ratio of mLSTM to sLSTM blocks. We place sLSTM blocks at positions [1], [1, 3], [1, 3], and [1, 3, 5] for the 16M, 48M, 110M, 206M, respectively.Across backbones, we use linear layers to encode continuous states, reward returns-to-go. The maximal state dimension across continuous control environments is 204 in our experiments. To use a shared linear embedding layer for continuous states, we pad states that have a lower number of dimensions to 204 dimensions using zeros. To encode image inputs on visual domains, we use the IMPALA-CNN and adopted by previous works on Procgen and Atari. Consequently, we do not make use of discretization of continuous states or patchification of images. This design choice significantly reduces the sequence length to only three tokens per time-step and consequently results in faster inference.For continuous actions, we make use of discretization and discretize of every action dimension into 256 uniformly-spaced bins. We experimented with lower / higher numbers of bins, but did not observe a benefit beyond 256 bins. Consequently, this resolution is sufficient for the environments we consider. We use a shared action head to predict the action bins of all continuous dimensions jointly. The maximum number of continuous action dimensions is 8 in our experiments, and consequently, the number of discrete action classes is 2048. In addition, there are 18 discrete actions originating from Atari and Procgen. Therefore, our action head learns to predict the correct action among the 2066 discrete classes. While different environments may have different action dimensions, the model predicts all action dimensions jointly. At inference time, the number of action dimensions of the current environment is known, and we extract the respective dimensions from the joint predictions. We opt for the shared action head representation, as this further speeds up inference and does not require autoregressive action prediction.For the Transformer baseline, we use global positional embeddings. For the recurrent backbones, we do not make use of positional encodings.Custom Kernels for XLSTMIn certain specific implementations, we leverage custom kernels for XLSTM to conduct our inference-speed comparisons. In particular, we compare four variants: recurrent-style inference with and without kernel acceleration, and chunkwise inference with and without kernel acceleration. In our experiments, every timestep contains three individual tokens. Consequently, regular recurrent-style inference requires iterating over the token sequence of length 3 in a loop, given the hidden state of the previous timestep. This requires three forward passes. In contrast, the chunkwise implementation operates on chunks of timesteps given a hidden state. Consequently, this only requires a single forward pass. We find that our chunkwise kernels result in considerably lower latencies. Interestingly, we find that for B=1, our chunkwise implementation without kernel acceleration is faster than the recurrent-style inference with kernel acceleration. However, as the batch size increases, this trend reverses. This highlights the importance of kernel acceleration for efficient inference.ExperimentsThe following experiments are based on a large-scale dataset comprising 432 tasks from six domains. We leverage datasets from prior works. For Atari, we extract 5M transitions per task from the DQN-Replay dataset released by Agarwal et al.. For Composuite, we leverage the datasets released by Hussing et al.. For Meta-World, we use 2M transitions per task released by Schmied et al.. For DMControl, we generate 10M transitions per task using task-specific RL agents. For Mimicgen, we use the datasets for the 21 tasks released by Mandlekar et al. and generate trajectories for the remaining 62 tasks. Finally, for Procgen, we extract 20M transitions from the datasets released by Schmied et al., 2024b. Our final dataset contains 3.4M trajectories and in total 894M transitions, as shown in the following table:DatasetTasksTrajectoriesMean Trj. LengthTotal TransitionsRepetitionsAtari41136K2733205M1.03×Composuite240480K500240M0.87×DMControl11110K1000110M1.92xMeta-World45450K20090M2.34×Mimicgen8383K30025M8.5×Procgen122185K144224M0.94×Total4323.4M-894M-We reserve an additional 39 tasks from the same domains for zero-shot evaluation.Atari and Procgen come with image observations and discrete action. In contrast, the remaining four domains exhibit state-based observations and continuous actions. Consequently, our experiments involve a mixture of state and action spaces as well as varying episode lengths. Periodically evaluating the trained agents on all 432 tasks sequentially is time-consuming and we, therefore, distributed the evaluation across GPUs and parallel processes.To conduct our main comparisons, we train our four backbone variants on the full training task mixture of 432 tasks. For each architecture backbone, we report performance scores for four model sizes: 16M, 48M, 108M, and 206M parameters. We train all models for 200K updates with a batch size of 128 and context length of 50 timesteps. All domains are represented with approximately equal proportion, resulting in 33K updates per domain.Performance Per DomainIn FIG. 3, we report the normalized scores for the 206M parameter models attained on all six domains. For Meta-World, DMControl, Mimicgen, Composuite, and Procgen we use data-normalized scores. For Atari, we report human-normalized scores. Overall, we observe that the xLSTM backbone outperforms competitors on three of the six domains, while all methods perform similarly on the remaining three domains. These experiments demonstrate that modern recurrent backbones can be attractive alternatives to the Transformer architecture for building large action models.Inference-Time Comparisonwe empirically examine the difference between xLSTM-based and Transformer-based agents at inference time. We report both latency and throughput. We focus our analysis on latency, as it is the more important dimension for real-time applications.We conduct all inference time tests on A100 GPUs with 40GB of RAM using 206M parameter models. For the Transformer, we use KV-caching and FlashAttention as supported by PyTorch. For xLSTM, we use recurrent-style inference using custom kernels to accelerate the computations. For both backbones, we use torch.compile. The Transformer with KV-caching has a linear time complexity per step and quadratic in the sequence length. In contrast, the XLSTM has a constant time complexity per step and linear in the sequence length. Therefore, we expect speed-ups especially for longer sequences and larger batch sizes. To ensure a fair comparison, we compare DT and xLSTM with the same number of layer blocks and increase the hidden size of xLSTM to match the number of parameters of DT.We conduct all inference time tests on the environment that exhibited the longest average episode lengths in our experiments, the Atari game Freeway. Every episode in Freeway lasts for 8192 steps, which is equivalent to 24576 tokens (s / rtg / r). We evaluate all models for 5 episodes and preserve the KV-cache / hidden state across episode boundaries. The reported latencies and throughputs are averaged across all evaluation episodes, except for the first episode, which we discard to exclude compilation times and prefilling. We opted for measuring the inference times during environment interaction, i.e., including simulator latency, rather than mere token generation.We measure latency by the average time (in seconds) taken to perform a single inference step with a fixed batch size B (lower is better). In FIG. 4, we report the latencies for varying context lengths, C\in[50,25600] and two batch sizes B\in\{1,16\}. Note that C is in time steps and every time step contains 3 tokens (state, reward-to-go, reward). Hence, the effective sequence length for the largest C is 76800. As expected, we find that the recurrent backbone attains lower inference latencies than the Transformer one.As the sequence length increases, DT runs out of memory due to the increasing size of the KV cache, as shown in FIG. 5. In contrast, the inference speeds for XLSTM are independent of the context length, and therefore enable significantly longer context lengths. This property is particularly interesting for in-context RL, which requires keeping multiple episodes in the context. Nevertheless, our experiments highlight that the materialization of the complexity advantage (quadratic vs. linear) depends on the device, model size, batch size and the context length.Throughput is measured by the total amount of inference steps performed per second for a model with a fixed context length. In FIG. 6, we report the throughputs for varying batch sizes, B\in[1,128] for a fixed context length of C=1600 Here, the batch size can be interpreted as the number of parallel environments the agent interacts with. As expected, we find that xLSTM attains considerably higher throughputs than the DT. The benefit of XLSTM increases with larger batch sizes. While the DT with quadratic complexity in the sequence length goes OOM for batch sizes above 64, the xLSTM with linear complexity can easily handle larger batch sizes. In both experiments, the recurrent XLSTM performs favorably over the Transformer backbone.Application in RoboticsOne target application of the large recurrent action model 102 of embodiments is robotics. Embodiments of the system 100 and method 200 described herein may implement one or more actions of the following control loop:1. Understand Intent: The system 100 processes a user's goal given in natural language (e.g., "Tidy up the desk").2. Decompose Task: The system 100 breaks the high-level goal into a logical sequence of actionable steps (e.g., find trash, pick it up, move to bin, release trash).3. Perceive Environment: The system 100 uses the robot's sensors (cameras, etc.) to see and understand the real-world space, locating objects and obstacles.4. Generate Commands or Code: The system 100 translates the action steps into low-level action commands 114 or code that the robot's hardware can execute.5. Execute and Adapt: The system 100 sends the action commands 114 to the robot 108 and monitors the outcome, using sensor feedback to correct errors and adjust the plan in real-time.The following are exemplary types of robot tasks supported by embodiments:• Manipulation & Assembly: Tasks include picking and placing components, welding, painting, and assembling products like cars and electronics. Modern robots can handle increasingly delicate and complex assembly, such as placing tiny components on a circuit board.• Logistics & Material Handling: In warehouses and distribution centers, robots excel at tasks like sorting packages, transporting goods on pallets (using Autonomous Mobile Robots or AMRs), and retrieving items from shelves for order fulfillment.• Inspection & Monitoring: Robots equipped with advanced sensors can perform detailed inspection tasks that are often repetitive or dangerous for humans. This includes inspecting infrastructure like bridges and pipelines, monitoring crop health in agriculture with drones, or checking for defects in manufactured goods.• Service & Assistance: Tasks include assisting surgeons in the operating room with high-precision movements, providing companionship and support in elder care, cleaning floors in homes and commercial spaces, and serving food in restaurants.• Exploration & Data Collection: Robots are invaluable for exploring environments that are inaccessible or hazardous to humans. This includes exploring deep-sea vents, navigating the surface of Mars to collect geological data, or mapping disaster sites to aid rescue efforts.The following are examples of action commands 114 for robot control that can be generated in certain embodiments:• Low-Level Commands (Hardware-Specific): These are the most basic instructions that tell the robot's hardware what to do. They are precise and direct.Ο Joint Control: set_joint_angle(joint_3, 90_degrees) - This command tells a specific joint in a robot arm to move to a precise angle.Ο Actuator Commands: set_gripper_voltage(5V) or gripper_close(70%) - These commands control the end-effector, like a gripper, telling it how much force to apply or how far to close.Ο Velocity Commands: set_wheel_velocity(left_motor, 0.5~m / s, right_motor, 0.5\_m / s) - This command instructs the motors of a mobile robot to turn at a specific speed, moving it forward.• Mid-Level Commands (Task-Specific Primitives): These commands abstract away the lowest-level details and represent a complete, simple action. They are often built from a sequence of low-level commands.Ο Movement Primitives: move_to_cartesian(x, y, z) - Instead of specifying each joint angle, this command tells a robot arm to move its end-effector to a specific coordinate in space.Grasping Primitives: pick_up(object_ID) - This command would trigger a pre-programmed sequence: move to the object's location, open the gripper, lower the arm, close the gripper, and lift.Ο Navigation Primitives: Maps_to(waypoint_B) - This tells a mobile robot to find and follow a path to a specific location, handling its own obstacle avoidance along the way.• High-Level Commands (Goal-Oriented): These commands are often expressed in natural language and describe the desired outcome, leaving the "how" up to the robot's Al or planning system (like a Large Action Model). The system then breaks this goal down into a series of mid and low-level commands.Ο Natural Language Instructions: "Get the water bottle from the kitchen table." - The robot must figure out how to navigate to the kitchen, identify the table, locate the bottle, and execute a grasp.Ο Complex Task Goals: "Inspect all the welds on this pipeline section and flag any anomalies." - The robot needs to plan a path for inspection, control its sensors, and run a defect detection algorithm.Ο Symbolic Goals: "Tidy_up(living_room)" - This highly abstract command requires the robot to understand what "tidy" means in that context and generate a long sequence of actions, such as finding objects that are out of place and returning them to their designated spots.Further Technical Application ExamplesEmbodiments of the disclosed method 200 and system 100 are applicable to a wide range of technical systems 108, including but not limited to industrial machinery, vehicles, robotics, process control systems, and building automation systems. Further examples may include:• Industrial Process Control & SCADAΟ Chemical Plant Batch Management: Translating a request like "Produce a 500-gallon batch of formula x-25^{\prime\prime} into a sequence of commands for valves, heaters, and mixers within a PLC or SCADA system, while monitoring sensor data for safety and quality.Ο Power Grid Load Balancing: Responding to real-time energy demand forecasts by adjusting output from different power sources (e.g., hydro, solar, gas turbines) to maintain grid stability.Smart Factory Re-tooling: Reconfiguring an assembly line for a new product by reprogramming machine controllers, adjusting conveyor speeds, and running calibration routines.• Network & Cloud Infrastructure ManagementΟ Automated Network Configuration: Executing commands like "Isolate the marketing department's VLAN from the engineering servers" by generating and applying the correct router and firewall rules.Intelligent Cloud Resource Scaling: Monitoring application performance metrics and user traffic to autonomously execute scaling commands, such as "Add three more web server instances to the load balancer pool and increase the database read replica count."Ο Dynamic Traffic Routing: Responding to network latency issues by analyzing traceroute data and rerouting traffic through different peering connections via BGP commands.• Cybersecurity OperationsΟ Automated Threat Hunting & Response: Acting on alerts like "Investigate suspicious outbound traffic from host server-1138" by querying logs, isolating the host from the network via firewall APIs, and scanning it for malware.Ο Vulnerability Remediation: Automatically generating and applying security patches or configuration changes in response to newly discovered vulnerabilities in the system's software stack.Ο Penetration Testing Automation: Simulating attacks on a network based on high-level goals like "Attempt to gain access to the customer database using common web application exploits."• Scientific & Research ComputingΟ Laboratory Automation: Controlling lab equipment to execute experiments based on a protocol, such as "Perform a serial dilution of the sample, then run it through the mass spectrometer and analyze the resulting peaks."Ο Computational Simulation Management: Steering complex physics or climate simulations by adjusting parameters in real-time based on intermediate results to test a hypothesis.Ο Telescope & Satellite Operation: Planning and executing observation schedules for astronomical telescopes or earth-observation satellites based on scientific proposals and changing weather conditions.Implementation in Hardware and SoftwareEmbodiments of the disclosed technology can be implemented in hardware, software, or both. FIG. 7 illustrates a schematic block diagram of computer hardware on which embodiments of the present disclosure can be implemented. As can be seen, a data processing apparatus 702 comprises one or more processors 704 and one or more memories 706 or storage devices.In embodiments, the data processing apparatus 702 may be a local computer device (e.g., personal computer, laptop, tablet computer or mobile phone), a remote computer device (e.g., remote server or server cluster), or a distributed computer system (e.g., a cloud computing system with one or more processors 704 and one or more memories 706 distributed at various locations, for example, at one or more local clients and / or one or more remote servers). The data processing apparatus 702 may comprise any circuit or combination of circuits.In some embodiments, the data processing apparatus 702 may be embedded within the technical system, such as the robot 108 shown in FIG. 1. For example, it is possible for the computational hardware responsible for implementing the large recurrent action model 102 to be physically integrated into the housing, chassis, or control unit of the technical system 108. Such an arrangement can facilitate direct communication between the data processing apparatus 702 and the various sensors, actuators, or subsystems of the technical system 108, potentially reducing latency and improving reliability in control operations.Alternatively, the data processing apparatus 702 could be realized as an onboard controller, such as an embedded processor, microcontroller, or system-on-chip, which is installed as part of the technical system's internal electronics. This configuration may be particularly advantageous in applications where real-time or near real-time processing is required, or where external network connectivity is limited or undesirable. By embedding the data processing apparatus 702, the system 108 may achieve greater autonomy and robustness, as control and inference tasks can be performed locally without reliance on remote servers or cloud-based resourcesIn embodiments, the one or more processors 704 may comprise any type of computational circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), multiple core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system may be a custom circuit, an application-specific integrated circuit (ASIC), or the like, such as, for example, one or more circuits (such as a communication circuit) for use in wireless devices like mobile telephones, tablet computers, laptop computers, two-way radios, and similar electronic systems.In embodiments, the one or more memories 706 or storage devices may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like.In embodiments, the data processing apparatus 702 may also include a display device 710, one or more speakers, and one or more input devices such as a keyboard 712, a computer mouse 714, trackball, touch screen, voice-recognition device, or any other device that permits a user to input information into and receive information from the data processing apparatus 702.In embodiments, the memory 706 stores a computer program 708. The computer program 708 may also be referred to as a program, software, a software application, an app, a module, a software module, a script, or code. The computer program 708 may be written in a programming language, including compiled or interpreted languages. The computer program 708 may be deployed in any form, including as a stand-alone product or as a module, component, subroutine, or other unit suitable for use in a computing environment, such as the data processing apparatus 702. The computer program 708 may implement some or all aspects of the methods and functionalities of the disclosed technology. Accordingly, embodiments of the disclosed technology can be implemented as a computer program (product) with program code, the program code being operative for performing one of the methods when the computer program runs on a computer. The program code may be stored on a machine-readable carrier.In embodiments, a data stream or a signal sequence may be provided, representing the computer program 708 for performing one of the methods and functionalities described herein. The data stream or the signal sequence may, for example, be configured to be transferred via a data communication connection, for example, via the internet.In embodiments, a data processing system, device or apparatus may be provided, e.g., in the form of a file server, configured to transfer the computer program 708 for performing one of the methods and functionalities described herein to a receiver. The receiver may itself comprise any type of data processing system, device or apparatus.While various aspects, embodiments and examples have been illustrated and described in the foregoing description and the drawings, such illustration and description is illustrative or exemplary and not restrictive. Variations to the disclosed aspects, embodiments and examples can be understood and effected by those skilled in the art in practicing the disclosed technology.Insofar as aspects, embodiments and examples are described, explicitly or implicitly, in the context of a product, apparatus, device or system, such description should be understood as also describing a corresponding process, method or use, and vice versa. Accordingly, aspects, embodiments and examples described in the context of a block, component or feature of a product, apparatus, device or system also represent a description of a corresponding method step or feature of a method or method step. Analogously, aspects, embodiments and examples described in the context of a method or method step also represent a description of a corresponding block, component or feature of a product, apparatus, device or system.Insofar as a method is described, explicitly or implicitly, as comprising actions of a particular entity or component, such description should be understood as also describing a complementary method of a counterpart entity or component. For example, a method described in terms of operations carried out by a client device also discloses a complementary method carried out by a server device that interacts with the client device, and vice versa. In this manner, the disclosed technology encompasses all perspectives of an interaction, regardless of whether a particular embodiment is drafted from one selected perspective.The order of execution of the operations in the described embodiments is not essential, unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and embodiments may include additional or fewer operations than those mentioned.In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single entity may fulfill the functions of several entities recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.Particular and preferred aspects of the disclosed technology are set out in the accompanying claims. Features from the dependent claims may be combined with features of the independent claims and with features of other dependent claims as technically appropriate and not merely as explicitly set out in the claims.

Claims

1. A system for controlling a technical system, comprising:a large recurrent action model implemented on a data processing apparatus and configured to receive inputs representing one or more of user commands or state information of the technical system, and to generate action commands for controlling the technical system;wherein the large recurrent action model comprises a recurrent inference module comprising a recurrent neural network or a state space model.

2. The system of claim 1, wherein the recurrent neural network is an extended long short-term memory.

3. The system of claim 1 or 2, wherein the state space model is a Mamba model.

4. The system of claim 2, wherein the extended long short-term memory comprises a plurality of blocks, and wherein the plurality of blocks comprises a mixture of mLSTM-based blocks and sLSTM-based blocks.

5. The system of claim 4, wherein a ratio of mLSTM-based blocks to sLSTM-based blocks in the plurality of blocks is 7:1.

6. The system of claim 2, wherein the extended long short-term memory comprises a plurality of blocks, and wherein all blocks in the plurality of blocks are mLSTM-based blocks.

7. The system of any one of the preceding claims, wherein the large recurrent action model comprises a plurality of encoders, each encoder configured to process inputs of a different modality.

8. The system of claim 7, wherein the plurality of encoders comprises a convolutional neural network configured to encode image inputs and a fully connected network configured to encode low-dimensional inputs.

9. The system of claim 8, wherein the large recurrent action model is configured to process the image inputs from the convolutional neural network without patchifying the images.

10. The system of claim 8 or 9, wherein the large recurrent action model is configured to process continuous state inputs from the fully connected network without tokenizing the continuous states.

11. The system of any one of the preceding claims, wherein the large recurrent action model is configured to discretize continuous action dimensions into a predefined number of bins.

12. The system of claim 11, wherein the predefined number of bins is 256.

13. The system of claim 11 or 12, further comprising a shared action head configured to jointly predict a plurality of action dimensions, wherein the plurality of action dimensions comprises the discretized continuous action dimensions.

14. The system of claim 13, wherein the shared action head is configured to perform non-autoregressive prediction of the plurality of action dimensions.

15. The system of any one of the preceding claims, wherein the recurrent inference module is configured to operate in a recurrent inference mode by maintaining a hidden state from a previous timestep to enable inference with linear-time complexity with respect to a sequence length of the inputs.

16. The system of claim 15, wherein the recurrent inference module is configured for chunkwise inference, wherein a single forward pass processes a chunk of tokens corresponding to a single timestep.

17. The system of any one of the preceding claims, wherein the large recurrent action model does not use timestep encodings.

18. The system of any one of the preceding claims, wherein the data processing apparatus is embedded in the technical system.

19. A method for controlling a technical system, the method comprising:receiving, at a large recurrent action model, inputs representing one or more of user commands or state information of the technical system;processing the inputs with the large recurrent action model to generate action commands, wherein the large recurrent action model comprises a recurrent inference module comprising a recurrent neural network or a state space model; andcontrolling the technical system based on the generated action commands.

20. The method of claim 19, wherein processing the inputs comprises using an extended long short-term memory as the recurrent neural network.

21. The method of claim 19 or 20, wherein processing the inputs comprises using a Mamba model as the state space model.

22. The method of any one of claims 19 to 21, further comprising:encoding image inputs using a convolutional neural network without patchifying the images; andencoding continuous state inputs using a fully connected network without tokenizing the continuous states.

23. The method of any one of claims 19 to 22, further comprising:discretizing continuous action dimensions of the technical system into a plurality of bins; andwherein generating the action commands comprises jointly predicting the discretized continuous action dimensions using a shared action head.

24. The method of any one of claims 19 to 23, wherein processing the inputs comprises maintaining a hidden state from a previous timestep to enable processing with linear-time complexity with respect to a sequence length of the inputs.

25. The method of claim 24, wherein processing the inputs comprises performing chunkwise inference, wherein a single forward pass processes a chunk of tokens corresponding to a single timestep.

26. A data processing apparatus, device or system comprising means for carrying out the method of any one of claims 1 to 25.

27. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any one of claims 1 to 25.

28. A robot, comprising:one or more sensors for acquiring state information about the robot and its environment;one or more actuators for performing actions; anda control system comprising the system of any one of claims 1 to 25, wherein the control system is configured to:receive the state information from the one or more sensors as inputs to the large recurrent action model; andcontrol the one or more actuators using the action commands generated by the large recurrent action model.

29. The robot of claim 28, wherein the one or more sensors comprise at least one of: a camera, a LiDAR sensor, a tactile sensor, a force-torque sensor, or an inertial measurement unit.

30. The robot of claim 28 or 29, wherein the one or more actuators comprise at least one of: a joint of a robotic arm, a gripper, or a wheel motor.

31. The robot of any one of claims 28 to 30, wherein the robot is one of: a robotic arm, a mobile robot, a humanoid robot, or a drone.

32. The robot of any one of claims 28 to 31, wherein the control system is configured to receive a user command in natural language, and wherein the large recurrent action model is configured to generate the action commands by decomposing the user command into a sequence of actionable steps for the one or more actuators.

33. The robot of any one of claims 28 to 32, wherein the action commands comprise low-level commands, including at least one of: a joint angle command, an actuator voltage command, or a motor velocity command.

Images