Continuous prediction method based on physical time, robot control method, and prediction device

Abstract

The application provides a continuous prediction method based on physical time, a robot control method and a prediction device, and relates to the technical field of artificial intelligence, and the method comprises the following steps: acquiring observation data of a current moment and target conditions for representing a prediction target; extracting state features of the observation data of the current moment as initial state features, and mapping the initial state features to a dynamic representation space to obtain initial dynamic representations; taking the initial dynamic representations as integral starting points, calling a common differential equation solver to integrate a velocity field defined by a pre-trained velocity prediction network on a physical time axis to obtain target dynamic representations corresponding to future target moments; and decoding the target dynamic representations in combination with the initial state features to generate state features of the future target moments. Through the method provided by the application, the time evolution of the physical world can be continuously modeled, and the dependence of traditional discrete prediction methods on a fixed sampling rate is overcome.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a continuous prediction method, robot control method and prediction device based on physical time. Background Technology

[0002] In the field of predictive modeling in artificial intelligence, especially in video-based physical world simulation and robot planning tasks, the mainstream technical approach is to discretize the time dimension, transform the continuously changing physical process into a sequence of image frames with fixed time intervals, and then use a scalar architecture such as recurrent neural networks, transformers, or diffusion models to predict future states.

[0003] While these methods have made significant progress on standard benchmarks, they essentially force continuously dynamic systems in physical time to be transformed into discrete, abruptly changing labeled sequences. This makes it difficult for the models to internalize the high-order physical laws implicit in the real world. They also have stringent requirements for the temporal alignment and sampling rate of training data, cannot natively handle irregular sampling or dropped frame data, and are prone to error accumulation and representation collapse when making long-term predictions, resulting in poor temporal consistency of the generated results. Meanwhile, latent world models, represented by joint embedding prediction architectures, alleviate some of the computational efficiency problems through learning in low-dimensional space, but are still based on the framework of discrete-time prediction. They not only fail to escape the above limitations, but also face the potential risk of representation collapse due to the indirect coupling supervision between the latent predictor and the encoder, requiring a trade-off between model expressiveness and training stability.

[0004] Therefore, how to construct a prediction method that can faithfully reflect the continuous-time nature of the physical world, support predictions at arbitrary time resolutions, and maintain stable and efficient expressive power during the learning process has become an urgent technical problem to be solved. Summary of the Invention

[0005] This invention provides a continuous prediction method, robot control method, and prediction device based on physical time, in order to overcome the deficiencies in the prior art.

[0006] This invention provides a continuous prediction method based on physical time, comprising: Obtain the observation data at the current moment and the target conditions used to characterize the predicted target; Extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; Using the initial dynamic representation as the starting point of integration, the ordinary differential equation solver is invoked to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis to obtain the target dynamic representation corresponding to the future target time. The velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs and outputs the dynamic change rate at the current time. The target dynamic representation is combined with the initial state features for decoding to generate the state features of the target at future time moments.

[0007] According to a continuous prediction method based on physical time provided by the present invention, the step of mapping the initial state features to a dynamic representation space to obtain an initial dynamic representation includes: The initial state features are input into the dynamic encoder, and the initial state features themselves are used as conditional signals. The dynamic encoder outputs the initial dynamic representation that focuses on characterizing dynamic change information. The step of decoding the dynamic representation of the target combined with the initial state features to generate the state features of the target at future time moments includes: The target dynamic representation and the initial state features are input into the reconstruction decoder. The reconstruction decoder uses the initial state features as the base frame information and fuses the change information contained in the target dynamic representation to reconstruct the state features of the future target moment.

[0008] According to the continuous prediction method based on physical time provided by the present invention, both the dynamic encoder and the reconstruction decoder are implemented based on the cross-attention mechanism; In the dynamic encoder, a set of learnable query tokens are configured, using the initial state features and the state features to be encoded as keys and values, and a corresponding dynamic representation is generated through cross-attention operations; In the reconstruction decoder, the initial state features are used as queries, and the dynamic representation of the input is used as keys and values ​​to generate state features through cross-attention operations.

[0009] According to a continuous prediction method based on physical time provided by the present invention, the training process of the velocity prediction network includes: Obtain a training dataset consisting of a sequence of state samples ordered by physical time, extract the state features at each time step, and use the state features of the first frame of the sequence as the initial state features for training. The initial state features for training and the state features at each time step are input into the dynamic encoder, and the dynamic representation sequence for training at each time step is generated based on the initial state features for training. Based on the state characteristics at each time step in the state sample sequence, the time derivative of the state characteristics is calculated, and the partial derivative of the state characteristics with respect to the state characteristics is mapped to the dynamic representation space using the dynamic encoder to obtain the potential velocity target value at each time step. The training dynamic representation at each time step, the corresponding time information, the initial dynamic representation for training, and the target conditions are input into the velocity prediction network to be trained. The predicted potential velocity is output, and the velocity loss function is constructed to optimize the velocity prediction network using the target potential velocity as the supervision signal.

[0010] According to a continuous prediction method based on physical time provided by the present invention, the step of calculating the time derivative of the state features based on the state features at each time point in the state sample sequence includes: A target differential filter is used to perform local polynomial fitting on the state sample sequence in the time dimension by taking a sliding window, and the smooth state feature time derivative is obtained by taking the derivative based on the fitted polynomial.

[0011] According to the present invention, a continuous prediction method based on physical time is provided. The time information received by the speed prediction network is the normalized time obtained after normalizing the actual physical time. The normalization process includes: dividing the actual physical time of the current moment by a preset normalized time window length; wherein the normalized time window length corresponds to the time span of the longest sample sequence in the training data, so that the value of the normalized time falls within a preset numerical range during training.

[0012] According to the present invention, a continuous prediction method based on physical time is provided, wherein the target condition includes a target state; After inputting the initial state features for training and the state features at each time step into the dynamic encoder, and generating the dynamic representation sequence for training at each time step based on the initial state features for training, the method further includes: If the normalization time corresponding to the current integration time exceeds the termination normalization time corresponding to the target condition, the dynamic representation is fixed to the dynamic representation corresponding to the target state, and the dynamic change rate of the velocity prediction network output tends to zero.

[0013] According to the present invention, a continuous prediction method based on physical time is provided. The velocity prediction network is a multilayer perceptron and is configured with multiple feature linear modulation layers; The feature linear modulation layer receives the time information and, based on the time information, scales and biases the feature map of the corresponding intermediate layer in the multilayer perceptron channel by channel to incorporate the time information into the network forward process.

[0014] According to a continuous prediction method based on physical time provided by the present invention, after mapping the initial state features to a dynamic representation space to obtain an initial dynamic representation, the method further includes: When calling the ordinary differential equation solver, the time integration step size is set to a negative value so that the integration proceeds in reverse physical time to obtain a retrospective dynamic representation corresponding to the past target time. The backtracking dynamic representation is combined with the initial state features for decoding to generate the state features of the past target time.

[0015] According to a continuous prediction method based on physical time provided by the present invention, after decoding the dynamic representation of the target combined with the initial state features to generate the state features of the future target time, the method further includes: The state features of the future target time are input into a pre-trained pixel decoder to output a predicted image frame; By repeatedly performing state feature prediction and pixel decoding at different future time points, a continuous sequence of image frames is generated, forming a future video.

[0016] The present invention also provides a robot control method, comprising: Acquire the robot's current observation image and the target conditions used to characterize the task objective; Using any of the above-described continuous prediction methods, the current observed image and the target conditions are used as input to generate a sub-target state feature sequence for multiple future time steps; The sub-target state feature sequence is successively input into a pre-trained policy network, and the corresponding control action sequence is output to control the robot to perform the task objective.

[0017] The present invention also provides a continuous prediction device based on physical time, comprising: The acquisition module is used to acquire the observation data at the current moment and the target conditions used to characterize the predicted target; The mapping module is used to extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; The representation module is used to call the ordinary differential equation solver to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis, using the initial dynamic representation as the starting point of integration, to obtain the target dynamic representation corresponding to the future target time; wherein, the velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs, and outputs the dynamic change rate at the current time. The prediction module is used to decode the dynamic representation of the target combined with the initial state features to generate the state features of the target at future time moments.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the physical time-based continuous prediction method and the robot control method as described above.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the physical time-based continuous prediction method and robot control method as described above.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the physical time-based continuous prediction method and the robot control method as described above.

[0021] This invention provides a continuous prediction method, robot control method, and prediction device based on physical time. It acquires observation data at the current moment and target conditions characterizing the predicted target; extracts the state features of the observation data at the current moment as initial state features, and maps these initial state features to a dynamic representation space to obtain an initial dynamic representation; using this initial dynamic representation as the starting point for integration, it calls an ordinary differential equation solver to integrate the velocity field defined by a pre-trained velocity prediction network along the physical time axis to obtain a target dynamic representation corresponding to the future target moment; wherein the velocity prediction network takes the current dynamic representation, the time information corresponding to the current integration moment, the initial dynamic representation, and the target conditions as inputs, and outputs the dynamic change rate at the current moment; the target dynamic representation is decoded in conjunction with the initial state features to generate the state features of the future target moment. Therefore, this invention generates the future state by integrating the velocity field along the physical time axis in the dynamic representation space using ordinary differential equations, making the prediction process independent of a fixed discrete time step, thus enabling continuous modeling of the temporal evolution of the physical world and overcoming the dependence of traditional discrete prediction methods on a fixed sampling rate. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the continuous prediction method based on physical time provided by the present invention.

[0024] Figure 2 This is a complete flowchart of the continuous prediction method based on physical time provided by the present invention.

[0025] Figure 3 This is a flowchart illustrating the robot control method provided by the present invention.

[0026] Figure 4 This is a schematic diagram of the structure of the continuous prediction device based on physical time provided by the present invention.

[0027] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The following description, in conjunction with the accompanying drawings, describes a continuous prediction method, robot control method, and prediction device based on physical time according to the present invention.

[0030] Figure 1 This is a flowchart illustrating the continuous prediction method based on physical time provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: Step 100: Obtain the observation data at the current moment and the target conditions used to characterize the predicted target.

[0031] Specifically, observation data refers to the raw information perceived by the system in the physical world, such as an RGB image frame of a robot's working scene or a segment of LiDAR scan data. Target conditions are guiding signals used to characterize the prediction of the final intention; they can be a target image at the end of a demonstration, a natural language instruction describing the semantics of the task, or other external information that can indicate the desired future state.

[0032] Step 200: Extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation.

[0033] Specifically, firstly, a pre-trained feature extractor (e.g., the DINOv2 visual transformer model) compresses the high-dimensional current observation data into compact state features, which are regarded as the initial state features of the entire prediction process in the time dimension. Then, the initial state features are fed into a dynamic encoder conditioned on itself. This encoder focuses on extracting essential information related to physical changes such as object motion and interaction, and filters out time-invariant details such as static background. Finally, it maps the data into a low-dimensional dynamic representation space, forming a compact vector that only captures what dynamic changes have occurred relative to the beginning, i.e., the initial dynamic representation.

[0034] Step 300: Using the initial dynamic representation as the starting point of integration, call the ordinary differential equation solver to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis to obtain the target dynamic representation corresponding to the future target time; wherein, the velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs, and outputs the dynamic change rate at the current time.

[0035] Specifically, this embodiment redefines future prediction as an initial value problem for an ordinary differential equation over continuous time: starting with the initial dynamic representation, a pre-trained ordinary differential equation solver (e.g., a fourth-order Runge-Kutta algorithm) is invoked to integrate along the real physical time direction. At each integration step, the solver invokes a velocity prediction network. This network is a lightweight neural network that takes four elements as input: the dynamic representation of the current integration step, the time information corresponding to the current integration moment, the initial dynamic representation at the beginning of the entire prediction process, and the target conditions. It outputs the rate of change of the dynamic representation at the current moment, which is the instantaneous velocity vector of the dynamic representation in the latent space. By continuously integrating this velocity field along the entire physical time axis, the solver can obtain the target dynamic representation that precisely corresponds to any future target moment.

[0036] In a preferred embodiment of the present invention, the velocity field is defined by the following ordinary differential equation, which describes the law governing the continuous evolution of the dynamic representation with physical time: in, t Represents physical time. z t For a moment t The dynamic representation,z 0 represents the initial dynamic representation. c As the target condition, v θ To predict the dynamic rate of change of the network output. Based on this equation, from the initial time to the target time... T The dynamic representation can be obtained through integration: The integration process is performed by an ordinary differential equation solver on a continuous physical time axis.

[0037] Step 400: Decode the target dynamic representation in combination with the initial state features to generate the state features of the future target moment.

[0038] Specifically, this dynamic representation of the target, along with the initial state features, is fed into a reconstruction decoder. This decoder uses the global scene structure and background information carried by the initial state features as the base frame, and corrects and supplements it with the cumulative change information contained in the dynamic representation of the target, thereby decoding and generating the state features for the future target moment. This state feature retains the structural integrity of the initial observation while accurately reflecting the physical changes that should occur in the future. If a trained pixel decoder is connected at the backend, this state feature can be directly rendered into a high-quality future prediction image frame. This entire process allows prediction to no longer be limited to predefined discrete frame intervals, enabling dynamic prediction at arbitrary resolution and in any direction across the continuous spectrum of physical time.

[0039] The above describes the steps of the continuous prediction method based on physical time provided by this invention. As can be seen from the above description, the continuous prediction method based on physical time provided by this invention involves acquiring observation data at the current moment and target conditions used to characterize the predicted target; extracting the state features of the observation data at the current moment as initial state features, and mapping the initial state features to a dynamic representation space to obtain an initial dynamic representation; using the initial dynamic representation as the starting point of integration, calling an ordinary differential equation solver to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis to obtain the target dynamic representation corresponding to the future target moment; wherein, the velocity prediction network takes the dynamic representation at the current moment, the time information corresponding to the current integration moment, the initial dynamic representation, and the target conditions as inputs, and outputs the dynamic change rate at the current moment; the target dynamic representation is decoded in combination with the initial state features to generate the state features of the future target moment. Therefore, this invention generates future states by integrating the velocity field along the physical time axis in the dynamic representation space using ordinary differential equations, so that the prediction process no longer depends on a fixed discrete time step, thus enabling continuous modeling of the temporal evolution of the physical world and overcoming the dependence of traditional discrete prediction methods on a fixed sampling rate.

[0040] Based on the above embodiments, in this embodiment, step 200 maps the initial state features to a dynamic representation space to obtain an initial dynamic representation, including: Step 210: Input the initial state features into the dynamic encoder, and use the initial state features themselves as a condition signal, so that the dynamic encoder outputs the initial dynamic representation that focuses on characterizing dynamic change information.

[0041] Step 400 decodes the target dynamic representation in combination with the initial state features to generate the state features of the future target time, including: Step 410: Input the target dynamic representation and the initial state features into the reconstruction decoder. The reconstruction decoder uses the initial state features as the base frame information and fuses the change information contained in the target dynamic representation to reconstruct the state features of the future target moment.

[0042] It should be noted that both the dynamic encoder and the reconstruction decoder are implemented based on a cross-attention mechanism. In the dynamic encoder, a set of learnable query tokens are configured, and the initial state features and the state features to be encoded are used as keys and values ​​to generate the corresponding dynamic representation through cross-attention operations. In the reconstruction decoder, the initial state features are used as queries, and the input dynamic representation is used as keys and values ​​to generate state features through cross-attention operations.

[0043] Specifically, in this embodiment, both the dynamic encoder and the reconstruction decoder employ a cross-attention architecture. The task of the dynamic encoder is to extract the truly time-evolution-related components from the state features. To this end, it internally maintains a set of learnable query tokens (i.e., trainable vector probes). These probes simultaneously focus on the initial state features and the current input state features. Through cross-attention operations, the static background information shared by both is subtracted or suppressed, extracting only the information representing the dynamic changes that have occurred in the current state relative to the initial state, ultimately condensing into a compact dynamic representation vector. It is worth noting that the dimension of this dynamic representation is much smaller than the dimension of the original state features. After predicting the target dynamic representation, the reconstruction decoder is responsible for re-injecting the change information into the static scene framework. Specifically, the initial state features are used as queries, and the target dynamic representation is used as a key and value for cross-attention operations. At this time, the complete scene layout, object appearance, and background texture carried by the initial state features serve as the base frame information, while the target dynamic representation serves as a change instruction, telling the decoder what kind of deformation, displacement, or occlusion relationship should occur at each spatial location, thereby accurately reconstructing the state features of the future target moment.

[0044] The physical time-based continuous prediction method provided in this embodiment eliminates the burden of describing the static appearance of the scene in the dynamic representation itself, allowing it to focus solely on modeling the physical change process, thus significantly improving the stability of long-term prediction.

[0045] Based on the above embodiments, in this embodiment, the training process of the velocity prediction network includes: Step S310: Obtain a training dataset consisting of a sequence of state samples ordered by physical time, and extract the state features at each time step. Use the state features of the first frame of the sequence as the initial state features for training.

[0046] Step S320: Input the initial state features for training and the state features at each time step into the dynamic encoder, and generate the dynamic representation sequence for training at each time step using the initial state features for training as a condition.

[0047] Step S330: Based on the state features at each time step in the state sample sequence, calculate the time derivative of the state features, and use the partial derivative of the state features with respect to the state features by the dynamic encoder to map the time derivative of the state features to the dynamic representation space to obtain the potential velocity target value at each time step.

[0048] Step S330 specifically includes: A target differential filter is used to perform local polynomial fitting on the state sample sequence in the time dimension by taking a sliding window, and the smooth state feature time derivative is obtained by taking the derivative based on the fitted polynomial.

[0049] Step S340: Input the training dynamic representation at each time point, the corresponding time information, the initial dynamic representation for training, and the target conditions into the velocity prediction network to be trained, output the predicted potential velocity, and use the potential velocity target value as a supervision signal to construct a velocity loss function to optimize the velocity prediction network.

[0050] Specifically, training first obtains a sequence of state samples arranged in physical time order from the dataset. Each frame is converted into state features using a pre-trained feature extractor, and the state features of the first frame of the sequence are designated as the initial state features for training. Subsequently, using the aforementioned dynamic encoder, the state features at each moment in the sequence are encoded conditioned on these initial state features, generating a dynamic representation sequence for training. The dynamic representation of each frame only carries information about the dynamic changes that have occurred relative to the starting frame, without containing any static scene content.

[0051] To obtain the target signal, i.e., the latent velocity target value, required for training the velocity prediction network, this embodiment introduces a direct first-order supervision strategy: First, the time derivative of each time step in the state feature sequence is calculated. Specifically, the Savitzky-Golay differential filter, a derivative method based on sliding window local polynomial fitting, is used. Within a time window containing several frames, a low-order polynomial is fitted to the change curve of the state feature, and then the derivative of the fitted polynomial is calculated. This filters out high-frequency noise while preserving the true change trend, resulting in a smooth state feature time derivative. Then, the partial derivative of the input state feature with respect to the dynamic encoder (i.e., the Jacobian matrix) is used to accurately project the time derivative in the state feature space onto the dynamic representation space through the Jacobian vector product operation, thereby obtaining the latent velocity target value corresponding to each time step. This operation utilizes the chain rule: the latent velocity is equal to the partial derivative of the encoder with respect to the state feature multiplied by the time derivative of the state feature.

[0052] Finally, the training dynamic representations at each time step, the corresponding time information, the initial dynamic representation for training, and the target conditions are all input into the velocity prediction network to be trained. This causes the network to output a predicted value of the dynamic change rate at that time step (i.e., the predicted latent velocity). The previously calculated target latent velocity value is used as a supervision signal, and the error between the two is calculated as the velocity loss function to directly optimize the parameters of the velocity prediction network. This direct and decoupled first-order supervision mechanism ensures that the learning of the velocity field no longer depends on the consistency constraints of latent variables at future time steps, thus fundamentally avoiding the risk of representation collapse.

[0053] In a preferred embodiment of the present invention, in order to obtain the potential velocity target value for supervising the velocity prediction network, the time derivative of the state features is mapped to the dynamic representation space using Jacobi-vector product projection, and the mathematical relationship is as follows: in, st For a moment t State characteristics, s 0 represents the initial state characteristic. f dyn is the dynamic encoder function. The time derivative of the state characteristic, This represents the approximate true latent velocity obtained after mapping (i.e., the supervised target). Due to the characteristics of the initial state... s 0 does not change over time, and this projection formula can be calculated simply by taking the partial derivative with respect to the current state characteristics.

[0054] Furthermore, the time information received by the velocity prediction network is a normalized time obtained by normalizing the actual physical time; the normalization process includes: dividing the actual physical time at the current moment by a preset normalized time window length; wherein, the normalized time window length corresponds to the time span of the longest sample sequence in the training data, so that the value of the normalized time falls within a preset numerical range during training.

[0055] The target condition includes a target state; after step S320 inputs the initial state features for training and the state features at each time step into the dynamic encoder, and generates the dynamic representation sequence for training at each time step using the initial state features for training as a condition, the method further includes: If the normalization time corresponding to the current integration time exceeds the termination normalization time corresponding to the target condition, the dynamic representation is fixed to the dynamic representation corresponding to the target state, and the dynamic change rate of the velocity prediction network output tends to zero.

[0056] Specifically, considering that the physical duration of different training sample sequences may vary greatly, directly inputting physical times with huge absolute differences into the network will lead to unstable gradient updates and difficulty in generalization. Therefore, the time information received by the velocity prediction network is normalized: the actual physical time at the current integration moment is divided by a preset normalized time window length. This window length is set to the physical time span corresponding to the longest sample sequence in the training dataset, so that the normalized time value mainly falls within the preset value range of zero to one during training, thereby providing the network with a unified, dimensionless time reference system.

[0057] Building upon this, for cases where the target conditions include a clearly defined target state, this embodiment also designs an endpoint convergence strategy: During the training phase, when the normalization time corresponding to the current integration moment of the ordinary differential equation solver exceeds the termination normalization time corresponding to the target condition, the system determines that the prediction has reached or exceeded the expected endpoint. At this point, it no longer updates the dynamic representation, but instead forces it to be fixed as the dynamic representation obtained after the target state is encoded by the dynamic encoder, and simultaneously forces the dynamic change rate of the velocity prediction network output in this region to tend towards zero. The physical meaning of this design is that once the system reaches the target state in the latent space, it should remain in that state and no longer evolve, which is equivalent to constructing a stable attractor in the velocity field. This processing not only eliminates invalid or conflicting supervision information that may be introduced into the training signal due to exceeding the target time, but also allows the learned latent velocity field to naturally form a zero-velocity convergence region near the target state. Thus, even if the number of integration steps exceeds the expectation during inference, the generated state feature sequence can smoothly terminate at the target state instead of diverging or drifting.

[0058] The physical time-based continuous prediction method provided in this embodiment directly supervises the velocity prediction network by extracting smooth latent velocity target values ​​directly from the state feature sequence, and introduces time normalization and endpoint convergence constraints to enable the model to stably learn continuous time dynamics, avoid representation collapse in discrete modeling, and ensure that the predicted trajectory smoothly converges to the target state.

[0059] Based on the above embodiments, in this embodiment, the velocity prediction network is a multilayer perceptron and is configured with multiple feature linear modulation layers; The feature linear modulation layer receives the time information and, based on the time information, scales and biases the feature map of the corresponding intermediate layer in the multilayer perceptron channel by channel to incorporate the time information into the network forward process.

[0060] Specifically, the velocity prediction network is implemented using a lightweight and computationally efficient multilayer perceptron, which incorporates multiple feature linear modulation layers (LMDs) specifically designed to organically inject continuously changing temporal information into the network's layer-by-layer forward inference process. In this embodiment, LMDs are inserted after each intermediate layer of the multilayer perceptron. Each LMD receives the temporal information corresponding to the current integration time as a conditional signal, maps it to a set of scaling and bias coefficients through a small subnetwork, and then scales and biases each channel of the intermediate layer feature map using channel-by-channel multiplication and addition. This design allows the LMDs to differentiate the scaling and translation of intermediate features based on time, even when the input dynamic representations are similar at different integration times. This enables the network to flexibly adjust the dynamic rate of its output according to the current time, thereby more accurately capturing the heterogeneous dynamic behaviors (such as acceleration, deceleration, and gradual stopping upon approaching a target) that the physical system may exhibit at different evolutionary stages. All of this is achieved within a network framework with extremely few parameters and extremely high inference speed.

[0061] The physical time-based continuous prediction method provided in this embodiment uses a lightweight multilayer perceptron combined with a feature linear modulation layer to inject time information into the network layer by layer. This enables the velocity prediction network to maintain high inference efficiency while adaptively adjusting the dynamic change rate according to different integration times, thereby accurately capturing the non-uniform dynamic behavior of the physical system in the continuous time evolution process.

[0062] Based on the above embodiments, in this embodiment, after step 200 maps the initial state features to the dynamic representation space to obtain the initial dynamic representation, the method further includes: When calling the ordinary differential equation solver, the time integration step size is set to a negative value so that the integration proceeds in reverse physical time to obtain a retrospective dynamic representation corresponding to the past target time. The backtracking dynamic representation is combined with the initial state features for decoding to generate the state features of the past target time.

[0063] Specifically, when generating state features for a past target time, the initial dynamic representation obtained by encoding the current observation is still used as the starting point. However, when calling the ordinary differential equation solver, the time integration step size is set to a negative value. Based on this negative step size, the ordinary differential equation solver iteratively calls the velocity prediction network in reverse along the physical time axis: in each inverse integration step, the solver queries the velocity prediction network for the dynamic change rate at that moment based on the current dynamic representation, the corresponding inverse time information, the initial dynamic representation, and the target conditions, and updates the dynamic representation by multiplying the rate by the negative time step size, thereby gradually backtracking to the past target time and obtaining the corresponding backtracked dynamic representation. Subsequently, this backtracked dynamic representation and the initial state features are fed into the reconstruction decoder. Using the initial state features as the base frame information and the inverse change information carried by the backtracked dynamic representation as the guide, the state features for the past target time are reconstructed, which can then be rendered as image frames of the past time.

[0064] The continuous prediction method based on physical time provided in this embodiment, compared with the traditional discrete prediction model which can only recursively predict frame by frame and has a rigid dependence on the time order, can simultaneously support prediction of any time in the future and the past by simply inverting the sign, which greatly expands the application potential of the model in scenarios such as video interpolation, motion backtracking, and causal reasoning.

[0065] Based on the above embodiments, in this embodiment, after step 400 decodes the target dynamic representation in combination with the initial state features to generate the state features of the future target time, the method further includes: The state features of the future target time are input into a pre-trained pixel decoder to output a predicted image frame; By repeatedly performing state feature prediction and pixel decoding at different future time points, a continuous sequence of image frames is generated, forming a future video.

[0066] Specifically, the pixel decoder is trained in the independent stage by combining L1 reconstruction loss in pixel space, similarity loss (such as LPIPS) in perceptual feature space, and generative adversarial loss. This enables it to faithfully recover rich visual information such as texture details, object contours, and scene lighting from compressed state features, without incurring the computational burden of pixel-level reconstruction during dynamic modeling. In actual inference, for each future time point to be predicted, the target dynamic representation at that moment is first obtained by integrating in the dynamic representation space using an ordinary differential equation solver. This dynamic representation is then fused with the initial state features by the reconstruction decoder to generate the state features for that moment. Finally, the state features are fed into the pixel decoder to output the predicted image frame for that moment. By specifying a series of equally or arbitrarily spaced future time points and repeatedly executing the above integration-decoding-rendering process, a complete sequence of temporally continuous and smoothly moving image frames can be generated, forming a complete future video.

[0067] The physical time-based continuous prediction method provided in this embodiment can output ultra-high-definition slow motion at a high frame rate, and can also achieve temporal super-resolution through dense sampling when the frame rate of the training data itself is low, overcoming the inherent limitation of traditional discrete prediction models that are strictly limited by the training frame interval.

[0068] Figure 2 This is a complete flowchart of the continuous prediction method based on physical time provided by the present invention. The following is in conjunction with... Figure 2 The continuous prediction method based on physical time provided by this invention will be fully described.

[0069] like Figure 2 As shown, the core of this invention is a continuous prediction architecture called Physical-time flow (PT-Flow). First, a pre-trained and frozen visual backbone network, the DINO encoder, processes the original high-dimensional image observations... x t Projecting this onto a compressed feature space, called the state space, yields the state representation. s t Next, a dynamic encoder with initial state conditionalization was introduced. f dyn The encoder is in its current state. s t As a query, in the initial state s Using 0 as both key and value, a cross-attention mechanism is used to extract purely dynamic change information, which is then encoded into a highly compact dynamic latent representation. z tThis process is called dynamic representation decoupling, and its purpose is to allow the latent representation to be decoupled. z t It focuses solely on the dynamic changes of the system, ignoring redundant information such as static background. Subsequently, a lightweight multilayer perceptron (MLP) was used as the speed network. v θ This is used to parameterize a latent velocity field defined over continuous physical time. To train this velocity field, this invention employs a direct first-order supervised method: firstly, a derivative filter is used to perform local polynomial regression on the state sequence to accurately estimate the true velocity of the state. Then, through a Jacobian-Vector Product (JVP) projection, the velocities in the state space are mapped to the latent space to obtain approximate true latent velocities as a supervision signal. This supervision signal directly optimizes the velocity network. v θ This makes its predicted potential speed consistent with z t Alignment with the true rate of change.

[0070] During the inference phase, given an initial observation and target conditions (such as an image target or a language instruction), this invention utilizes an ordinary differential equation solver (ODE solver) (e.g., fourth-order Runge-Kutta method RK4) in normalized physical time. τ The learned velocity field v θ Integrating is performed to predict the potential state at any future time point. Finally, an initial state-conditional dynamics decoder is used. g dyn Compare the predicted latent representation with the static initial state. s 0-fusion reconstructs the state-space representation, which is then processed by the image decoder. g obs Decode to generate the final predicted video frames. This enables continuous-time, high-fidelity future prediction.

[0071] Figure 3 This is a flowchart illustrating the robot control method provided by the present invention, as shown below. Figure 3 As shown, the method includes the following: Step 500: Obtain the robot's current observation image and the target conditions used to characterize the task objective.

[0072] Step 600: Using any of the above-described continuous prediction methods, with the current observed image and the target conditions as input, generate a sub-target state feature sequence for multiple future time steps.

[0073] Step 700: Input the sub-target state feature sequence into the pre-trained policy network one by one, and output the corresponding control action sequence to control the robot to perform the task target.

[0074] It should be noted that, in this embodiment, the aforementioned continuous prediction method is specifically applied to a robot control scenario.

[0075] Specifically, in step 500, the robot acquires real-time observation images of the current working scene through its onboard visual sensors (such as a depth camera or an RGB camera); at the same time, the system receives a target condition to characterize the task objective, such as a target image showing the expected position that the object should reach when the task is completed, or a natural language instruction.

[0076] In step 600, the system invokes the continuous prediction method described in any of the preceding embodiments, using the current observed image as the starting state and the task target condition as the endpoint guidance. By integrating the pre-trained velocity field using ordinary differential equations in the dynamic representation space, a sequence of sub-target state features for multiple future time steps is generated. The sub-target state feature sequence refers to a series of intermediate state features sampled at specific time intervals along a latent space path that evolves from the current state along the learned physical dynamics to the target state. Each intermediate state feature corresponds to a visually reachable and physically plausible sub-target image, and together they depict the complete motion trajectory of how the robot should gradually approach and ultimately complete the task.

[0077] In step 700, the state features of these sub-targets are sequentially input into a pre-trained policy network. This policy network takes the state features as perceptual input and the corresponding control actions as output, sequentially outputting a sequence of control actions matching each sub-target. This drives the robot to smoothly execute operations along the planned latent space path, ultimately completing the specified task objective. Compared to the end-to-end control paradigm that directly maps current observations to the final action, this hierarchical control architecture, mediated by continuous latent space sub-targets, provides richer intermediate supervision signals and smoother guidance trajectories, significantly improving the success rate and execution stability of long-term complex operation tasks.

[0078] It should be noted that the prediction framework based on continuous physical time modeling of this invention is not limited to robot operation scenarios, but can also be extended to a wider range of embodied intelligence and environmental simulation applications. For example, in autonomous driving scenarios, observation data can be multi-view images and LiDAR point clouds around the vehicle, and the target condition can be the endpoint coordinates of the navigation path or the predicted trajectory of the preceding vehicle. This method can predict the future state sequence of the vehicle and surrounding traffic participants on a continuous time axis. In the fields of weather forecasting or physical simulation, observation data can be the distribution maps of physical quantities such as temperature and pressure fields at the current moment, and the target condition can be the evolution label of an extreme weather event. This method can extrapolate the evolution process of physical fields over continuous time at arbitrary time granularity, achieving high temporal resolution climate simulation. In human-machine collaboration or augmented reality scenarios, this method can perform continuous-time motion prediction and interpolation of human actions, enabling virtual characters or collaborative robots to more naturally and smoothly predict and respond to the intentions of human partners. In short, any task that needs to learn the continuous-time dynamic evolution law from discrete observations and predict future states accordingly can benefit from the unified prediction paradigm provided by this invention, and this invention does not impose any particular limitations on it.

[0079] The physical time-based continuous prediction device provided by the present invention will be described below. The physical time-based continuous prediction device described below and the physical time-based continuous prediction method described above can be referred to in correspondence.

[0080] Figure 4 This is a schematic diagram of the structure of the continuous prediction device based on physical time provided by the present invention, as shown below. Figure 4 As shown, the continuous prediction device based on physical time provided by the present invention includes: The acquisition module 401 is used to acquire the observation data at the current moment and the target conditions used to characterize the predicted target; The mapping module 402 is used to extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; The representation module 403 is used to call the ordinary differential equation solver to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis, using the initial dynamic representation as the starting point of integration, to obtain the target dynamic representation corresponding to the future target time; wherein, the velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs, and outputs the dynamic change rate at the current time. The prediction module 404 is used to decode the target dynamic representation in combination with the initial state features to generate the state features of the target at future time.

[0081] The present invention provides a continuous prediction device based on physical time. It acquires observation data at the current moment and target conditions characterizing the predicted target; extracts the state features of the observation data at the current moment as initial state features, and maps these initial state features to a dynamic representation space to obtain an initial dynamic representation; using this initial dynamic representation as the starting point for integration, it calls an ordinary differential equation solver to integrate the velocity field defined by a pre-trained velocity prediction network along the physical time axis to obtain a target dynamic representation corresponding to the future target moment; wherein the velocity prediction network takes the current dynamic representation, the time information corresponding to the current integration moment, the initial dynamic representation, and the target conditions as inputs, and outputs the dynamic change rate at the current moment; the target dynamic representation is decoded in conjunction with the initial state features to generate the state features of the future target moment. Therefore, the present invention generates the future state by integrating the velocity field along the physical time axis in the dynamic representation space using ordinary differential equations, making the prediction process independent of a fixed discrete time step, thus enabling continuous modeling of the temporal evolution of the physical world and overcoming the dependence of traditional discrete prediction methods on a fixed sampling rate.

[0082] Based on the above embodiments, in this embodiment, the mapping module 402 is specifically used for: The initial state features are input into the dynamic encoder, and the initial state features themselves are used as conditional signals. The dynamic encoder outputs the initial dynamic representation that focuses on characterizing dynamic change information. The step of decoding the dynamic representation of the target combined with the initial state features to generate the state features of the target at future time moments includes: The target dynamic representation and the initial state features are input into the reconstruction decoder. The reconstruction decoder uses the initial state features as the base frame information and fuses the change information contained in the target dynamic representation to reconstruct the state features of the future target moment.

[0083] Based on the above embodiments, in this embodiment, both the dynamic encoder and the reconstruction decoder are implemented based on the cross-attention mechanism; In the dynamic encoder, a set of learnable query tokens are configured, using the initial state features and the state features to be encoded as keys and values, and a corresponding dynamic representation is generated through cross-attention operations; In the reconstruction decoder, the initial state features are used as queries, and the dynamic representation of the input is used as keys and values ​​to generate state features through cross-attention operations.

[0084] Based on the above embodiments, in this embodiment, the device further includes a training module, specifically used for: Obtain a training dataset consisting of a sequence of state samples ordered by physical time, extract the state features at each time step, and use the state features of the first frame of the sequence as the initial state features for training. The initial state features for training and the state features at each time step are input into the dynamic encoder, and the dynamic representation sequence for training at each time step is generated based on the initial state features for training. Based on the state characteristics at each time step in the state sample sequence, the time derivative of the state characteristics is calculated, and the partial derivative of the state characteristics with respect to the state characteristics is mapped to the dynamic representation space using the dynamic encoder to obtain the potential velocity target value at each time step. The training dynamic representation at each time step, the corresponding time information, the initial dynamic representation for training, and the target conditions are input into the velocity prediction network to be trained. The predicted potential velocity is output, and the velocity loss function is constructed to optimize the velocity prediction network using the target potential velocity as the supervision signal.

[0085] Based on the above embodiments, in this embodiment, the device further includes a computing module, specifically used for: A target differential filter is used to perform local polynomial fitting on the state sample sequence in the time dimension by taking a sliding window, and the smooth state feature time derivative is obtained by taking the derivative based on the fitted polynomial.

[0086] Based on the above embodiments, in this embodiment... The time information received by the speed prediction network is the normalized time obtained after normalizing the actual physical time. The normalization process includes: dividing the actual physical time of the current moment by a preset normalized time window length; wherein the normalized time window length corresponds to the time span of the longest sample sequence in the training data, so that the value of the normalized time falls within a preset numerical range during training.

[0087] Based on the above embodiments, in this embodiment, the target condition includes the target state; The device further includes a detection module, specifically used for: The process involves inputting the initial state features for training and the state features at each time step into the dynamic encoder, and generating the dynamic representation sequence for training at each time step using the initial state features as a condition. If the normalization time corresponding to the current integration time exceeds the termination normalization time corresponding to the target condition, the dynamic representation is fixed to the dynamic representation corresponding to the target state, and the dynamic change rate of the velocity prediction network output tends to zero.

[0088] Based on the above embodiments, in this embodiment... The velocity prediction network is a multilayer perceptron and is configured with multiple feature linear modulation layers; The feature linear modulation layer receives the time information and, based on the time information, scales and biases the feature map of the corresponding intermediate layer in the multilayer perceptron channel by channel to incorporate the time information into the network forward process.

[0089] Based on the above embodiments, in this embodiment, the device further includes a first generation module, specifically used for: After mapping the initial state features to the dynamic representation space to obtain the initial dynamic representation, When calling the ordinary differential equation solver, the time integration step size is set to a negative value so that the integration proceeds in reverse physical time to obtain a retrospective dynamic representation corresponding to the past target time. The backtracking dynamic representation is combined with the initial state features for decoding to generate the state features of the past target time.

[0090] Based on the above embodiments, in this embodiment, the device further includes a second generation module, specifically used for: After decoding the dynamic representation of the target combined with the initial state features to generate the state features of the target at future time moments. The state features of the future target time are input into a pre-trained pixel decoder to output a predicted image frame; By repeatedly performing state feature prediction and pixel decoding at different future time points, a continuous sequence of image frames is generated, forming a future video.

[0091] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device can be a robot or other electronic device, and may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logic instructions from the memory 530 to execute a physical time-based continuous prediction and robot control method, including: Obtain the observation data at the current moment and the target conditions used to characterize the predicted target; Extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; Using the initial dynamic representation as the starting point of integration, the ordinary differential equation solver is invoked to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis to obtain the target dynamic representation corresponding to the future target time. The velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs and outputs the dynamic change rate at the current time. The target dynamic representation is combined with the initial state features to decode and generate the state features of the target at future time. Acquire the robot's current observation image and the target conditions used to characterize the task objective; Using any of the above-described continuous prediction methods, the current observed image and the target conditions are used as input to generate a sub-target state feature sequence for multiple future time steps; The sub-target state feature sequence is successively input into a pre-trained policy network, and the corresponding control action sequence is output to control the robot to perform the task objective.

[0092] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in at least one embodiment of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the physical time-based continuous prediction method and robot control method provided by the above methods, including: Obtain the observation data at the current moment and the target conditions used to characterize the predicted target; Extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; Using the initial dynamic representation as the starting point of integration, the ordinary differential equation solver is invoked to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis to obtain the target dynamic representation corresponding to the future target time. The velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs and outputs the dynamic change rate at the current time. The target dynamic representation is combined with the initial state features to decode and generate the state features of the target at future time. Acquire the robot's current observation image and the target conditions used to characterize the task objective; Using any of the above-described continuous prediction methods, the current observed image and the target conditions are used as input to generate a sub-target state feature sequence for multiple future time steps; The sub-target state feature sequence is successively input into a pre-trained policy network, and the corresponding control action sequence is output to control the robot to perform the task objective.

[0094] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the physical time-based continuous prediction method and robot control method provided by the above methods, including: Obtain the observation data at the current moment and the target conditions used to characterize the predicted target; Extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; Using the initial dynamic representation as the starting point of integration, the ordinary differential equation solver is invoked to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis to obtain the target dynamic representation corresponding to the future target time. The velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs and outputs the dynamic change rate at the current time. The target dynamic representation is combined with the initial state features to decode and generate the state features of the target at future time. Acquire the robot's current observation image and the target conditions used to characterize the task objective; Using any of the above-described continuous prediction methods, the current observed image and the target conditions are used as input to generate a sub-target state feature sequence for multiple future time steps; The sub-target state feature sequence is successively input into a pre-trained policy network, and the corresponding control action sequence is output to control the robot to perform the task objective.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A continuous prediction method based on physical time, characterized in that, include: Obtain the observation data at the current moment and the target conditions used to characterize the predicted target; Extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; Using the initial dynamic representation as the starting point of integration, the ordinary differential equation solver is invoked to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis to obtain the target dynamic representation corresponding to the future target time. The velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs and outputs the dynamic change rate at the current time. The target dynamic representation is combined with the initial state features for decoding to generate the state features of the target at future time moments.

2. The continuous prediction method based on physical time according to claim 1, characterized in that, The step of mapping the initial state features to the dynamic representation space to obtain the initial dynamic representation includes: The initial state features are input into the dynamic encoder, and the initial state features themselves are used as conditional signals. The dynamic encoder outputs the initial dynamic representation that focuses on characterizing dynamic change information. The step of decoding the dynamic representation of the target combined with the initial state features to generate the state features of the target at future time moments includes: The target dynamic representation and the initial state features are input into the reconstruction decoder. The reconstruction decoder uses the initial state features as the base frame information and fuses the change information contained in the target dynamic representation to reconstruct the state features of the future target moment.

3. The continuous prediction method based on physical time according to claim 2, characterized in that, Both the dynamic encoder and the reconstruction decoder are implemented based on the cross-attention mechanism; In the dynamic encoder, a set of learnable query tokens are configured, using the initial state features and the state features to be encoded as keys and values, and a corresponding dynamic representation is generated through cross-attention operations; In the reconstruction decoder, the initial state features are used as queries, and the dynamic representation of the input is used as keys and values ​​to generate state features through cross-attention operations.

4. The continuous prediction method based on physical time according to claim 1, characterized in that, The training process of the velocity prediction network includes: Obtain a training dataset consisting of a sequence of state samples ordered by physical time, extract the state features at each time step, and use the state features of the first frame of the sequence as the initial state features for training. The initial state features for training and the state features at each time step are input into the dynamic encoder, and the dynamic representation sequence for training at each time step is generated based on the initial state features for training. Based on the state characteristics at each time step in the state sample sequence, the time derivative of the state characteristics is calculated, and the partial derivative of the state characteristics with respect to the state characteristics is mapped to the dynamic representation space using the dynamic encoder to obtain the potential velocity target value at each time step. The training dynamic representation at each time step, the corresponding time information, the initial dynamic representation for training, and the target conditions are input into the velocity prediction network to be trained. The predicted potential velocity is output, and the velocity loss function is constructed to optimize the velocity prediction network using the target potential velocity as the supervision signal.

5. The continuous prediction method based on physical time according to claim 4, characterized in that, The step of calculating the time derivative of the state features based on the state features at each time step in the state sample sequence includes: A target differential filter is used to perform local polynomial fitting on the state sample sequence in the time dimension by taking a sliding window, and the smooth state feature time derivative is obtained by taking the derivative based on the fitted polynomial.

6. The continuous prediction method based on physical time according to claim 4, characterized in that, The time information received by the speed prediction network is the normalized time obtained after normalizing the actual physical time. The normalization process includes: dividing the actual physical time of the current moment by a preset normalized time window length; wherein the normalized time window length corresponds to the time span of the longest sample sequence in the training data, so that the value of the normalized time falls within a preset numerical range during training.

7. The continuous prediction method based on physical time according to claim 6, characterized in that, The target conditions include the target state; After inputting the initial state features for training and the state features at each time step into the dynamic encoder, and generating the dynamic representation sequence for training at each time step based on the initial state features for training, the method further includes: If the normalization time corresponding to the current integration time exceeds the termination normalization time corresponding to the target condition, the dynamic representation is fixed to the dynamic representation corresponding to the target state, and the dynamic change rate of the velocity prediction network output tends to zero.

8. The continuous prediction method based on physical time according to claim 1, characterized in that, The velocity prediction network is a multilayer perceptron and is configured with multiple feature linear modulation layers; The feature linear modulation layer receives the time information and, based on the time information, scales and biases the feature map of the corresponding intermediate layer in the multilayer perceptron channel by channel to incorporate the time information into the network forward process.

9. The continuous prediction method based on physical time according to claim 1, characterized in that, After mapping the initial state features to the dynamic representation space to obtain the initial dynamic representation, the method further includes: When calling the ordinary differential equation solver, the time integration step size is set to a negative value so that the integration proceeds in reverse physical time to obtain a retrospective dynamic representation corresponding to the past target time. The backtracking dynamic representation is combined with the initial state features for decoding to generate the state features of the past target time.

10. The continuous prediction method based on physical time according to any one of claims 1 to 9, characterized in that, After decoding the target dynamic representation in conjunction with the initial state features to generate the state features of the future target time, the method further includes: The state features of the future target time are input into a pre-trained pixel decoder to output a predicted image frame; By repeatedly performing state feature prediction and pixel decoding at different future time points, a continuous sequence of image frames is generated, forming a future video.

11. A robot control method, characterized in that, include: Acquire the robot's current observation image and the target conditions used to characterize the task objective; Using the continuous prediction method according to any one of claims 1 to 10, a sub-target state feature sequence for multiple future time steps is generated, with the current observed image and the target conditions as input. The sub-target state feature sequence is successively input into a pre-trained policy network, and the corresponding control action sequence is output to control the robot to perform the task objective.

12. A continuous prediction device based on physical time, characterized in that, include: The acquisition module is used to acquire the observation data at the current moment and the target conditions used to characterize the predicted target; The mapping module is used to extract the state features of the observation data at the current moment as initial state features, and map the initial state features to the dynamic representation space to obtain the initial dynamic representation; The representation module is used to call the ordinary differential equation solver to integrate the velocity field defined by the pre-trained velocity prediction network on the physical time axis, using the initial dynamic representation as the starting point of integration, to obtain the target dynamic representation corresponding to the future target time; wherein, the velocity prediction network takes the dynamic representation at the current time, the time information corresponding to the current integration time, the initial dynamic representation, and the target conditions as inputs, and outputs the dynamic change rate at the current time. The prediction module is used to decode the dynamic representation of the target combined with the initial state features to generate the state features of the target at future time moments.

Images